[inference] Add support for inference connectors #204541
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pgayvallet
added
release_note:skip
|
/ci |
|
/ci |
|
/ci |
|
/ci |
|
/ci |
|
/ci |
|
/ci |
|
/ci |
|
/ci |
|
/ci |
pgayvallet
commented
Dec 18, 2024
Comment on lines
+46
to
+57
| export function isSupportedConnector(connector: RawConnector): connector is RawInferenceConnector { | ||
| if (!isSupportedConnectorType(connector.actionTypeId)) { | ||
| return false; | ||
| } | ||
| if (connector.actionTypeId === InferenceConnectorType.Inference) { | ||
| const config = connector.config ?? {}; | ||
| if (config.taskType !== COMPLETION_TASK_TYPE) { | ||
| return false; | ||
| } | ||
| } | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Checking if a connector is compatible is no longer just based on its type, so I had to create that new check logic.
For inference connectors, we might eventually want to filter based on the provider, but for now I feel like filtering on completion tasks should be sufficient
There was a problem hiding this comment.
moved to x-pack/platform/packages/shared/ai-infra/inference-common/src/connectors.ts
Comment on lines
+25
to
+30
| export const inferenceAdapter: InferenceConnectorAdapter = { | ||
| chatComplete: ({ | ||
| executor, | ||
| system, | ||
| messages, | ||
| toolChoice, |
There was a problem hiding this comment.
The adapter working with the inference connector. Very similar to the existing openAI adapter, which is why most of the in/out processing logic has been factorized.
There was a problem hiding this comment.
deleted because the o11y assistant is now using the helpers exposed from @kbn/inference-common
| { | ||
| method: 'POST', | ||
| path: `_inference/completion/${this.inferenceId}/_unified`, | ||
| body: { ...params.body, n: undefined }, // exclude n param for now, constant is used on the inference API side | ||
| }, | ||
| { | ||
| asStream: true, | ||
| meta: true, | ||
| signal: params.signal, |
There was a problem hiding this comment.
propagating the signal was missing, fixed that
Comment on lines
+198
to
+202
| // errors should be thrown as it will not be a stream response | ||
| if (response.statusCode >= 400) { | ||
| const error = await streamToString(response.body as unknown as Readable); | ||
| throw new Error(error); | ||
| } |
There was a problem hiding this comment.
Errors from the API call were not caught and simply streamed to the consumer, which was fairly problematic.
Fixed by checking the status code and throwing for error code.
The response format remain unchanged for successful calls (only returning the streaming body)
|
Pinging @elastic/appex-ai-infra (Team:AI Infra) |
legrego
approved these changes
Dec 18, 2024
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
YulNaumenko
approved these changes
Dec 18, 2024
|
When I try doing a test run with inference connector, it works when I try the first time: screenshot
But if I run it again I get: screenshot
|
|
The Obs AI Assistant is mostly working, except when I ask about alerts or slos: This doesn't occur when I use the "regular" OpenAI connector |
This is a known issue, which ES ML team are working on to fix elastic/elasticsearch#119000 |
I believe this one is related to another issues ML team is tracking https://github.com/elastic/ml-team/issues/1441 |
neptunian
approved these changes
Dec 19, 2024
…e-connector-support
joemcelroy
approved these changes
Dec 20, 2024
…e-connector-support
|
Confirmed that the errors are coming from elastic/elasticsearch#119000, so I'll consider the PR is fine to merge |
|
Starting backport for target branches: 8.x |
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
Unknown metric groupsAPI count
History
|
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Dec 23, 2024
## Summary ~Depends on~ elastic#200249 merged! Fix elastic#199082 - Add support for the `inference` stack connectors to the `inference` plugin (everything is inference) - Adapt the o11y assistant to use the `inference-common` utilities for connector filtering / compat checking ## How to test **1. Starts ES with the unified completion feature flag** ```sh yarn es snapshot --license trial ES_JAVA_OPTS="-Des.inference_unified_feature_flag_enabled=true" ``` **2. Enable the inference connector for Kibana** In the Kibana config file: ```yaml xpack.stack_connectors.enableExperimental: ['inferenceConnectorOn'] ``` **3. Start Dev Kibana** ```sh node scripts/kibana --dev --no-base-path ``` **4. Create an inference connector** Go to `http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`, create an inference connector - Type: `AI connector` then - Service: `OpenAI` - API Key: Gwzk... Kidding, please ping someone - Model ID: `gpt-4o` - Task type: `completion` -> save **5. test the o11y assistant** Use the assistant as you would do for any other connector (just make sure the inference connector is selected as the one being used) and do your testing. --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 3dcae51)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Dec 23, 2024
…5078) # Backport This will backport the following commits from `main` to `8.x`: - [[inference] Add support for inference connectors (#204541)](#204541) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Pierre Gayvallet","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-23T09:20:42Z","message":"[inference] Add support for inference connectors (#204541)\n\n## Summary\r\n\r\n~Depends on~ #200249 merged!\r\n\r\nFix https://github.com/elastic/kibana/issues/199082\r\n\r\n- Add support for the `inference` stack connectors to the `inference`\r\nplugin (everything is inference)\r\n- Adapt the o11y assistant to use the `inference-common` utilities for\r\nconnector filtering / compat checking\r\n\r\n## How to test\r\n\r\n**1. Starts ES with the unified completion feature flag**\r\n\r\n```sh\r\nyarn es snapshot --license trial ES_JAVA_OPTS=\"-Des.inference_unified_feature_flag_enabled=true\"\r\n```\r\n\r\n**2. Enable the inference connector for Kibana**\r\n\r\nIn the Kibana config file:\r\n```yaml\r\nxpack.stack_connectors.enableExperimental: ['inferenceConnectorOn']\r\n```\r\n\r\n**3. Start Dev Kibana**\r\n\r\n```sh\r\nnode scripts/kibana --dev --no-base-path\r\n```\r\n\r\n**4. Create an inference connector**\r\n\r\nGo to\r\n`http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`,\r\ncreate an inference connector\r\n\r\n- Type: `AI connector`\r\n\r\nthen\r\n\r\n- Service: `OpenAI`\r\n- API Key: Gwzk... Kidding, please ping someone\r\n- Model ID: `gpt-4o`\r\n- Task type: `completion`\r\n\r\n-> save\r\n\r\n**5. test the o11y assistant**\r\n\r\nUse the assistant as you would do for any other connector (just make\r\nsure the inference connector is selected as the one being used) and do\r\nyour testing.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"3dcae5144034a146068566e920ade2e57d9abd08","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Obs AI Assistant","backport:version","Team:AI Infra","v8.18.0"],"title":"[inference] Add support for inference connectors","number":204541,"url":"https://github.com/elastic/kibana/pull/204541","mergeCommit":{"message":"[inference] Add support for inference connectors (#204541)\n\n## Summary\r\n\r\n~Depends on~ #200249 merged!\r\n\r\nFix https://github.com/elastic/kibana/issues/199082\r\n\r\n- Add support for the `inference` stack connectors to the `inference`\r\nplugin (everything is inference)\r\n- Adapt the o11y assistant to use the `inference-common` utilities for\r\nconnector filtering / compat checking\r\n\r\n## How to test\r\n\r\n**1. Starts ES with the unified completion feature flag**\r\n\r\n```sh\r\nyarn es snapshot --license trial ES_JAVA_OPTS=\"-Des.inference_unified_feature_flag_enabled=true\"\r\n```\r\n\r\n**2. Enable the inference connector for Kibana**\r\n\r\nIn the Kibana config file:\r\n```yaml\r\nxpack.stack_connectors.enableExperimental: ['inferenceConnectorOn']\r\n```\r\n\r\n**3. Start Dev Kibana**\r\n\r\n```sh\r\nnode scripts/kibana --dev --no-base-path\r\n```\r\n\r\n**4. Create an inference connector**\r\n\r\nGo to\r\n`http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`,\r\ncreate an inference connector\r\n\r\n- Type: `AI connector`\r\n\r\nthen\r\n\r\n- Service: `OpenAI`\r\n- API Key: Gwzk... Kidding, please ping someone\r\n- Model ID: `gpt-4o`\r\n- Task type: `completion`\r\n\r\n-> save\r\n\r\n**5. test the o11y assistant**\r\n\r\nUse the assistant as you would do for any other connector (just make\r\nsure the inference connector is selected as the one being used) and do\r\nyour testing.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"3dcae5144034a146068566e920ade2e57d9abd08"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204541","number":204541,"mergeCommit":{"message":"[inference] Add support for inference connectors (#204541)\n\n## Summary\r\n\r\n~Depends on~ #200249 merged!\r\n\r\nFix https://github.com/elastic/kibana/issues/199082\r\n\r\n- Add support for the `inference` stack connectors to the `inference`\r\nplugin (everything is inference)\r\n- Adapt the o11y assistant to use the `inference-common` utilities for\r\nconnector filtering / compat checking\r\n\r\n## How to test\r\n\r\n**1. Starts ES with the unified completion feature flag**\r\n\r\n```sh\r\nyarn es snapshot --license trial ES_JAVA_OPTS=\"-Des.inference_unified_feature_flag_enabled=true\"\r\n```\r\n\r\n**2. Enable the inference connector for Kibana**\r\n\r\nIn the Kibana config file:\r\n```yaml\r\nxpack.stack_connectors.enableExperimental: ['inferenceConnectorOn']\r\n```\r\n\r\n**3. Start Dev Kibana**\r\n\r\n```sh\r\nnode scripts/kibana --dev --no-base-path\r\n```\r\n\r\n**4. Create an inference connector**\r\n\r\nGo to\r\n`http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`,\r\ncreate an inference connector\r\n\r\n- Type: `AI connector`\r\n\r\nthen\r\n\r\n- Service: `OpenAI`\r\n- API Key: Gwzk... Kidding, please ping someone\r\n- Model ID: `gpt-4o`\r\n- Task type: `completion`\r\n\r\n-> save\r\n\r\n**5. test the o11y assistant**\r\n\r\nUse the assistant as you would do for any other connector (just make\r\nsure the inference connector is selected as the one being used) and do\r\nyour testing.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"3dcae5144034a146068566e920ade2e57d9abd08"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: Pierre Gayvallet <[email protected]>
stratoula
pushed a commit
to stratoula/kibana
that referenced
this pull request
Jan 2, 2025
## Summary ~Depends on~ elastic#200249 merged! Fix elastic#199082 - Add support for the `inference` stack connectors to the `inference` plugin (everything is inference) - Adapt the o11y assistant to use the `inference-common` utilities for connector filtering / compat checking ## How to test **1. Starts ES with the unified completion feature flag** ```sh yarn es snapshot --license trial ES_JAVA_OPTS="-Des.inference_unified_feature_flag_enabled=true" ``` **2. Enable the inference connector for Kibana** In the Kibana config file: ```yaml xpack.stack_connectors.enableExperimental: ['inferenceConnectorOn'] ``` **3. Start Dev Kibana** ```sh node scripts/kibana --dev --no-base-path ``` **4. Create an inference connector** Go to `http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`, create an inference connector - Type: `AI connector` then - Service: `OpenAI` - API Key: Gwzk... Kidding, please ping someone - Model ID: `gpt-4o` - Task type: `completion` -> save **5. test the o11y assistant** Use the assistant as you would do for any other connector (just make sure the inference connector is selected as the one being used) and do your testing. --------- Co-authored-by: kibanamachine <[email protected]>
benakansara
pushed a commit
to benakansara/kibana
that referenced
this pull request
Jan 2, 2025
## Summary ~Depends on~ elastic#200249 merged! Fix elastic#199082 - Add support for the `inference` stack connectors to the `inference` plugin (everything is inference) - Adapt the o11y assistant to use the `inference-common` utilities for connector filtering / compat checking ## How to test **1. Starts ES with the unified completion feature flag** ```sh yarn es snapshot --license trial ES_JAVA_OPTS="-Des.inference_unified_feature_flag_enabled=true" ``` **2. Enable the inference connector for Kibana** In the Kibana config file: ```yaml xpack.stack_connectors.enableExperimental: ['inferenceConnectorOn'] ``` **3. Start Dev Kibana** ```sh node scripts/kibana --dev --no-base-path ``` **4. Create an inference connector** Go to `http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`, create an inference connector - Type: `AI connector` then - Service: `OpenAI` - API Key: Gwzk... Kidding, please ping someone - Model ID: `gpt-4o` - Task type: `completion` -> save **5. test the o11y assistant** Use the assistant as you would do for any other connector (just make sure the inference connector is selected as the one being used) and do your testing. --------- Co-authored-by: kibanamachine <[email protected]>
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Jan 13, 2025
## Summary ~Depends on~ elastic#200249 merged! Fix elastic#199082 - Add support for the `inference` stack connectors to the `inference` plugin (everything is inference) - Adapt the o11y assistant to use the `inference-common` utilities for connector filtering / compat checking ## How to test **1. Starts ES with the unified completion feature flag** ```sh yarn es snapshot --license trial ES_JAVA_OPTS="-Des.inference_unified_feature_flag_enabled=true" ``` **2. Enable the inference connector for Kibana** In the Kibana config file: ```yaml xpack.stack_connectors.enableExperimental: ['inferenceConnectorOn'] ``` **3. Start Dev Kibana** ```sh node scripts/kibana --dev --no-base-path ``` **4. Create an inference connector** Go to `http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`, create an inference connector - Type: `AI connector` then - Service: `OpenAI` - API Key: Gwzk... Kidding, please ping someone - Model ID: `gpt-4o` - Task type: `completion` -> save **5. test the o11y assistant** Use the assistant as you would do for any other connector (just make sure the inference connector is selected as the one being used) and do your testing. --------- Co-authored-by: kibanamachine <[email protected]>
viduni94
pushed a commit
to viduni94/kibana
that referenced
this pull request
Jan 23, 2025
## Summary ~Depends on~ elastic#200249 merged! Fix elastic#199082 - Add support for the `inference` stack connectors to the `inference` plugin (everything is inference) - Adapt the o11y assistant to use the `inference-common` utilities for connector filtering / compat checking ## How to test **1. Starts ES with the unified completion feature flag** ```sh yarn es snapshot --license trial ES_JAVA_OPTS="-Des.inference_unified_feature_flag_enabled=true" ``` **2. Enable the inference connector for Kibana** In the Kibana config file: ```yaml xpack.stack_connectors.enableExperimental: ['inferenceConnectorOn'] ``` **3. Start Dev Kibana** ```sh node scripts/kibana --dev --no-base-path ``` **4. Create an inference connector** Go to `http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors`, create an inference connector - Type: `AI connector` then - Service: `OpenAI` - API Key: Gwzk... Kidding, please ping someone - Model ID: `gpt-4o` - Task type: `completion` -> save **5. test the o11y assistant** Use the assistant as you would do for any other connector (just make sure the inference connector is selected as the one being used) and do your testing. --------- Co-authored-by: kibanamachine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Depends on#200249 merged!Fix #199082
inferencestack connectors to theinferenceplugin (everything is inference)inference-commonutilities for connector filtering / compat checkingHow to test
1. Starts ES with the unified completion feature flag
yarn es snapshot --license trial ES_JAVA_OPTS="-Des.inference_unified_feature_flag_enabled=true"2. Enable the inference connector for Kibana
In the Kibana config file:
3. Start Dev Kibana
4. Create an inference connector
Go to
http://localhost:5601/app/management/insightsAndAlerting/triggersActionsConnectors/connectors, create an inference connectorAI connectorthen
OpenAIgpt-4ocompletion-> save
5. test the o11y assistant
Use the assistant as you would do for any other connector (just make sure the inference connector is selected as the one being used) and do your testing.