[SIEM Rule Migration] Minor prompt improvements #201941
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
P1llus
added
release_note:skip
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
P1llus
added
backport:version
…atterns in translation
💚 Build Succeeded
Metrics [docs]
History
cc @P1llus |
|
Starting backport for target branches: 8.x |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added
the
backport missing
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
semd
pushed a commit
to semd/kibana
that referenced
this pull request
Nov 30, 2024
## Summary Adding some minor improvements and modifies the format on some of the prompts used in the siem_migration process, a few specifics below: 1. Moved prebuilt rules matching to JSON output parsing, as its less prone for issues on some models. 2. Moved some of the formatting of the prompts to focus on xml-like tags, as it often gives good results on multiple models. Also context that is relevant to perform an action is moved to system, while context related to the question (and to the response) is kept in the human prompt. 3. Moved process_query to ChatPromptTemplate, moved the collection of ResourceContext outside of the prompt because of this. (cherry picked from commit eb87889) # Conflicts: # x-pack/plugins/security_solution/server/lib/siem_migrations/rules/task/agent/nodes/match_prebuilt_rule/match_prebuilt_rule.ts
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
semd
added a commit
that referenced
this pull request
Nov 30, 2024
) # Backport This will backport the following commits from `main` to `8.x`: - [[SIEM Rule Migration] Minor prompt improvements (#201941)](#201941) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Marius Iversen","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-27T15:30:34Z","message":"[SIEM Rule Migration] Minor prompt improvements (#201941)\n\n## Summary\n\nAdding some minor improvements and modifies the format on some of the\nprompts used in the siem_migration process, a few specifics below:\n\n1. Moved prebuilt rules matching to JSON output parsing, as its less\nprone for issues on some models.\n2. Moved some of the formatting of the prompts to focus on xml-like\ntags, as it often gives good results on multiple models. Also context\nthat is relevant to perform an action is moved to system, while context\nrelated to the question (and to the response) is kept in the human\nprompt.\n3. Moved process_query to ChatPromptTemplate, moved the collection of\nResourceContext outside of the prompt because of this.","sha":"eb87889934bbe693b6fa410a366a1e46ad136242","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport missing","v9.0.0","Team: SecuritySolution","backport:version","v8.18.0"],"number":201941,"url":"https://github.com/elastic/kibana/pull/201941","mergeCommit":{"message":"[SIEM Rule Migration] Minor prompt improvements (#201941)\n\n## Summary\n\nAdding some minor improvements and modifies the format on some of the\nprompts used in the siem_migration process, a few specifics below:\n\n1. Moved prebuilt rules matching to JSON output parsing, as its less\nprone for issues on some models.\n2. Moved some of the formatting of the prompts to focus on xml-like\ntags, as it often gives good results on multiple models. Also context\nthat is relevant to perform an action is moved to system, while context\nrelated to the question (and to the response) is kept in the human\nprompt.\n3. Moved process_query to ChatPromptTemplate, moved the collection of\nResourceContext outside of the prompt because of this.","sha":"eb87889934bbe693b6fa410a366a1e46ad136242"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201941","number":201941,"mergeCommit":{"message":"[SIEM Rule Migration] Minor prompt improvements (#201941)\n\n## Summary\n\nAdding some minor improvements and modifies the format on some of the\nprompts used in the siem_migration process, a few specifics below:\n\n1. Moved prebuilt rules matching to JSON output parsing, as its less\nprone for issues on some models.\n2. Moved some of the formatting of the prompts to focus on xml-like\ntags, as it often gives good results on multiple models. Also context\nthat is relevant to perform an action is moved to system, while context\nrelated to the question (and to the response) is kept in the human\nprompt.\n3. Moved process_query to ChatPromptTemplate, moved the collection of\nResourceContext outside of the prompt because of this.","sha":"eb87889934bbe693b6fa410a366a1e46ad136242"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Marius Iversen <[email protected]>
kibanamachine
removed
the
backport missing
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Dec 12, 2024
## Summary Adding some minor improvements and modifies the format on some of the prompts used in the siem_migration process, a few specifics below: 1. Moved prebuilt rules matching to JSON output parsing, as its less prone for issues on some models. 2. Moved some of the formatting of the prompts to focus on xml-like tags, as it often gives good results on multiple models. Also context that is relevant to perform an action is moved to system, while context related to the question (and to the response) is kept in the human prompt. 3. Moved process_query to ChatPromptTemplate, moved the collection of ResourceContext outside of the prompt because of this.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adding some minor improvements and modifies the format on some of the prompts used in the siem_migration process, a few specifics below: