Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rules migration] Retry failed translations (#11383) #204619

Merged

Conversation

e40pud
Copy link
Contributor

@e40pud e40pud commented Dec 17, 2024

Summary

Internal link to the feature details

These changes add a functionality which allows user to retry failed migration rules.

Other tasks and fixes

  • Integrated MigrationReadyPanel and MigrationProgressPanel to show migration's ready and running states
  • Migration stats pooling issue caused by waiting while there are no pending migrations left. If any other operation triggers startPooling during the waiting it will be ignored and thus latest stats will never come back.

Note

This feature needs siemMigrationsEnabled experimental flag enabled to work.

Testing note

  1. Make sure you have a SIEM migration with failed rules
  2. Open that migration via Security > Rules > SIEM Rules Migrations > {#MIGRATION_WITH_FAILED_RULES}
  3. You should see a Reprocess rules (#) button which triggers failed rules reprocessing

Screen recording

Screen.Recording.2024-12-17.at.17.53.37.mov

@e40pud e40pud added release_note:skip Skip the PR/issue when compiling release notes Team:Threat Hunting Security Solution Threat Hunting Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Dec 17, 2024
@e40pud e40pud requested a review from semd December 17, 2024 17:01
@e40pud e40pud self-assigned this Dec 17, 2024
@e40pud e40pud requested a review from a team as a code owner December 17, 2024 17:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

@elastic-vault-github-plugin-prod elastic-vault-github-plugin-prod bot requested a review from a team as a code owner December 17, 2024 17:22
Copy link
Contributor

@semd semd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Thanks @e40pud, good work 💯

@elastic-vault-github-plugin-prod elastic-vault-github-plugin-prod bot requested a review from a team as a code owner December 19, 2024 14:35
@semd semd removed the request for review from a team December 19, 2024 16:25
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 6523 6524 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 22.1MB 22.1MB +5.3KB

History

cc @e40pud

@e40pud e40pud merged commit e4586da into elastic:main Jan 8, 2025
8 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12670450680

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 8, 2025
…204619)

## Summary

[Internal link](elastic/security-team#10820)
to the feature details

These changes add a functionality which allows user to retry failed
migration rules.

### Other tasks and fixes

* Integrated `MigrationReadyPanel` and `MigrationProgressPanel` to show
migration's `ready` and `running` states
* Migration stats pooling issue caused by waiting while there are no
pending migrations left. If any other operation triggers `startPooling`
during the waiting it will be ignored and thus latest stats will never
come back.

> [!NOTE]
> This feature needs `siemMigrationsEnabled` experimental flag enabled
to work.

### Testing note

1. Make sure you have a SIEM migration with failed rules
2. Open that migration via `Security > Rules > SIEM Rules Migrations >
{#MIGRATION_WITH_FAILED_RULES}`
3. You should see a `Reprocess rules (#)` button which triggers failed
rules reprocessing

## Screen recording

https://github.com/user-attachments/assets/d33dc4a0-1791-4869-aa8d-b0322b5f19c3

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit e4586da)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

crespocarlos pushed a commit to crespocarlos/kibana that referenced this pull request Jan 8, 2025
…204619)

## Summary

[Internal link](elastic/security-team#10820)
to the feature details

These changes add a functionality which allows user to retry failed
migration rules.

### Other tasks and fixes

* Integrated `MigrationReadyPanel` and `MigrationProgressPanel` to show
migration's `ready` and `running` states
* Migration stats pooling issue caused by waiting while there are no
pending migrations left. If any other operation triggers `startPooling`
during the waiting it will be ignored and thus latest stats will never
come back.

> [!NOTE]  
> This feature needs `siemMigrationsEnabled` experimental flag enabled
to work.

### Testing note

1. Make sure you have a SIEM migration with failed rules
2. Open that migration via `Security > Rules > SIEM Rules Migrations >
{#MIGRATION_WITH_FAILED_RULES}`
3. You should see a `Reprocess rules (#)` button which triggers failed
rules reprocessing

## Screen recording


https://github.com/user-attachments/assets/d33dc4a0-1791-4869-aa8d-b0322b5f19c3

---------

Co-authored-by: kibanamachine <[email protected]>
kibanamachine added a commit that referenced this pull request Jan 9, 2025
…205878)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Rules migration] Retry failed translations (#11383)
(#204619)](#204619)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ievgen
Sorokopud","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-08T12:21:17Z","message":"[Rules
migration] Retry failed translations (#11383) (#204619)\n\n##
Summary\r\n\r\n[Internal
link](https://github.com/elastic/security-team/issues/10820)\r\nto the
feature details\r\n\r\nThese changes add a functionality which allows
user to retry failed\r\nmigration rules.\r\n\r\n### Other tasks and
fixes\r\n\r\n* Integrated `MigrationReadyPanel` and
`MigrationProgressPanel` to show\r\nmigration's `ready` and `running`
states\r\n* Migration stats pooling issue caused by waiting while there
are no\r\npending migrations left. If any other operation triggers
`startPooling`\r\nduring the waiting it will be ignored and thus latest
stats will never\r\ncome back.\r\n\r\n> [!NOTE] \r\n> This feature needs
`siemMigrationsEnabled` experimental flag enabled\r\nto work.\r\n\r\n###
Testing note\r\n\r\n1. Make sure you have a SIEM migration with failed
rules\r\n2. Open that migration via `Security > Rules > SIEM Rules
Migrations >\r\n{#MIGRATION_WITH_FAILED_RULES}`\r\n3. You should see a
`Reprocess rules (#)` button which triggers failed\r\nrules
reprocessing\r\n\r\n## Screen
recording\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d33dc4a0-1791-4869-aa8d-b0322b5f19c3\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"e4586dac809ebfe618223aae33e9723e6cc38384","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Threat
Hunting","Team: SecuritySolution","backport:prev-minor"],"title":"[Rules
migration] Retry failed translations
(#11383)","number":204619,"url":"https://github.com/elastic/kibana/pull/204619","mergeCommit":{"message":"[Rules
migration] Retry failed translations (#11383) (#204619)\n\n##
Summary\r\n\r\n[Internal
link](https://github.com/elastic/security-team/issues/10820)\r\nto the
feature details\r\n\r\nThese changes add a functionality which allows
user to retry failed\r\nmigration rules.\r\n\r\n### Other tasks and
fixes\r\n\r\n* Integrated `MigrationReadyPanel` and
`MigrationProgressPanel` to show\r\nmigration's `ready` and `running`
states\r\n* Migration stats pooling issue caused by waiting while there
are no\r\npending migrations left. If any other operation triggers
`startPooling`\r\nduring the waiting it will be ignored and thus latest
stats will never\r\ncome back.\r\n\r\n> [!NOTE] \r\n> This feature needs
`siemMigrationsEnabled` experimental flag enabled\r\nto work.\r\n\r\n###
Testing note\r\n\r\n1. Make sure you have a SIEM migration with failed
rules\r\n2. Open that migration via `Security > Rules > SIEM Rules
Migrations >\r\n{#MIGRATION_WITH_FAILED_RULES}`\r\n3. You should see a
`Reprocess rules (#)` button which triggers failed\r\nrules
reprocessing\r\n\r\n## Screen
recording\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d33dc4a0-1791-4869-aa8d-b0322b5f19c3\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"e4586dac809ebfe618223aae33e9723e6cc38384"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204619","number":204619,"mergeCommit":{"message":"[Rules
migration] Retry failed translations (#11383) (#204619)\n\n##
Summary\r\n\r\n[Internal
link](https://github.com/elastic/security-team/issues/10820)\r\nto the
feature details\r\n\r\nThese changes add a functionality which allows
user to retry failed\r\nmigration rules.\r\n\r\n### Other tasks and
fixes\r\n\r\n* Integrated `MigrationReadyPanel` and
`MigrationProgressPanel` to show\r\nmigration's `ready` and `running`
states\r\n* Migration stats pooling issue caused by waiting while there
are no\r\npending migrations left. If any other operation triggers
`startPooling`\r\nduring the waiting it will be ignored and thus latest
stats will never\r\ncome back.\r\n\r\n> [!NOTE] \r\n> This feature needs
`siemMigrationsEnabled` experimental flag enabled\r\nto work.\r\n\r\n###
Testing note\r\n\r\n1. Make sure you have a SIEM migration with failed
rules\r\n2. Open that migration via `Security > Rules > SIEM Rules
Migrations >\r\n{#MIGRATION_WITH_FAILED_RULES}`\r\n3. You should see a
`Reprocess rules (#)` button which triggers failed\r\nrules
reprocessing\r\n\r\n## Screen
recording\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d33dc4a0-1791-4869-aa8d-b0322b5f19c3\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"e4586dac809ebfe618223aae33e9723e6cc38384"}}]}]
BACKPORT-->

Co-authored-by: Ievgen Sorokopud <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting Security Solution Threat Hunting Team v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants