[Siem migrations] Implement UI service and migrations polling #201503

semd · 2024-11-23T15:38:08Z

Summary

Sends "Rule migration complete" notifications from anywhere in the Security Solution app, whenever a rule migration finishes, with a link to the migrated rules.

The polling logic has been encapsulated in the new siemMigrations.rules service so the request loop is centralized in one place. The value updates are broadcasted using the latestStats$ observable.
It will only keep requesting while there are running migrations and will stop automatically when no more migrations are running.

The reusable useLatestStats hook has been created for the UI components to consume. This approach allows multiple components to listen and update their content automatically with every rule migration stats update, having only one request loop running.

The polling will only start if it's not already running and only if the SIEM migration functionality is available, which includes:

Experimental flag enabled
Enterprise license
TODO: feature capability check (RBAC issue)

The polling will try to start when:

Automatically with the Security Solution application starts
The first render of every page that uses useLatestStats hook.
TODO: A new migration is created from the onboarding page (issue)

Tests will be implemented in this task

Example

A Rule migration finishes while using Timeline in the Alerts page:

rule_migrations_notifications.mov

elasticmachine · 2024-11-23T15:38:11Z

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

…ns/ui_service

e40pud

Looks great!! We will need to add route handler to be able to open a specific migration.

e40pud · 2024-11-25T09:28:30Z

x-pack/plugins/security_solution/public/siem_migrations/rules/service/success_notification.tsx

+
+const SuccessToastContent: React.FC<{ migration: RuleMigrationStats }> = ({ migration }) => {
+  const { migration_id: path, rules } = migration;
+  const navigation = { deepLinkId: SecurityPageName.siemMigrationsRules, path };


We will need to add a route to handle migration_id as a path parameter (somewhere here x-pack/plugins/security_solution/public/siem_migrations/routes.tsx). Right now we only open a default page where we would pick the first available migration and use it.

bojanasan · 2024-11-25T09:31:03Z

@semd looks great! I'll work with @benironside on copy. When possible it would be great to update the styling.

mock link

angorayc · 2024-11-25T13:57:38Z

...security_solution/server/lib/siem_migrations/rules/data/rule_migrations_data_rules_client.ts

@@ -226,30 +229,34 @@ export class RuleMigrationsDataRulesClient extends RuleMigrationsDataBaseClient
        throw error;
      });

-    const { pending, processing, completed, lastUpdatedAt, failed } = result.aggregations ?? {};
+    // const { pending, processing, completed, lastUpdatedAt, createdAt, failed }


We don't need this line anymore, right?

elasticmachine · 2024-11-25T16:34:27Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 96be006

Failed CI Steps

Serverless Rule Management - Prebuilt Rules - Security Solution Cypress Tests #1

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	6259	6263	+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	13.3MB	13.4MB	+6.6KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	86.4KB	86.6KB	+152.0B

Unknown metric groups

async chunk count

id	before	after	diff
`securitySolution`	100	101	+1

ESLint disabled line counts

id	before	after	diff
`securitySolution`	555	556	+1

Total ESLint disabled count

id	before	after	diff
`securitySolution`	640	641	+1

History

cc @semd

kibanamachine · 2024-11-25T17:48:24Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12015799904

…c#201503) ## Summary Sends "Rule migration complete" notifications from anywhere in the Security Solution app, whenever a rule migration finishes, with a link to the migrated rules. The polling logic has been encapsulated in the new `siemMigrations.rules` service so the request loop is centralized in one place. The value updates are broadcasted using the `latestStats$` observable. It will only keep requesting while there are _running_ migrations and will stop automatically when no more migrations are _running_. The reusable `useLatestStats` hook has been created for the UI components to consume. This approach allows multiple components to listen and update their content automatically with every rule migration stats update, having only one request loop running. The polling will only start if it's not already running and only if the SIEM migration functionality is available, which includes: - Experimental flag enabled - _Enterprise_ license - TODO: feature capability check (RBAC [issue](elastic/security-team#11262)) The polling will try to start when: - Automatically with the Security Solution application starts - The first render of every page that uses `useLatestStats` hook. - TODO: A new migration is created from the onboarding page ([issue](elastic/security-team#10667)) Tests will be implemented in [this task](elastic/security-team#11256) ## Example A Rule migration finishes while using Timeline in the Alerts page: https://github.com/user-attachments/assets/aac2b2c8-27fe-40d5-9f32-0bee74c9dc6a (cherry picked from commit b6586a9)

kibanamachine · 2024-11-25T17:53:23Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…201503) (#201664) # Backport This will backport the following commits from `main` to `8.x`: - [[Siem migrations] Implement UI service and migrations polling (#201503)](#201503)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Sergi Massaneda <[email protected]>

…c#201503) ## Summary Sends "Rule migration complete" notifications from anywhere in the Security Solution app, whenever a rule migration finishes, with a link to the migrated rules. The polling logic has been encapsulated in the new `siemMigrations.rules` service so the request loop is centralized in one place. The value updates are broadcasted using the `latestStats$` observable. It will only keep requesting while there are _running_ migrations and will stop automatically when no more migrations are _running_. The reusable `useLatestStats` hook has been created for the UI components to consume. This approach allows multiple components to listen and update their content automatically with every rule migration stats update, having only one request loop running. The polling will only start if it's not already running and only if the SIEM migration functionality is available, which includes: - Experimental flag enabled - _Enterprise_ license - TODO: feature capability check (RBAC [issue](elastic/security-team#11262)) The polling will try to start when: - Automatically with the Security Solution application starts - The first render of every page that uses `useLatestStats` hook. - TODO: A new migration is created from the onboarding page ([issue](elastic/security-team#10667)) Tests will be implemented in [this task](elastic/security-team#11256) ## Example A Rule migration finishes while using Timeline in the Alerts page: https://github.com/user-attachments/assets/aac2b2c8-27fe-40d5-9f32-0bee74c9dc6a

semd added 2 commits November 23, 2024 16:06

implement UI polling in new siem_migrations service

c7eca1f

remove comments

493fdff

semd added release_note:skip Skip the PR/issue when compiling release notes Team:Threat Hunting Security Solution Threat Hunting Team backport:version Backport to applied version labels v8.18.0 labels Nov 23, 2024

semd self-assigned this Nov 23, 2024

semd requested review from a team as code owners November 23, 2024 15:38

semd added 2 commits November 25, 2024 09:54

lazy load service

9b01918

Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…

463706e

…ns/ui_service

semd changed the title ~~[Siem migrations] Implement UI polling with~~ [Siem migrations] Implement UI service and stats polling Nov 25, 2024

semd changed the title ~~[Siem migrations] Implement UI service and stats polling~~ [Siem migrations] Implement UI service and migrations polling Nov 25, 2024

e40pud approved these changes Nov 25, 2024

View reviewed changes

add consistent seq number to migrations

f30c903

angorayc approved these changes Nov 25, 2024

View reviewed changes

fix types

96be006

semd merged commit b6586a9 into elastic:main Nov 25, 2024
46 checks passed

kibanamachine added the v9.0.0 label Nov 25, 2024

kibanamachine mentioned this pull request Nov 25, 2024

[8.x] [Siem migrations] Implement UI service and migrations polling (#201503) #201664

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Siem migrations] Implement UI service and migrations polling #201503

[Siem migrations] Implement UI service and migrations polling #201503

semd commented Nov 23, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Nov 23, 2024

e40pud left a comment

e40pud Nov 25, 2024

bojanasan commented Nov 25, 2024 •

edited

Loading

angorayc Nov 25, 2024

elasticmachine commented Nov 25, 2024 •

edited

Loading

async chunk count

ESLint disabled line counts

Total ESLint disabled count

kibanamachine commented Nov 25, 2024

kibanamachine commented Nov 25, 2024

[Siem migrations] Implement UI service and migrations polling #201503

[Siem migrations] Implement UI service and migrations polling #201503

Conversation

semd commented Nov 23, 2024 • edited by kibanamachine Loading

Summary

Example

elasticmachine commented Nov 23, 2024

e40pud left a comment

Choose a reason for hiding this comment

e40pud Nov 25, 2024

Choose a reason for hiding this comment

bojanasan commented Nov 25, 2024 • edited Loading

angorayc Nov 25, 2024

Choose a reason for hiding this comment

elasticmachine commented Nov 25, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Async chunks

Page load bundle

async chunk count

ESLint disabled line counts

Total ESLint disabled count

History

kibanamachine commented Nov 25, 2024

kibanamachine commented Nov 25, 2024

💚 All backports created successfully

Questions ?

semd commented Nov 23, 2024 •

edited by kibanamachine

Loading

bojanasan commented Nov 25, 2024 •

edited

Loading

elasticmachine commented Nov 25, 2024 •

edited

Loading