Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SecuritySolution][Onboarding] Siem migration start card and flyout - Part 1 #202886

Merged
merged 40 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
60fbe6f
add support for topics on the onboarding page
semd Nov 22, 2024
3985477
reorganize directories inside components
semd Nov 22, 2024
03b5034
solve conflict
semd Nov 22, 2024
376f2d6
Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…
semd Nov 25, 2024
318d79e
Merge remote-tracking branch 'upstream' into 10667/siem_migrations/on…
semd Nov 26, 2024
8557994
resources schema id
semd Nov 26, 2024
da3938c
update and add tests
semd Nov 26, 2024
132c67b
resources type fix
semd Nov 27, 2024
94c56c8
add connector selection and auto-resume stopped migrations within the…
semd Nov 28, 2024
a0702c9
Merge remote-tracking branch 'origin/10667/siem_migrations/onboarding…
semd Nov 28, 2024
1d1f1da
Merge branch 'main' into 10667/siem_migrations/onboarding_ui
elasticmachine Nov 28, 2024
cee50d7
initial implementation
semd Nov 28, 2024
5a9a06a
extract common styles into single places
semd Nov 28, 2024
f2beaf3
format code
semd Nov 28, 2024
9f305c9
Merge branch '10667/siem_migrations/onboarding_ui' into 10667/siem_mi…
semd Nov 28, 2024
4ce992d
conflict solve
semd Nov 28, 2024
8b60935
cleaning
semd Nov 29, 2024
f09344b
solve conflict
semd Nov 29, 2024
ef6c17d
upload panel styles
semd Nov 29, 2024
5cb839d
flyout scafolding
semd Nov 29, 2024
ecadb11
solve conflicts
semd Nov 30, 2024
d79dc47
add upload input
semd Dec 1, 2024
a486424
basic workflow working
semd Dec 3, 2024
0ce93d9
Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…
semd Dec 3, 2024
b02a569
no connector selected callout
semd Dec 3, 2024
216a6dc
solve conflicts
semd Dec 3, 2024
0d6ae1f
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Dec 4, 2024
6e63ffa
add langsmith trace options
semd Dec 4, 2024
86210fc
Merge remote-tracking branch 'refs/remotes/origin/10667/siem_migratio…
semd Dec 4, 2024
ac6e675
lint fixes
semd Dec 4, 2024
dcaffe2
[CI] Auto-commit changed files from 'yarn openapi:generate'
kibanamachine Dec 4, 2024
0957440
organize get and filter parameters
semd Dec 5, 2024
5cdf539
conflicts solved
semd Dec 5, 2024
09b8b5a
Merge remote-tracking branch 'refs/remotes/origin/10667/siem_migratio…
semd Dec 5, 2024
ca33b12
type fixes
semd Dec 5, 2024
0af2d10
typo
semd Dec 5, 2024
349915b
restore batch size
semd Dec 5, 2024
eb87b4b
Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…
semd Dec 5, 2024
9e548ff
use Security common NonEmptyString
semd Dec 5, 2024
aa06fe8
try/catch NDJson parser
semd Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,7 @@ import type {
ResolveTimelineResponse,
} from './timeline/resolve_timeline/resolve_timeline_route.gen';
import type {
CreateRuleMigrationRequestParamsInput,
CreateRuleMigrationRequestBodyInput,
CreateRuleMigrationResponse,
GetAllStatsRuleMigrationResponse,
Expand Down Expand Up @@ -686,7 +687,7 @@ If a record already exists for the specified entity, that record is overwritten
this.log.info(`${new Date().toISOString()} Calling API CreateRuleMigration`);
return this.kbnClient
.request<CreateRuleMigrationResponse>({
path: '/internal/siem_migrations/rules',
path: replaceParams('/internal/siem_migrations/rules/{migration_id}', props.params),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the type perspective, props.params.migration_id could be undefined. Is this an actual case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is auto-generated code from the API schema. But yes, I checked the props.params type here is:
{ migration_id?: string | undefined; }.

headers: {
[ELASTIC_HTTP_VERSION_HEADER]: '1',
},
Expand Down Expand Up @@ -2267,6 +2268,7 @@ export interface CreateRuleProps {
body: CreateRuleRequestBodyInput;
}
export interface CreateRuleMigrationProps {
params: CreateRuleMigrationRequestParamsInput;
body: CreateRuleMigrationRequestBodyInput;
}
export interface CreateTimelinesProps {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ export const SIEM_MIGRATIONS_PATH = '/internal/siem_migrations' as const;
export const SIEM_RULE_MIGRATIONS_PATH = `${SIEM_MIGRATIONS_PATH}/rules` as const;

export const SIEM_RULE_MIGRATIONS_ALL_STATS_PATH = `${SIEM_RULE_MIGRATIONS_PATH}/stats` as const;
export const SIEM_RULE_MIGRATION_CREATE_PATH =
`${SIEM_RULE_MIGRATIONS_PATH}/{migration_id?}` as const;
export const SIEM_RULE_MIGRATION_PATH = `${SIEM_RULE_MIGRATIONS_PATH}/{migration_id}` as const;
export const SIEM_RULE_MIGRATION_START_PATH = `${SIEM_RULE_MIGRATION_PATH}/start` as const;
export const SIEM_RULE_MIGRATION_RETRY_PATH = `${SIEM_RULE_MIGRATION_PATH}/retry` as const;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,27 @@
import { z } from '@kbn/zod';
import { ArrayFromString } from '@kbn/zod-helpers';

import { NonEmptyString, ConnectorId, LangSmithOptions } from '../../common.gen';
import {
OriginalRule,
ElasticRulePartial,
RuleMigrationTranslationResult,
RuleMigrationComments,
RuleMigrationTaskStats,
OriginalRule,
RuleMigration,
RuleMigrationTranslationStats,
RuleMigrationResourceData,
RuleMigrationResourceType,
RuleMigrationResource,
} from '../../rule_migration.gen';
import { NonEmptyString, ConnectorId, LangSmithOptions } from '../../common.gen';

export type CreateRuleMigrationRequestParams = z.infer<typeof CreateRuleMigrationRequestParams>;
export const CreateRuleMigrationRequestParams = z.object({
migration_id: NonEmptyString.optional(),
});
export type CreateRuleMigrationRequestParamsInput = z.input<
typeof CreateRuleMigrationRequestParams
>;

export type CreateRuleMigrationRequestBody = z.infer<typeof CreateRuleMigrationRequestBody>;
export const CreateRuleMigrationRequestBody = z.array(OriginalRule);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,38 +4,7 @@ info:
version: '1'
paths:
# Rule migrations APIs

/internal/siem_migrations/rules:
post:
summary: Creates a new rule migration
operationId: CreateRuleMigration
x-codegen-enabled: true
x-internal: true
description: Creates a new SIEM rules migration using the original vendor rules provided
tags:
- SIEM Rule Migrations
requestBody:
required: true
content:
application/json:
schema:
type: array
items:
$ref: '../../rule_migration.schema.yaml#/components/schemas/OriginalRule'
responses:
200:
description: Indicates migration have been created correctly.
content:
application/json:
schema:
type: object
required:
- migration_id
properties:
migration_id:
description: The migration id created.
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'

put:
summary: Updates rules migrations
operationId: UpdateRuleMigration
Expand Down Expand Up @@ -81,95 +50,64 @@ paths:
type: boolean
description: Indicates rules migrations have been updated.

/internal/siem_migrations/rules/{migration_id}/install:
post:
summary: Installs translated migration rules
operationId: InstallMigrationRules
/internal/siem_migrations/rules/stats:
get:
summary: Retrieves the stats for all rule migrations
operationId: GetAllStatsRuleMigration
x-codegen-enabled: true
description: Installs migration rules
x-internal: true
description: Retrieves the rule migrations stats for all migrations stored in the system
tags:
- SIEM Rule Migrations
parameters:
- name: migration_id
in: path
required: true
schema:
description: The migration id to isnstall rules for
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
requestBody:
required: true
content:
application/json:
schema:
type: array
items:
description: The rule migration id
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
responses:
200:
description: Indicates rules migrations have been installed correctly.
description: Indicates rule migrations have been retrieved correctly.
content:
application/json:
schema:
type: object
required:
- installed
properties:
installed:
type: boolean
description: Indicates rules migrations have been installed.
type: array
items:
$ref: '../../rule_migration.schema.yaml#/components/schemas/RuleMigrationTaskStats'

/internal/siem_migrations/rules/{migration_id}/install_translated:
## Specific rule migration APIs

/internal/siem_migrations/rules/{migration_id}:
post:
summary: Installs all translated migration rules
operationId: InstallTranslatedMigrationRules
summary: Creates a new rule migration
operationId: CreateRuleMigration
x-codegen-enabled: true
description: Installs all translated migration rules
x-internal: true
description: Creates a new SIEM rules migration using the original vendor rules provided
tags:
- SIEM Rule Migrations
parameters:
- name: migration_id
in: path
required: true
required: false
schema:
description: The migration id to install translated rules for
description: The migration id to create rules for
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
requestBody:
required: true
content:
application/json:
schema:
type: array
items:
$ref: '../../rule_migration.schema.yaml#/components/schemas/OriginalRule'
responses:
200:
description: Indicates rules migrations have been installed correctly.
description: Indicates migration have been created correctly.
content:
application/json:
schema:
type: object
required:
- installed
- migration_id
properties:
installed:
type: boolean
description: Indicates rules migrations have been installed.

/internal/siem_migrations/rules/stats:
get:
summary: Retrieves the stats for all rule migrations
operationId: GetAllStatsRuleMigration
x-codegen-enabled: true
x-internal: true
description: Retrieves the rule migrations stats for all migrations stored in the system
tags:
- SIEM Rule Migrations
responses:
200:
description: Indicates rule migrations have been retrieved correctly.
content:
application/json:
schema:
type: array
items:
$ref: '../../rule_migration.schema.yaml#/components/schemas/RuleMigrationTaskStats'

## Specific rule migration APIs

/internal/siem_migrations/rules/{migration_id}:
migration_id:
description: The migration id created.
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
get:
summary: Retrieves all the rules of a migration
operationId: GetRuleMigration
Expand Down Expand Up @@ -222,6 +160,73 @@ paths:
204:
description: Indicates the migration id was not found.

/internal/siem_migrations/rules/{migration_id}/install:
post:
summary: Installs translated migration rules
operationId: InstallMigrationRules
x-codegen-enabled: true
description: Installs migration rules
tags:
- SIEM Rule Migrations
parameters:
- name: migration_id
in: path
required: true
schema:
description: The migration id to install rules for
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
requestBody:
required: true
content:
application/json:
schema:
type: array
items:
description: The rule migration id
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
responses:
200:
description: Indicates rules migrations have been installed correctly.
content:
application/json:
schema:
type: object
required:
- installed
properties:
installed:
type: boolean
description: Indicates rules migrations have been installed.

/internal/siem_migrations/rules/{migration_id}/install_translated:
post:
summary: Installs all translated migration rules
operationId: InstallTranslatedMigrationRules
x-codegen-enabled: true
description: Installs all translated migration rules
tags:
- SIEM Rule Migrations
parameters:
- name: migration_id
in: path
required: true
schema:
description: The migration id to install translated rules for
$ref: '../../common.schema.yaml#/components/schemas/NonEmptyString'
responses:
200:
description: Indicates rules migrations have been installed correctly.
content:
application/json:
schema:
type: object
required:
- installed
properties:
installed:
type: boolean
description: Indicates rules migrations have been installed.

/internal/siem_migrations/rules/{migration_id}/start:
put:
summary: Starts a rule migration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,7 @@ import { z } from '@kbn/zod';
* A string that is not empty and does not contain only whitespace
*/
export type NonEmptyString = z.infer<typeof NonEmptyString>;
export const NonEmptyString = z
.string()
.min(1)
.regex(/^(?! *$).+$/);
export const NonEmptyString = z.string().min(1).regex(/\S+/);
semd marked this conversation as resolved.
Show resolved Hide resolved

/**
* The GenAI connector id to use.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ components:
schemas:
NonEmptyString:
type: string
pattern: ^(?! *$).+$
pattern: \S+
minLength: 1
description: A string that is not empty and does not contain only whitespace
ConnectorId:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,19 @@ import { NonEmptyString } from './common.gen';
export type OriginalRuleVendor = z.infer<typeof OriginalRuleVendor>;
export const OriginalRuleVendor = z.literal('splunk');

/**
* The original rule annotations containing additional information.
*/
export type OriginalRuleAnnotations = z.infer<typeof OriginalRuleAnnotations>;
export const OriginalRuleAnnotations = z
.object({
/**
* The original rule Mitre Attack IDs.
*/
mitre_attack: z.array(z.string()).optional(),
})
.catchall(z.unknown());

/**
* The original rule to migrate.
*/
Expand All @@ -40,23 +53,23 @@ export const OriginalRule = z.object({
/**
* The original rule name.
*/
title: z.string(),
title: NonEmptyString,
/**
* The original rule description.
*/
description: z.string(),
/**
* The original rule query.
*/
query: z.string(),
query: z.string().min(1),
/**
* The original rule query language.
*/
query_language: z.string(),
/**
* The original rule Mitre Attack technique IDs.
* The original rule annotations containing additional information.
*/
mitre_attack_ids: z.array(z.string()).optional(),
annotations: OriginalRuleAnnotations.optional(),
});

/**
Expand Down
Loading