Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] Updates test plans for importing and exporting prebuilt rules #204889

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[Security Solution] Updates test plans for importing and exporting prebuilt rules #204889

wants to merge 4 commits into from
+141 −14

Conversation

dplumlee
Copy link
Contributor

Summary

Addresses #202079

Updates the existing import and export rule test plans to include front end tests as well as more exhaustive coverage of the prebuilt rule customization milestone 3 epic

@dplumlee dplumlee added release_note:skip Skip the PR/issue when compiling release notes test-plan v9.0.0 Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area v8.18.0 labels Dec 19, 2024
@dplumlee dplumlee self-assigned this Dec 19, 2024
@dplumlee dplumlee requested a review from a team as a code owner December 19, 2024 06:22
@dplumlee dplumlee requested a review from nikitaindik December 19, 2024 06:22
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

@dplumlee dplumlee added the backport:version Backport to applied version labels label Dec 19, 2024
@banderror banderror requested a review from pborgonovi December 23, 2024 19:09
@banderror
Copy link
Contributor

@pborgonovi Please review this one

nikitaindik
nikitaindik reviewed Jan 3, 2025
View reviewed changes
Copy link
Contributor

@nikitaindik nikitaindik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @dplumlee! Nice work! 👍

I have reviewed and can confirm that most of the cases from the ticket are covered. However, I couldn’t find explicit test cases for these:

  • Converting custom rules to prebuilt rules on upgrade
  • Users can import custom rules with rule_id equal to that of a not-installed prebuilt rule (issue)
  • Incorrect is_customized Value on Re-Import of Non-Customized Prebuilt Rule (issue)

I also left a few comments. Whenever you have time, please take a look

@dplumlee
Copy link
Contributor Author

dplumlee commented Jan 3, 2025 •
edited
Loading

@nikitaindik

Incorrect is_customized Value on Re-Import of Non-Customized Prebuilt Rule (#202613)

This was one of the bugs related to the missing base version so I believe it'd be covered by the first scenario: Importing an unmodified prebuilt rule with a matching rule_id and version.

Users can import custom rules with rule_id equal to that of a not-installed prebuilt rule (#180198)

For this one, I'm not sure what the expected action should be? (failing, changing rule_id, etc.) I think there needs to be more discussion in the ticket and then we can write the test case for the expected outcome.

Converting custom rules to prebuilt rules on upgrade

For this one did you mean customized rules? If so, I can add that - I think it'd be good to add a similar test in the rule upgrade test plan as well

@pborgonovi
Copy link
Contributor

Thanks for addressing the comments @dplumlee

I'll follow up on new bug I raised while reviewing this plan.

The coverage looks good to me! :)

nikitaindik
nikitaindik reviewed Jan 8, 2025
View reviewed changes
...ins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md
Given the import payload contains a custom rule with a matching rule_id and version
And the overwrite flag is set to true
When the user imports the rule
Then the rule should be created or updated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If imported custom rule has rule_id and version that match the currently installed rule, then it's already created and has to be updated. Is my understanding correct? If yes, then I think we can remove the mention of "created".

nikitaindik
nikitaindik reviewed Jan 8, 2025
View reviewed changes
...ins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md
```Gherkin
Given the import payload contains a prebuilt rule with a matching rule_id but no matching version
And the overwrite flag is set to true
When the user imports the rule
Then the rule should be created or updated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps here it should be just "created", without "updated"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nikitaindik nikitaindik nikitaindik left review comments

@pborgonovi pborgonovi pborgonovi approved these changes

At least 1 approving review is required to merge this pull request.

Assignees

@dplumlee dplumlee

Labels
backport:version Backport to applied version labels Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test-plan v8.18.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@dplumlee @elasticmachine @banderror @pborgonovi @nikitaindik