Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Expand rule upgrade review functionality test plan and coverage to include all rule fields #205217

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

jpdjere
Copy link
Contributor

@jpdjere jpdjere commented Dec 27, 2024

Partially resolves: #202078

Summary

  • This PR expands the rule upgrade review functionality (performed by the /upgrade/_review endpoint) test plan and coverage to include all diffable fields:
    • Extends: upgrade.md test plan to list all diffable fields that should be tested
    • Refactors and extends existing upgrade_review_prebuilt_rules.[ALGORITHM]_fields.ts integration test suites to actually test reviewing upgrades for all possible diffable fields.
    • Adds new upgrade_review_prebuilt_rules.simple_fields.ts for testing fields which still have the simple diff algorithm, which was missing.

Flaky test run

🕐 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7636

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

@jpdjere jpdjere self-assigned this Dec 27, 2024
@jpdjere jpdjere added test release_note:skip Skip the PR/issue when compiling release notes test-plan v9.0.0 Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area backport:version Backport to applied version labels v8.18.0 labels Dec 27, 2024
@jpdjere jpdjere marked this pull request as ready for review December 27, 2024 19:00
@jpdjere jpdjere requested review from a team as code owners December 27, 2024 19:00
@jpdjere jpdjere requested a review from maximpn December 27, 2024 19:00
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

@jpdjere jpdjere changed the title Expand rule upgrade review functionality test plan and coverage to include all rule fields [Security Solution] Expand rule upgrade review functionality test plan and coverage to include all rule fields Dec 27, 2024
@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7636

[❌] x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/configs/ess.config.ts: 74/100 tests passed.
[✅] x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/configs/serverless.config.ts: 100/100 tests passed.

see run history

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

cc @jpdjere

@maximpn maximpn marked this pull request as draft January 8, 2025 11:48
@elasticmachine
Copy link
Contributor

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:version Backport to applied version labels Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test test-plan v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Security Solution] Tests for prebuilt rule upgrade workflow
4 participants