[Security Solution] Unable to unset field value upon rules upgrade #203634
Assignees
Labels
8.18 candidate
bug
Fixes for quality problems that affect the customer experience
Feature:Prebuilt Detection Rules
Security Solution Prebuilt Detection Rules area
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
Team:Detection Rule Management
Security Detection Rule Management Team
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
v8.18.0
Comments
maximpn
added
8.18 candidate
bug
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
This was referenced Dec 10, 2024
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Dec 20, 2024
…stic#204231) **Resolves: elastic#203634 ## Summary This PR fixes bugs blocking unsetting optional rule field values in rule upgrade workflow. ## Details Changes here cover 3 groups of fields optional, string fields allowing empty strings and array fields allowing empty arrays. It was verified that fields in that groups allow to unset the value. The following issues were fixed - inability to set an empty string or `setup` and `note` fields It required adding `stripEmptyFields: false` for rule upgrade fields edit form. - inability to unset `timestamp_override` field Timestamp override form deserializer was fixed. - inability to unset `alert_suppression` Alert Suppression was excluded from special special fields list always upgrading to the current value. It's expected Alert Suppression won't be included in Prebuilt Rules delivered in prebuilt rules packages. The only way to get this setting and have it included in rule upgrade flyout is editing a prebuilt rule by a user with a sufficient licence. The following fields were verified and fixed where necessary ### Optional fields - ✅ `investigation_fields` - ✅ `rule_name_override` -⚠️ `timestamp_override` (field's form deserializer was fixed) - ✅ `timeline_template` - ✅ `building_block` -⚠️ `alert_suppression` (the field was excluded from special special fields list always upgrading to the current value) - ✅ `threat_indicator_path` (empty value resets to default `threat.indicator`) ### String fields allowing empty strings -⚠️ `note` (required adding `stripEmptyFields: false` to the form) -⚠️ `setup` (required adding `stripEmptyFields: false` to the form) ### Array fields allowing empty arrays - ✅ `tags` - ✅ `references` - ✅ `false_positives` - ✅ `threat` - ✅ `related_integrations` - ✅ `required_fields` - ✅ `severity_mapping` - ✅ `risk_score_mapping` ## Screenshots  https://github.com/user-attachments/assets/bd36e5ba-e7fb-4733-a792-ea5435d579e2 ## How to test? - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml` - Clear Elasticsearch data - Run Elasticsearch and Kibana locally (do not open Kibana in a web browser) - Install an outdated version of the `security_detection_engine` Fleet package ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1 ``` - Install prebuilt rules ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform ``` - Customize one or more rules (change fields to see them in rule upgrade workflow) - Open Rule upgrade for the rule(s) - Unset field values - Upgrade rule(s) --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 54989a5)
stratoula
pushed a commit
to stratoula/kibana
that referenced
this issue
Jan 2, 2025
…stic#204231) **Resolves: elastic#203634 ## Summary This PR fixes bugs blocking unsetting optional rule field values in rule upgrade workflow. ## Details Changes here cover 3 groups of fields optional, string fields allowing empty strings and array fields allowing empty arrays. It was verified that fields in that groups allow to unset the value. The following issues were fixed - inability to set an empty string or `setup` and `note` fields It required adding `stripEmptyFields: false` for rule upgrade fields edit form. - inability to unset `timestamp_override` field Timestamp override form deserializer was fixed. - inability to unset `alert_suppression` Alert Suppression was excluded from special special fields list always upgrading to the current value. It's expected Alert Suppression won't be included in Prebuilt Rules delivered in prebuilt rules packages. The only way to get this setting and have it included in rule upgrade flyout is editing a prebuilt rule by a user with a sufficient licence. The following fields were verified and fixed where necessary ### Optional fields - ✅ `investigation_fields` - ✅ `rule_name_override` -⚠️ `timestamp_override` (field's form deserializer was fixed) - ✅ `timeline_template` - ✅ `building_block` -⚠️ `alert_suppression` (the field was excluded from special special fields list always upgrading to the current value) - ✅ `threat_indicator_path` (empty value resets to default `threat.indicator`) ### String fields allowing empty strings -⚠️ `note` (required adding `stripEmptyFields: false` to the form) -⚠️ `setup` (required adding `stripEmptyFields: false` to the form) ### Array fields allowing empty arrays - ✅ `tags` - ✅ `references` - ✅ `false_positives` - ✅ `threat` - ✅ `related_integrations` - ✅ `required_fields` - ✅ `severity_mapping` - ✅ `risk_score_mapping` ## Screenshots  https://github.com/user-attachments/assets/bd36e5ba-e7fb-4733-a792-ea5435d579e2 ## How to test? - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml` - Clear Elasticsearch data - Run Elasticsearch and Kibana locally (do not open Kibana in a web browser) - Install an outdated version of the `security_detection_engine` Fleet package ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1 ``` - Install prebuilt rules ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform ``` - Customize one or more rules (change fields to see them in rule upgrade workflow) - Open Rule upgrade for the rule(s) - Unset field values - Upgrade rule(s) --------- Co-authored-by: Elastic Machine <[email protected]>
benakansara
pushed a commit
to benakansara/kibana
that referenced
this issue
Jan 2, 2025
…stic#204231) **Resolves: elastic#203634 ## Summary This PR fixes bugs blocking unsetting optional rule field values in rule upgrade workflow. ## Details Changes here cover 3 groups of fields optional, string fields allowing empty strings and array fields allowing empty arrays. It was verified that fields in that groups allow to unset the value. The following issues were fixed - inability to set an empty string or `setup` and `note` fields It required adding `stripEmptyFields: false` for rule upgrade fields edit form. - inability to unset `timestamp_override` field Timestamp override form deserializer was fixed. - inability to unset `alert_suppression` Alert Suppression was excluded from special special fields list always upgrading to the current value. It's expected Alert Suppression won't be included in Prebuilt Rules delivered in prebuilt rules packages. The only way to get this setting and have it included in rule upgrade flyout is editing a prebuilt rule by a user with a sufficient licence. The following fields were verified and fixed where necessary ### Optional fields - ✅ `investigation_fields` - ✅ `rule_name_override` -⚠️ `timestamp_override` (field's form deserializer was fixed) - ✅ `timeline_template` - ✅ `building_block` -⚠️ `alert_suppression` (the field was excluded from special special fields list always upgrading to the current value) - ✅ `threat_indicator_path` (empty value resets to default `threat.indicator`) ### String fields allowing empty strings -⚠️ `note` (required adding `stripEmptyFields: false` to the form) -⚠️ `setup` (required adding `stripEmptyFields: false` to the form) ### Array fields allowing empty arrays - ✅ `tags` - ✅ `references` - ✅ `false_positives` - ✅ `threat` - ✅ `related_integrations` - ✅ `required_fields` - ✅ `severity_mapping` - ✅ `risk_score_mapping` ## Screenshots  https://github.com/user-attachments/assets/bd36e5ba-e7fb-4733-a792-ea5435d579e2 ## How to test? - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml` - Clear Elasticsearch data - Run Elasticsearch and Kibana locally (do not open Kibana in a web browser) - Install an outdated version of the `security_detection_engine` Fleet package ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1 ``` - Install prebuilt rules ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform ``` - Customize one or more rules (change fields to see them in rule upgrade workflow) - Open Rule upgrade for the rule(s) - Unset field values - Upgrade rule(s) --------- Co-authored-by: Elastic Machine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
It's impossible to remove optional field upon rule upgrade. UI shows field's value as removed but rule upgrade request fails with an error.
Steps to reproduce:
Setup GuideUpdate rulebuttonExpected behavior: Rule upgrades successfully.
Actual behavior: Rule fails to upgrade.
Screenshots:
Diffable rule optional fields
There are different rule field groups allowing empty values of different kind. Users should be able to set (reset to) a propitiate field's empty value.
Optional fields
investigation_fieldsrule_name_overridetimestamp_overridetimeline_templatebuilding_blockalert_suppressionthreat_indicator_pathNote
Diffable rule allows empty
data_sourcebut rule edit forms don't allow empty value. It should NOT be possible to set an empty value fordata_source.String fields allowing empty strings
notesetupmachine_learning_job_id? (Check an empty string should be allowed)Array fields allowing empty arrays
tagsreferencesfalse_positivesthreatrelated_integrationsrequired_fieldsSetup the environment
prebuiltRulesCustomizationEnabledfeature flag is enabledserver.restrictInternalApis: falsetokibana.dev.yamlsecurity_detection_engineFleet packageThe text was updated successfully, but these errors were encountered: