[Request] 8 New Endpoint Security rules

[caitlinbetz]

Description

We are creating 8 new, optional, Elastic Defend (Endpoint) promotion rules (https://github.com/elastic/security-team/issues/6287). These will be 4 Detection & 4 Prevention rules for Behavior Protection, Malware, Ransomware, & Memory protection (8 total).

Today, when a user installs Elastic Defend, we automatically enable the "Endpoint Security" promotion rule which ensures alerts are properly generated from Defend (https://www.elastic.co/guide/en/security/master/detection-engine-overview.html). However, using a single promotion rule for all the Elastic Endpoint security alerts implies that all alerts from the endpoint (prevention or detection alerts) are handled the same way. Users must manually inspect each alert’s metadata to determine if it was preventive or only detection. In addition, users can't configure different actions (endpoint response actions or otherwise) based on the alert type. These additional endpoint security rules provide more of this flexibility.

Enabling the single Endpoint Security rule by default upon installation of Defend will continue to be the default behavior. These 8 new rules will be optional - user can manually enable these in the Rules section of the app.

I don't believe we have any content in our Defend focused pages about rules (I believe the main mention is on the page noted above, https://www.elastic.co/guide/en/security/master/detection-engine-overview.html). It could be beneficial to add something to the install or policy pages regarding how they can use these different rules.

Background & resources

• PRs:
• Issues/metas: https://github.com/elastic/security-team/issues/6287
• Point of contact: @approksiu @caitlinbetz
• Test environments:

Which documentation set does this change impact?

ESS and serverless

ESS release

8.16

Serverless release

TBD

Feature differences

No changes between ESS/Serverless

API docs impact

TBD

Prerequisites, privileges, feature flags

No response

[caitlinbetz]

@approksiu - can you confirm:

• If user has all 8 new rules enabled, and the existing endpoint security rule, they will receive duplicate alerts
• there is no downside/user impact to enabling all 8 new rules (with original endpoint security rule disabled) - they will receive appropriate alerts based on defend policy config
• no differences between serverless and ESS? (when will these be made available for serverless?)

[approksiu]

@caitlinbetz yes, all you mention is correct.

[nicpenning]

👀

[joepeeples]

Looking at this PR's changes, it looks like the existing "Endpoint Security" rule is also being renamed to "Elastic Defend":
https://github.com/elastic/detection-rules/pull/3533/files#diff-4a21e3b3b3623f10693707595df4e471e04bd1cbb129ce9f6385b3e67096423eL22-R22

@approksiu @caitlinbetz @Mikaayenson Could someone confirm this?

[approksiu]

Looking at this PR's changes, it looks like the existing "Endpoint Security" rule is also being renamed to "Elastic Defend": https://github.com/elastic/detection-rules/pull/3533/files#diff-4a21e3b3b3623f10693707595df4e471e04bd1cbb129ce9f6385b3e67096423eL22-R22

@approksiu @caitlinbetz @Mikaayenson Could someone confirm this?

@joepeeples Yes. This rule only looks at Defend alerts.

[natasha-moore-elastic]

Hi @caitlinbetz, @approksiu

We have a few more follow-up questions:

• What’s the earliest Stack version where the new rules (AND the renamed “Elastic Defend” rule) will be available? For example, could a user on Stack 8.15, 8.14, etc. get these rules as part of their latest out-of-band prebuilt rule updates?

• Are there API changes that need to be documented, and which versions of the API docs are affected? (For anything 8.16+, the new API reference generated from OpenAPI spec is the canonical source, but if the rules will be available in 8.15 and earlier as per above, we also need to update the legacy AsciiDoc API docs.)

• If a user already has the “Endpoint Security” rule installed, when they get the new rules will they need to manually confirm that the “Elastic Defend” rule will overwrite it? We had to do this when importing the .ndjson rules as a test, but don’t know how the process will work in production.

• The description and the linked issue call these “promotion” rules, which historically isn’t a concept we’ve covered in docs. What exactly is a “promotion” rule (as opposed to other rule types like prebuilt), and is that something we should start documenting?

• The linked dev issue mentions exceptions, but we’re still unclear how exceptions will work, or how to explain them in docs. Previously we called them “Endpoint exceptions” because they were associated with the “Endpoint Security” rule, but now we have 9 rules doing similar things — are their exceptions still called “Endpoint exceptions” for all 9 rules?

• We noticed that alerts created by both the legacy “Elastic Defend” rule AND the new rules look virtually the same in the Alerts table — you have to manually open each alert details flyout to see the actual name of the rule that created the alert. This doesn’t seem ideal so we’re wondering if this is part of the metadata tweaks that Mika mentioned are still ongoing. We need to know the exact names that’ll appear throughout the UI before we can finalize any docs.

  2024-11-04_14-58-32.mp4

[approksiu]

1. What’s the earliest Stack version where the new rules (AND the renamed “Elastic Defend” rule) will be available? For example, could a user on Stack 8.15, 8.14, etc. get these rules as part of their latest out-of-band prebuilt rule updates?

These rules will only be available from 8.16 onwards, as they depend on a data field provided by Elastic Defend from 8.16. We can release these rules out-of-band once the docs are ready - rules release is more flexible than the stack release.

2. Are there API changes that need to be documented, and which versions of the API docs are affected? (For anything 8.16+, the new API reference generated from OpenAPI spec is the canonical source, but if the rules will be available in 8.15 and earlier as per above, we also need to update the legacy AsciiDoc API docs.)

There are no API changes AFAIK.

3. If a user already has the “Endpoint Security” rule installed, when they get the new rules will they need to manually confirm that the “Elastic Defend” rule will overwrite it? We had to do this when importing the .ndjson rules as a test, but don’t know how the process will work in production.

The name update will be shown to them in the update flyout and they can proceed with updating the rule as with the other updates. When you import rules, there is a different workflow.

4. The description and the linked issue call these “promotion” rules, which historically isn’t a concept we’ve covered in docs. What exactly is a “promotion” rule (as opposed to other rule types like prebuilt), and is that something we should start documenting?

If we don't mention it in the rules themselves, there is no need to document it at the moment. Promotion means it take an alert from logs and "promotes" it to the ELastic Security alert, basically creating an alert document.

5. The linked dev issue mentions exceptions, but we’re still unclear how exceptions will work, or how to explain them in docs. Previously we called them “Endpoint exceptions” because they were associated with the “Endpoint Security” rule, but now we have 9 rules doing similar things — are their exceptions still called “Endpoint exceptions” for all 9 rules?

We have a special shared exception list "Endpoint exceptions", which makes sure exceptions are applied on Elastic Defend as well. These new rules will be using the same exception list "Endpoint exceptions", so if user switches to using the new rules and stops using Endpont Security rule, the exceptions will continue working.

6. We noticed that alerts created by both the legacy “Elastic Defend” rule AND the new rules look virtually the same in the Alerts table — you have to manually open each alert details flyout to see the actual name of the rule that created the alert. This doesn’t seem ideal so we’re wondering if this is part of the metadata tweaks that Mika mentioned are still ongoing. We need to know the exact names that’ll appear throughout the UI before we can finalize any docs.

These rules typically overwrite rule name with the the alert/rule name value from the original incoming alert (in this case from Elastic Defend), here the behavior is not changing, and we plan to address this in the future when we can show original Elastic Defent rules in the SIEM. At the moment users benefit from ability to set more specific actions based on whether the malicious activity was prevented or just detected.

@natasha-moore-elastic let me know if there are more questions.