[Request]: Documenting no-data settings for the custom threshold rule

[maryam-saeidi]

Description

In the custom threshold rule, we have 2 settings:

1. Alert on no data (alertOnNoData)
2. Alert on missing group (alertOnGroupDisappear)

At each given time, only one of these settings can be used depending on whether there is a group by field for the rule or not. In this PR, we tried to improve the UX for using these settings by only showing one setting in the UI and changing the related underlying setting depending on whether there is a group by field or not.

Here is a description related to these settings that we would like to include in this document:
In the API, we have 2 settings related to no data:

1. alertOnNoData: this is used when we don't have any group by and will trigger a no alert data if the condition doesn’t report any data over the expected time period or if the rule fails to query Elasticsearch. (Essentially, it means something is wrong, and we don't have data to evaluate the related threshold)
2. alertOnGroupDisappeer: When we have group by fields and after seeing a group, if that group does not report any data, we trigger a missing group no data alert.
   Here is an example scenario when we have host.name as the group by field for CPU usage above 80%:

   • During the first rule execution, we have 2 hosts that are reporting data: host-1 and host-2

   • In the next rule execution, host-1 does not report any data, so we trigger a no-data alert for host-1

   • In the next execution, if host-1 starts reporting data again, we will have 2 scenarios:

     • If the host-1 reports data for CPU usage and it is above the threshold of 80%, then we don't trigger a new alert, but we change the existing alert from no-data to a triggered alert that breaches the threshold. It is important to keep in mind that we don't send any notifications in this case because there is still an ongoing issue.
     
     • If the host-1 reports CPU usage below the threshold of 80%, then we change the alert status to recovered.

It would be great to also include information about how users can untrack a group that is decommissioned (related ticket). For example, maybe in the above scenario, host-1 is decommissioned so we don't need the related alert anymore. In that case, user can select that alert in the alert table and use mark as untrack action:

In the UI, we are now using only one setting, and we enable the related API setting according to the group by field. Here, you can see that we also adjust the related tooltip based on whether a group by field is selected or not.

Resources

PR: elastic/kibana#188300

Related issues:

• [Custom threshold] Bring back no data setting and set it to false by default kibana#188229
• [AO] Adding a way to allow users to remove one of the groups if it does not exist anymore in the threshold rule kibana#161621

Which documentation set does this change impact?

Stateful and Serverless

Feature differences

This feature is identical in both environments.

What release is this request related to?

8.16

Collaboration model

The documentation team

Point of contact.

Main contact: @maryam-saeidi

Stakeholders:

• @maciejforcone
• @jasonrhodes
• @vinaychandrasekhar