Redesign how state attributes are excluded from history

[emontnemery]

Background

Integrations can today specify state attributes to exclude from recorder history by implementing a recorder platform which implements a function def exclude_attributes(hass: HomeAssistant) -> set[str]:

This design has some drawbacks:

• Recorder is notified about integration platforms loaded via a signal, which means state events may already be recorded before the corresponding integration platform is known to the recorder causing unwanted attributes to be recorded
  • This race has caused several bugs, and to avoid it integrations with recorder platforms need to call function async_process_integration_platform_for_component before creating entities.
• The granularity of excluded state attributes is on integration level. This is sufficient for base platforms, e.g. sensor, but not suitable for integrations which provide multiple types of entities, e.g. unifiprotect.

Proposal

To improve the granularity of attributes to be excluded from history:

• Replace the recorder platforms with two new attribute on the Entity base class:
  • _entity_component_unrecorded_attributes: frozenset[str] - This should be set by base component entity classes, e.g. LightEntity
  • _unrecorded_attributes: frozenset[str] - This should be set by derived platform classes, e.g. HueLight to exclude additional, integration specific, attributes from recording.

To solve the race:

• Include a reference to excluded attributes in state change events

Draft implementation in home-assistant/core#100239

Alternative, rejected proposal

Replace the recorder platforms with a new entry in manifest.json (See core PR#99283

• This solves the races but does not improve the granularity of excluded state attributes

[frenck]

Replace the recorder platforms with a new attribute on the Entity base class, _recorder_excluded_attributes: frozenset[str]

So, there is 1 property for both the integration provided entity implementation and the base entity?

For the example given in the linked PR that contains automation, that makes sense and would work.
For light, the entity component has, for example, excluded min_mireds & max_mireds. If an integration wants to extend this for a light entity it provides, does that make the integration responsible for handling current values from the entity component?

../Frenck

[emontnemery]

Yes, the suggestion is to use a single property.

An integration which wants to exclude additional attributes should do this:

class MyLightEntity(LightEntity):
    _recorder_excluded_attributes = LightEntity._recorder_excluded_attributes | {"additional_attribute"}

[emontnemery]

As discussed, this is now split in two attributes: one to be set by entity components, one to be set by platforms.

[AdrianGarside]

I'm trying to use this to stop the image_last_updated from spamming the log book for an image that updates every 1-2 seconds. But adding that to _entity_component_unrecorded_attributes isn't helping. Am I missing something or is what I want to do not possible?

[gjohansson-ST]

This is the discussion for the architectural proposal and as such no the right place for your question.
I would suggest you try in the devs channel in discord if you need help on how to use this in an integration.
Thanks

[Robbe-B]

I'm still wondering, what if we want to exclude the state itself from the recorder instead of an attribute of the state ?

I know we can do this via filters afterwards in the recorder yaml file.
But an integration itself is probably not able to do this at this point?

[AdrianGarside]

Not that I’ve been able to work out by searching documentation or looking at the core HA code.

The filters afterwards is only partially functionally. You can filter by entity glob successfully but filtering of exact entities doesn’t work.

[frenck]

Merged into Core, can thus be closed now.

[autoSteve]

The _entity_component_unrecorded_attributes and _unrecorded_attributes must be declared as class attributes; instance attributes will be ignored.

This is unfortunate, @emontnemery, but probably has an explanation. It's a cool implementation, but no more flexible than that it replaced.

What is it that I am trying to do? To exclude recording certain entity attributes with variable naming on startup. It can't be done with this implementation. Wildcards would be handy as the attributes all start with the same prefix, but I do not believe that exists.

Nice work, though.