Profile V2

[frsantos]

Hi @davidrapan

I've been following the recent changes for name standardization, and as I've already created a profile myself for my inverter, I know how hard and repetitive task is to do it. Also, as you said in #73

And at the end I really don't care about the naming or simply how are the entities called. My only goal is to have the same name for entities with the same purpose (across all profile files) for clarity so it's easier and cleaner for people who are using several different devices (sure I'm now one of them so call me out for being a biased! 😆).

So, I've been thinking in a V2 profile that is easier to fill. We know we all have more or less the same sensors/entities, some profiles have more, some have less, some have unique ones but most have the same and we can standardize most of the attributes of the common ones.

For example, we all have PV Power that is composed of 1 or more "PVn Power", so we can predefine some entities for all profiles and let the profile just fill in the gaps:

# Default profile: predefined entities in most profiles, only missing information about how to retrieve the data
version: 2

default:
  update_interval: 5
  # code: must be defined on profiles
  digits: 6

parameters:
  - group: pv
    name: 'Photovoltaic panels'
    power:
      # Not sure if this would be useful, but may the definition in the profile and the entity name should be different?      
      entity_name: pv_power 
      name: "PV Power"
      description: "Combined power of all inputs"
      update_frequency: realtime
      class: "power"
      state_class: "measurement"
      uom: "W"
      icon: "mdi:solar-power-variant"
      aggregate:
        # When an entity is composed of several items, how to name the others
        name: "PV#{index} Power"
        # By default sum all the item values, but we may need other complex operations
        operation: sum

    current:
      entity_name: pv_current
      name: "PV Current"
      update_frequency: realtime
      class: "current"
      state_class: "measurement"
      uom: "A"
      icon: "mdi:solar-power"
      aggregate:
        name: "PV#{index} Current"
        opearation: none # We don't need a total, just the items themselves

    voltage:      
      entity_name: "pv#{index}_voltage"
      name: ~ # No aggregation for voltage, just the individual panel values, so no aggregation entity is created
      update_interval: realtime
      class: "voltage"
      state_class: "measurement"
      uom: "V"
      icon: "mdi:solar-power"
      aggregate:
        item_name: "PV#{index} Voltage"
        aggregation: none # We don't need a total, just the items themselves

  - group: inverter_info
    name: "Inverter information"
    serial_number: 
      entity_name: serial_number
      name: "Device Serial Number"
      update_frequency: 600
      class: ""
      state_class: "measurement"
      uom: ""
      icon: "mdi:wrench"

And then each profile would just fill in from and how the data is retrieved:

# kstar profile
info:
  manufacturer: KSTAR
  model: Hybrid Inverter

default:
  update_interval: 20 # Overwrites default
  code: 0x04          # Specifies missing value in default
  # digits: 6         # Not defined, so it takes default                     

- parameters:
  - group: pv
    # Fill in missing information about pv_power
    pv_power:
      # All items have the same rule/scale
      rule: integer # Use names instead of codes to define rules, way more intuitive
      items:
        - registers: [3024]
        - registers: [3024]

    pv_current:
      items:
        # Each item have specific rule/scale
        - registers: [3024]
          rule: integer
          scale: 0.1
        - registers: [3024]
          rule: signed_integer
          scale: 0.01

    # pv_voltage not available on this profile, so no entity is defined

  - group: inverter_info
    serial_number:
        registers: [3228, 3229, 3230, 3231, 3232, 3233, 3234, 3235, 3236, 3237, 3238]
        code: 0x03  # Define the function code for this data request
        rule: string

  # This profile has a very specific group of entities that can't be generalized into the default profile
  - group: my_specific_group
    my_entity:
      entity_name: my_entity
      name: "My specific inverter option"
      update_frequency: 20
      class: "class"
      state_class: "measurement"
      uom: "unit"
      icon: "mdi:icon"
      registers: [5000, 5001, 5002]
      rule: string

So, each profile would deep merge with the default profile to get the final profile, resulting in a very powerful profile definition while retaining a very standard list of entities for all profiles. Also, the default profile serves as documentation about what info is needed for a new profile.

Of course, this can integrate my other suggestion #48 about the inverter information to allow auto selecting the profile

What do you think?

[davidrapan]

Hey, first off it's nice to see someone taking the initiative on this standardization as it's not something where many see immediate value, haha. 😆

What I was doing so far was let's say mapping "the wide area" when comes to the sensors and find those which fulfills the same purpose, but have different naming.

Your idea seems reasonable. When I was thinking about it initially I thought of having the scheme hardcoded and then in the files just say what's there and what's not and in which registers +-.

Both implementation are great next stepping stone. But in the long run I would even try to do some sort of a register scanning to identify the device as you suggested in the #48 and then even locate registers but that would be very hard as the Solarman protocol, the Sticks and even modbus are real cr*p in my opinion and I' starting to hate this trio with every day more and more. 😆

But if that would be somewhat successful my end goal would be to satisfy requirements to be added as component of the HA but current implementation which is using those standalone files is everything but suitable for that.

That being said I'm currently writing small tools (scripts) for browsing the definition profiles all of them at once and looking for sensors with common names or even same registers to give us overview of the whole thing so when I'll be done it can be used to finish the design you started. 👍

[frsantos]

Hey, first off it's nice to see someone taking the initiative on this standardization as it's not something where many see immediate value, haha. 😆

Yeah, you know the saying: "There are only two hard things in Computer Science..."

But if that would be somewhat successful my end goal would be to satisfy requirements to be added as component of the HA but current implementation which is using those standalone files is everything but suitable for that.

And what are the requirements, may I ask?

That being said I'm currently writing small tools (scripts) for browsing the definition profiles all of them at once and looking for sensors with common names

👍

or even same registers

Do you think there can be common registers? I would say each system assigned the registers starting from scratch and there is no common ones.

to give us overview of the whole thing so when I'll be done it can be used to finish the design you started. 👍

There are some improvements to the current profile schema that can be added incrementally, and then we can move to a more default one like this proposal. For example:

• Using a single update_interval: key for both realtime and a number (can be backwards compatible allowing both syntax for a while)
• Defining the rule by name instead of number (integer, string, ...) (also backwards compatible)
• Maybe the scale can be made more user friendly? Not sure
• code is also very generic and not descriptive enough (code for what?), maybe function_code to mimic the modbus terminology (Sebastian used mb_fuction_code which is still cryptic, but at least refers to modbus).
• class, state_class, platform: hard for a new user to differentiate among those. Maybe I'm still not too much into HA (still a lot to learn, solarman was the first I integrated), but I have a hard time to identify these, and I just copy/paste from one to another.

I don't know, maybe a good simple wiki page would help instead of renaming the keys, but still the code values would improve readability and bug spotting in the profiles.

One good thing about having a default profile is that we can group keys by purpose, without needing to repeat on each custom profile, only the default profile would be verbose but very descriptive. In the following example, more understandable though more verbose, we define the entity and aggregate in the default, and let the source be defined per profile.

    current:
      entity:
        id: pv_current
        name: "PV Current"
        icon: "mdi:solar-power"

        platform: "sensor"
        state_class: "measurement"
        class: "current"
        uom: "A"
        digits: 6

      aggregate:
        name: "PV#{index} Current"
        operation: none # We don't need a total, just the items themselves

      source:
        registers: [1, 2]
        function_code: 3
        rule: integer
        scale: 0.1
        update_frequency: realtime

And taking this even further, the default profile can be even split into different files for better organisation:

schema/
  info.yml
  pv.yml
  inverter.yml
  battery.yml
  grid.yml
  status_and_errors.yml

They would be joined together into a single file for custom profile, though: profiles.kstar_hybrid.yml.

[davidrapan]

And what are the requirements, may I ask?

I don't think they are set in stone or anything like that but I think that our "profile files" are what would not be accepted.

Do you think there can be common registers? I would say each system assigned the registers starting from scratch and there is no common ones.

Yes. In fact I know for sure. For example as far as I know all Deye inverters have Serial Numbers or some other static parameters of the device in the same registers. So some are the same at least within single brand.

Using a single update_interval: key for both realtime and a number (can be backwards compatible allowing both syntax for a while)

Yeah, realtime was predecessor to the update_interval, so... As during the early development the update_interval: 5 was not yet set in stone constant. But at the end I would somehow like to avoid setting that number anyway as it would still be very redundant, imo.

Defining the rule by name instead of number (integer, string, ...) (also backwards compatible)

There is a lot of thinks which can be done about that one. So let's not rush this one yet.

code is also very generic and not descriptive enough (code for what?), maybe function_code to mimic the modbus terminology (Sebastian used mb_fuction_code which is still cryptic, but at least refers to modbus).

Both still works. My thougths were that mb_fuction_code contained too much information for no reason. And it's also prepared for construction like this:

code:
  read: 0x03
  write: 0x10

class, state_class, platform: hard for a new user to differentiate among those. Maybe I'm still not too much into HA (still a lot to learn, solarman was the first I integrated), but I have a hard time to identify these, and I just copy/paste from one to another.

Those are HA terms. Except class, it should be device_class and that's already workable replacement. So I for sure want to stick with HA terms for clarity from HA's PoV.

I don't know, maybe a good simple wiki page would help instead of renaming the keys, but still the code values would improve readability and bug spotting in the profiles.

At the end I have strong feeling that there should be a way how to get rid of setting the code all together.

And taking this even further, the default profile can be even split into different files for better organization:

That's for sure. But I think better will be to have the default profile hardcoded in the code instead in yamls.

[davidrapan]

Hey @frsantos cause you are so eager into this don't you want to start making some sort of a graph of common sensors (it can also involve the less common just be stated as such) so it can be used for planning of the v2?

🎉

[frsantos]

Hi @davidrapan

I may, yes, but don't expect it done in a timely manner. My problem is I can only spend some time here and there when I don't have work or family chores, which is not really much. I'm really astonished the time you can spend on this integration, it's like you are on a paid support (thank you very much!). I remember my time when I didn't have family that I could spend hours or the whole weekend hacking around.

I'll have it in mind, though.

[davidrapan]

Sure thing! 😉

[frsantos]

I tried claude.ai to do the work for me. However, even though it seems clever enough to do it, it can't access internet by default, and pasting just a few profiles exceeds the maximum chat length allowed 😭

Google Gemini or Github Copilot didn't seem clever enough to do it.