Features proposal from @munzili

[raomin]

@munzili started this #110 PR. I create this issue to discuss here the long feature list he's proposing.
Feel free to contribute to the discussion but please have a look at the PR #110 first

Thank you very much @munzili for starting this work.
There are some pretty nice things in there.

Before getting to the details, I would suggest first to agree on the changes by discussing it here.
Once we have the changes agreed on, prioritized and broken down in sets, we can work on them in small PRs.
I would much prefer this than a big one.

We can keep the breaking ones in a separate branch, yes.

Now just some quick feedbacks on the suggested improvements:

there should not be any code changes needed. Just download a precompiled firmware and config the ESP32 over Wifi/Webserver.

I like this very much but there is significant work to design the web UI with all the configuration we have. But many we can always query and return all possible values for a selected heat pump. This could be the v1 of the ESPAlthermaEasy.
The v2 would log and graph the data without the need for HomeAssistant...

added cooling modus control. If correctly connected, both heating and cooling can be controlled

Cooling mode I guess. Low on priority for me: it would however require one extra relay that cannot be supported by the m5 for something you'll do a couple of time per year. Plus, if the ESP is down the result can be unpredictable.

added ESP32 WROOM dev board

Ok, non breaking, quick win

fixed state remembering for SG1 and SG2

Ok, non breaking, quick win

changed setup process. Instead of modifying a file(setup.h and a altherma*.h definition file), witch is registered in git and creates modification messages/conflicts on pull, user should copy a local version witch gets ignored from git and modify it. This way a pull will not create a conflict. This should be done for setup.h and the desired altherma*.h register file

I understand the motivation behind this, it makes sense. I imagine that many users are at their technical limit with the configuration process. I doubt that they would actually clone the repo and git fetch new commits that are not bringing significant updates... Maybe I'm wrong but I imagine that most users are downloading the zip of the repo and it is then easier to edit 2 files than copying and editing. So I would make it low on the priority.

extracted EEPROM methods into own header file

Ok, non breaking, quick win

added CAN lib to be used in future version (Read Rotex/Altherma CAN Bus also)

That is a nice one definitely. I got myself a TTGO Can to play with the native CAN busses on the display pannel (not the P1P2). I expect we can find the same bus. But I have no time to explore so help will be greatly appreciated.

renamed MySerial to a more clear description SerialX10A

Ok, non breaking, quick win

refactored topic names for callback to have same semantic

I don't understand this one, I'll have a look at the code if you did it already.

added header makros to avoid recursive include

I'll have a look at the code

[andreapier]

Hi, about SG state persistence, I think we should first ask ourselves if this is something needed and actually useful.
I wouldn't want my ESPAltherma to restore SG state after a reboot or a reset. We cannot know for sure how long it took the device to get back up and running. Maybe it is 12:00 with full PV power, so SG3 mode is active, then a sudden blackout happens and it takes 10 hours to recover. I doubt someone wants to be at SG3 at 22:00.
So, if you are going to implement this, please make it at least optional. A simple flag will do.

[munzili]

Before getting to the details, I would suggest first to agree on the changes by discussing it here. Once we have the changes agreed on, prioritized and broken down in sets, we can work on them in small PRs. I would much prefer this than a big one.

We can keep the breaking ones in a separate branch, yes.

I absolutely agree with you. I think prioritize them and break them in small PR is a good way. One big PR was not my final intention - just to show how it may work together :)

I like this very much but there is significant work to design the web UI with all the configuration we have. But many we can always query and return all possible values for a selected heat pump. This could be the v1 of the ESPAlthermaEasy.

I think creating the UI/HTML and Webserver code is the easy part. Problem we may have is saving the configuration in the dynamic EEPROM. As far as i know, with the Arduino ESP32 lib the EEPROM is limited to 4kb. So it will not be possible to store the asltherma*.h definitions as currently are. Maybe we need to create for all known registers in the altherma*.h files, unique ID's and match them into a language id. For example:

1 = 0x10,0,217,1,-1
2 = 0x10,1,304,1,-1
3 = 0x10,6,114,2,1,
...

and in the definition files only use (based on correct model)
1 = "Operation Mode"
2 = "Defrost Operation"
3 = "Target Evap. Temp."
...

On the WebUI the User could choose the Heat pump model and the register number he needs. So there will be only 1 byte stored for the model and 1 byte for each chosen register number.

The v2 would log and graph the data without the need for HomeAssistant...

A log and graph are also a nice idea for the WebUI. But we would need a place to store the data. EEPROM hasn't enough place for it and will overwrite it too often, i think.

Cooling mode I guess. Low on priority for me: it would however require one extra relay that cannot be supported by the m5 for something you'll do a couple of time per year. Plus, if the ESP is down the result can be unpredictable.

Yea, cooling mode is correct :) Well, it depends on what you want to do. If you don't need smart grid and only want to switch the modes than this also can be done with the m5. Like me i only need the SG1, no need for SG2 - so i still can control heating, cooling and SG. Of course, if all is needed you need another board (like the ESP32-WROOM) with more GPIO. Will also be needed for CAN support i think - but that is another story.

I understand the motivation behind this, it makes sense. I imagine that many users are at their technical limit with the configuration process. I doubt that they would actually clone the repo and git fetch new commits that are not bringing significant updates... Maybe I'm wrong but I imagine that most users are downloading the zip of the repo and it is then easier to edit 2 files than copying and editing. So I would make it low on the priority.

Well, if we really get a WebUI for the settings, the setup.h and altherma*.h files don't need any modifications or copy anymore. So maybe the WebUI should be the priority instead of the modifications i did for the setup/settings. And you are right, most users will just download the zip from the repo.

That is a nice one definitely. I got myself a TTGO Can to play with the native CAN busses on the display pannel (not the P1P2). I expect we can find the same bus. But I have no time to explore so help will be greatly appreciated.

As already mentioned, my basic idea was to implement the existing commands/protocols from the pyHSPU project as those are well tested already. I personally have a Rotex HSPU Compact 516 11kW from 2015. So i would connect the ESP32 with a CAN transceiver (https://www.amazon.com/SN65HVD230-Board-Connecting-Communication-Development/dp/B00KM6XMXO/) to the J13 connector. The informations there should be published to MQTT too and be configurable in the WebUI interface as well.

Hi, about SG state persistence, I think we should first ask ourselves if this is something needed and actually useful.
I wouldn't want my ESPAltherma to restore SG state after a reboot or a reset. We cannot know for sure how long it took the device to get back up and running. Maybe it is 12:00 with full PV power, so SG3 mode is active, then a sudden blackout happens and it takes 10 hours to recover. I doubt someone wants to be at SG3 at 22:00.
So, if you are going to implement this, please make it at least optional. A simple flag will do.

Good point. Maybe the SG persistence is realy obsolete.

[munzili]

so, based on the feedback i started to cancel my current work and restarted it based on a webui with configs. When this update is done, other parts like CAN bus integration, ... can be done. I created another branch for this

https://github.com/munzili/ESPAltherma/tree/feature-webui

The current status is WIP!! The WebUI interface is 90% done an gives a preview how it will look and work - a lot of the backend work is still open. All the existing definitions files are redefined in a .json format. Its possible via the WebUI to Upload the needed definition file and use it to fetch the parameters. The end-user can than choose witch parameters to use. Also it will be possible to fetch all datas on the config page to give the end-user a preview of the datas, that will be fetched. Also custom parameters can be added or definition files be mixed. They will be stored on LittleFS

Things Todo:

• Finish WebUI to actually save the given settings and use them
• Finish the "Load parameter values" feature to show what kind of values will be generated
• Reduce app size (with the ESPAsyncWebServer memory gets up to 90%-95% used). Have a lot of opinions in my mind how a reduction can be archived.
• Finish backend to use the new generated config file. Here still needs to be done a lot of work (atm just temp changes done to compile correctly and let WebUI work)
• When all thing work, adding new settings from my other branch like defining a pin for cooling mode, SG state remembering, CAN bus settings (for future), ...
• allow to access the WebUI with connected WIFI (atm the ESP32 generates a WIFI where a connection is possible).

[munzili]

[raomin]

Good work @munzili!
I believe I would try to keep the interface a bit simpler: choose a model (name of the file) and a language and a version: light, normal or full.
Light would give the minimal parameters you would expect to be interested in (temperatures, consumption etc.). Full would give them all.
I would not show the parameter list in the UI, too complex. Also, I would use a select for the PIN selection.

The definition files should go in json files (as you did) then parsed by ArduinoJSON lib. Same for the resulting configuration file, indeed all this stored in littlefs.
To reduce the size of the json definition files we could:

• gzip it (would need a lib to uzip - not sure if we gain anything in the end)
• extract the labels of each language in specific files and reference them into each model file (huge gain)

I would also gz compress all the html files, ESPAsyncWebServer serves them automagically, this should all fit nicely in a 4mb...

[munzili]

Currently im rewriting the WebUI to allow choosing presets (light, normal, full, ...) which will be defined in the model's file. But its quit hard to tell what prams should go to light or normal. I could use some help/suggestions here per model.

I rewrite the JSON model definitions to include those presets like:
"Model": "Altherma HPSU6 Ultra",
"Language": "English",
"Presets": {
"Light": [12, 24, 45, 66, 136, 201], // array contains the array index of the parameters
"Normal": [],
"All": []
},
...

[munzili]

So, after some rework on the WebUI this is the current status. Everything should work already, just didn't connect it yet to the head pump for deeper testing. Will be the next step. Also still need some recommendation/help for defining the correct presets per model.

All pins get preselected based on the ESP32 model, also only Pins are available that exists on the board. Defaults per board can be extra configured. Added possibility to choose a Wifi from a list in range instead of entering manually the SSID. Also, the HTML/JS is gziped and whole app uses about 60% of flash space (changed partition table). All parameters and config should be now UTF8 compliant.

[DieterTHeck]

Hi @munzili

I am pretty much interested in particular in your CAN bus proposal.
Let me know if I can assist in testing.

[munzili]

Hi @munzili

I am pretty much interested in particular in your CAN bus proposal. Let me know if I can assist in testing.

Thx, at the moment I'm busy in my job and can't do much coding and testing right now. I hope i can continue with the project in a month or so

[DieterTHeck]

Hi @munzili

I guess the job still keeps you busy?
Is there a realistic chance that you'll resume your improvement work in the near future?

[munzili]

So, i have updated the PR with my newst version. See PR for more infos