Changes 2023.05.08

* Added: Possibility to add `config.ini` to the root of a USB flash drive on install via the USB method by @mr-manuel
* Changed: Daly BMS - Improved driver stability by @transistorgit & @mr-manuel
* Changed: Improved install workflow via USB flash drive by @mr-manuel

.github/workflows/release-beta.yml

@@ -28,7 +28,8 @@ jobs:
             --exclude bms/battery_template.py \
             --exclude bms/revov.py \
             --exclude bms/test_max17853.py \
-            etc/dbus-serialbattery/
+            etc/dbus-serialbattery/ \
+            rc/
 
       - name: Release
         uses: softprops/action-gh-release@v1

.github/workflows/release.yml

@@ -27,7 +27,8 @@ jobs:
             --exclude bms/battery_template.py \
             --exclude bms/revov.py \
             --exclude bms/test_max17853.py \
-            etc/dbus-serialbattery/
+            etc/dbus-serialbattery/ \
+            rc/
 
       - name: Release
         uses: softprops/action-gh-release@v1

CHANGELOG.md

@@ -37,6 +37,7 @@
 * Added: JKBMS BLE - Show if balancing is active and which cells are balancing by @mr-manuel
 * Added: JKBMS BLE - Show serial number and "User Private Data" field that can be set in the JKBMS App to identify the BMS in a multi battery environment by @mr-manuel
 * Added: JKBMS BLE driver by @baranator
+* Added: Possibility to add `config.ini` to the root of a USB flash drive on install via the USB method by @mr-manuel
 * Added: Post install notes by @mr-manuel
 * Added: Read charge/discharge limits from JKBMS by @mr-manuel
 * Added: Recalculation interval in linear mode for CVL, CCL and DCL by @mr-manuel
@@ -55,7 +56,7 @@
 * Changed: `reinstall-local.sh` to recreate `/data/conf/serial-starter.d`, if deleted by `disable.sh` --> to check if the file `conf/serial-starter.d` could now be removed from the repository by @mr-manuel
 * Changed: Added QML to `restore-gui.sh` by @mr-manuel
 * Changed: Bash output by @mr-manuel
-* Changed: Daly BMS - Improved driver stability by @transistorgit
+* Changed: Daly BMS - Improved driver stability by @transistorgit & @mr-manuel
 * Changed: Default config file by @ppuetsch
   * Added missing descriptions to make it much clearer to understand by @mr-manuel
   * Changed name from `default_config.ini` to `config.default.ini` https://github.com/Louisvdw/dbus-serialbattery/pull/412#issuecomment-1434287942 by @mr-manuel
@@ -75,6 +76,7 @@
 * Changed: Fixed black lint errors by @mr-manuel
 * Changed: Fixed cell balancing background for cells 17-24 by @mr-manuel
 * Changed: Fixed Time-To-Go is not working, if `TIME_TO_SOC_VALUE_TYPE` is set to other than `1` https://github.com/Louisvdw/dbus-serialbattery/pull/424#issuecomment-1440511018 by @mr-manuel
+* Changed: Improved install workflow via USB flash drive by @mr-manuel
 * Changed: Improved JBD BMS soc calculation https://github.com/Louisvdw/dbus-serialbattery/pull/439 by @aaronreek
 * Changed: Logging to get relevant data by @mr-manuel
 * Changed: Many code improvements https://github.com/Louisvdw/dbus-serialbattery/pull/393 by @ppuetsch

docs/docs/general/install.md

@@ -60,9 +60,13 @@ In [VRM](https://vrm.victronenergy.com/) look under the device list for your ins
 
 1. Download and copy the [latest release](https://github.com/Louisvdw/dbus-serialbattery/releases) `venus-data.tar.gz` to the root of a USB flash drive that is in FAT32 format (a SD card is also an option for GX devices, but not for Raspberry Pi).
 
-2. Plug the flash drive/SD into the Venus device and reboot. It will automatically extract and install to the correct locations and try the driver on any connected devices.
+2. OPTIONAL: Create a `config.ini` file in the root of your USB flash drive with your custom settings. Put `[DEFAULT]` in the first line of the file and add all the values you want to change below. You only have to insert the values you want to change, all other values are fetched from the `config.default.ini`. In the [`config.default.ini`](https://github.com/Louisvdw/dbus-serialbattery/blob/jkbms_ble/etc/dbus-serialbattery/config.default.ini) you find all possible settings that you can copy over and change.
 
-3. Reboot the GX (in the Remote Console go to `Settings` → `General` → `Reboot?`).
+   > If you put a `config.ini` in the root of the USB flash drive, then an existing `config.ini` will be overwritten.
+
+3. Plug the flash drive/SD into the Venus device and reboot. It will automatically extract and install to the correct locations and try the driver on any connected devices.
+
+4. Reboot the GX (in the Remote Console go to `Settings` → `General` → `Reboot?`).
 
 
 ### Install over SSH

etc/dbus-serialbattery/bms/daly.py

@@ -3,7 +3,7 @@
 from utils import open_serial_port, logger
 import utils
 from struct import unpack_from, pack_into
-from time import sleep
+from time import sleep, time
 from datetime import datetime
 from re import sub
 
@@ -19,12 +19,11 @@ def __init__(self, port, baud, address):
         self.cell_min_no = None
         self.cell_max_no = None
         self.poll_interval = 1000
-        self.poll_step = 0
         self.type = self.BATTERYTYPE
         self.has_settings = 1
         self.reset_soc = 0
         self.soc_to_set = None
-        self.busy = False
+        self.runtime = 0  # TROUBLESHOOTING for no reply errors
 
     # command bytes [StartFlag=A5][Address=40][Command=94][DataLength=8][8x zero bytes][checksum]
     command_base = b"\xA5\x40\x94\x08\x00\x00\x00\x00\x00\x00\x00\x00\x81"
@@ -53,96 +52,122 @@ def test_connection(self):
         # The result or call should be unique to this BMS. Battery name or version, etc.
         # Return True if success, False for failure
         result = False
-        logger.info("> test_connection: start")
         try:
-            logger.info("> test_connection: open_serial_port")
             with open_serial_port(self.port, self.baud_rate) as ser:
-                logger.info("> test_connection: read_battery_code")
-                self.read_battery_code(ser)
-
-                logger.info("> test_connection: read_status_data")
                 result = self.read_status_data(ser)
-
-                logger.info("> test_connection: read_soc_data")
                 self.read_soc_data(ser)
+                self.read_battery_code(ser)
                 self.reset_soc = (
                     self.soc
                 )  # set to meaningful value as preset for the GUI
 
-                logger.info("> test_connection: read_soc_data")
-
         except Exception as err:
             logger.error(f"Unexpected {err=}, {type(err)=}")
             result = False
 
-        logger.info("> test_connection: end")
         return result
 
     def get_settings(self):
-        logger.info("> get_settings: start")
         self.capacity = utils.BATTERY_CAPACITY
-        logger.info("> get_settings: open_serial_port")
         with open_serial_port(self.port, self.baud_rate) as ser:
-            logger.info("> get_settings: read_capacity")
             self.read_capacity(ser)
-
-            logger.info("> get_settings: read_production_date")
             self.read_production_date(ser)
 
         self.max_battery_charge_current = utils.MAX_BATTERY_CHARGE_CURRENT
         self.max_battery_discharge_current = utils.MAX_BATTERY_DISCHARGE_CURRENT
-        logger.info("> get_settings: end")
         return True
 
     def refresh_data(self):
-        logger.info("> refresh_data: start")
         result = False
 
-        if self.busy:
-            logger.info("> refresh_data: CURRENTLY BUSY!")
-            return False
-        self.busy = True
-
         # Open serial port to be used for all data reads instead of opening multiple times
         try:
-            logger.info("> refresh_data: open_serial_port")
             with open_serial_port(self.port, self.baud_rate) as ser:
-                logger.info("> refresh_data: read_soc_data")
                 result = self.read_soc_data(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_soc_data - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                logger.info("> refresh_data: read_fed_data")
                 result = result and self.read_fed_data(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_fed_data - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                logger.info("> refresh_data: read_cell_voltage_range_data")
                 result = result and self.read_cell_voltage_range_data(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_cell_voltage_range_data - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                logger.info("> refresh_data: write_soc_and_datetime")
                 self.write_soc_and_datetime(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: write_soc_and_datetime - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                if self.poll_step == 0:
-                    logger.info("> refresh_data: read_alarm_data")
-                    result = result and self.read_alarm_data(ser)
-
-                    logger.info("> refresh_data: read_temperature_range_data")
-                    result = result and self.read_temperature_range_data(ser)
+                result = result and self.read_alarm_data(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_alarm_data - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                elif self.poll_step == 1:
-                    logger.info("> refresh_data: read_cells_volts")
-                    result = result and self.read_cells_volts(ser)
+                result = result and self.read_temperature_range_data(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_temperature_range_data - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                    logger.info("> refresh_data: read_balance_state")
-                    result = result and self.read_balance_state(ser)
+                result = result and self.read_cells_volts(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_cells_volts - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                    # else:  # A placeholder to remind this is the last step. Add any additional steps before here
-                    # This is last step so reset poll_step
-                    self.poll_step = -1
+                result = result and self.read_balance_state(ser)
+                if self.runtime > 0.200:  # TROUBLESHOOTING for no reply errors
+                    logger.info(
+                        "  |- refresh_data: read_balance_state - result: "
+                        + str(result)
+                        + " - runtime: "
+                        + str(self.runtime)
+                        + "s"
+                    )
 
-                self.poll_step += 1
         except OSError:
             logger.warning("Couldn't open serial port")
 
-        self.busy = False
-        logger.info("> refresh_data: end")
+        if not result:  # TROUBLESHOOTING for no reply errors
+            logger.info("refresh_data: result: " + str(result))
         return result
 
     def read_status_data(self, ser):
@@ -414,6 +439,8 @@ def read_balance_state(self, ser):
             self.cells[i].balance = True if bitdata & mask else False
             mask >>= 1
 
+        return True
+
     def read_temperature_range_data(self, ser):
         minmax_data = self.read_serial_data_daly(ser, self.command_minmax_temp)
         # check if connection success
@@ -452,12 +479,11 @@ def read_capacity(self, ser):
             return False
 
         (capacity, cell_volt) = unpack_from(">LL", capa_data)
-        self.capacity = (
-            capacity / 1000
-            if capacity and capacity != "" and capacity > 0
-            else utils.BATTERY_CAPACITY
-        )
-        return True
+        if capacity and capacity > 0:
+            self.capacity = capacity / 1000
+            return True
+        else:
+            return False
 
     # new
     def read_production_date(self, ser):
@@ -580,6 +606,11 @@ def read_serialport_data(
         length_size=None,
     ):
         try:
+            # wait shortly, else the Daly is not ready and throws a lot of no reply errors
+            # if you see a lot of errors, try to increase in steps of 0.005
+            sleep(0.020)
+
+            time_start = time()
             ser.flushOutput()
             ser.flushInput()
             ser.write(command)
@@ -591,13 +622,14 @@ def read_serialport_data(
                 elif length_size.upper() == "I" or length_size.upper() == "L":
                     length_byte_size = 4
 
-            start_time = datetime.now()
             toread = ser.inWaiting()
 
             while toread < (length_pos + length_byte_size):
                 sleep(0.005)
                 toread = ser.inWaiting()
-                if (datetime.now() - start_time).microseconds > 250000:
+                time_run = time() - time_start
+                if time_run > 0.500:
+                    self.runtime = time_run
                     logger.error(">>> ERROR: No reply - returning")
                     return False
 
@@ -625,7 +657,9 @@ def read_serialport_data(
                 res = ser.read(packetlen - len(data))
                 data.extend(res)
                 sleep(0.005)
-                if (datetime.now() - start_time).microseconds > 500000:
+                time_run = time() - time_start
+                if time_run > 0.500:
+                    self.runtime = time_run
                     logger.error(
                         ">>> ERROR: No reply - returning [len:"
                         + str(len(data))
@@ -635,6 +669,7 @@ def read_serialport_data(
                     )
                     return False
 
+            self.runtime = time_run
             return data
 
         except Exception as e:
@@ -680,17 +715,17 @@ def write_soc_and_datetime(self, ser):
         logger.info(f"write soc {self.soc_to_set}%")
         self.soc_to_set = None  # Reset value, so we will set it only once
 
+        time_start = time()
         ser.flushOutput()
         ser.flushInput()
         ser.write(cmd)
 
         toread = ser.inWaiting()
-        count = 0
         while toread < 13:
             sleep(0.005)
             toread = ser.inWaiting()
-            count += 1
-            if count > 50:
+            time_run = time() - time_start
+            if time_run > 0.500:
                 logger.warning("write soc: no reply, probably failed")
                 return False
 

etc/dbus-serialbattery/disable.sh

@@ -3,27 +3,32 @@
 # remove comment for easier troubleshooting
 #set -x
 
-DRIVERNAME=dbus-serialbattery
-
 # handle read only mounts
 sh /opt/victronenergy/swupdate-scripts/remount-rw.sh
 
-# remove files
-rm -f /data/conf/serial-starter.d/$DRIVERNAME.conf
+# remove files, don't use variables here, since on an error the whole /opt/victronenergy gets deleted
+rm -f /data/conf/serial-starter.d/dbus-serialbattery.conf
+rm -rf /service/dbus-serialbattery.*
 rm -rf /service/dbus-blebattery.*
 
-# kill driver, if running. It gets restarted by the service daemon
-pkill -f "python .*/$DRIVERNAME.py"
-
 # remove install script from rc.local
-sed -i "/sh \/data\/etc\/$DRIVERNAME\/reinstall-local.sh/d" /data/rc.local
+sed -i "/bash \/data\/etc\/dbus-serialbattery\/reinstall-local.sh/d" /data/rc.local
 
 
 ### needed for upgrading from older versions | start ###
 # remove old drivers before changing from dbus-blebattery-$1 to dbus-blebattery.$1
 rm -rf /service/dbus-blebattery-*
 # remove old install script from rc.local
-sed -i "/sh \/data\/etc\/$DRIVERNAME\/reinstalllocal.sh/d" /data/rc.local
+sed -i "/sh \/data\/etc\/dbus-serialbattery\/reinstalllocal.sh/d" /data/rc.local
+sed -i "/sh \/data\/etc\/dbus-serialbattery\/reinstall-local.sh/d" /data/rc.local
 # remove old entry from rc.local
 sed -i "/sh \/data\/etc\/dbus-serialbattery\/installble.sh/d" /data/rc.local
 ### needed for upgrading from older versions | end ###
+
+
+# kill serial starter, to reload changes
+pkill -f "/opt/victronenergy/serial-starter/serial-starter.sh"
+
+# kill driver, if running
+pkill -f "serialbattery"
+pkill -f "blebattery"

etc/dbus-serialbattery/reinstall-local.sh

@@ -46,7 +46,7 @@ if [ ! -f $filename ]; then
     echo "#!/bin/bash" >> $filename
     chmod 755 $filename
 fi
-grep -qxF "sh /data/etc/$DRIVERNAME/reinstall-local.sh" $filename || echo "sh /data/etc/$DRIVERNAME/reinstall-local.sh" >> $filename
+grep -qxF "bash /data/etc/$DRIVERNAME/reinstall-local.sh" $filename || echo "bash /data/etc/$DRIVERNAME/reinstall-local.sh" >> $filename
 
 # add empty config.ini, if it does not exist to make it easier for users to add custom settings
 filename=/data/etc/$DRIVERNAME/config.ini