Connection aborts after firmware update #83

Markkuuss · 2019-05-24T06:14:55Z

Since I updated my Raspberry (Linux 4.19.42-v7+ #1219; newest released kernel), the Miflora sensor is no longer accessible after some time. In the beginning, when I started the service, it works fine for some time, but after a while the Miflora sensor is no longer accessible. It looks similar to what @aymeric106 described in #80.

The sensor no longer returns any values. Even a restart of the service or even of the complete system does not result in the connection being restored.

By rebooting several times, connecting to the Android App, removing the battery, etc. I made it work again somehow. But it is not an unique problem. The battery is 99% full and it has happened several times after the update.

I still have no idea what it could be. Do any of you have a similar problem after upgrading the kernel?

Markkuuss · 2019-05-24T06:44:40Z

I just noticed that if the sensor is not reachable again and I execute the following command, it returns an error message:

sudo hcitool lescan

LE Scan ...
62:2B:4F:27:9D:6C (unknown)
62:2B:4F:27:9D:6C (unknown)
C4:7C:8D:67:3A:CC (unknown)
C4:7C:8D:67:3A:CC Flower care
Disable scan failed: Input/output error

netzwerkgoettin · 2019-06-16T22:59:50Z

Hi @Markkuuss,

I can confirm this behaviour, same kernel. I also rebooted, restarted and so on, but no difference – I only got this:

[2019-06-17 00:39:43] Initial connection to Mi Flora sensor "Konferenzpflanze" (C4:7C:8D:67:4C:9E) failed.

[2019-06-17 00:39:43] Announcing Mi Flora devices to MQTT broker for auto-discovery ...
[2019-06-17 00:39:43] Retrieving data from sensor "Konferenzpflanze" ...
[2019-06-17 00:40:13] Retrying ...
[2019-06-17 00:40:44] Failed to retrieve data from Mi Flora sensor "Konferenzpflanze" (C4:7C:8D:67:4C:9E), success rate: 0%

[2019-06-17 00:40:44] Sleeping (300 seconds) ...

Interesting: I have a running Home Assistant instance, miflora configured as a sensor; it stopped working, too – HA only reports messages like these:

2019-06-17 00:15:35 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.orchidee_battery is taking over 10 seconds
2019-06-17 00:16:05 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.orchidee_conductivity is taking over 10 seconds
2019-06-17 00:35:36 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.orchidee_moisture is taking over 10 seconds

The iPhone App Flower Care detects this miflora device, connects it and retrieves the data! So what – at least for the moment – helped was this: placing miflora within spitting distance to RPi. I mean: not more than 10 or perhaps up to 20cm. So I finally got it working again (both miflora-mqtt-daemon and Home Assistant instance!).

Adding sensor to device list and testing connection ...
Name:          "Konferenzpflanze"
Internal name: "Konferenzpflanze"
Device name:   "Flower care"
MAC address:   C4:7C:8D:67:4C:9E
Firmware:      3.1.9
[2019-06-17 00:46:01] Initial connection to Mi Flora sensor "Konferenzpflanze" (C4:7C:8D:67:4C:9E) successful

[2019-06-17 00:46:01] Announcing Mi Flora devices to MQTT broker for auto-discovery ...
[2019-06-17 00:46:01] Retrieving data from sensor "Konferenzpflanze" ...
[2019-06-17 00:46:01] Result: {"moisture": 0, "light": 110, "battery": 100, "conductivity": 112, "temperature": 29.3}
[2019-06-17 00:46:01] publishing to mqtt topic "homeassistant/sensor/konferenzpflanze/state"

[2019-06-17 00:46:02] Sleeping (300 seconds) ...

That's really crazy. Does distance make a difference for you, too? I just started to search for 4.19.42-v7+, bluetooth and problem and there seem to be some... so probably it's a more general problem. But it's way too late now for further researching... 😴

Cheers,
Marianne

ThomDietrich · 2019-06-19T15:16:27Z

Hey guys, Hey Marianna!
This is crazy! In general be aware that updating the RPi firmware (rpi-update) is not generally a good idea. It's recommended to stay with the default. Could you try reverting? I believe these instructions might help: https://www.raspberrypi.org/documentation/linux/kernel/updating.md

Markkuuss · 2019-06-20T09:46:27Z

I have changed the firmware of my Raspberry back to the kernel with the last kernel version 4.14:
Hexxeh/rpi-firmware@a08ece3

So it works again.

The command for this is:
sudo rpi-update a08ece3

@ThomDietrich: But what happens if I download the current Raspbian Image? Then the default firmware has a kernel higher than 4.14 and Miflora won't work anymore?

Markkuuss · 2019-07-01T07:29:37Z

I tested it again yesterday. With the current stable kernel 1.20190517, which comes with the standard Raspbian update/upgrade process (sudo apt-get dist-upgrade), miflora-mqtt-daemon no longer works.

The previous stable kernel version 1.20190401 (https://github.com/raspberrypi/firmware/releases/tag/1.20190401) is available with a downgrade via rpi-update:
sudo BRANCH=stable rpi-update 45a2e771e7272781e62ae92322734c2b90e0268a
So it works fine again.

@sysadmama: I didn't notice any difference in the distance. In the beginning it runs with the current kernel at exactly the same distance as with the old kernel. But if it runs longer, after a certain time no connection can be established. This is also reproduceable.

krikk · 2019-07-10T20:29:21Z

it looks like this is the problem why it is not working on my brand new raspberry pi 4...

Markkuuss · 2019-07-11T05:52:49Z

it looks like this is the problem why it is not working on my brand new raspberry pi 4...

Probably that's the problem. Post your Linux version - the output of the command "uname -a".

krikk · 2019-07-11T16:57:59Z

i don't understand why, but it started working again... so forget my post...

[18:55:25] root@raspi4:/opt/miflora-mqtt-daemon# uname -a Linux raspi3plus 4.19.50-v7l+ #895 SMP Thu Jun 20 16:03:42 BST 2019 armv7l GNU/Linux

milkpirate · 2019-07-28T07:24:06Z

I had similar problem with not working connections or working for a while an then not anymore. Since the downgrade (like @Markkuuss) suggested, everthing seems fine. Will do further testing...

lionhe1966 · 2019-08-21T09:00:53Z

I am experiencing the same problem on a raspberry pi v3 with Stretch since a long time, but it seems to me that my kernel is older than the one reported here.
My kernel is

Linux raspberrypi2 4.14.79-v7+ #1159 SMP Sun Nov 4 17:50:20 GMT 2018 armv7l GNU/Linux

I implemented a rule in OpenHAB (apt-get version 2.4 installed on raspbian stretch) that restarts bluetooth and miflora services.
It happens more ore less daily and it seems related to the weakness of the signal (with low batteries it occurs more often?)

I updated right now to the version suggested a few post above, and in my case it does not seem a downgrade, rather it is an upgrade.

Linux raspberrypi2 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l GNU/Linux

I will see how it proceed and will report back
(Edit:So far, after 4 days, no restart of bluetooth service was required)

mvalla · 2019-09-22T06:33:48Z

in my case the daemon works on a RaPi with Raspbian (no OH installed) and kernel:
Linux raspberrypi 4.19.57-v7+ #1244 SMP Thu Jul 4 18:45:25 BST 2019 armv7l GNU/Linux
Success rate 100%.

the daemon does not work if activated on another RaPi with openhabian and kernel:
Linux openhab 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

the 2 RaPis (both are model 3b+) are positioned close to each other so position is not part of the problem.
I even tested switching the SD card of the two RaPi, and the behaviour becomes switched, so it does not depend on the hardware either.

Markkuuss · 2019-11-25T07:46:09Z

Any news here? Has anyone made it stable with the latest firmware?

I'm afraid my attempts were all negative. After some time no more data will be read.

cropab · 2020-02-03T13:27:09Z

Any news here? Has anyone made it stable with the latest firmware?

I'm afraid my attempts were all negative. After some time no more data will be read.

Tried it on Pi 0, 3 and 4. Same issue. Its even the same in Node Red.

a-x- · 2020-05-02T16:30:34Z

I have an error too. I installed this library first time

raspberry pi zero w (with ble),
uname -a
Linux 4.19.75+ #1270 Tue Sep 24 18:38:54 BST 2019 armv6l GNU/Linux

cat /etc/*-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"

xiaomi mi flower care firmware: 3.2.2

bluez is the newest version (5.50-1.2~deb10u1+rpt1).

...
Adding sensor to device list and testing connection ...
Name:          "flower_3"
[2020-05-02 19:19:45] Initial connection to Mi Flora sensor "flower_3" (C4:7C:8D:6A:65:12) failed due to exception: 

[2020-05-02 19:19:45] Announcing Mi Flora devices to MQTT broker for auto-discovery ...
Traceback (most recent call last):
  File "miflora-mqtt-daemon.py", line 293, in <module>
    'sw_version': flora['firmware']
KeyError: 'firmware'

bluetooth daemon has some warnings too: sap-server: Operation not permitted (1)

● bluetooth.service - Bluetooth service
   Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)   Active: active (running) since Sat 2020-05-02 18:52:33 MSK; 55min ago
     Docs: man:bluetoothd(8)
 Main PID: 366 (bluetoothd)
   Status: "Running"
   Memory: 3.0M
   CGroup: /system.slice/bluetooth.service
           └─366 /usr/lib/bluetooth/bluetoothd

May 02 18:52:33 orpik2 systemd[1]: Starting Bluetooth service...
May 02 18:52:33 orpik2 bluetoothd[366]: Bluetooth daemon 5.50
May 02 18:52:33 orpik2 bluetoothd[366]: Starting SDP server
May 02 18:52:33 orpik2 systemd[1]: Started Bluetooth service.
May 02 18:52:33 orpik2 bluetoothd[366]: Bluetooth management interface 1.14 initialized
May 02 18:52:34 orpik2 bluetoothd[366]: Sap driver initialization failed.
May 02 18:52:34 orpik2 bluetoothd[366]: sap-server: Operation not permitted (1)

——

little more debug info (print(flora_poller.__dict__)):

{'_mac': 'C4:7C:8D:6A:65:12', '_bt_interface': <btlewrap.base.BluetoothInterface object at 0xb60c0bb0>, '_cache': None, '_cache_timeout': datetime.timedelta(seconds=299), '_last_read': None, '_fw_last_read': None, 'retries': 3, 'ble_timeout': 10, 'lock': <unlocked _thread.lock object at 0xb60ce140>, '_firmware_version': None, 'battery': None}

sudo hcitool lescan
LE Scan ...
C4:7C:8D:6A:65:12 (unknown)
C4:7C:8D:6A:65:12 Flower care

more exceptions (I commented some except branches and revealed some exception)

  File "/usr/local/lib/python3.7/dist-packages/btlewrap/bluepy.py", line 27, in _func_wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/btlewrap/bluepy.py", line 56, in connect
    self._peripheral = Peripheral(mac, iface=iface, addrType=self.address_type)
  File "/usr/local/lib/python3.7/dist-packages/bluepy/btle.py", line 391, in __init__
    self._connect(deviceAddr, addrType, iface)
  File "/usr/local/lib/python3.7/dist-packages/bluepy/btle.py", line 439, in _connect
    raise BTLEDisconnectError("Failed to connect to peripheral %s, addr type: %s" % (addr, addrType), rsp)
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral C4:7C:8D:6A:66:FC, addr type: public

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "miflora-mqtt-daemon.py", line 212, in <module>
    flora_poller.fill_cache()
  File "/usr/local/lib/python3.7/dist-packages/miflora/miflora_poller.py", line 85, in fill_cache
    firmware_version = self.firmware_version()
  File "/usr/local/lib/python3.7/dist-packages/miflora/miflora_poller.py", line 127, in firmware_version
    with self._bt_interface.connect(self._mac) as connection:
  File "/usr/local/lib/python3.7/dist-packages/btlewrap/base.py", line 47, in __enter__
    self._backend.connect(self._mac)
  File "/usr/local/lib/python3.7/dist-packages/btlewrap/bluepy.py", line 33, in _func_wrapper
    raise BluetoothBackendException() from last_error
btlewrap.base.BluetoothBackendException

but it’s still not clear

I also tried to pair with xiaomi flower care devices via bluetoothctl:

pair C4:7C:8D:6A:66:F2
Attempting to pair with C4:7C:8D:6A:66:F2
[CHG] Device C4:7C:8D:6A:66:F2 Connected: yes
Failed to pair: org.bluez.Error.AuthenticationFailed
[CHG] Device C4:7C:8D:6A:66:F2 Connected: no

I keep devices nearby

a-x- · 2020-05-04T19:16:13Z

UPD. I found my mistake. I must run it with sudo!

Make errors clear, please! It took many hours of my life. Good example: another miflower api project: https://github.com/vrachieru/xiaomi-flower-care-api. It said I must run with sudo. When I tried it here too, it started works.
Also, add please note about sudo into readme.

Any way, big thank you for your work!

ThomDietrich · 2020-05-07T09:31:45Z

Hey @a-x-

thanks for the exhaustive analysis. Let me just clarify that root privileges are not needed for the daemon. Also see the service definition: https://github.com/ThomDietrich/miflora-mqtt-daemon/blob/master/template.service#L8 I wonder if there were changes in buster leading to missing permissions, e.g. users not being added to the bluetooth group. Would you be willing to check this?

Quick note: The KeyError: 'firmware' was just fixed in #118 but isn't really related to your issue.

Make errors clear, please!

I thought we already do that. Which error message is missing in your opinion? A pull request would be highly appreciated!

Markkuuss · 2020-06-12T09:22:20Z

I had similar problem with not working connections or working for a while an then not anymore. Since the downgrade (like @Markkuuss) suggested, everthing seems fine. Will do further testing...

Were you able to find anything else in the meantime, @cropab and @milkpirate ?

I'm currently at kernel version 4.19.66-v7+ #1253. Unfortunately it doesn't work after a while. Then I have to do a reset every time: sudo hciconfig hci0 reset

Markkuuss · 2020-06-18T19:52:14Z

I think I have now found the solution. With the firmware, which is currently distributed as a stable with Raspbian Stretch (1.20190819 / 4.19.66), the Bluetooth does not work correctly on my Raspberry Pi 3b+. The problem existed since the first update to 4.19.

I have now updated to the last available version 4.19.118 (1.20200512: kernel: Latest 4.19). Since then miflora runs stable again, at least it runs 24h without problems. This was unthinkable before.

The command for the update to the last 4.19 is
sudo rpi-update 866751bfd023e72bd96a8225cf567e03c334ecc4

Markkuuss mentioned this issue May 24, 2019

Initial connection to Mi Flora sensor [...] failed (was working previously on a pi zero) #80

Open

Markkuuss mentioned this issue Jul 1, 2019

Gatttool is deprecated #70

Open

Markkuuss closed this as completed Jun 18, 2020

Markkuuss mentioned this issue Jun 18, 2020

bluetooth fails to connect after some time #120

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Connection aborts after firmware update #83

Connection aborts after firmware update #83

Markkuuss commented May 24, 2019

Markkuuss commented May 24, 2019

netzwerkgoettin commented Jun 16, 2019

ThomDietrich commented Jun 19, 2019

Markkuuss commented Jun 20, 2019

Markkuuss commented Jul 1, 2019

krikk commented Jul 10, 2019

Markkuuss commented Jul 11, 2019

krikk commented Jul 11, 2019

milkpirate commented Jul 28, 2019

lionhe1966 commented Aug 21, 2019 •

edited

Loading

mvalla commented Sep 22, 2019

Markkuuss commented Nov 25, 2019

cropab commented Feb 3, 2020

a-x- commented May 2, 2020 •

edited

Loading

a-x- commented May 4, 2020 •

edited

Loading

ThomDietrich commented May 7, 2020

Markkuuss commented Jun 12, 2020

Markkuuss commented Jun 18, 2020

Connection aborts after firmware update #83

Connection aborts after firmware update #83

Comments

Markkuuss commented May 24, 2019

Markkuuss commented May 24, 2019

netzwerkgoettin commented Jun 16, 2019

ThomDietrich commented Jun 19, 2019

Markkuuss commented Jun 20, 2019

Markkuuss commented Jul 1, 2019

krikk commented Jul 10, 2019

Markkuuss commented Jul 11, 2019

krikk commented Jul 11, 2019

milkpirate commented Jul 28, 2019

lionhe1966 commented Aug 21, 2019 • edited Loading

mvalla commented Sep 22, 2019

Markkuuss commented Nov 25, 2019

cropab commented Feb 3, 2020

a-x- commented May 2, 2020 • edited Loading

a-x- commented May 4, 2020 • edited Loading

ThomDietrich commented May 7, 2020

Markkuuss commented Jun 12, 2020

Markkuuss commented Jun 18, 2020

lionhe1966 commented Aug 21, 2019 •

edited

Loading

a-x- commented May 2, 2020 •

edited

Loading

a-x- commented May 4, 2020 •

edited

Loading