Add Hunter BTT (BLE) as plugin

[BasvanH]

Hi,

I'm reverse engineering the BLE protocol for controlling my Hunter BTT irrigation valve. As Im already using ESPEasy within my home ecosystem, I was thinking about writing a plugin for it.

Im using the NimBLE library to communicate with the Valve. I want to readout the battery status (done). I wish control open and close of the valve (done). And I wish to change it operating mode, schema / manual / disabled (WIP).

Plugin features:

• The plugin should support those features above.
• Additionally Im thinking about a BLE scan window so you can select your BTT as we need it's MAC address to communicate to. Manually enter the MAC could also be possible.

I currently own the single valve version, there's also a double valve version. I can add support but I then need to BLE traffic dumps.

My questions:

• Because the value is not really a sensor but a device, does this fit well within this ESPEasy project?
• Im a beginner in Arduino, Im able to build the parts for communication with the devices but not sure if I can write the whole plugin so perhaps some help is needed.

Regards,
Bastiaan

[TD-er]

Well it is just like a switch, so it would fit perfectly well in the ESPEasy architecture...

However (there is always one of those ;) ) we currently have not any Bluetooth related code integrated in ESPEasy so far.
Problem with Bluetooth on ESP32-xx is that it is using the same radio as WiFi.
So it is an exclusive-OR to use either Bluetooth or WiFi.
It can be done as some other projects already have shown, but it will be quite tricky to get the timings right between interacting with Bluetooth and WiFi or else either one may get a poor connection.

This is one of the reasons I have not yet looked into Bluetooth myself to integrate it into ESPEasy.

I am currently doing a complete rewrite of the WiFi/network related code, so I can keep this in mind.

N.B. not all ESP32's support the same Bluetooth. So that's another caveat for adding Bluetooth support.

Does your test setup use both WiFi and Bluetooth?
Is it loosing its WiFi connection, or acting 'sluggish' when communicating with Bluetooth?
You could try to let a ping run from any other computer in your network to the ESP and then see if it is still stable while interacting with Bluetooth.

[BasvanH]

Im currently testing/developing with the ESP32 connected with USB cable, so not via WiFi yet, I was not aware of this challenge. For this project it would be mandatory to use WiFi in combination with Bluetooth. I started this project to extend the very short Bluetooth range of this value over WiFi and off course add automation via ESPEasy.

These are my board details:

• PLATFORM: Espressif 32 (6.9.0) > Espressif ESP32 Dev Module
• HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
• Chip is ESP32-D0WD-V3 (revision v3.1)
• Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
• Crystal is 40MHz

I will include WiFi in my test to see what happens.

[BasvanH]

I did some testing. I enabled Wifi and Webserver and created a simple page that triggers some BLE commands. When BLE is connected, the webpage is very unstable, not usable. When BLE is disconnected, the page works fine. After some researching I got it working stable with these code parts:

#include <WiFi.h>
#include <WebServer.h>
#include <NimBLEDevice.h>

WebServer server(80);
// WebServer task for core 0
void WiFiTask(void * parameter) {
    for (;;) {
        server.handleClient();
        delay(10);
    }
}
void setup() {
    // Start webserver on core 0
    server.begin();
    xTaskCreatePinnedToCore(WiFiTask, "WiFiTask", 10000, NULL, 1, NULL, 0);
}

From what I understand it puts Webserver on core 0 where also wifi is serviced from. BLE is serviced from core 1. This works for me on a 250 lines sketch. Ping to ESP32 is fluctuating a lot, between 6 and 500ms.

[TD-er]

I expect the only way to get this stable is to let the WiFi negotiate a longer DTIM with the AP and keeping track of when a beacon interval is about to happen.
Inbetween those beacon intervals you can then handle Bluetooth.

But I think the only real stable option is to use Ethernet when using Bluetooth.

[BasvanH]

And what about using an HM10 Bluetooth module? It has support for BLE. You can't use an BLE library, but I can create the AT commands to reach the same goal. Do you see any problems with using that module?

[tonhuisman]

Do you see any problems with using that module?

It feels a bit backward to use an external BLE module on a unit that has BLE on-board (at least the ESP32 series), but it will probably be the simplest solution to communicate with Bluetooth devices.

You'd probably have to use the Serial Proxy (P087) or Serial Server (P020) plugins to control that module and receive/process the response in rules.

Not sure what the response speed would be though, but they seem to communicate at 115200 baud, so that should be fast enough for most types of sensors, once they are connected.

[BasvanH]

With BLE speed is not very essential. Just ordered some HM10's and will do some testing.

[uzi18]

@BasvanH do you have list of AT commands for this module?

[tonhuisman]

@uzi18 I found this page quite helpful: https://www.martyncurrey.com/hm-10-bluetooth-4ble-modules/#HM-10%20-%20AT%20commands

[uzi18]

@tonhuisman but BLE module is not a problem here.
To talk to Hunter BTT you need to know protocol ;)

Note.: The HM-10 is Bluetooth version 4.0 only. This means it cannot connect to Bluetooth 2/2.1 modules such as the HC-06 and HC-05.

[BasvanH]

@uzi18 I already have reversed engineered part of the Hunter BTT BLE protocol. Im able to manually control the valve by giving x minutes to run. And I can manually close it.

Open, run for 5 minutes: Service ID fcc0 / Characteristic fcd9

69 03 01 00 05

69 Manual control
03 Dont know
01 Open
00 05 5 minutes, max is 6 hours = 03 60

Close: Service ID fcc0 / Characteristic fcd9

69 03 00 00 00

69 Manual control
03 Dont know
01 Close
00 00 But doenst matter when closing

Im busy gatchering the data for switching between Disabled / Manual and Schedule.

If you want to do reverse engineering yourself, grab an android phone. Enable developer tools and enable HCI snooping. Reboot your phone. Then open Hunter BTT app and perform your actions, note down the time you execute those actions. Then pull the debug logs and drop them in Wireshark. There you can see the BLE packets and their service ID's and Characteristics.

//Edit
Hunter BTT = BLE = Bluetooth v4. so HM10 (with firmware v700 and higher) will work. Avoid the clone ones (no second crystal), as those you probably cant upgrade to latest firmware version v709 which you need to have BLE support.

[uzi18]

Nice, will compare these data with what have found on my own.

[BasvanH]

If you found any, please share it with the community.

[uzi18]

@BasvanH did you find any kind of ACK from BTT about new state?

[BasvanH]

No ACK, but it advertises the active state in service fcc0. When you WRITE for example ON with 5 minutes 69 03 01 00 05, you will see this value been published in that service. And, when a minute passes you will see it changing to 69 03 01 00 04. And when all minutes have passed it changes to 69 03 00 00 00.

[BasvanH]

I see there is a Write response.

This is the sequence:
Write Request

Frame 6892: 17 bytes on wire (136 bits), 17 bytes captured (136 bits)
    Encapsulation type: Bluetooth H4 with linux header (99)
    Arrival Time: Nov 11, 2024 10:51:31.499658000 West-Europa (standaardtijd)
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1731318691.499658000 seconds
    [Time delta from previous captured frame: 0.795861000 seconds]
    [Time delta from previous displayed frame: 0.795861000 seconds]
    [Time since reference or first frame: 50199.894685000 seconds]
    Frame Number: 6892
    Frame Length: 17 bytes (136 bits)
    Capture Length: 17 bytes (136 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    Point-to-Point Direction: Sent (0)
    [Protocols in frame: bluetooth:hci_h4:bthci_acl:btl2cap:btatt]
Bluetooth
    [Source: Google_a6:e7:d3 (74:74:46:a6:e7:d3)]
    [Destination: TexasIns_52:a5:e8 (30:45:11:52:a5:e8)]
Bluetooth HCI H4
    [Direction: Sent (0x00)]
    HCI Packet Type: ACL Data (0x02)
Bluetooth HCI ACL Packet
    .... 0000 0000 0001 = Connection Handle: 0x001
    ..00 .... .... .... = PB Flag: First Non-automatically Flushable Packet (0)
    00.. .... .... .... = BC Flag: Point-To-Point (0)
    Data Total Length: 12
    Data
    [Connect in frame: 6498]
    [Disconnect in frame: 7003]
    [Source BD_ADDR: Google_a6:e7:d3 (74:74:46:a6:e7:d3)]
    [Source Device Name: Pixel 7a]
    [Source Role: Unknown (0)]
    [Destination BD_ADDR: TexasIns_52:a5:e8 (30:45:11:52:a5:e8)]
    [Destination Device Name: BTT A5E8]
    [Destination Role: Unknown (0)]
    [Current Mode: Unknown (-1)]
Bluetooth L2CAP Protocol
    Length: 8
    CID: Attribute Protocol (0x0004)
Bluetooth Attribute Protocol
    Opcode: Write Request (0x12)
        0... .... = Authentication Signature: False
        .0.. .... = Command: False
        ..01 0010 = Method: Write Request (0x12)
    Handle: 0x004d (Unknown: Unknown)
        [Service UUID: Unknown (0xfcc0)]
        [UUID: Unknown (0xfcd9)]
    Value: 6903000005

Then you receive a Handle Value Notification:

Frame 6894: 15 bytes on wire (120 bits), 15 bytes captured (120 bits)
    Encapsulation type: Bluetooth H4 with linux header (99)
    Arrival Time: Nov 11, 2024 10:51:31.601932000 West-Europa (standaardtijd)
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1731318691.601932000 seconds
    [Time delta from previous captured frame: 0.058694000 seconds]
    [Time delta from previous displayed frame: 0.058694000 seconds]
    [Time since reference or first frame: 50199.996959000 seconds]
    Frame Number: 6894
    Frame Length: 15 bytes (120 bits)
    Capture Length: 15 bytes (120 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    Point-to-Point Direction: Received (1)
    [Protocols in frame: bluetooth:hci_h4:bthci_acl:btl2cap:btatt]
Bluetooth
    [Source: TexasIns_52:a5:e8 (30:45:11:52:a5:e8)]
    [Destination: Google_a6:e7:d3 (74:74:46:a6:e7:d3)]
Bluetooth HCI H4
    [Direction: Rcvd (0x01)]
    HCI Packet Type: ACL Data (0x02)
Bluetooth HCI ACL Packet
    .... 0000 0000 0001 = Connection Handle: 0x001
    ..10 .... .... .... = PB Flag: First Automatically Flushable Packet (2)
    00.. .... .... .... = BC Flag: Point-To-Point (0)
    Data Total Length: 10
    Data
    [Connect in frame: 6498]
    [Disconnect in frame: 7003]
    [Source BD_ADDR: TexasIns_52:a5:e8 (30:45:11:52:a5:e8)]
    [Source Device Name: BTT A5E8]
    [Source Role: Unknown (0)]
    [Destination BD_ADDR: Google_a6:e7:d3 (74:74:46:a6:e7:d3)]
    [Destination Device Name: Pixel 7a]
    [Destination Role: Unknown (0)]
    [Current Mode: Unknown (-1)]
Bluetooth L2CAP Protocol
    Length: 6
    CID: Attribute Protocol (0x0004)
Bluetooth Attribute Protocol
    Opcode: Handle Value Notification (0x1b)
        0... .... = Authentication Signature: False
        .0.. .... = Command: False
        ..01 1011 = Method: Handle Value Notification (0x1b)
    Handle: 0x0032 (Unknown: Unknown)
        [Service UUID: Unknown (0xfcc0)]
        [UUID: Unknown (0xfcd1)]
    Value: 610102

And when it's written, you receive a Write Response

Frame 6895: 10 bytes on wire (80 bits), 10 bytes captured (80 bits)
    Encapsulation type: Bluetooth H4 with linux header (99)
    Arrival Time: Nov 11, 2024 10:51:31.604472000 West-Europa (standaardtijd)
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1731318691.604472000 seconds
    [Time delta from previous captured frame: 0.002540000 seconds]
    [Time delta from previous displayed frame: 0.002540000 seconds]
    [Time since reference or first frame: 50199.999499000 seconds]
    Frame Number: 6895
    Frame Length: 10 bytes (80 bits)
    Capture Length: 10 bytes (80 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    Point-to-Point Direction: Received (1)
    [Protocols in frame: bluetooth:hci_h4:bthci_acl:btl2cap:btatt]
Bluetooth
    [Source: TexasIns_52:a5:e8 (30:45:11:52:a5:e8)]
    [Destination: Google_a6:e7:d3 (74:74:46:a6:e7:d3)]
Bluetooth HCI H4
    [Direction: Rcvd (0x01)]
    HCI Packet Type: ACL Data (0x02)
Bluetooth HCI ACL Packet
    .... 0000 0000 0001 = Connection Handle: 0x001
    ..10 .... .... .... = PB Flag: First Automatically Flushable Packet (2)
    00.. .... .... .... = BC Flag: Point-To-Point (0)
    Data Total Length: 5
    Data
    [Connect in frame: 6498]
    [Disconnect in frame: 7003]
    [Source BD_ADDR: TexasIns_52:a5:e8 (30:45:11:52:a5:e8)]
    [Source Device Name: BTT A5E8]
    [Source Role: Unknown (0)]
    [Destination BD_ADDR: Google_a6:e7:d3 (74:74:46:a6:e7:d3)]
    [Destination Device Name: Pixel 7a]
    [Destination Role: Unknown (0)]
    [Current Mode: Unknown (-1)]
Bluetooth L2CAP Protocol
    Length: 1
    CID: Attribute Protocol (0x0004)
Bluetooth Attribute Protocol
    Opcode: Write Response (0x13)
        0... .... = Authentication Signature: False
        .0.. .... = Command: False
        ..01 0011 = Method: Write Response (0x13)
    [Handle: 0x004d (Unknown: Unknown)]
        [Service UUID: Unknown (0xfcc0)]
        [UUID: Unknown (0xfcd9)]
    [Request in Frame: 6892]