BACnet Proxy Improvements

[acedrew]

Introduction

Based on conversations with the PNNL team, I've started this discussion to document improvements and issues with the BACnet proxy that could be resolved during the process of updating the upstream BACPypes Library. Please feel free to add additional observations or issues here.

Scope:

The BACnet proxy has not had significant improvements in several years, and there are some fundamental issues to address.
I'll try to categorize them as Standards Compliance, Technology Robustness, and Feature Improvements.

Standards Compliance

• Currently the BACnet Device config requires a device address, this is counter to the BACnet Standard, where an ID is meant to be unique to a BACnet Network in the standard. In the real world, especially with BACnet/IP, you will find non-unique IDs, typically behind different IP addresses. So we need to maintain the ability to point to a particular address with an ID, but by default, we should be able to perform a WHOIS with the ID limited to that device ID to determine the routable address. Coincidentally, this WHOIS is already happening, but it's sent to the target address, not broadcast, likely it's as simple as adding a branch that broadcasts if the the target address is not set. There will be a need to setup another workflow to manage when a device moves addresses and fails to respond to a query, and this behavior should probably be behind a flag, and have some form of rate limiting, in order to eliminate pesky broadcast messages on unreliable networks.

Technology Robustness

• Currently there are many unhandled exceptions happening in the driver and bacnet_proxy, and they do not seem to be communicating well about what's actually going wrong on a particular failed scrape. There are defined enumerations in BACpypes library for all of the BACnet Standard failure codes, and we should decode those for the users, further, some of them can be automatically resolved by switching to read_property from read_property_multiple for instance.
• Migrate to the AsyncIO version of the BACPypes library, gaining performance and reliability improvements from the upstream library, as well as slightly simplified API

Feature Improvements

• Automatic backoff or interval skipping after timeouts for a particular device
• Automatic backoff for a network segment for devices behind the same router (include discovery of network topology)
• Failure type dependent handling of failures.

[bonicim]

@acedrew Are the categories ordered according to priority? In other words, should dev time focus on Standards Compliance, followed by Tech Robustness, etc?

[acedrew]

That feels like questions for above my pay grade/perhaps too early to prioritize, hopefully other folks can contribute from their experience and perspective as well before we try to rank them.

…

On Wed, Mar 16, 2022 at 1:53 PM Mark Bonicillo ***@***.***> wrote: @acedrew <https://github.com/acedrew> Are the categories ordered according to priority? In other words, should dev time focus on Standards Compliance, followed by Tech Robustness, etc? — Reply to this email directly, view it on GitHub <#2925 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHOUBYIQW7AWVUOKGK3XI3VAINYDANCNFSM5QZD64XQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bonicim]

Introduction

Based on conversations with the PNNL team, I've started this discussion to document improvements and issues with the BACnet proxy that could be resolved during the process of updating the upstream BACPypes Library. Please feel free to add additional observations or issues here.

Scope:

The BACnet proxy has not had significant improvements in several years, and there are some fundamental issues to address. I'll try to categorize them as Standards Compliance, Technology Robustness, and Feature Improvements.

Standards Compliance

• Currently the BACnet Device config requires a device address, this is counter to the BACnet Standard, where an ID is meant to be unique to a BACnet Network in the standard. In the real world, especially with BACnet/IP, you will find non-unique IDs, typically behind different IP addresses. So we need to maintain the ability to point to a particular address with an ID, but by default, we should be able to perform a WHOIS with the ID limited to that device ID to determine the routable address. Coincidentally, this WHOIS is already happening, but it's sent to the target address, not broadcast, likely it's as simple as adding a branch that broadcasts if the the target address is not set. There will be a need to setup another workflow to manage when a device moves addresses and fails to respond to a query, and this behavior should probably be behind a flag, and have some form of rate limiting, in order to eliminate pesky broadcast messages on unreliable networks.

Technology Robustness

• Currently there are many unhandled exceptions happening in the driver and bacnet_proxy, and they do not seem to be communicating well about what's actually going wrong on a particular failed scrape. There are defined enumerations in BACpypes library for all of the BACnet Standard failure codes, and we should decode those for the users, further, some of them can be automatically resolved by switching to read_property from read_property_multiple for instance.
• Migrate to the AsyncIO version of the BACPypes library, gaining performance and reliability improvements from the upstream library, as well as slightly simplified API

Feature Improvements

• Automatic backoff or interval skipping after timeouts for a particular device
• Automatic backoff for a network segment for devices behind the same router (include discovery of network topology)
• Failure type dependent handling of failures.

@acedrew Regarding Tech Robustness, which Asyncio version of BACpypes are you referring to? I was looking through their read-the-docs and the Releases page, but couldn't confirm which version has it. Is it safe to assume that v0.18.6 (the latest version) uses asyncio?

[bbartling]

(running into some formatting issues, I cant seem to update below, sorry for this)

Just throwing this out there as I assume there is no easy button, but if there were could we just use BAC0? Christrian Trembley seems to keep his BAC0 app up with whatever version Joel Bender is producing in BACpypes. It seems like BAC0 is just a BACpypes wrapper and someone already is updating it quite nicely : )

BACnet global for whois with BAC0:

import BAC0
import time

# define BAC0 app
bacnet = BAC0.lite()

# BACnet scan network
time.sleep(1)
devices = bacnet.whois(global_broadcast=True)
device_mapping = {}
for device in devices:
    if isinstance(device, tuple):
        device_mapping[device[1]] = device[0]
        print("Detected device %s with address %s" % (str(device[1]), str(device[0])))
print(device_mapping)
print((str(len(device_mapping)) + " devices discovered on network."))

I've been playing around with an app where all the person does is feed an f string some data to perform BACnet read, writes, releases. Like this:

  read_vals = f'{address} {object_type} {object_instance} presentValue'
  print("Excecuting BACnet read statement:", read_vals)
  read_result = bacnet.read(read_vals)

Could we ever just easy button BAC0 into VOLTTRON where the BAC0 author keeps updating his app for whatever is happening in BACpypes? (like BACnet secure, etc..)

I dont understand all that goes into g event or asyncio but I haven't had the need to perform anything more than a BACnet read, release, or write in my experience as a field technician for building automation. Some concept idea:

class BacNetWorker(PydanticView):
    async def do_things(action,address,object_type,object_instance, **kwargs):
        value = kwargs.get('value', None)
        priority = kwargs.get('priority', None)

        if action == "read":
            try:
                read_vals = f'{address} {object_type} {object_instance} presentValue'
                print("Excecuting BACnet read statement:", read_vals)
                read_result = bacnet.read(read_vals)
                if isinstance(read_result, str):
                    pass
                else:
                    read_result = round(read_result,2)
                return read_result
            except Exception as error:
                return "error"
      
        elif action == "write":
            try:
                write_vals = f'{address} {object_type} {object_instance} presentValue {value} - {priority}'
                print("Excecuting BACnet write statement:", write_vals)
                bacnet.write(write_vals)
                return write_vals          
            except Exception as error:
                return "error"

        elif action == "release":
            try:    
                release_vals = f'{address} {object_type} {object_instance} presentValue null - {priority}'
                print("Excecuting BACnet release statement:", release_vals)
                bacnet.write(release_vals)
                return release_vals 
            except Exception as error:
                return "error"
                
        else:
            return "server error on BACnet opts"

I know BAC0 is synchronous code, so maybe a bad idea? But the code above with the BACnet worker function works in a asnycio hobby project where I can call the synchronous BAC0 function with async function style coding. I can replicate what VOLTTRON does read write release points.

I'd love to be involved with these efforts but I get lost real fast under the hood in BACpypes.

[bbartling]

On the BAC0 Gitter Christian said they may merge BAC0 into BACpypes at some future point in time.

Christian Tremblay @ChristianTremblay 11:59
Eventually. Maybe. bacpypes3 will be (is) async … but it means a lot of work so we’ll see. Could we merge BAC0 inside it ? Future will tell.

[acedrew]

The challenge is BACnet is complex. And to have effective performance at scale, we need the full complexity of BACpypes available to meet the use cases. We especially need the callback mechanisms in BACpypes to handle broadcast traffic, COV, etc. In my experience, BACpypes is pretty complex, but it maps pretty directly to the 1500 page ASHARE 135 standard.
The other potential would be to move the bacnet proxy to something Steve Karg's http://bacnet.sourceforge.net/ to enable higher performance.

[bbartling]

Cool stuff @acedrew . For VOLTTRON would this BACnet protocol stack look like its written in C just run in its own container alongside of VOLTTRON? Seems updated, like 6 days ago: https://sourceforge.net/projects/bacnet/

[ChristianTremblay]

@bbartling, thanks for the act of trust :) but for any project of the scale of Volttron they would be best served by directly using bacpypes3 or karg's stack. Having someone between them and bacpypes3 (or karg's) would probably bring more problems as their use case may require things that I don't. And you don't want to wait that Christian finds some time to fix a bug that makes you lost money.

And as Andrew said, BACnet is complex. It's not the typical API or server-client architecture. Add to that all the edge-cases of asyncio... my weekends and night shifts are all set for the next 5 years...

Good thing is that the community is growing. The more people will use bacpypes(3), the more solid it will become and good practices will be shared amongst implementations.

and not too fast on the merge :0) Joel's stuff is level 356 and I'm still learning to walk. And bacpypes have always been this huge toolbox that allow you to build what you want. In its actual form, BAC0 is way too opiniated... So let's pretend I never said that. But if bacpypes3 offers a simple complete command line / script feature... BAC0 will become useless... or at least very different.

[jkempf42]

Hi,

Just want to reiterate the problem I'm seeing with the Yabe and Chipkin device simulators. Basically, regardless of whether they receive a message on the unicast subnet address (ex. 192.162.7.1) , broadcast subnet address (ex. 192.162.7.255), or all points broadcast address (255.255.255.255), they respond on the all points broadcast address. The ASHRE 2020 spec does not specify on which address a response should be made; in particular, it does not specify that the unicast address should be favored to avoid excessive broadcast. I don't know if this is a concern for actual devices as opposed to simulators, but given that the behavior is spec complaint it might be. Chipkin, for example, sells a BACnet stack for embedded use.

So it might be helpful if the BACnet proxy could have an option to listen on multiple addresses in case a device responded on a different address than the unicast address.

Thanx.

      jak

[jkempf42]

In the area of feature improvement, may I suggest simplifying the split between the BACnet Proxy agent and the BACnet driver? If I understand the BACnet driver correctly, from the documentation, it needs to have the IP address of a BACnet device (not the BACnet Proxy agent but one actual device, the documenation is not clear what to do if you have more than one device). And there needs to be a BACnet proxy agent per UDP port on which Volttron is expecting BACnet traffic, with the IP address of the local Volttron platform (VCP agent). Why not instead have the BACnet driver take a list of IP addresses and UDP ports for BACnet/IP devices, paired naturally with schema for the objects and object properties, in JSON rather than CSV (to simplify configuration file formats)? My inspiration here is the Thingsboard Gateway, an open source Python gateway with drivers for many different device protocols that is part of the Thingsboard IoT framework. If the BACnet devices happen to be on an MS/TP network, then the device type will be a BACnet router which will route from the MS/TP network to Volttron on the IP network. With the current architecture, it is unclear why the split is needed. Thanx.

[bbartling]

My experience:

the scanning scripts require the manually configuration for the computers IP address ----> run the scans with appropriate file outputs ----> then to save the BACnet scan data (CSV info) for the platform driver config ---> install BACnet proxy ---> install the platform driver ----> start BACnet proxy/platform driver it all takes off via the platform driver knowing that the device types are BACnet.

And if you have to modify the BACnet networks config (CSV files for the platform driver) you have to stop the BACnet proxy before using any of the scanning scripts to rescan the BACnet network.

Its a daunting processes, I wish the auto scan script was easy to run, there are so my arguments to make it go that for myself renders it useless.

So my former life was a building automation field technician where this industry uses all canned GUI software to pull in BACnet data and create your BACnet network on a canned platform supervisory level controller. I know there is people out there that have developed the VOLTTRON BACnet processes with a GUI to have the same look and feel of a BAS platform. Two things I wish the community could go to would be a GUI for this process and the auto scan script to be painless/easy to use. I do think there is power in auto scanning the entire BACnet network at one time (if used appropriately/carefully not the bring the system down) and then to have some simple method GUI or JSON like you mentioned to trim down the point count for the platform driver to only capture what data/BACnet points you need.

[bonicim]

FYI, I am currently tracking and scoping improvements to all things BACnet in Volttron (e.g. BACnet proxy agent, BACnet driver, BACnet scripts). https://github.com/orgs/VOLTTRON/projects/6/views/12

[acedrew]

@Jak42 can you clarify what device you're referring to? The device_address in the registry config (which can be a json file if you like, nothing forcing CSV there) is simply the address that resolves to the device_id. This is one of the things that should be resolved, it's not compliant with the standard, it's nice to be able to specify the address to eliminate additional WHO-IS broadcasts, but currently the driver fires them off anyway (limited to the device_id range, so not too bad for the network). What should happen is if the device_address is specified, it should be used, if not, the BACnet address that is reported by the WHO-IS call should be accepted. Further, if a device stops responding at a particular address, a WHO-IS should be issued periodically and the resulting BACnet address used for future unicast communications.

For your other point, the BACnet Proxy should listen for broadcast traffic, this can be verified by running ss -tunpl and noticing that the proxy PID is listening on both the address in the configuration and the broadcast address for the network as indicated by the CIDR notation in the config. If it's not listening for broadcast traffic you can see the format for the proxy device address in the BACpypes docs.

Your note about MS/TP doesn't really make sense, as the BACnet internetworking handles resolving devices, regardless of what network segment they might be on, provided they're routable.
There's a lot to BACnet, obvious by the 1300 pages in the standard, but it's important specifically to remember the difference in BACnet/IP and BACnet/Ethernet which drives a lot of the complexity incapsulated in the split driver/proxy. Also, the BACnet proxy acts as a device on the network and you can participate in the BACnet network, not just as polling device, but as a readable device as well.

[bbartling]

Can python subprocess ever be used in VOLTTRON for BACnet? I can read and write BACnet points using the sample scripts in the BACpypes repo. For example with the ReadProperty.py in the BACpypes repo I can call it with another script and pass in my BACnet device/point address:

import subprocess
read = "echo 'read 12345:2 analogInput:2 presentValue' | python ReadProperty.py"

sensor_reading = subprocess.check_output(read, shell=True)

print("sensor reading is: ", float(sensor_reading))

prints sensor reading is: 66.06999206542969 which is a value from a mstp device on my test bench.

Would we need a proxy at all if we can just make a read, write, release script to call from subprocess?

[jkempf42]

Ben, My understanding from the discussion with Robert a few weeks ago is that the proxy is needed in case some of the BACnet/IP devices use ports other than the default 47808. But it seems that a Python thread in the Platform Driver agent could be used if some of the devices don't use the default port, with a separate thread for a BACnet driver on each port. Of course, this would require that the nondefault ports be included in the Platform Driver agent configuration file, but that would be required in the current case for the BACnet Proxy agent anyway. The Thingsboard Gateway uses this architecture (here's a link <https://github.com/thingsboard/thingsboard-gateway> to the github repo). Having a BACnet proxy for each BACnet device is kind of awkward, since it makes configuration of a lean system difficult. However, getting rid of the BACnet Proxy agent won't get rid of the need to preconfigure the Platform Driver agent's BACnet driver with the points and properties it needs to monitor, since as we also discussed, currently Volttron has no building metadata API for filtering the points to only those the particular deployment wants to monitor. Thanx. jak

…

On Sat, May 28, 2022 at 8:10 AM Ben Bartling ***@***.***> wrote: Can python subprocess ever be used in VOLTTRON for BACnet? I can read and write BACnet points using the sample scripts in the BACpypes repo. For example with the ReadProperty.py <https://github.com/JoelBender/bacpypes/blob/master/samples/ReadProperty.py> in the BACpypes repo I can call it with another script and pass in my BACnet device/point address: import subprocess read = "echo 'read 12345:2 analogInput:2 presentValue' | python ReadProperty.py" sensor_reading = subprocess.check_output(read, shell=True) print("sensor reading is: ", float(sensor_reading)) prints sensor reading is: 66.06999206542969 Would we need a proxy at all if we can just make a read, write, release script to call from subprocess? — Reply to this email directly, view it on GitHub <#2925 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GQ2AQU7CHRLCMZI643VMIZOLANCNFSM5QZD64XQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bbartling]

Hey Jak, does thingsboard support writing the BACnet device? For example could you create some sort of algorithm on thingsboard that will write to the BACnet system? Or is it only for retrieving data?

[bbartling]

One thing that is sort of cool is Joel Bender has another repo for BACnet called BACpypes snapshot:
https://github.com/JoelBender/bacpypes-snapshot

I think this is similar for when using VOLTTRON initially on a BACnet system someone needs to scan the network inside the building with the VOLTTRON bacnet scanning scripts where the effort is to create all the CSV files for the platform.driver. This process is daunting and I hope the setup process for BACnet can also be improved.

When using BACpypes snapshot, running python snapshot.py bensfurnace for an mstp device on a testbench at my house where this scripts opens up the console where I can then run a whois on a range of devices IDs:
> whois 201200 201202

In current volttron processes I think this would be similar to running this below to generate the config files:
python grab_bacnet_config.py 201201 --registry-out-file registry_configs/201201.csv --driver-out-file ./devices/201201

In BACpypes-snapshot, I can replay this scanning process of my test bench device: python dump.py bensfurnace 201201 and see alot of information about this device like all of the points, point readings, all BACnet object types that the device supports. The dump file is large enough I just created a gist.

Why not fork BACpypes-snapshot to replace the current scanning scripts? This process of using the BACpypes console seems way easier than running scanning scripts...

[jkempf42]

Yes, you can either send a message over the MQTT bus or use a HTTP REST from the Thingboard Web UI, see here <https://thingsboard.io/docs/iot-gateway/config/bacnet/>for BACnet gateway configuration. jak

…

On Mon, May 30, 2022 at 6:30 AM Ben Bartling ***@***.***> wrote: Hey Jak, does thingsboard support writing the BACnet device? For example could you create some sort of algorithm on thingsboard that will write to the BACnet system? Or is it only for retrieving data? — Reply to this email directly, view it on GitHub <#2925 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GXUB4CSCQLFU7SEZ5DVMS7G7ANCNFSM5QZD64XQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>