-
Notifications
You must be signed in to change notification settings - Fork 6
Teltonika AVL Parameter Parser ‐ What, How, When, and Why
This page will introduce you to the Teltonika AVL parameter ID parser, why it is required, how it works, what it produces as output, and how it is used within OpenRemote. The Teltonika AVL parameter ID parser (parser for the rest of the document) is an integral part of the OpenRemote Teltonika parameter integration.
You can find the repository for this tool here.
Teltonika transfers its data by providing a parameter ID and then that parameter's value. Each Teltonika GPS Tracker model may have different IDs for different devices, and different sets of parameters for different devices. For example, an advanced tracker with CAN connectivity will not have the same amount of available parameters as a device that does not interface with the car other than the car's battery. It could also have different parameters for Hence, we have written a custom Python program that is able to retrieve the parameter data. By running that script for your model name, you are able to retrieve the official parameter data from Teltonika's website.
Teltonika Telematics has a long device list, both end of life and currently supported. Those devices are crammed with sensors and calculations to provide an enormous amount of data to the user. To maintain a standard across the entire product line, Teltonika has created their own communication protocols for devices to communicate with servers. Those protocols, called codecs, are specialized to transmit the data generated by the device to the server with the smallest possible overhead. A new codec that was introduced was Codec JSON, that sends the device data as a JSON file, with the format that loosely adheres to AWS IoT Core Shadow payloads. Each payload is transmitted via MQTT and is received by OpenRemote, where it is interpreted and converted into a new Asset state with upserted attributes. The JSON payload received, contains some parameters that are always included (timestamp, priority, coordinates, altitude, angle, visible satellite count, speed, event parameter), and the rest are key-value pairs, with an integer as a key and any primitive as a value.
Those key-value pairs are the parameter data that the device sends over. The key of the key-value pair is the parameter ID, and the value is the value of that parameter that the device found.
Parameter IDs are in essence the unique ID for identifying parameters in the list of key-value pairs. Those IDs are generated by Teltonika Telematics, and each model type has their own list of parameters, and each parameter has its own ID. Here's the parameter list for the FMC003, FMB920, and FMB640.
To assure 100% data integrity within OpenRemote, meaning that 100% of each payload is understood and converted to an Asset containing all of the correct Attributes with the correct metadata, we require to know what each parameter ID actually represents; Instead of naming an attribute "389" and storing the corresponding payload value as a String, we need to know that the parameter actually is the total mileage of the car according to the OBD interface. If we know that, we can then convert it into a number for use with the Rules Engine, add a label to it for people to be able to read what the value says, and add constraints, a description, use the multiplier, etc. (TBD).
Instead of hardcoding the 20 or 30 most used parameter IDs and only reading those parameters from the device's payload, we use the Parser (remember what I said at the top?) to automatically parse the Teltonika table and save the information to a Pandas DataFrame. From there on out, we can easily just export the parameters to a JSON file, Excel, ODT, Word, PDF, you name it.
To generate the file, head to this repository, and use the Jupyter notebook by changing the device variable to the device you plan on using. Once done, it will generate a JSON file with a list of every single parameter that can be sent for that device.
You can add a new device at any time by creating a new Teltonika Model Configuration Asset, setting the model number, and pasting the contents of that JSON file within the "Parameter Data" attribute, and then updating the attribute. OpenRemote will automatically read the parameter IDs, save them to its database, append the small amount of parameters that are string-based into the list, and that will be used for every device with the model number you set.
Using that file, we generate a list of key-value pairs, with the key being the parameter ID and the value being the parameter data, like Name, Description, minimum and maximum values, multiplier, etc.
When a new payload comes in, we attempt to parse the list of parameters, and we match the keys of those parameters (the parameter IDs) to the parameter values (the data of the parameters). We then use that parameter data to properly parse the value sent in the payload to a legible Attribute.
There are some limitations with this approach. First, it seems as though the wiki pages with the parameter data are written by hand, leading to some typos. These typos are minimal, but they are still possible and have been found, so we cannot truly trust specific parameter data, such as minimum and maximum value constraints. For this reason, all available parameter data is parsed and stored, but:
- Constraints (minimum and maximum) are not being applied)
- Units are not being applied (No support for custom units of measurement)
- Description is not shown (No support for Attribute description other than labels)
To automatically be able to parse the data supplied by the tracker for any type of Teltonika device, other than being able to communicate with the device and retrieve its data, we will also need to be able to decode the message's contents.