The Pangolin API mirrors - as closely as possible - the existing API of AsyncMqttClient to enable users to migrate code quickly. However the AsyncMqttClient API has simple errors, redundant and flawed functions. For that reason some small changes have necessarily been made just to be able to compile correctly coded examples.
It also omits some necessary functionality which Pangolin has added. New users must be warned: what you see at this version of Pangolin is not good, fully functional nor error-free, it exists as it is for a very specific purpose, and will change a fair degree at the next release to correct all the problems described here.
The starting point then is to look first at the AsyncMqttClient API documentation
Many of the functions are simply wrong and appear to have been written with the misundertanding that MQTT payloads are "strings". They are not. The discussion of what is a "string" gets even experienced programmers heated: this is not "C programming for beginners". You should know the difference between a char* and byte* or uint8_t already.
MQTT payloads are NOT "strings" of any form or description and CANNOT be treated as such without serious problems and potential crashes
ALL MQTT payloads are length-described blocks of arbitrary bytes aka "BLOB" (Binary Large OBjects). They don't have to be large - mostly they contain only a few characters of temperature or some other sensor reading etc, but the term is a good one because it describes ALL payloads including the tiny ones.
The correct C/C++ datatype for such data is either byte or uint8_t. It is NOT, never has been and never will be char.
Therefore, any reference to payload data that is not a uint8_t or a uint8_t* pointer to that data is also WRONG. The biggest problem in assuming tht "its the same as a char*" is that 99% of C string functions use char* for the simple fact they operate on C strings and MQTT payloads are NOT C strings.
Seeing a char* then can easliy seduce the programmer into thinking what it points to is a string as 99.9% of the times he has seen it before, it does. Thus "Oh, char* - must be a string then, I'll pass it to strlen or strcpy or strXXXanything else" - but if you do any of those things with an MQTT payload, your code will probably crash, because it is NOT a C string.
That fact alone should be enough to remind you to always address payload data with a uint8_t* - if it isn't, you are going to have problems.
The MQTT specification has little more to say on the payload than this:
3.3.3 Payload
The Payload contains the Application Message that is being published. The content and format of the data is application specific
Think about "application specific": it means any app can put anything it wants in there... so why do some people assume it's a string? It's not, it's whatever the application that sent it chose to put in there. That's why it has no structure or type - that is up to you. It's just a "BLOB" of bytes whose correct C / C++ type is uint8_t[]. It is up to your code to check / validate / confirm that the contents are as expected. Failure to do so, or a faulty assumption about its contents will almost certaily lead to corrupt data at best and at worst, a crash.
Packet Ids are generated by a client and a server as a necessary part of the MQTT protocol "handshake" process. Client and server each have different sequences and they have no meaning whatsoever to the user.
As such, there is nothing safe, useful or informative any user can do with one. Debating their presence in the AsyncMqttClient API only raises more questions than it can answer: All references to them in the API will be removed at next release.
- new getMaxPayloadSize
Required to prevent attemps to send payload bigger than the lib can handle. AsyncMqttClient has no way of telling you what that figure is, and will just refuse or crash if you try. This is also the maximun value of any inbound packets. Any attempt in either direction will call the new onError callback with info on the size of the offending packet.
- new onError callbacxk
Provides much-needed handling of abnormal conditions simply ignored by AsyncMqttClient. The prototype is:
void onError(uint8_t code,int info);Where code is one of the following:
enum PANGO_FAILURE : uint8_t {
SUBSCRIBE_FAIL,
INBOUND_QOS_FAIL,
OUTBOUND_QOS_FAIL,
INBOUND_QOS_ACK_FAIL,
OUTBOUND_QOS_ACK_FAIL,
INBOUND_PUB_TOO_BIG,
OUTBOUND_PUB_TOO_BIG,
BOGUS_PACKET,
BOGUS_ACK
};info has a different meaning depending on the error.
- onMessage
To resolve the issues outlined above, the payload parameter has already been changed to the correct type uint8_t*: you may need to change your existing code.
There is no reason why the qos, dup and retain values should be inside a struct, let alone one with a 33-character name of AsyncMqttClientMessageProperties The name has been shortened for now to PANGO_PROPS and will be removed completely at the next release with those inner fields having their own distinct existence
AsyncMqttClient does not reassemble large inbound packets, it expects you to do all the hard work which is non-trivial. It is also a major source of bugs and part of the reason QoS1 & 2 do not work properly on AsyncMqttClient . Read more
Pangolin automatically ressembles any packet that will safely fit into memory and delivers you the whole packet once with the correct size. If it gets a "killer packet", it ignores all the fragments for you and then calls the new onError function to tell you the size of the bullet you just dodged.
For this reason, index and total are redundant
The next version is likely to look like:
void onMessage(const char* topic, uint8_t* payload, size_t length,uint8_t qos, bool dup, bool retain);- onMqttConnect
Passing the sessionPresent flag is redundant, since there is nothing the user can/should do differently no matter its value. It should be (but is not, see List of Bugs ) used inetrnally by the library to manage session recovery: it has no other use and will be removed at next release.
- onPublish, onSubscribe, unSubscribe callbacks
Have no practical purpose or function. See the notes above on packet Ids.
Obviously the functions that set these callbacks are now even more pointless. All 6 removed at next release withiout any loss of functionality to the library.
- publish
Perhaps equal first place with subscribe as the most use / important function in the API, so it's worrying that it contains errors and misinformation
Firstly, there is the necessary change to the correct uint8_t type for the payload which will require a change to existing code.
qos, retain and dup: Why are not the members of 33-character struct AsyncMqttClientMessageProperties required to be in one here? For the same reason already outlined above: Because there is no reason for them to be in one at all.
Now let's consider dup
The MQTT specification has this to say about dup:
3.3.1.1 DUP
...
If the DUP flag is set to 0, it indicates that this is the first occasion that the Client or Server has attempted to send this MQTT PUBLISH Packet. If the DUP flag is set to 1, it indicates that this might be re-delivery of an earlier attempt to send the Packet. The DUP flag MUST be set to 1 by the Client or Server when it attempts to re-deliver a PUBLISH Packet [MQTT-3.3.1.-1]. The DUP flag MUST be set to 0 for all QoS 0 messages [MQTT-3.3.1-2].
Is the user in control of any of that, or is it what he/she expects the library to be doing for hime/her?
The DUP flag in the outgoing PUBLISH packet is set independently to the incoming PUBLISH packet, its value MUST be determined solely by whether the outgoing PUBLISH packet is a retransmission [MQTT-3.3.1-3].
Note the use of "solely" - i.e only by something thing that can know when it is necessary: the library itself. Allowing the user to set it can only be described as pointless and potentially dangerous.
Not only is there no circumstance under which you should ever touch it, you absolutely should never be allowed to. If you even think of touching it, you would break QoS1. If QoS1 worked, that is.
New overloads to publish are added that make it easy to supply a payload of std::string or Arduino String without having to worry about calculating its length
setWill
Firstly it has a payload, so everything you have already learned about those also applies. It has already hd the incorrect char* changed to uint8_t* so - again - your existing code may need to change.
It also has a fatal bug. See Bugs
- onDisconnect
Has the same pointlessly-long-name issue: AsyncMqttClientDisconnectReason albeit only 30 chracters this time.
It describes amd renames what its just a uint8_t which is wrong anyway because to get the fullest information from the many valid disconnect reasons, it needs to be a signed int8_t.
LwIP produces error codes with negative value and these get fed back up through ESPAsyncTCP eventually to this library, which also has a few of its own valid reasons. How to tell the difference? All of Pangolin's are +ve of course, but now if you get a rare underlying TCP error which will help in diagnosing problems, you will at least get told the reason, unlike AsyncMqttClient which just either ignores it, lets its escalate to a confusingly different error or simply pretends it didn't happen.
The positive, library generated disconnect reasons are taken from:
TCP_DISCONNECTED = 0,
// MQTT_UNACCEPTABLE_PROTOCOL_VERSION = 1,
MQTT_IDENTIFIER_REJECTED = 2,
MQTT_SERVER_UNAVAILABLE = 3,
MQTT_MALFORMED_CREDENTIALS = 4,
MQTT_NOT_AUTHORIZED = 5,
// ESP8266_NOT_ENOUGH_SPACE = 6,
TLS_BAD_FINGERPRINT = 7,
TCP_TIMEOUT,
};
AsyncMqttClient is totally lacking in any funtionality to unpack or otherwise handle an incoming payload, which we now know is always just a BLOB. Pangoline remedies this by providing a few nifty functions that will save you a lot of time.
They all live in the PANGO namespace which means to call them you need to write e.g.
PANGO::dumphex(payload,length);Which will show something like this:
22:08:58.506 -> Address: 0x3FFF07AF len: 0x64 (100)
22:08:58.506 -> [0x3FFF07AF] 0x00000000: 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX
22:08:58.506 -> [0x3FFF07BF] 0x00000010: 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX
22:08:58.506 -> [0x3FFF07CF] 0x00000020: 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX
22:08:58.539 -> [0x3FFF07DF] 0x00000030: 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX
22:08:58.539 -> [0x3FFF07EF] 0x00000040: 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX
22:08:58.539 -> [0x3FFF07FF] 0x00000050: 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 58 XXXXXXXXXXXXXXXX
22:08:58.539 -> [0x3FFF080F] 0x00000060: 58 58 58 58 XXXX
In the real world when you subscribe to a topic, you generally know what is the format of the data it contains. Usually that is because you have decided it! That fact alone however does not stop:
-
People like me loading up any old MQTT client app e.g. MQTT-spy and throwing a huge packet of random junk at you under the topic that should have your cat's heartbeat per minute in it as a c-style string of less than 4 digits.
-
Badly written MQTT client libraries can corrupt data in-flight that was valid when someone else sent it.
The point is if you don't validate the packet, no-one else is going to. And if you don't validate it and simply assume it has in it what it ought to, well then you are designing-in failure. ALWAYS VALIDATE YOUR PACKETS!
Pangolin provides the following assistance in addition to the hex dumper already mentioned:
char* payloadToCstring(uint8_t* data,size_t len);
int payloadToInt(uint8_t* data,size_t len);
std::string payloadToStdstring(uint8_t* data,size_t len);
String payloadToString(uint8_t* data,size_t len);
All of them are designed to be called with PANGO::xxx(payload, length) inside the onMessage callback, but will work anywhere in your code with any blob + known length. There are plenty of examples in the er... examples.
char* payloadToCstring(uint8_t* data,size_t len);
Returns a char* that you can pass to any C string function only if the packet actually does in fact contain a valid NUL-terminated c-string. If it doesn't then all hell will break loose probably leading to an exception and a crash.
In addition you MUST free the returned pointer before exiting your function or you will cause a memory leak. You have been warned
Happily, none of the other functions require anything fancy like that.
int payloadToInt(uint8_t* data,size_t len);
Returns an int only if the packet actually does in fact contain a valid NUL-terminated c-string consiting of numeric digits. If it doesn't then all hell etc... If it' a valid string but contains alpah characters you will get either zero some bogus value.
std::string payloadToStdstring(uint8_t* data,size_t len);
Does what is says on he tin if valid c-string etc - you know the drill by now.
String payloadToString(uint8_t* data,size_t len);
Same as above but returns an Arduino String (capital "S") if you re daft enoiugh to want to use one.]