Added documentation for protocol and slightly refactored it (#2)

eclipse-ecal · Mar 21, 2024 · 32a8347 · 32a8347
1 parent ab6f608
commit 32a8347
Show file tree

Hide file tree

Showing 5 changed files with 168 additions and 51 deletions.
diff --git a/README.md b/README.md
@@ -4,7 +4,19 @@
 
 # ecaludp
 
-ecaludp is the underlying implementation for UDP traffic in eCAL. It transparently fragments and reassembles messages to provide support for big messages. 
+ecaludp is the underlying implementation for UDP traffic in eCAL. It transparently **fragments and reassembles** messages to provide support for big messages. An ecaludp socket is **not limited** to the ordinary UDP datagram size of ~64KiB. It can transport messages **up to 4 GiB**.
+
+ecaludp has **npcap support** for efficient receiving of multicast traffic in Windows. For that, the [udpcap](https://github.com/eclipse-ecal/udpcap) library is used.
+
+ecaludp requires C++14.
+
+## Sample Projects
+
+ecalupd features an asio-style API. Check out the following samples to see its usage:
+
+- [ecaludp_sample](samples/ecaludp_sample/src/main.cpp): The regular ecaludp socket API
+
+- [ecaludp_npcap_sample](samples/ecaludp_sample_npcap/src/main.cpp) The (slightly different) npcap socket API. Only available for Windows.
 
 ## Dependencies
 
@@ -29,6 +41,31 @@ When building the **tests**, the following dependency is required:
 |----------------|-------------|-------------------------|
 | [Googletest](https://github.com/google/googletest) | [BSD-3](https://github.com/google/googletest/blob/main/LICENSE) | [git submodule](https://github.com/eclipse-ecal/ecaludp/tree/master/thirdparty) |
 
+## How to checkout and build
+
+1. Install cmake and git / git-for-windows
+
+2. Checkout this repo and the asio submodule
+    ```console
+    git clone https://github.com/eclipse-ecal/ecaludp.git
+    cd ecaludp
+    git submodule init
+    git submodule update
+    ```
+
+3. CMake the project _(check the next section for available CMake options)_
+    ```console
+    mkdir _build
+    cd _build
+    cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=_install
+    ```
+
+4. Build the project
+    - Linux: `make`
+    - Windows: Open `_build\ecaludp.sln` with Visual Studio and build the example project
+
+5. Check the functionality with the `udpcap_sample /.exe` sample project!
+
 ## CMake Options
 
 You can set the following CMake Options to control how ecaludp is built:
@@ -42,4 +79,84 @@ You can set the following CMake Options to control how ecaludp is built:
 | `ECALUDP_USE_BUILTIN_RECYCLE`| `BOOL`| `ON` | Use the builtin steinwurf::recycle submodule. If set to `OFF`, recycle must be available from somewhere else (e.g. system libs). |
 | `ECALUDP_USE_BUILTIN_UDPCAP`| `BOOL`| `ON`<br>_(when building with npcap)_ | Use the builtin udpcap submodule. Only needed if `ECALUDP_ENABLE_NPCAP` is `ON`. If set to `OFF`, udpcap must be available from somewhere else (e.g. system libs). Setting this option to `ON` will also use the default dependencies of udpcap (npcap-sdk, pcapplusplus). |
 | `ECALUDP_USE_BUILTIN_GTEST`| `BOOL`| `ON` <br>_(when building tests)_ | Use the builtin GoogleTest submodule. Only needed if `FINEFTP_SERVER_BUILD_TESTS` is `ON`. If set to `OFF`, GoogleTest must be available from somewhere else (e.g. system libs). |
-| `ECALUDP_LIBRARY_TYPE` | `STRING` |             | Controls the library type of Ecaludp by injecting the string into the `add_library` call. Can be set to STATIC / SHARED / OBJECT. If set, this will override the regular `BUILD_SHARED_LIBS` CMake option. If not set, CMake will use the default setting, which is controlled by `BUILD_SHARED_LIBS`. |
+| `ECALUDP_LIBRARY_TYPE` | `STRING` |             | Controls the library type of Ecaludp by injecting the string into the `add_library` call. Can be set to STATIC / SHARED / OBJECT. If set, this will override the regular `BUILD_SHARED_LIBS` CMake option. If not set, CMake will use the default setting, which is controlled by `BUILD_SHARED_LIBS`. |
+
+## Protocol Specification (Version 5)
+
+An ecaludp message consists of one or multiple datagrams. How many datagrams that will be is determined by the fragmentation.
+
+Each datagram carries a header starting at byte 0. The header is defined in [header_v5.h](ecaludp/src/protocol/header_v5.h). Alien datagrams can be eliminated by comparing the `magic` bytes with a predefined value.
+Some datagrams may also carry payload directly after the header.
+
+| size   | Name       | Explanation                                             |
+|--------|------------|---------------------------------------------------------|
+| 32 bit | `magic`      | User-defined binary data. Used for identifying and dropping alien traffic. |
+| 8 bit  | `version`    | Header version. Must be `5` for protocol version 5     |
+| 24 bit | _reserved_ | Must be sent as 0.     |
+| 32 bit <br> little-endian | `type` | Datagram type. Must be one of: <br> `1`: fragmented_message_info <br> `2`: fragment <br> `3`: non_fragmented_message   |
+| 32 bit <br> signed little-endian | `id`       | Random ID to match fragmented parts of a message. Used differently depending on the message type:<br> - `type == fragmented_message_info (1)`: The Random ID that this fragmentation info will be applied to <br> - `type == fragment (2)`: The Random ID that this fragment belongs to. Used to match fragments to their fragmentation info <br> - `type == non_fragmented_message (3)`: Unused field. Must be sent as -1. Must not be evaluated.|
+| 32 bit <br> unsigned little-endian | `num` | Fragment number. Used differently depending on the datagram type: <br> - `type == fragmented_message_info (1)`: Amount of fragments that this message was split into. <br> - `type == fragment (2)`: The number of this fragment <br> - `type == non_fragmented_message (3)`: Unused field. Must be sent as `1`. Must not be evaluated. |
+| 32 bit <br> unsigned little-endian | `len` | Payload length. Used differently depending on the datagram type. The payload must start directly after the header. <br> - `type == fragmented_message_info (1)`: Length of the original message before it got fragmented. Messages of this type must not carry any payload themselves. <br> - `type == fragment (2)`: The payload lenght of this fragment <br> - `type == non_fragmented_message (3)`: The payload length of this message |
+| `len` bytes | _payload_ | Payload of the message or fragment.
+
+### Message Types
+
+There are two different types of messages that can be sent: Messages that are fragmented and messages that are not fragmented.
+
+1. **Non-fragmented data**
+   - The entire message **consists of 1 datagram** carrying both a header and the payload.
+   - The header looks as follows:
+        - `type` is set to `non_fragmented_message (3)`
+        - `id` is `-1`
+        - `num` is `1`
+        - `length` is the amount of payload bytes following after the header
+
+2. **Fragmented data**
+
+    A message which had to be fragmented into $n \in \mathbb{N}_0$ parts **consists of $n+1$ datagrams**:
+
+    - **$1\times$ Fragmentation info**
+        - The first datagram carries the fragmentation info.
+        - The header looks as follows:
+            - `type` is set to `fragmented_message_info (1)`
+            - `id` is a random number. It is used to match the fragments to their fragmentation info and therefore must be unique for each fragmented message.
+            - `num` is the amount of fragments that the message was split into (i.e. $n$)
+            - `length` is the length of the original message before it got fragmented
+        - This datagram must not carry any payload.
+
+    - **$n\times$ Fragments**
+        - The following $n$ datagrams carry the fragments.
+        - The header looks as follows:
+            - `type` is set to `fragment (2)`
+            - `id` is the random number that was used in the fragmentation info
+            - `num` is the number of this fragment (i.e. $1$ to $n$)
+            - `length` is the length of the payload of this fragment.
+        - The payload of each fragment is a part of the original message.
+
+### Communication diagram
+
+The following diagram shows the communication between the sender and the receiver. The sender sends a non-fragmented message and a fragmented message. The fragmented message consists of n fragments.
+
+```
+  Sender                              Receiver
+    |                                    |
+    |  Non-fragmented message            |
+    |----------------------------------->|
+    |                                    |
+    |  Fragmentation info (n fragments)  |
+    |----------------------------------->|
+    |                                    |
+    |  Fragment 1                        |
+    |----------------------------------->|
+    |                                    |
+    |  Fragment 2                        |
+    |----------------------------------->|
+    |                                    |
+    |  ...                               |
+    |----------------------------------->|
+    |                                    |
+    |  Fragment (n)                      |
+    |----------------------------------->|
+    |                                    |
+    |                                    |
+```
diff --git a/ecaludp/src/protocol/datagram_builder_v5.cpp b/ecaludp/src/protocol/datagram_builder_v5.cpp
@@ -102,8 +102,8 @@ namespace ecaludp
 
       header_ptr->version = 5;
 
-      header_ptr->type     = static_cast<ecaludp::v5::message_type_uint32t>(
-                                htole32(static_cast<uint32_t>(ecaludp::v5::message_type_uint32t::msg_type_non_fragmented_message)));
+      header_ptr->type     = static_cast<ecaludp::v5::datagram_type_uint32t>(
+                                htole32(static_cast<uint32_t>(ecaludp::v5::datagram_type_uint32t::datagram_type_non_fragmented_message)));
       header_ptr->id       = htole32(int32_t(-1));   // -1 => not fragmented
       header_ptr->num      = htole32(uint32_t(1));    // 1 => only 1 fragment
       header_ptr->len      = htole32(static_cast<uint32_t>(total_size)); // denotes the length of the payload of this message only
@@ -168,8 +168,8 @@ namespace ecaludp
 
         fragment_info_header_ptr->version = 5;
 
-        fragment_info_header_ptr->type     = static_cast<ecaludp::v5::message_type_uint32t>(
-                                                htole32(static_cast<int32_t>(ecaludp::v5::message_type_uint32t::msg_type_fragmented_message_info)));
+        fragment_info_header_ptr->type     = static_cast<ecaludp::v5::datagram_type_uint32t>(
+                                                htole32(static_cast<int32_t>(ecaludp::v5::datagram_type_uint32t::datagram_type_fragmented_message_info)));
         fragment_info_header_ptr->id       = htole32(message_id);
         fragment_info_header_ptr->num      = htole32(needed_fragment_count);
         fragment_info_header_ptr->len      = htole32(total_size); // denotes the length of the entire payload
@@ -202,8 +202,8 @@ namespace ecaludp
 
           header_ptr->version = 5;
 
-          header_ptr->type     = static_cast<ecaludp::v5::message_type_uint32t>(
-                                    htole32(static_cast<uint32_t>(ecaludp::v5::message_type_uint32t::msg_type_fragment)));
+          header_ptr->type     = static_cast<ecaludp::v5::datagram_type_uint32t>(
+                                    htole32(static_cast<uint32_t>(ecaludp::v5::datagram_type_uint32t::datagram_type_fragment)));
           header_ptr->id       = htole32(message_id);
           header_ptr->num      = htole32(static_cast<uint32_t>(datagram_list.size() - 2)); // -1, because the first datagram is the fragmentation info
           header_ptr->len      = htole32(static_cast<uint32_t>(0));                        // denotes the length of the entire payload

diff --git a/ecaludp/src/protocol/header_v5.h b/ecaludp/src/protocol/header_v5.h
@@ -21,41 +21,41 @@ namespace ecaludp
 {
   namespace v5
   {
-    enum class message_type_uint32t : uint32_t
+    enum class datagram_type_uint32t : uint32_t
     {
-      msg_type_unknown                 = 0,
-      msg_type_fragmented_message_info = 1, // former name: msg_type_header
-      msg_type_fragment                = 2, // former name: msg_type_content
-      msg_type_non_fragmented_message  = 3  // former name: msg_type_header_with_content
+      datagram_type_unknown                 = 0,
+      datagram_type_fragmented_message_info = 1, // former name: msg_type_header
+      datagram_type_fragment                = 2, // former name: msg_type_content
+      datagram_type_non_fragmented_message  = 3  // former name: msg_type_header_with_content
     };
 
     #pragma pack(push, 1)
     struct Header
     {
-      char                 magic[4];
-
-      uint8_t              version;   /// Header version. Must be 5 for this version 5 header
-      uint8_t              reserved1; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future.
-      uint8_t              reserved2; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future.
-      uint8_t              reserved3; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future.
-
-      message_type_uint32t type;      /// The message type. See message_type_uint32t for possible values
-
-      int32_t              id;        /// Random ID to match fragmented parts of a message (Little-endian). Used differently depending on the message type:
-                                      ///   - msg_type_fragmented_message_info: The Random ID that this fragmentation info will be applied to
-                                      ///   - msg_type_fragment: The Random ID that this fragment belongs to. Used to match fragments to their fragmentation info
-                                      ///   - msg_type_non_fragmented_message: Unused field. Must be sent as -1. Must not be evaluated.
-
-      uint32_t             num;       /// Fragment number (Little-endian). Used differently depending on the message type:
-                                      ///   - msg_type_fragmented_message_info: Amount of fragments that this message was split into.
-                                      ///   - msg_type_fragment: The number of this fragment
-                                      ///   - msg_type_non_fragmented_message: Unused field. Must be sent as 1. Must not be evaluated.
-
-      uint32_t             len;       /// Payload length (Little-endian). Used differently depending on the message type. The payload must start directly after the header.
-                                      ///   - msg_type_fragmented_message_info: Length of the original message before it got fragmented.
-                                      ///                                       Messages of this type must not carry any payload themselves.
-                                      ///   - msg_type_fragment: The payload lenght of this fragment
-                                      ///   - msg_type_non_fragmented_message: The payload length of this message
+      char                  magic[4];
+
+      uint8_t               version;   /// Header version. Must be 5 for this version 5 header
+      uint8_t               reserved1; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future.
+      uint8_t               reserved2; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future.
+      uint8_t               reserved3; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future.
+
+      datagram_type_uint32t type;      /// The datagram type. See datagram_type_uint32t for possible values
+
+      int32_t               id;        /// Random ID to match fragmented parts of a message (Little-endian). Used differently depending on the datagram type:
+                                       ///   - datagram_type_fragmented_message_info: The Random ID that this fragmentation info will be applied to
+                                       ///   - datagram_type_fragment: The Random ID that this fragment belongs to. Used to match fragments to their fragmentation info
+                                       ///   - datagram_type_non_fragmented_message: Unused field. Must be sent as -1. Must not be evaluated.
+
+      uint32_t              num;       /// Fragment number (Little-endian). Used differently depending on the datagram type:
+                                       ///   - datagram_type_fragmented_message_info: Amount of fragments that this message was split into.
+                                       ///   - datagram_type_fragment: The number of this fragment
+                                       ///   - datagram_type_non_fragmented_message: Unused field. Must be sent as 1. Must not be evaluated.
+
+      uint32_t              len;       /// Payload length (Little-endian). Used differently depending on the datagram type. The payload must start directly after the header.
+                                       ///   - datagram_type_fragmented_message_info: Length of the original message before it got fragmented.
+                                       ///                                       Messages of this type must not carry any payload themselves.
+                                       ///   - datagram_type_fragment: The payload lenght of this fragment
+                                       ///   - datagram_type_non_fragmented_message: The payload length of this message
     };
     #pragma pack(pop)
   }

diff --git a/ecaludp/src/protocol/reassembly_v5.cpp b/ecaludp/src/protocol/reassembly_v5.cpp
@@ -53,18 +53,18 @@ namespace ecaludp
       const auto* header = reinterpret_cast<ecaludp::v5::Header*>(buffer->data());
 
       // Each message type must be handled differently
-      if (static_cast<ecaludp::v5::message_type_uint32t>(le32toh(static_cast<uint32_t>(header->type)))
-                 == ecaludp::v5::message_type_uint32t::msg_type_fragmented_message_info)
+      if (static_cast<ecaludp::v5::datagram_type_uint32t>(le32toh(static_cast<uint32_t>(header->type)))
+                 == ecaludp::v5::datagram_type_uint32t::datagram_type_fragmented_message_info)
       {
         return handle_datagram_fragmented_message_info(buffer, sender_endpoint, error);
       }
-      else if (static_cast<ecaludp::v5::message_type_uint32t>(le32toh(static_cast<uint32_t>(header->type)))
-                 == ecaludp::v5::message_type_uint32t::msg_type_fragment)
+      else if (static_cast<ecaludp::v5::datagram_type_uint32t>(le32toh(static_cast<uint32_t>(header->type)))
+                 == ecaludp::v5::datagram_type_uint32t::datagram_type_fragment)
       {
         return handle_datagram_fragment(buffer, sender_endpoint, error);
       }
-      else if (static_cast<ecaludp::v5::message_type_uint32t>(le32toh(static_cast<uint32_t>(header->type)))
-                 == ecaludp::v5::message_type_uint32t::msg_type_non_fragmented_message)
+      else if (static_cast<ecaludp::v5::datagram_type_uint32t>(le32toh(static_cast<uint32_t>(header->type)))
+                 == ecaludp::v5::datagram_type_uint32t::datagram_type_non_fragmented_message)
       {
         return handle_datagram_non_fragmented_message(buffer, error);
       }