diff --git a/README.md b/README.md index 529b482..b3270af 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,19 @@ # ecaludp -ecaludp is the underlying implementation for UDP traffic in eCAL. It transparently fragments and reassembles messages to provide support for big messages. +ecaludp is the underlying implementation for UDP traffic in eCAL. It transparently **fragments and reassembles** messages to provide support for big messages. An ecaludp socket is **not limited** to the ordinary UDP datagram size of ~64KiB. It can transport messages **up to 4 GiB**. + +ecaludp has **npcap support** for efficient receiving of multicast traffic in Windows. For that, the [udpcap](https://github.com/eclipse-ecal/udpcap) library is used. + +ecaludp requires C++14. + +## Sample Projects + +ecalupd features an asio-style API. Check out the following samples to see its usage: + +- [ecaludp_sample](samples/ecaludp_sample/src/main.cpp): The regular ecaludp socket API + +- [ecaludp_npcap_sample](samples/ecaludp_sample_npcap/src/main.cpp) The (slightly different) npcap socket API. Only available for Windows. ## Dependencies @@ -29,6 +41,31 @@ When building the **tests**, the following dependency is required: |----------------|-------------|-------------------------| | [Googletest](https://github.com/google/googletest) | [BSD-3](https://github.com/google/googletest/blob/main/LICENSE) | [git submodule](https://github.com/eclipse-ecal/ecaludp/tree/master/thirdparty) | +## How to checkout and build + +1. Install cmake and git / git-for-windows + +2. Checkout this repo and the asio submodule + ```console + git clone https://github.com/eclipse-ecal/ecaludp.git + cd ecaludp + git submodule init + git submodule update + ``` + +3. CMake the project _(check the next section for available CMake options)_ + ```console + mkdir _build + cd _build + cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=_install + ``` + +4. Build the project + - Linux: `make` + - Windows: Open `_build\ecaludp.sln` with Visual Studio and build the example project + +5. Check the functionality with the `udpcap_sample /.exe` sample project! + ## CMake Options You can set the following CMake Options to control how ecaludp is built: @@ -42,4 +79,84 @@ You can set the following CMake Options to control how ecaludp is built: | `ECALUDP_USE_BUILTIN_RECYCLE`| `BOOL`| `ON` | Use the builtin steinwurf::recycle submodule. If set to `OFF`, recycle must be available from somewhere else (e.g. system libs). | | `ECALUDP_USE_BUILTIN_UDPCAP`| `BOOL`| `ON`
_(when building with npcap)_ | Use the builtin udpcap submodule. Only needed if `ECALUDP_ENABLE_NPCAP` is `ON`. If set to `OFF`, udpcap must be available from somewhere else (e.g. system libs). Setting this option to `ON` will also use the default dependencies of udpcap (npcap-sdk, pcapplusplus). | | `ECALUDP_USE_BUILTIN_GTEST`| `BOOL`| `ON`
_(when building tests)_ | Use the builtin GoogleTest submodule. Only needed if `FINEFTP_SERVER_BUILD_TESTS` is `ON`. If set to `OFF`, GoogleTest must be available from somewhere else (e.g. system libs). | -| `ECALUDP_LIBRARY_TYPE` | `STRING` | | Controls the library type of Ecaludp by injecting the string into the `add_library` call. Can be set to STATIC / SHARED / OBJECT. If set, this will override the regular `BUILD_SHARED_LIBS` CMake option. If not set, CMake will use the default setting, which is controlled by `BUILD_SHARED_LIBS`. | \ No newline at end of file +| `ECALUDP_LIBRARY_TYPE` | `STRING` | | Controls the library type of Ecaludp by injecting the string into the `add_library` call. Can be set to STATIC / SHARED / OBJECT. If set, this will override the regular `BUILD_SHARED_LIBS` CMake option. If not set, CMake will use the default setting, which is controlled by `BUILD_SHARED_LIBS`. | + +## Protocol Specification (Version 5) + +An ecaludp message consists of one or multiple datagrams. How many datagrams that will be is determined by the fragmentation. + +Each datagram carries a header starting at byte 0. The header is defined in [header_v5.h](ecaludp/src/protocol/header_v5.h). Alien datagrams can be eliminated by comparing the `magic` bytes with a predefined value. +Some datagrams may also carry payload directly after the header. + +| size | Name | Explanation | +|--------|------------|---------------------------------------------------------| +| 32 bit | `magic` | User-defined binary data. Used for identifying and dropping alien traffic. | +| 8 bit | `version` | Header version. Must be `5` for protocol version 5 | +| 24 bit | _reserved_ | Must be sent as 0. | +| 32 bit
little-endian | `type` | Datagram type. Must be one of:
`1`: fragmented_message_info
`2`: fragment
`3`: non_fragmented_message | +| 32 bit
signed little-endian | `id` | Random ID to match fragmented parts of a message. Used differently depending on the message type:
- `type == fragmented_message_info (1)`: The Random ID that this fragmentation info will be applied to
- `type == fragment (2)`: The Random ID that this fragment belongs to. Used to match fragments to their fragmentation info
- `type == non_fragmented_message (3)`: Unused field. Must be sent as -1. Must not be evaluated.| +| 32 bit
unsigned little-endian | `num` | Fragment number. Used differently depending on the datagram type:
- `type == fragmented_message_info (1)`: Amount of fragments that this message was split into.
- `type == fragment (2)`: The number of this fragment
- `type == non_fragmented_message (3)`: Unused field. Must be sent as `1`. Must not be evaluated. | +| 32 bit
unsigned little-endian | `len` | Payload length. Used differently depending on the datagram type. The payload must start directly after the header.
- `type == fragmented_message_info (1)`: Length of the original message before it got fragmented. Messages of this type must not carry any payload themselves.
- `type == fragment (2)`: The payload lenght of this fragment
- `type == non_fragmented_message (3)`: The payload length of this message | +| `len` bytes | _payload_ | Payload of the message or fragment. + +### Message Types + +There are two different types of messages that can be sent: Messages that are fragmented and messages that are not fragmented. + +1. **Non-fragmented data** + - The entire message **consists of 1 datagram** carrying both a header and the payload. + - The header looks as follows: + - `type` is set to `non_fragmented_message (3)` + - `id` is `-1` + - `num` is `1` + - `length` is the amount of payload bytes following after the header + +2. **Fragmented data** + + A message which had to be fragmented into $n \in \mathbb{N}_0$ parts **consists of $n+1$ datagrams**: + + - **$1\times$ Fragmentation info** + - The first datagram carries the fragmentation info. + - The header looks as follows: + - `type` is set to `fragmented_message_info (1)` + - `id` is a random number. It is used to match the fragments to their fragmentation info and therefore must be unique for each fragmented message. + - `num` is the amount of fragments that the message was split into (i.e. $n$) + - `length` is the length of the original message before it got fragmented + - This datagram must not carry any payload. + + - **$n\times$ Fragments** + - The following $n$ datagrams carry the fragments. + - The header looks as follows: + - `type` is set to `fragment (2)` + - `id` is the random number that was used in the fragmentation info + - `num` is the number of this fragment (i.e. $1$ to $n$) + - `length` is the length of the payload of this fragment. + - The payload of each fragment is a part of the original message. + +### Communication diagram + +The following diagram shows the communication between the sender and the receiver. The sender sends a non-fragmented message and a fragmented message. The fragmented message consists of n fragments. + +``` + Sender Receiver + | | + | Non-fragmented message | + |----------------------------------->| + | | + | Fragmentation info (n fragments) | + |----------------------------------->| + | | + | Fragment 1 | + |----------------------------------->| + | | + | Fragment 2 | + |----------------------------------->| + | | + | ... | + |----------------------------------->| + | | + | Fragment (n) | + |----------------------------------->| + | | + | | +``` \ No newline at end of file diff --git a/ecaludp/src/protocol/datagram_builder_v5.cpp b/ecaludp/src/protocol/datagram_builder_v5.cpp index 45c5c47..daf3ae8 100644 --- a/ecaludp/src/protocol/datagram_builder_v5.cpp +++ b/ecaludp/src/protocol/datagram_builder_v5.cpp @@ -102,8 +102,8 @@ namespace ecaludp header_ptr->version = 5; - header_ptr->type = static_cast( - htole32(static_cast(ecaludp::v5::message_type_uint32t::msg_type_non_fragmented_message))); + header_ptr->type = static_cast( + htole32(static_cast(ecaludp::v5::datagram_type_uint32t::datagram_type_non_fragmented_message))); header_ptr->id = htole32(int32_t(-1)); // -1 => not fragmented header_ptr->num = htole32(uint32_t(1)); // 1 => only 1 fragment header_ptr->len = htole32(static_cast(total_size)); // denotes the length of the payload of this message only @@ -168,8 +168,8 @@ namespace ecaludp fragment_info_header_ptr->version = 5; - fragment_info_header_ptr->type = static_cast( - htole32(static_cast(ecaludp::v5::message_type_uint32t::msg_type_fragmented_message_info))); + fragment_info_header_ptr->type = static_cast( + htole32(static_cast(ecaludp::v5::datagram_type_uint32t::datagram_type_fragmented_message_info))); fragment_info_header_ptr->id = htole32(message_id); fragment_info_header_ptr->num = htole32(needed_fragment_count); fragment_info_header_ptr->len = htole32(total_size); // denotes the length of the entire payload @@ -202,8 +202,8 @@ namespace ecaludp header_ptr->version = 5; - header_ptr->type = static_cast( - htole32(static_cast(ecaludp::v5::message_type_uint32t::msg_type_fragment))); + header_ptr->type = static_cast( + htole32(static_cast(ecaludp::v5::datagram_type_uint32t::datagram_type_fragment))); header_ptr->id = htole32(message_id); header_ptr->num = htole32(static_cast(datagram_list.size() - 2)); // -1, because the first datagram is the fragmentation info header_ptr->len = htole32(static_cast(0)); // denotes the length of the entire payload diff --git a/ecaludp/src/protocol/header_v5.h b/ecaludp/src/protocol/header_v5.h index 957cfa0..99395d5 100644 --- a/ecaludp/src/protocol/header_v5.h +++ b/ecaludp/src/protocol/header_v5.h @@ -21,41 +21,41 @@ namespace ecaludp { namespace v5 { - enum class message_type_uint32t : uint32_t + enum class datagram_type_uint32t : uint32_t { - msg_type_unknown = 0, - msg_type_fragmented_message_info = 1, // former name: msg_type_header - msg_type_fragment = 2, // former name: msg_type_content - msg_type_non_fragmented_message = 3 // former name: msg_type_header_with_content + datagram_type_unknown = 0, + datagram_type_fragmented_message_info = 1, // former name: msg_type_header + datagram_type_fragment = 2, // former name: msg_type_content + datagram_type_non_fragmented_message = 3 // former name: msg_type_header_with_content }; #pragma pack(push, 1) struct Header { - char magic[4]; - - uint8_t version; /// Header version. Must be 5 for this version 5 header - uint8_t reserved1; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future. - uint8_t reserved2; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future. - uint8_t reserved3; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future. - - message_type_uint32t type; /// The message type. See message_type_uint32t for possible values - - int32_t id; /// Random ID to match fragmented parts of a message (Little-endian). Used differently depending on the message type: - /// - msg_type_fragmented_message_info: The Random ID that this fragmentation info will be applied to - /// - msg_type_fragment: The Random ID that this fragment belongs to. Used to match fragments to their fragmentation info - /// - msg_type_non_fragmented_message: Unused field. Must be sent as -1. Must not be evaluated. - - uint32_t num; /// Fragment number (Little-endian). Used differently depending on the message type: - /// - msg_type_fragmented_message_info: Amount of fragments that this message was split into. - /// - msg_type_fragment: The number of this fragment - /// - msg_type_non_fragmented_message: Unused field. Must be sent as 1. Must not be evaluated. - - uint32_t len; /// Payload length (Little-endian). Used differently depending on the message type. The payload must start directly after the header. - /// - msg_type_fragmented_message_info: Length of the original message before it got fragmented. - /// Messages of this type must not carry any payload themselves. - /// - msg_type_fragment: The payload lenght of this fragment - /// - msg_type_non_fragmented_message: The payload length of this message + char magic[4]; + + uint8_t version; /// Header version. Must be 5 for this version 5 header + uint8_t reserved1; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future. + uint8_t reserved2; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future. + uint8_t reserved3; /// Must be sent as 0. The old implementation used this byte as 4-byte version (little endian), but never checked it. Thus, it may be used in the future. + + datagram_type_uint32t type; /// The datagram type. See datagram_type_uint32t for possible values + + int32_t id; /// Random ID to match fragmented parts of a message (Little-endian). Used differently depending on the datagram type: + /// - datagram_type_fragmented_message_info: The Random ID that this fragmentation info will be applied to + /// - datagram_type_fragment: The Random ID that this fragment belongs to. Used to match fragments to their fragmentation info + /// - datagram_type_non_fragmented_message: Unused field. Must be sent as -1. Must not be evaluated. + + uint32_t num; /// Fragment number (Little-endian). Used differently depending on the datagram type: + /// - datagram_type_fragmented_message_info: Amount of fragments that this message was split into. + /// - datagram_type_fragment: The number of this fragment + /// - datagram_type_non_fragmented_message: Unused field. Must be sent as 1. Must not be evaluated. + + uint32_t len; /// Payload length (Little-endian). Used differently depending on the datagram type. The payload must start directly after the header. + /// - datagram_type_fragmented_message_info: Length of the original message before it got fragmented. + /// Messages of this type must not carry any payload themselves. + /// - datagram_type_fragment: The payload lenght of this fragment + /// - datagram_type_non_fragmented_message: The payload length of this message }; #pragma pack(pop) } diff --git a/ecaludp/src/protocol/reassembly_v5.cpp b/ecaludp/src/protocol/reassembly_v5.cpp index ef43420..7673b13 100644 --- a/ecaludp/src/protocol/reassembly_v5.cpp +++ b/ecaludp/src/protocol/reassembly_v5.cpp @@ -53,18 +53,18 @@ namespace ecaludp const auto* header = reinterpret_cast(buffer->data()); // Each message type must be handled differently - if (static_cast(le32toh(static_cast(header->type))) - == ecaludp::v5::message_type_uint32t::msg_type_fragmented_message_info) + if (static_cast(le32toh(static_cast(header->type))) + == ecaludp::v5::datagram_type_uint32t::datagram_type_fragmented_message_info) { return handle_datagram_fragmented_message_info(buffer, sender_endpoint, error); } - else if (static_cast(le32toh(static_cast(header->type))) - == ecaludp::v5::message_type_uint32t::msg_type_fragment) + else if (static_cast(le32toh(static_cast(header->type))) + == ecaludp::v5::datagram_type_uint32t::datagram_type_fragment) { return handle_datagram_fragment(buffer, sender_endpoint, error); } - else if (static_cast(le32toh(static_cast(header->type))) - == ecaludp::v5::message_type_uint32t::msg_type_non_fragmented_message) + else if (static_cast(le32toh(static_cast(header->type))) + == ecaludp::v5::datagram_type_uint32t::datagram_type_non_fragmented_message) { return handle_datagram_non_fragmented_message(buffer, error); } diff --git a/tests/ecaludp_private_test/src/fragmentation_v5_test.cpp b/tests/ecaludp_private_test/src/fragmentation_v5_test.cpp index 58aedee..0a22a78 100644 --- a/tests/ecaludp_private_test/src/fragmentation_v5_test.cpp +++ b/tests/ecaludp_private_test/src/fragmentation_v5_test.cpp @@ -75,7 +75,7 @@ TEST(FragmentationV5Test, NonFragmentedMessage) // Check the header auto* header = reinterpret_cast(binary_buffer->data()); ASSERT_EQ(header->version, 5); - ASSERT_EQ(le32toh(static_cast(header->type)), 3u /* = ecaludp::v5::message_type_uint32t::msg_type_non_fragmented_message */); + ASSERT_EQ(le32toh(static_cast(header->type)), 3u /* = ecaludp::v5::datagram_type_uint32t::datagram_type_non_fragmented_message */); ASSERT_EQ(le32toh(header->id), -1); ASSERT_EQ(le32toh(header->num), 1); ASSERT_EQ(le32toh(header->len), hello_world.size()); @@ -135,7 +135,7 @@ TEST(FragmentationV5Test, FragmentedMessage) auto* header_1 = reinterpret_cast(binary_buffer_1->data()); auto common_id = le32toh(header_1->id); ASSERT_EQ(header_1->version, 5); - ASSERT_EQ(le32toh(static_cast(header_1->type)), 1u /* = ecaludp::v5::message_type_uint32t::msg_type_fragment_info */); + ASSERT_EQ(le32toh(static_cast(header_1->type)), 1u /* = ecaludp::v5::datagram_type_uint32t::msg_type_fragment_info */); ASSERT_EQ(le32toh(header_1->num), 2); ASSERT_EQ(le32toh(header_1->id), common_id); ASSERT_EQ(le32toh(header_1->len), message_size); @@ -143,7 +143,7 @@ TEST(FragmentationV5Test, FragmentedMessage) // Check the header of the first fragment auto* header_2 = reinterpret_cast(binary_buffer_2->data()); ASSERT_EQ(header_2->version, 5); - ASSERT_EQ(le32toh(static_cast(header_2->type)), 2u /* = ecaludp::v5::message_type_uint32t::msg_type_fragment */); + ASSERT_EQ(le32toh(static_cast(header_2->type)), 2u /* = ecaludp::v5::datagram_type_uint32t::datagram_type_fragment */); ASSERT_EQ(le32toh(header_2->id), common_id); ASSERT_EQ(le32toh(header_2->num), 0); ASSERT_EQ(le32toh(header_2->len), 100 - sizeof(ecaludp::v5::Header)); @@ -151,7 +151,7 @@ TEST(FragmentationV5Test, FragmentedMessage) // Check the header of the last fragment auto* header_3 = reinterpret_cast(binary_buffer_3->data()); ASSERT_EQ(header_3->version, 5); - ASSERT_EQ(le32toh(static_cast(header_3->type)), 2u /* = ecaludp::v5::message_type_uint32t::msg_type_fragment */); + ASSERT_EQ(le32toh(static_cast(header_3->type)), 2u /* = ecaludp::v5::datagram_type_uint32t::datagram_type_fragment */); ASSERT_EQ(le32toh(header_3->id), common_id); ASSERT_EQ(le32toh(header_3->num), 1); ASSERT_EQ(le32toh(header_3->len), message_size - (100 - sizeof(ecaludp::v5::Header))); @@ -303,7 +303,7 @@ TEST(FragmentationV5Test, SingleFragmentFragmentation) auto* header_1 = reinterpret_cast(binary_buffer_1->data()); auto common_id = le32toh(header_1->id); ASSERT_EQ(header_1->version, 5); - ASSERT_EQ(le32toh(static_cast(header_1->type)), 1u /* = ecaludp::v5::message_type_uint32t::msg_type_fragment_info */); + ASSERT_EQ(le32toh(static_cast(header_1->type)), 1u /* = ecaludp::v5::datagram_type_uint32t::msg_type_fragment_info */); ASSERT_EQ(le32toh(header_1->num), 1); ASSERT_EQ(le32toh(header_1->id), common_id); ASSERT_EQ(le32toh(header_1->len), hello_world.size()); @@ -311,7 +311,7 @@ TEST(FragmentationV5Test, SingleFragmentFragmentation) // Check the header of the first fragment auto* header_2 = reinterpret_cast(binary_buffer_2->data()); ASSERT_EQ(header_2->version, 5); - ASSERT_EQ(le32toh(static_cast(header_2->type)), 2u /* = ecaludp::v5::message_type_uint32t::msg_type_fragment */); + ASSERT_EQ(le32toh(static_cast(header_2->type)), 2u /* = ecaludp::v5::datagram_type_uint32t::datagram_type_fragment */); ASSERT_EQ(le32toh(header_2->id), common_id); ASSERT_EQ(le32toh(header_2->num), 0); ASSERT_EQ(le32toh(header_2->len), hello_world.size()); @@ -377,7 +377,7 @@ TEST(FragmentationV5Test, ZeroByteMessage) // Check the header auto* header = reinterpret_cast(binary_buffer->data()); ASSERT_EQ(header->version, 5); - ASSERT_EQ(le32toh(static_cast(header->type)), 3u /* = ecaludp::v5::message_type_uint32t::msg_type_non_fragmented_message */); + ASSERT_EQ(le32toh(static_cast(header->type)), 3u /* = ecaludp::v5::datagram_type_uint32t::datagram_type_non_fragmented_message */); ASSERT_EQ(le32toh(header->id), -1); ASSERT_EQ(le32toh(header->num), 1); ASSERT_EQ(le32toh(header->len), 0); @@ -749,7 +749,7 @@ TEST(FragmentationV5Test, FaultyFragmentedMessages) { auto faulty_binary_buffer_2 = std::make_shared(*binary_buffer_2); auto* header = reinterpret_cast(faulty_binary_buffer_2->data()); - header->type = static_cast(1000); + header->type = static_cast(1000); ecaludp::Error error = ecaludp::Error::ErrorCode::GENERIC_ERROR; auto message = reassembly.handle_datagram(faulty_binary_buffer_2, sender_endpoint, error);