Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with packing a map larger than 15 elements. #1073

Open
lunarvi opened this issue Jun 4, 2023 · 0 comments
Open

Issue with packing a map larger than 15 elements. #1073

lunarvi opened this issue Jun 4, 2023 · 0 comments

Comments

@lunarvi
Copy link

lunarvi commented Jun 4, 2023

Hey!

I'm trying to pack a map into a msgpack object and serialize it.

This is the code I'm using:

msgpack::type::variant ObjectNode::toMsgPack() const {
    std::map<msgpack::type::variant, msgpack::type::variant> packed_map;
    for (const auto &pair: children) {
        packed_map.insert({std::move(msgpack::type::variant(pair.first)), std::move(pair.second->toMsgPack())});
    }
    return msgpack::type::variant(std::move(packed_map));
}

At smaller map sizes, everything seems to work fine. However, I've noticed that the moment I try to pack a map larger than 15 elements -> which changes the internal type from fixmap to map16 if I understand correctly -> the output is incorrect.

I tried dissecting the output in hex:

| Fixmap(1el) | 3ch string | "OBJ"  | 16 el fixmap | 0->1, 1->1...                                                                              |
|-------------|------------|--------|--------------|--------------------------------------------------------------------------------------------|
| 81          | a3         | 4f424a | 8f           | a13001a13101a13201a13301a13401a13501a13601a13701a13801a13901a14101a14201a14301a14401a14501 |

(the correct output from a 15 elements long map)

| Fixmap(1el) | 3ch string | "OBJ"  | map16 | map length | 0->1, 1->1...                                                                                    |
|-------------|------------|--------|-------|------------|--------------------------------------------------------------------------------------------------|
| 81          | a3         | 4f424a | de    | 2010       | a13001a13101a13201a13301a13401a13501a13601a13701a13801a13901a14101a14201a14301a14401a14501a14601 |

Output after adding a new key->value pair. (incorrect)

| Fixmap(1el) | 3ch string | "OBJ"  | map16 | map length | 0->1, 1->1...                                                                                    |
|-------------|------------|--------|-------|------------|--------------------------------------------------------------------------------------------------|
| 81          | a3         | 4f424a | de    | 0010       | a13001a13101a13201a13301a13401a13501a13601a13701a13801a13901a14101a14201a14301a14401a14501a14601 |

^The output when I tried using the json->msgpack converter from msgpack's site. (correct)

As you can see, it appears that msgpack::pack() in my case mishandles the length of the map, putting it at 8208(2^13+16?) instead of 16.
I've spent a good while thinking about it & trying to look through the docs, but I've no idea what could cause this.

To be clear, mpgpack-c itself still seems to be able to unpack the map just fine, even if it doesn't seem to conform to the format spec. However I'm slightly worried about whether this would be transferable between various environments, hence - if possible - I'd like to address it.

Would anyone be able to point me in the right direction?
I'd be extremely grateful^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant