Core-dump when converting messags with key-ed member

[t0ny-peng]

I have a message that has a key-ed data member. Generating a Python file from it yields core dump.

CycloneDDS version: eclipse-cyclonedds/cyclonedds@b6fe21d
Build command:

cmake -DCMAKE_INSTALL_PREFIX=$(realpath ../install) -DBUILD_EXAMPLES=ON -DBUILD_IDLC=ON -DENABLE_TYPELIB=ON -DENABLE_TYPE_DISCOVERY=NO -DENABLE_TOPIC_DISCOVERY=NO -DENAB
LE_SOURCE_SPECIFIC_MULTICAST=NO -DBUILD_IDLC_XTESTS=NO ..

Python binding version: 2023.8.24 https://pypi.org/project/cyclonedds-nightly/2023.8.24/

# MsgBuffer.idl
module MyNamespace
{
  struct MsgBuffer
  {
    long id_;
    sequence<octet> data_;
    #pragma keylist MsgBuffer id_
  };
};

Command:

<path-to>/idlc \
  -l py \
  -Wno-implicit-extensibility \
  -o. \
  MsgBuffer.idl

Segmentation fault (core dumped)

[t0ny-peng]

Uh, might be the way how key-ed field is handled differently for the python binding generator? Adding @key before the field name works just fine. Still any official clarification would be much appreciated!

[eboasson]

It looks like a straightforward bug in the handling of #pragma keylist only (#219 seems to fix it), it is simply handled in a completely separate path from the @key annotation. The @key annotation was introduced with the XTypes standard, before that, there was no standardized way of defining keys. Today we support #pragma keylist for backwards compatibility more than anything.

The reason #pragma keylist is handled entirely differently from the @key is capable of expressing some (weird) configurations of key fields that the @key annotation cannot express (it is strictly more expressive), and these were supported in Cyclone before we started supporting XTypes.

Part of XTypes is type discovery and type assignability checking. For that we need to represent the types in the standard way (the noxious TypeInformation and TypeObject) and that includes the key annotations. What we decided to do is to drop support for any key specification other than what XTypes can express, and so we now check the #pragma keylist specifications for any incompatibilities. Fortunately, no-one was doing anything not supportable in XTypes!

In case you're curious: this was supported by Cyclone pre-XTypes but fundamentally incompatible with how XTypes represents keys:

struct inner { long a, b; };
struct outer { inner x, y; };
#pragma keylist outer x.a, y.b

for which IDLC now reports Conflicting keys in keylist directive for type 'inner'

[t0ny-peng]

@eboasson I see. Thanks for the explanation. Makes perfect sense. If we are about to use it in a project, is there a recommended version combination that are compatible? E.g., Core in 0.10.4, CXX in 0.10.4 and Python binding in 0.10.2? Of course I'd love to use 0.11.0 but it's not released yet🙏

[eboasson]

Yes, 0.10.4/0.10.4/0.10.2 is good. I've only recently realised the Python package should be bumped to, even if the sources have not changed: because that way the pre-built pypi packages can use the newer core library. But ... I need some time to figure out the process ...

I was considering doing an 11.0¹ alpha 1 tag soon. Once I merge eclipse-cyclonedds/cyclonedds#1676 and no-one complains about it, to be precise. I've reviewed it, played with it and fixed a few odds and ends on my own fork (still to be reviewed by the PR author). It is a big one, but I'll have to bite the bullet one day and I think it is at the stage where I don't gain anything more by waiting longer. Once there is tag/branch, a lot of things become possible, among others trying to get ROS 2 "rolling" to switch over. And then we can finally leave the 0.10.x behind. 🙂

Footnotes

1. 11.0 as in: dropping the "0." because it's been stuck at zero for five years, it's been very stable and I don't foresee it going up in the next five years either ... ↩