Cross-compilation issue

[jdiez17]

Hi, thank you for making r2r, it's a joy to use.

I'm trying to cross-compile a ROS2 workspace including r2r nodes.
Cross-compiling ROS2 at all is already complicated (we just got the ros/meta-ros OE layer cross-compilation SDK to work) - let's add Rust to the mix...

I have some interface packages (action/msg) that I'd like to use in my r2r nodes. I've already cross-compiled them and sourced the install env.
I get the following (abbreviated) error:

 Compiling r2r v0.9.2
     Running `rustc --crate-name build_script_build --edition=2021 /opt/raccoon/0.4/sysroots/cortexa53-crypto-poky-linux/home/cargo/registry/src/index.crates.io-6f17d22bba15001f/r2r-0.9.2/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=251 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=4c0f760af2461753 -C extra-filename=-4c0f760af2461753 --out-dir /home/rosdev/ros2_ws/target/debug/build/ [COMMAND ABBREVIATED]`
error: linking with `x86_64-pokysdk-linux-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/lib/rustlib/x86_64-pokysdk-linux-gnu/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/sbin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/sbin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin/../x86_64-pokysdk-linux/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux-musl:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/sbin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/sbin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin/../x86_64-pokysdk-linux/bin:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux:/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux-musl:/vscode/vscode-server/bin/linux-x64/65edc4939843c90c34d61f4ce11704f09d3e5cb6/bin/remote-cli:/home/rosdev/.local/bin:/home/rosdev/.venv/bin:/opt/ros/humble/bin:/home/rosdev/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "x86_64-pokysdk-linux-gcc" "/tmp/rustcUHzfaB/symbols.o" "/home/rosdev/ros2_ws/target/debug/build/r2r-4c0f760af2461753/build_script_build-4c0f760af2461753.build_script_build.3b62d96f2f76d02a-cgu.0.rcgu.o" "/home/rosdev/ros2_ws/target/debug/build/r2r-4c0f760af2461753/build_script_build-4c0f760af2461753.build_script_build.3b62d96f2f76d02a-cgu.1.rcgu.o" "[COMMAND ABBREVIATED] laction_msgs__rosidl_typesupport_introspection_c" "-laction_msgs__rosidl_generator_c" "-lunique_identifier_msgs__rosidl_typesupport_c" "-lunique_identifier_msgs__rosidl_typesupport_introspection_c" "-lunique_identifier_msgs__rosidl_generator_c" "-lstd_msgs__rosidl_typesupport_c" "-lstd_msgs__rosidl_typesupport_introspection_c" "-lstd_msgs__rosidl_generator_c" "-lrcl_interfaces__rosidl_typesupport_c" "-lrcl_interfaces__rosidl_typesupport_introspection_c" "-lrcl_interfaces__rosidl_generator_c" "-lbuiltin_interfaces__rosidl_typesupport_c" "-lbuiltin_interfaces__rosidl_typesupport_introspection_c" "-lbuiltin_interfaces__rosidl_generator_c" "-lrcl" "-lrcl_logging_spdlog" "-lrcl_yaml_param_parser" "-lrcutils" "-lrmw" "-lrmw_implementation" "-lrosidl_typesupport_c" "-lrosidl_runtime_c" "-ldl" "-lc" "-lm" "-lrt" "-lpthread" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/lib/rustlib/x86_64-pokysdk-linux-gnu/lib" "-o" "/home/rosdev/ros2_ws/target/debug/build/r2r-4c0f760af2461753/build_script_build-4c0f760af2461753" "-Wl,--gc-sections" "-pie" "-nodefaultlibs"
  = note: /opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/lib/gcc/x86_64-pokysdk-linux/13.3.0/../../../../x86_64-pokysdk-linux/bin/ld: skipping incompatible /opt/raccoon/0.4/sysroots/cortexa53-crypto-poky-linux/usr/opt/ros/humble/lib/libaction_msgs__rosidl_typesupport_c.so when searching for -laction_msgs__rosidl_typesupport_c
          /opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/lib/gcc/x86_64-pokysdk-linux/13.3.0/../../../../x86_64-pokysdk-linux/bin/ld: skipping incompatible /opt/raccoon/0.4/sysroots/cortexa53-crypto-poky-linux/usr/opt/ros/humble/lib/libaction_msgs__rosidl_typesupport_c.so when searching for -laction_msgs__rosidl_typesupport_c
          /opt/raccoon/0.4/sysroots/x86_64-pokysdk-linux/usr/lib/gcc/x86_64-pokysdk-linux/13.3.0/../../../../x86_64-pokysdk-linux/bin/ld: cannot find -laction_msgs__rosidl_typesupport_c: No such file or directory
          [... many more "skipping incompatible X.so" errors ...]

What seems to be happening is that Rust is compiling r2r/build.rs for the dev machine's architecture (understandable), but then it tries to link all of the interface library dependencies to the build script binary.
This is a problem, because the libraries it's trying to link to are only available as arm64 libraries.

I'm not sure why the build script should be linked with those dependency libraries. Maybe you have an idea?

[jdiez17]

I looked a bit more into how r2r handles message generation and found that it differs from the ros2_rust approach. You use the rosidl_introspection_* functions to generate Rust wrappers, while ros2_rust does it by looking at the IDL/header files.

In the end I solved my cross-compilation issue by compiling the interface libraries for both the build host and target architectures.

[m-dahl]

Hi,

Sorry I didn't manage to answer until now. Yes I guess this is a downside of using the introspection function for generating the wrappers, that you need to have them build also on the host. It would be elegant if this was not necessary but I am not sure if its technically possible the way it is set up now.

[jdiez17]

No worries. I wonder if it would be possible to use bindgen to generate wrappers for the interface libraries generated by rosidl. I won’t look at this short term since we have a working solution by compiling the libs for the host before cross compiling the nodes. But I wanted to ask if you see any downsides or obstacles to changing r2r to use this approach.

Thanks again for this lovely crate!

[m-dahl]

Do you mean by relying on what bindgen spits out to guess the types etc? Maybe possible, we do something like that for constants defined in the messages. But it has some limitations, see #47. I suspect one would have to parse the idl files in the end to get the information we need (which we can do of course, and would improve some things like the constants -- but I have no intention of doing it at the moment).