Packaging for foreign architectures

[LucienMorey]

Hello!

Is there a way to use this tool to package something for a CPU architecture different from the host system?

I have a RiscV board with somewhat less grunt than a Raspberry Pi. As no RiscV ROS debs are publicly available, I have been cross-compiling the core of ROS2 along with any packages I develop and then copying them to the machine. This has significantly reduced build times compared to compiling on the board natively or emulating the architecture to do a "native" build.

The next workflow improvement I am interested in is packaging debs to limit the number of source builds for an even faster turnaround to testing. I have done some playing around, and there might be a way to do so if I modify the deb generator template to look for the foreign rootfs when bloom does its internal rebuild for packaging.

Either way, I thought it would be good to ask if there was ever any discussion around this point and there is another way to do things that I haven't seen yet.

[cottsay]

I don't believe there's anything in Bloom that is architecture-specific. The deb and RPM metadata generated by this tool should be usable on any architecture supported by the downstream build process.

For that downstream build process, given that Ubuntu distributes debs for riscv64, it should be possible to build ROS deb packages using the existing metadata.

I've been working on a new tool called colcon-ros-buildfarm that would allow you to reproduce the official ROS buildfarm process on a local host. It should be possible to fork the official ros_buildfarm_config repository for the ROS 2 buildfarm and add a job targeting riscv64. I don't have access to any RiscV systems to try this on to verify, but assuming docker or podman work on your system, it SHOULD work.

As far as officially producing debs for riscv64 for inclusion in ROS repositories, it's really a conversation about policy, resource allocation, and ability to execute riscv64 workloads.

[LucienMorey]

I don't believe there's anything in Bloom that is architecture-specific. The deb and RPM metadata generated by this tool should be usable on any architecture supported by the downstream build process.

Yep. I agree. The actual colcon build process happening under the hood determines the package architecture. Are you aware of a mechanism to inject cmake flags easily through bloom? If I can do that, then I will have full control of what is being built.

I've been working on a new tool called colcon-ros-buildfarm that would allow you to reproduce the official ROS buildfarm process on a local host. It should be possible to fork the official ros_buildfarm_config repository for the ROS 2 buildfarm and add a job targeting riscv64. I don't have access to any RiscV systems to try this on to verify, but assuming docker or podman work on your system, it SHOULD work.

Nice. I will take a look.

As far as officially producing debs for riscv64 for inclusion in ROS repositories, it's really a conversation about policy, resource allocation, and ability to execute riscv64 workloads.

Totally fair. Part of me is surprised that cross-compilation isn't a more prominent workflow in the ecosystem. It would limit the type of box required to build things to just x86, alleviating any pain in scaling up to support other architecture builds in the future. But also, the number of users that aren't just on arm or x86 is probably so small that it doesn't warrant the effort when there are other fish to fry.

[cottsay]

The actual colcon build process happening under the hood determines the package architecture.

This is a nuanced point, but beyond the colcon-ros-buildfarm extension, colcon is actually not involved in building system packages at all. We invoke CMake and/or setuptools directly as the system would. We use colcon to build packages from source on a developer workstation, but colcon would provide no benefit for individually building a single package like we do in our debian and RPM system package jobs.

a mechanism to inject cmake flags easily through bloom?

When bloom generates the metadata, it is possible to inject patches into the process. Keep in mind, however, that this is done by the package maintainers when they generate the metadata. Setting up an environment where you can modify an existing package's metadata is exceedingly difficult to do, and essentially involves forking the entire ROS distribution.