Support better interop with rpm-ostree

[jlebon]

Currently, bootc cannot be used on systems managed by rpm-ostree as soon as any rpm-ostree client-side features are enabled (e.g. package layering or initramfs regeneration). This makes it harder to migrate systems from one to the other. The replacement for package layering is building container images, but that requires changing workflows and assuming new responsibilities. Also, initramfs regeneration and kernel argument manipulations still require rpm-ostree today.

But also for OS maintainers of rpm-ostree-based variants, trying to migrate users from one to the other would be much less dramatic if there were a window of time when both were supported. It would allow moving to bootc for updates, without breaking users currently using layering. Notably in upstream communities like FCOS, Silverblue, and IoT where rpm-ostree is today very popular. To a lesser degree in downstream systems like OCP, where we have more control on the implementation details and are already working on a container-native replacement to some of the features backed by rpm-ostree for now but we definitely have instances of people doing rpm-ostree-specific things behind the MCO's back today.

Implementation-wise, I think we could have a bit like

[bootc]
deployer=rpm-ostree deploy
deployer-status=rpm-ostree status --booted --json

in the origin, which would tell bootc switch/bootc upgrade to run the command in the deployer key instead of ostree_sysroot_stage_tree(), passing the container ref as an argument (or the OSTree commit so that bootc is still in charge of the downloading and importing? basically force rpm-ostree to operate in offline mode). And bootc status would add under the BootEntry object a new key like deployer containing the deployer command, and deployerStatus containing the output of the command which is expected to return JSON (but obviously displayed as YAML by bootc). The idea being that bootc status still captures the rpm-ostree-specific state here.

rpm-ostree could also learn to look for the bootc group in the origin and rpm-ostree upgrade/rpm-ostree deploy could tell the user to use bootc instead.

(Obviously this is structured in a way that doesn't actually require bootc to know anything about rpm-ostree and which could in theory be reused for other things that could be driving ostree today.)

[cgwalters]

I definitely want to improve the experience here. I would say though what I think would be even more valuable though is being able to tell the story where e.g. dnf can take over some of the role of things around stuff like package layering.

Anyways though I think the challenge is that in practice anything doing stuff like package layering is going to need to in practice operate with rpm-ostree (or in general whatever higher level build tool is driving bootc) to inspect state. Shoveling the rpm-ostree status --json into bootc status just feels weird. (But still, not opposed)

I can certainly imagine hooking things such that bootc upgrade turns around and invokes some other command...but how valuable in practice is that? I'm a bit uncertain. There is on one hand a lot of value in doing something useful with "finger compatibility" I guess.

[jlebon]

I definitely want to improve the experience here. I would say though what I think would be even more valuable though is being able to tell the story where e.g. dnf can take over some of the role of things around stuff like package layering.

👍 Indeed, I think that's where we want to go eventually.

Anyways though I think the challenge is that in practice anything doing stuff like package layering is going to need to in practice operate with rpm-ostree (or in general whatever higher level build tool is driving bootc) to inspect state. Shoveling the rpm-ostree status --json into bootc status just feels weird. (But still, not opposed)

Yeah, unsure about the JSON bit myself too. It's definitely not meant to be as user-facing as the rest of the bootc status output. But it's nice too to have it be right there. Not sure. My original idea was to just have a field like deployerStatusCommand: rpm-ostree status or something that directs people to the underlying command to use. None of this is great, it's just a temporary thing to try to make migration smoother.

I can certainly imagine hooking things such that bootc upgrade turns around and invokes some other command...but how valuable in practice is that? I'm a bit uncertain.

Today, not much. But the idea in theory is that we'd be able to enjoy the feature sets of both tools (e.g. #100).

[ericcurtin]

FWIW, I think many people would migrate to the maintain a Containerfile approach, like having a Container in some directory locally and just use bootc to build the Containerfile (and deploy the locally built image) when we want to install/remove packages etc.:

FROM quay.io/fedora/fedora-bootc:40
RUN dnf install -y list_of_packages

one can take advantage of layering when you add a new package, just add a dnf install as a separate RUN command and we can just re-use the cached layers. People can do ":latest" if they are the type that like to upgrade as soon as possible, etc.

In general dnf is a more complete rpm package managing tool.

Maybe a best-effort generator could be useful to run on an rpm-ostree system to turn it into an approximate Containerfile (and show users the new way) or documentation on how to do it manually...

People like the Containerfile format, the recent surge in popularity of Ollama is just one example of this:

https://trends.google.com/trends/explore?date=today%205-y&q=Gpt4all,ollama,llamma.cpp&hl=en-GB

I also feel this flow would make things easier for kernel developers also, kernel developers struggle with rpm-ostree.

[jlebon]

Closing in favour of coreos/rpm-ostree#4994 which proposes the solution opposite to the one proposed here, which is to have rpm-ostree call bootc.

[ericcurtin]

I got a notification on this since it was closed. On an rpm-ostree based Fedora Kinoite 40 machine, I recently upgraded via bootc using:

sudo bootc switch quay.io/fedora/fedora-kinoite:41
sudo reboot
# I temporarily lost some layered packages which I re-installed via:
sudo rpm-ostree install alacritty bootc clang cmake distrobox dnf gcc hip-devel hipblas-devel hyperfine keepassxc make nvtop qemu-kvm rocblas-devel rocm-clinfo rocm-hip rocm-hip-devel rocm-opencl rocm-smi rocminfo the_silver_searcher vim

This worked pretty flawlessly.