Handling of /etc/fstab

[mvo5]

While moving into the brave new bootc install to-filesystem world for bootc-image-builder I noticed that we need a way to write a "correct" /etc/fstab (this might actually be a larger topic but let's focus on this use-case).

Currently bib uses a fairly standard partition table [0] but the booted image generated with bootc install to-filesystem will only contain /boot in the fstab (it looks like it's using a hardcoded method here [1]?). But it seems preferable to have a complete fstab with "/", "/boot" and "/boot/efi" [2]. bib/osbuild have ways to generate the correct fstab but there is currently no way (afaict) to put it into the deployment.

Interestingly it seems even though /boot is part of fstab it's not mounted (but that might be a fedora:eln issue)

So my questions are:

1. Should we fix this on the bootc level (probably given that bootc install to-disk writes the same (incomplete) /etc/fstab ? Probably yes so that install to-disk has the right information but the "install to-filesystem" case is more tricky
2. Bib could use install: Add --copy-etc #267 and push the bib generated /etc/fstab this way. The downside here is that /etc/fstab will be untracked state. Which is bad because if the bootc image ever changes their opinion about mounts it can no longer do it via /etc/fstab (but systemd mount units are a thing).
3. This touches on a more general issue - the mess that /etc is today (in general on linux), partly admin owned state, partly state shipped by packages/bootc images. But here it seems to me like /etc/fstab should not be something the bootc owns, it should be user configuration that is injected from bib or bootc install (we won't solve the general problem short term but finding a way for /etc/fstab as a starting point and maybe even establishing a pattern would be a nice start).

I hope this makes sense, I still have huge gaps in my understanding of the details of ostree (sorry for that).

[0] https://github.com/osbuild/bootc-image-builder/blob/main/bib/cmd/bootc-image-builder/partition_tables.go#L19
[1] https://github.com/containers/bootc/blob/main/lib/src/install.rs#L600
[2] Also with the goal of bootc to be a generic tool it seems users will want to customize /etc/fstab and have their /backup mounts in there etc.

[mvo5]

One more quick note related to this and #267 (comment):

[..]

And now that we effectively have 3 sources for /etc content:

upstream OS
initial provisioning state
current state

We can diff all three of these.

I wonder if we cannot simply have a dir for bootc images under deploy that contains the initial provision state (maybe this exists already?) and then the order is simply that files in this provision "shadow" upstream OS files and that's it. This way a factory reset can go back to the original state easily and the user can check initial-state customizations equally easily (manually). At least for /etc and /var this seems just fine to not bother with diff or automatic merges as we should really work towards a a world where /etc/ is empty on an upstream OS image (obviously this will take a bit of time :)

[cgwalters]

Currently bib uses a fairly standard partition table [0] but the booted image generated with bootc install to-filesystem will only contain /boot in the fstab (it looks like it's using a hardcoded method here [1]?). But it seems preferable to have a complete fstab with "/", "/boot" and "/boot/efi" [2].

IMO the model of having / in /etc/fstab is really legacy because it's always been duplicative information with the effectively requisite root= kernel argument. The tension has been around root mount options, but even that is problematic because setting root mount options in /etc/fstab only works for options that can be changed post-mount, or it requires a "mount, parse /etc/fstab, unmount, remount" flow (AFAIK systemd and dracut never did this). There's a rootopts= kernel argument to match the root= one to avoid the circularity.

I think the classic case here is XFS and the prjquota option which needs to be set at mount time...in coreos we basically bypassed this with https://github.com/coreos/fedora-coreos-config/blob/e51777611b7963f76e490e682ff8d8ea4b6ef9d1/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-rootflags.sh#L29 which is ugly.

as we should really work towards a a world where /etc/ is empty on an upstream OS image (obviously this will take a bit of time :)

Yeah, that's been a goal for many years; there's still some stragglers in popular software that make it problematic for general purpose systems.

Anyways, so besides the root mount, as you noted bootc today writes an entry for /boot itself (that said, I think it'd be better to support this being owned by the OS in a nicer way, like having bootc customize a systemd boot.mount unit perhaps). Or maybe even encourage the OS content to have a mechanism to auto-discover it.

(This gets into the discoverable partition spec, but last I looked that worked well in virt setups but causes problems on bare metal where one might have legacy/unused OS installs that happen to be on other block devices).

For /boot/efi - ultimately I think the ESP should be owned by a dedicated tool (e.g. bootupd as it is for bootc), and bootupd already has logic to mount it explicitly by type/label discovery. It's not necessary to unconditionally auto-mount it.

Also, I think in general we need to support arbitrary machine-local mutable state; today the way Anaconda installs work for example it just writes stuff into /etc; and nothing stops one from doing something similar "manually" after a bootc install to-filesystem too. #267 is just formalizing this - and effectively encouraging it more. But again I don't see a world where we would require it.

Ultimately I think the most practical thing for those who want to avoid unmanaged state is going as far as possible towards a "transient /etc" flow, and baking bootstrap configuration into the OS image.

---

Backing up to a high level, for just having bib use bootc install to-filesystem (i.e. closing the basics of this issue) - per above are we really blocked?

There's certainly some things to flesh out of course, like whether tools like anaconda/bib should append to /etc/fstab for the root (and hmm, this issue strongly relates to ostreedev/ostree#3193 ) or whether it should use rootflags= karg. I personally prefer the latter because it breaks the circularity - and in practice it's not that common to change the root mount options post-install, and so handling that unusual case via kernel arguments doesn't lose out too much on ergonomics.

[mvo5]

Thanks for your excellent feedback. I agree, let's not block on this, I will move forward.

[cgwalters]

There's a bunch of stuff in this issue...let's try to split it into more actionable dedicated issues. I made one just for /boot.

[cgwalters]

While there's good discussion in this issue it isn't actionable in itself, we have a tracker for the /boot one.