systemd-boot: New /boot data format

[ElvishJerricco]

The systemd-boot-builder.py script stores information about the profile, generation, and specialisation in the file name of loader entry .conf files. This has led to some issues where these names are improperly generated or parsed, e.g. the recent revert of boot counting: #334526.

I suggest a new format for these entry files. There should be a JSON file of a format something like this:

{
  "$profile": {
    "$generation": {
      "entry.conf": "$path_to_entry.conf",
      "files": [
        "$path_to_kernel"
        "$path_to_initrd"
        ...
      ]
    }
  }
}

This will make it easier to identify the locations of entry files as well as kernels, initrds, etc. to garbage collect.

When boot counting is reintroduced, the entry file name will refer to the name if it did not have any boot count included.

The entry name becomes arbitrary, giving us slightly better control over sorting.

This JSON file could have an ever-increasing number in its name, so that updates can be atomic without breaking the ability to GC the old one. When updating, make the new JSON file with the higher number, add the new files to the ESP, GC the old files from the ESP, and finally remove the old JSON file with the lower number. This way all the files that could need GC'ing will always have their corresponding JSON file present.

This is just a rough idea. And migration is going to be the tricky part. @alois31 recommended taking inspiration from when lanzaboote had to do something similar:

• The migration: Stop most overwriting in the ESP nix-community/lanzaboote#204
• Rewriting thin UKI stubs: UKI Compatibility for systemd 255 nix-community/lanzaboote#302

/cc @JulienMalka @Mic92 @r-vdp @NixOS/systemd

[Atemu]

Don't we already have the bootspec which fulfils a similar function?

[arianvp]

It sounds to me we should (finally) rewrite systemd-boot-builder to properly use bootspec instead. I think there is some prior work for this in the lanzaboote repo if memory serves me well

[JulienMalka]

systemd-boot-builder does use bootspec. I think the difference here is that essentially the bootspec is stateless whereas if I understand correctly this additionnal data would be stateful,

[ElvishJerricco]

@Atemu, @arianvp Yes, the idea here is not to encode the same information as bootspec. It's instead to manage the stateful aspects of where and how we store boot entries in the ESP.