Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Advanced partitioning customizations (COMPOSER-2399) #1041

Merged
merged 13 commits into from
Nov 20, 2024

Commits on Nov 14, 2024

  1. blueprint: unify handling of size customization parsing

    Size values in the blueprint can be defined as either integers,
    representing bytes, or strings with SI unit prefixes for bytes.
    The conversion for the Filesystem.MinSize happens in both the JSON and
    TOML unmarshallers.  With the new partitioning customizations we will
    have more conversions of the same type.
    
    This commit extracts the handling into a generic function for
    reusability and convenience.
    
    This changes the error messages.  Tests have been updated accordingly.
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    567f093 View commit details
    Browse the repository at this point in the history
  2. blueprint: disk customization

    - New structure that's more powerful than the existing array of
      FilesystemCustomization.
    - The general structure defines a (minimum) size for the overall disk
      and an array of partitions.
    - Each partition has one of three types:
      - plain: a plain disk partition, meaning partitions with traditional
        filesystem payloads (xfs, ext4, etc).
      - lvm: a partition containing an LVM volume group, which in turn
        will contain one or more LVM logical volumes.
      - btrfs: a partition containing a btrfs volume, which in turn will
        contain one or more subvolumes.
    - The new FilesystemTypedCustomization struct supports a label and
      fs_type and no minsize, compared to the old FilesystemCustomization.
      This struct is used to define both the payload of a plain partition
      and the payload of an LVM logical volume.
    - The PartitionCustomization has a type and size and embeds three
      substructures, one for each partition type.  Decoding/unmarshalling of
      the PartitionCustomization is handled by a "typeSniffer" which first
      reads the value of the "type" field and then deserialises the whole
      object into a struct that only contains the fields valid for that
      partition type.  This ensures that no fields are set for the
      substructure of a different type than the one defined in the "type"
      fields.
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    a2df2e2 View commit details
    Browse the repository at this point in the history
  3. blueprint: add custom TOML unmarshaller for PartitionCustomization

    The unmarshaller decodes "type" and "minsize" manually and reuses the
    decode* functions from the JSON unmarshaller for each subtype based on
    the value of "type".
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    b9ad80d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5c0fe88 View commit details
    Browse the repository at this point in the history
  5. blueprint: validate DiskCustomization

    Add validation functions for DiskCustomization.  Three types of
    validation are added:
    - Validate(): checks for functionally invalid configurations (bad
      mountpoint names or invalid structures, etc).  The docstring lists all
      properties that are validated.
    - ValidateLayoutConstraints(): checks for limitations we imposed
      ourselves as a matter of policy and support on the structure of the
      partition table.  This includes disallowing, for example, multiple LVM
      volume groups, multiple btrfs volumes, or combinations of the two.
      While these layouts are technically valid for a partitioned disk, we
      currently do not support them.
    - CheckDiskMountpointsPolicy(): Checks that none of the mountpoints are
      disallowed by the given PathPolicy.
    
    Tests added for all validators.
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    9d72478 View commit details
    Browse the repository at this point in the history
  6. disk: extract partition type ID selection into a function

    Define a nested map:
    [partition table type] -> [partition type name] -> [partition type ID]
    and use it from a private function getPartitionTypeIDfor() for
    convenient partition type ID selection.
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    da345e6 View commit details
    Browse the repository at this point in the history
  7. disk: NewCustomPartitionTable() function

    New function that creates a partition table from scratch based on the
    new disk customization.
    
    First, it creates any necessary partitions for the selected boot mode
    (BIOS boot, ESP, or both).
    Then it determines if a /boot partition is needed based on the
    partitions and mountpoints defined in the disk customizations.
    Then, it iterates through partitions and creates each one in the order
    its defined.
    Finally, it creates a root filesystem if one wasn't already defined,
    resizes any necessary volumes, relayouts the table (setting all
    partition starts and offsets), and generates UUIDs all entities.
    
    Because of the new way the boot partition is handled, the
    EnsureBootPartition() function was changed to always add a boot
    partition and was renamed to AddBootPartition() to match the new
    behaviour.  The tests have been updated accordingly.
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e324102 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    29542a1 View commit details
    Browse the repository at this point in the history
  9. distro/fedora: pass all customizations to getPartitionTable()

    Preparing the function to use the new PartitioningCustomization when
    set.
    achilleas-k committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    21fc141 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. Configuration menu
    Copy the full SHA
    58944b1 View commit details
    Browse the repository at this point in the history
  2. distro/fedora: add minimum directory sizes to image types

    For all image types that don't specify a requiredPartitionSizes, add the
    defaults that are used in the NewPartitionTable() so that
    NewCustomPartitionTable() has similar behaviour.
    
    The only image type that explicitly doesn't specify minimum sizes is the
    iot-raw-image, which has a pre-existing empty map.
    achilleas-k committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    2f81245 View commit details
    Browse the repository at this point in the history
  3. distro/fedora: check partitioning customizations

    Add checks for Partitioning customizations in Fedora's checkOptions()
    function.  The function now returns an error if:
    - Partitioning is defined at the same time as Filesystem
    - Partitioning is defined for ostree image types (commit and container).
    - The mountpoints in Partitioning violate the mountpoint policies for
      the image type.
    achilleas-k committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    a72afb6 View commit details
    Browse the repository at this point in the history
  4. test: add partitioning build configs

    One for each variant:
    - Plain
    - Plain + btrfs
    - Plain + lvm
    achilleas-k committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    6ba8dad View commit details
    Browse the repository at this point in the history