Debug logs enhancements around disk quotas

This release...

1. ...adds debug logs (can be seen by setting grootfs.log_level to debug) around the disk quota application process.

Cache Management improvements

This release...

1. ...changes the way we do cache management (previously called graph cleanup):
   • Cache is now considered only the set of image layers that are not currently used.
   • Dropped grootfs.graph_cleanup_threshold_in_mb BOSH property.
   • Added grootfs.cache_size_bytes instead.
   • If you are currently setting grootfs.graph_cleanup_threshold_in_mb to any value but 0, we recommend converting that value to bytes and providing it to grootfs.cache_size_bytes instead.
2. ...emits new cache management related metric: grootfs.UnusedLayersSize.

0.27.0

This release...

1. ...drops some BOSH properties:
   • grootfs.external_logdevice_size_in_mb which refers to the ability to use a separate loop device for the XFS log.
   • grootfs.persistent_image_list and grootfs.store_size_bytes: both in dropped in favour of the new cache management approach which will be governed by grootfs.cache_size_bytes and a sensible default.
2. ...improves the security of the set-UID binary which is used by the Overlay-XFS driver (tardis) when GrootFS is used in rootless mode.
3. ...introduces grootfs.experimental_rootless_mode which is linked to Garden's respective BOSH property. NOTICE: If you have garden.experimental_rootless_mode turned on, you will need to recreate your Diego Cells.
4. ...allows root use to consume GrootFS even if the helper binaries (drax and tardis) are not set-UID.
5. ...enables GrootFS to download remote layers from an mTLS HTTP store.

0.26.0

This release...

• Extends support for OCI Images to fit the needs of the OCI Buildpacks feature narrative that Garden is working on.
• Adds retry loops for fetches from Docker registries (in case Docker registries intermittently fail to serve blobs).
• Finishes work for rootless overlay-xfs driver.
• Changes the format of grootfs create to a partial OCI runtime spec format.

0.25.0

...this release:

• Adds OCI image support to GrootFS. Local OCI Images can be used by running: grootfs create oci://<Path to OCI Image> <Image ID>.
• Extends the rootless Overlay-XFS driver. It can now delete container images.
• Fixes chain ID calculation for OCI and Docker images, This removes a small potential cache poisoning exploit.
• Unpacks layers inside a chroot jail in order to avoid having layers write files outside the store.

v0.24.0 - Better performance around `grootfs stats`

This release was meant to be v0.22.0 but due to Github and subsequent CI issues we had to jump some release numbers.

This release...

• Removes unnecessary cache flushing from grootfs stats. This should improve general file system performance in Grooted Diego cells.
• Includes some more work on the rootless use case for Overlay-XFS:
  • Local tar-balls can be used with disk quotas.
  • Metrics can be obtained.

v0.21.0

...this release:

New Features

• Adds a new flag, called --rootless, to grootfs init-store. By calling grootfs init-store --rootless user:group you can create a store owned by a non-root user.
  • The new store will use UID and GID mappings found in /etc/subuid and /etc/subgid.
• Makes grootfs init-store use the underlying XFS filesystem if --store-size-bytes is passed with value 0.
  • No need to always create a loop device.
• Introduces a new BOSH property, called grootfs-diagnostics.use_persistent_disk, which instructs GrootFS to use the underlying persistent disk for its store.
  • BOSH persistent disks are mounted in /var/vcap/store.
  • BOSH jobs need to require the persistent disk to be formatted as an XFS filesystem.

Fixes

• Fixes a concurrency / idempotency issue found in grootfs clean and grootfs create --with-clean.
• Fixes short id collision for the Overlay-XFS driver.
• Cleans up projectids in the Overlay-XFS driver.

v0.20.0 - init-store enhancements and debug job for XFS

• Extends grootfs init-store to take care of creating and mounting a backing filesystem for the store. The optional flag --store-size-bytes controls the size of the filesystem.
• Drops --uid-mapping and --gid-mapping from grootfs create. The UID and GID mappings for the store must now be specified as part of grootfs init-store.
• Makes GrootFS able to optionally report an extensive set of system metrics that are aimed to help us study and improve the way GrootFS uses XFS.
• Improves store validation logic.
• Removes functionality that was lazily initializing the store upon first grootfs create call.
• Introduces the grootfs-debug BOSH job which is designed to store XFS internal logs when it finds processes being in D state (disk wait or uninterruptible sleep) for a long time. This is to help us investigate further issues with stuck D-state processes that we have experienced in production environments.
• Adds migration path for old (pre-v0.20.0) store formats.

v0.19.0 - Awesome performance

...this release

• Enhances grootfs create logs with tar-ball location.
• Adds feature flags for XFS mount options and support of XFS external log device.
• Significantly improves the locking strategy for parallel create and clean operations. Story.
• Uses noatime and nobarrier XFS mount options by default when used with the overlay-xfs driver.

v0.18.0 - JSON output by default!

• Drop the --json / --no-json flags from the GrootFS CLI. Output is always in JSON format.
• Drop the create.json configuration file option since the respective flag is gone.
• Drop the grootfs.json_output BOSH release property since the respective configuration flag option is gone.

Notice: GrootFS v0.18.0 can only be used with Garden runC v1.6.0+.