devcontainer-cache-build

Build devcontainer images using remote Docker cache. This repo contains definitions for a complete workflow around the devcontainer image, including configuration for local building and cache population by CI pipeline.

Usage

This repo defines tooling for the entire lifecycle of a devcontainer cache. Based on the context of the script call, devcontainer-cache-build-initialize supplies build configuration to target cache to refs for each event in that lifecycle, as described below:

graph TD;
  gha_main_workflow(GitHub Actions main workflow) <--1--> main_cache(registry/image-cache:main-layer)
  gha_pr_workflow(GitHub Actions PR workflow) <--2--> branch_cache(registry/image-cache:branch-layer)
  devcontainer_build(devcontainer build)
  devcontainer_build <--3--> local_branch_cache(registry/image-cache:local-branch-layer)
  main_cache --4-->  gha_pr_workflow & devcontainer_build
  branch_cache --5--> gha_main_workflow & devcontainer_build
  local_branch_cache --6--> gha_main_workflow & gha_main_workflow
  subgraph Registry
    main_cache
    branch_cache
    local_branch_cache
  end
  subgraph GitHub Actions
    gha_main_workflow
    gha_pr_workflow
  end

Loading

1. GitHub Actions default branch workflows populate the cache for each layer, under the main ref.
2. GitHub Actions PR workflows populate the cache for each layer, under the ref for the PR branch name.
3. Devcontainer builds populate the cache for each layer, under the ref for the local branch name, with a local- prefix.
4. The main ref cache for each layer is used for builds in all contexts.
5. The branch ref cache for each layer is used for builds in all contexts.
6. The local-branch ref cache for each layer is used for builds in all contexts.

Initialize script

The initialize script replaces build in .devcontainer/devcontainer.json, and computes appropriate configuration for the devcontainer image Docker build command.

Initialize script basic usage

To use the script, add "initializeCommand": "curl https://raw.githubusercontent.com/rcwbr/devcontainer-cache-build/0.1.0/devcontainer-cache-build-initialize | bash" to your devcontainer.json, or to scripts referenced by it. For example, you might replace

{
  ...
  "build": {
    "dockerfile": "Dockerfile"
  },
  ...
}

with

{
  ...
  "initializeCommand": ".devcontainer/initialize",
  "image": "my-project-devcontainer"
  ...
}

and .devcontainer/initialize:

#!/bin/bash

export DEVCONTAINER_IMAGE=my-project-devcontainer
curl https://raw.githubusercontent.com/rcwbr/devcontainer-cache-build/0.1.0/devcontainer-cache-build-initialize | bash

Initialize script specific version usage

A specific version of the script may be used by adjusting the URL in the curl command. The format is https://raw.githubusercontent.com/rcwbr/devcontainer-cache-build/<version ref>/devcontainer-cache-build-initialize, where <version ref> may be any valid version reference from this repo.

Initialize script bake usage

To use the devcontainer-cache-build-initialize script with bake, the recommended usage is to set the DEVCONTAINER_BUILD_ADDITIONAL_ARGS and DEVCONTAINER_DEFINITION_FILES vars to leverage bake file partials as defined in the dockerfile-partials repository.

If using a custom bake file, the config must contain the following configuration:

variable "DEVCONTAINER_OUTPUTS" {
  default = ""
}
variable "DEVCONTAINER_CACHE_FROMS" {
  default = ""
}
variable "DEVCONTAINER_CACHE_TOS" {
  default = ""
}

target "default" {
  dockerfile = ".devcontainer/Dockerfile"
  cache-from = [
    for cache_from in split(" ", trimspace("${DEVCONTAINER_CACHE_FROMS}")):
    "${cache_from}-base"
  ]
  cache-to = [
    for cache_to in split(" ", trimspace("${DEVCONTAINER_CACHE_TOS}")):
    "${cache_to}-base"
  ]
  output = split(" ", trimspace("${DEVCONTAINER_OUTPUTS}"))
}

Initialize script inputs

The devcontainer-cache-build-initialize script reads several environment variables as configuration. To provide an empty value for an input and avoid the default, set the variables to "_NONE_".

Variable	Required	Default	Effect
DEVCONTAINER_IMAGE	✓	N/A	The tag applied to the image build
DEVCONTAINER_BUILD_ADDITIONAL_ARGS	✗	N/A	Arbitrary additional args forwarded to the build or bake command
DEVCONTAINER_CACHE_FROMS	✗	type=registry,ref=[DEVCONTAINER_REGISTRY]-cache:[current git branch name sanitized] type=registry,ref=[DEVCONTAINER_REGISTRY]-cache:local-[current git branch name sanitized] type=registry,ref=[DEVCONTAINER_REGISTRY]-cache:[DEVCONTAINER_DEFAULT_BRANCH_NAME, sanitized]	Each cache-from arg to be applied to the image build, space separated
DEVCONTAINER_CACHE_TOS	✗	type=registry,rewrite-timestamp=true,mode=max,ref=[DEVCONTAINER_REGISTRY]-cache:[local-][current git branch name sanitized]	Each cache-to arg to be applied to the image build, space separated. The default value includes local- applied as a version prefix unless CI=true
DEVCONTAINER_CONTEXT	✗	.	The build context for the image
DEVCONTAINER_DEFAULT_BRANCH_NAME	✗	main	The branch name from which to always pull cache
DEVCONTAINER_DEFINITION_TYPE	✗	build	The image definition type, basic Docker build (build) or Bake (bake)
DEVCONTAINER_DEFINITION_FILES	✗	.devcontainer/Dockerfile, or .devcontainer/bake.hcl if DEVCONTAINER_DEFINITION_TYPE is bake	The Dockerfile or bake config file path(s) for the image build, space separated
DEVCONTAINER_INITIALIZE_PID	✗	N/A	If defined, must be set to the process ID of the command provided to the devcontainer.json initializeCommand (often $PPID). Used to determine whether the context of the initializeCommand call is a new container bringup, based on the presence of the the --expect-existing-container argument
DEVCONTAINER_OUTPUTS	✗	type=image,name=[DEVCONTAINER_REGISTRY],push=[DEVCONTAINER_PUSH_IMAGE]	Each output arg to applt to the image build, space separated
DEVCONTAINER_PUSH_IMAGE	✗	false	Whether to push the image to the provided registry (requires DEVCONTAINER_REGISTRY)
DEVCONTAINER_REGISTRY	✗	DEVCONTAINER_IMAGE	The registry for the image and/or cache

The image build leverages any values provided or computed to DEVCONTAINER_CACHE_FROM as cache inputs.

Initialize script outputs

The devcontainer-cache-build-initialize script image build produces several outputs.

• An image in the local daemon image store (the image output type) at the name given by DEVCONTAINER_IMAGE
• Image build cache output published as specified by DEVCONTAINER_CACHE_TO

Initialize script GitHub container registry setup

Configuring the devcontainer-cache-build-initialize script with a plain image name results in targeting outputs and cache to DockerhHub by default. Setting the DEVCONTAINER_REGISTRY to ghcr.io/[your user/org name] instead allows you to target the GitHub container registry instead. To set up a container repository for this in a local environment, use docker login against ghcr.io. For GitHub Codespaces environments, use the following steps:

1. Create a Personal Access Token with write:packages scope for the repository to which the image belongs.
2. Add the token as a Codespace secret, to the repository to which the Codespaces environment belongs.
3. Use the secret variable to docker login in the Codespace environment by adding a login command to the devcontainer initializeCommand in advance of the curl to the intialize script, e.g.:

# .devcontainer/initialize

echo $DEVCONTAINER_CACHE_BUILD_DEVCONTAINER_INITIALIZE | docker login ghcr.io --username $GITHUB_USER --password-stdin
...
curl https://raw.githubusercontent.com/rcwbr/devcontainer-cache-build/0.1.0/devcontainer-cache-build-initialize | bash

GitHub Actions workflow

Leveraging the entire lifecycle of the devcontainer cache requires applying a CI/CD workflow to prepopulate cache. This may be achieved via the reusable workflow defined in .github/workflows/devcontainer-cache-build.yaml, e.g.:

on: push
jobs:
  devcontainer-cache-build:
    uses: rcwbr/devcontainer-cache-build/.github/workflows/[email protected]
    permissions:
      packages: write

The default behavior of the workflow provides arguments for use with the useradd Dockerfile partial for Codespaces user provisioning. These arguments must be forwarded to the devcontainer-cache-build-initialize script, e.g. via the DEVCONTAINER_BUILD_ADDITIONAL_ARGS variable:

# .devcontainer/initialize

export DEVCONTAINER_BUILD_ADDITIONAL_ARGS="$@"
curl https://raw.githubusercontent.com/rcwbr/devcontainer-cache-build/0.1.0/devcontainer-cache-build-initialize | bash

Contributing

CI/CD

This repo uses the release-it-gh-workflow, with the conventional-changelog image defined at any given ref, as its automation.

It uses its own reusable devcontainer cache build workflow to pre-populate the devcontainer cache.

Settings

The GitHub repo settings for this repo are defined as code using the Probot settings GitHub App. Settings values are defined in the .github/settings.yml file. Enabling automation of settings via this file requires installing the app.

The settings applied are as recommended in the release-it-gh-workflow usage, including tag and branch protections, GitHub App and environment authentication, and required checks.