Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: in What's New, clarified custom publishing behavior #50

Merged
merged 3 commits into from
Apr 9, 2024
Merged

docs: in What's New, clarified custom publishing behavior #50

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
36c2a9c
docs: in What's New, clarified behavior
mbshields Apr 8, 2024
7d6e670
docs: Make What's New the first page when Get Started
mbshields Apr 8, 2024
1ce9c68
docs: updated the substitutions section, etc in the stacker file article
mbshields Apr 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 41 additions & 77 deletions docs/reference/stacker_file.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,29 @@ processed in stacker as:

1 2 3

That is, variables of the form `$FOO` or `${FOO}` are supported, and variables
with `${FOO:default}` a default value will evaluate to their default if not
specified on the command line. It is an error to specify a `${FOO}` style
without a default; to make the default an empty string, use `${FOO:}`.
In order to avoid conflict with bash or POSIX shells in the `run` section, only placeholders with two braces are supported, e.g. `${{FOO}}`. Placeholders with a default value like `${{FOO:default}}` will evaluate to their default if not specified on the command line or in a substitution file.

In addition to substitutions provided on the command line, the following
variables are also available with their values from either command
line flags or stacker-config file.
Using a `${{FOO}}` placeholder without a default will result in an error if there is no substitution provided. If you want an empty string in that case, use an empty default: `${{FOO:}}`.

In order to avoid confusion, it is also an error if a placeholder in the shell style (`$FOO` or `${FOO}`) is found when the same key has been provided as a substitution either via the command line (for example, `--substitute FOO=bar`) or in a substitution file. An error will be printed that explains how to rewrite it. For example:

error "A=B" was provided as a substitution and unsupported placeholder "${A}" was found. Replace "${A}" with "${{A}}" to use the substitution.

Substitutions can also be specified in a yaml file given with the argument `--substitute-file`, with any number of key: value pairs:

FOO: bar
BAZ: bat

In addition to substitutions provided on the command line or a file, the following variables are also available with their values from either command line flags or stacker-config file.

STACKER_STACKER_DIR config name 'stacker_dir', cli flag '--stacker-dir'-
STACKER_ROOTFS_DIR config name 'rootfs_dir', cli flag '--roots-dir'
STACKER_OCI_DIR config name 'oci_dir', cli flag '--oci-dir'


The stacker build environment has the following environment variables
available for reference:

* `STACKER_LAYER_NAME`: the name of the layer being built. `STACKER_LAYER_NAME`
will be `my-build` when the `run` section below is executed.
* `STACKER_LAYER_NAME`: the name of the layer being built. `STACKER_LAYER_NAME` will be `my-build` when the `run` section below is executed.

```yaml
my-build:
Expand All @@ -38,8 +42,7 @@ available for reference:

### `from`

The `from` directive describes the base image that stacker will start from. It
takes the form:
The `from` directive describes the base image that stacker will start from. It takes the form:

from:
type: $type
Expand All @@ -50,16 +53,13 @@ takes the form:
Some directives are irrelevant depending on the type. Supported types are:

`docker`: `url` is required, `insecure` is optional. When `insecure` is
specified, stacker attempts to connect via http instead of https to the Docker
Hub.
specified, stacker attempts to connect via http instead of https to the Docker Hub.

`tar`: `url` is required, everything else is ignored.

`oci`: `url` is required, of the form `path:tag`. This uses the OCI image at
`url` (which may be a local path).
`oci`: `url` is required, and must be a local OCI layout URI of the form `oci:/local/path/image:tag`

`built`: `tag` is required, everything else is ignored. `built` bases this
layer on a previously specified layer in the stacker file.
`built`: `tag` is required, everything else is ignored. `built` bases this layer on a previously specified layer in the stacker file.

`scratch`: which is an empty rootfs and can be used to host statically built binaries.

Expand All @@ -73,15 +73,15 @@ Will import a file or directory from the local filesystem. If the file or direct

http://example.com/foo.tar.gz

Will import foo.tar.gz and make it available in `/stacker`. Note that stacker will NOT update this file unless the cache is cleared, to avoid excess network usage. That means that updates after the first time stacker downloads the file will not be reflected.
Will import foo.tar.gz and make it available in `/stacker`. Note that stacker will NOT update this file unless the cache is cleared, to avoid excess network usage. This means that updates after the first time stacker downloads the file will not be reflected.

stacker://$name/path/to/file

Will grab /path/to/file from the previously built image `$name`.

#### `import hash`

Each entry in the `import` directive also supports specifying the hash (sha256sum) of import source, for all the three forms presented above. For example:
Each entry in the `imports` directive also supports specifying the hash (sha256sum) of import source, for all the three forms presented above. For example:
```
imports:
- path: config.json
Expand All @@ -91,7 +91,7 @@ imports:
- path: stacker://$name/path/to/file
hash: f805b012017bc769a....
```
Before copying the file, stacker will check that the file's hash matches the given value. For file imports, the source file is hashed at build time. For HTTP imports, the value returned by the server in the `X-Checksum-Sha256` HTTP header is checked first. If that value matches, the file is downloaded and then hashed and compared again.
Before copying or downloading the file, stacker will check that the file's hash matches the given value. For file imports, the source file is hashed at build time. For HTTP imports, the value returned by the server in the `X-Checksum-Sha256` HTTP header is checked first. If that value matches, the file is downloaded and then hashed and compared again.

`stacker build` supports the flag `--require-hash`, which will cause a build error if any HTTP(S) remote imports do not have a hash specified, in all transitively included stacker YAMLs.

Expand All @@ -106,9 +106,7 @@ imports:

#### `import dest`

The `import` directive also supports specifying the destination path (specified
by `dest`) in the resulting container image, where the source file (specified
by `path`) will be copyed to. For example:
The `import` directive also supports specifying the destination path (specified by `dest`) in the resulting container image, where the source file (specified by `path`) will be copied to. For example:
```
imports:
- path: config.json
Expand All @@ -119,40 +117,31 @@ imports:

The deprecated `import` directive operates like `imports` except that the entries in the `import` array will be placed into `/stacker` instead of `/stacker/imports`.

See https://github.com/project-stacker/stacker/issues/571 for timeline and migration info.

### `overlay_dirs`
This directive works only with OverlayFS backend storage.

The `overlay_dirs` directive describes the directories (content) from the host that should be
available in the container's filesystem. It preserves all file/dirs attributes but no
owner or group.
The `overlay_dirs` directive describes the directories (content) from the host that should be available in the container's filesystem. It preserves all file/dirs attributes but not owner or group.

```
overlay_dirs:
- source: /path/to/directory
dest: /usr/local/ ## optional arg, default is '/'
- source: /path/to/directory2
```
This example will result in all the files/dirs from the host's /path/to/directory
to be available under container's /usr/local/ and all the files/dirs from the host's
/path/to/directory2 to be available under container's /
This example will result in all the files/dirs from the host's /path/to/directory to be available under container's /usr/local/ and all the files/dirs from the host's /path/to/directory2 to be available under container's /


### `environment`, `labels`, `working_dir`, `volumes`, `cmd`, `entrypoint`, `user`

These correspond exactly to the similarly named bits in the [OCI image
config
spec](https://github.com/opencontainers/image-spec/blob/master/config.md#properties),
and are available for users to pass variables through to the runtime environment
of the image.
config spec](https://github.com/opencontainers/image-spec/blob/master/config.md#properties),
and are available for users to pass variables through to the runtime environment of the image.

### `generate_labels`

The `generate_labels` entry is similar to `run` in that it contains a list of
commands to run inside the generated rootfs. It runs after the `run` section is
done, and its mutations to the filesystem are not recorded, except in one case
`/oci-labels`. `/oci-labels` is a special directory where this code can write a
file, and the name of the file will be the OCI label name, and the content will
be the label content.
The `generate_labels` entry is similar to `run` in that it contains a list of commands to run inside the generated rootfs. It runs after the `run` section is done, and its mutations to the filesystem are not recorded, except in one case: `/oci-labels`. `/oci-labels` is a special directory where this code can write a file, and the name of the file will be the OCI label name, and the content will be the label content.

### `build_env` and `build_env_passthrough`

Expand All @@ -162,9 +151,7 @@ build environment.
`build_env`: A dictionary of environment variable definitions. The values will be present in the build's environment.

`build_env_passthrough`: A list of regular expressions that work as a
filter on which environment variables should be passed through from the current
env into the container. To let all variables through, simply set
`build_env_passthrough`: `[".*"]`
filter on which environment variables should be passed through from the current env into the container. To let all variables through, simply set `build_env_passthrough`: `[".*"]`

If `build_env_passthrough` is not set, the default behavior is to allow
through proxy variables `HTTP_PROXY, HTTPS_PROXY, FTP_PROXY, http_proxy, https_proxy, ftp_proxy`.
Expand All @@ -173,38 +160,27 @@ Values in the `build_env` override values passed through via `build_env_passthro

### `full_command`

Because of the odd behavior of `cmd` and `entrypoint` (and the inherited nature
of these from previous stacker layers), `full_command` provides a way to set
the full command that will be executed in the image, clearing out any previous
`cmd` and `entrypoint` values that were set in the image.
Because of the odd behavior of `cmd` and `entrypoint`, and the inherited nature of these from previous stacker layers, `full_command` provides a way to set the full command that will be executed in the image, clearing out any previous `cmd` and `entrypoint` values that were set in the image.

### `build_only`

`build_only`: indicates whether or not to include this layer in the final OCI
image. This can be useful in conjunction with an import from this layer in
another image, if you want to isolate the build environment for a binary but
not include all of its build dependencies.
`build_only`: indicates whether or not to include this layer in the final OCI image. This can be useful in conjunction with an import from this layer in another image, if you want to isolate the build environment for a binary but not include all of its build dependencies.

### `binds`

`binds`: specifies bind mounts from the host to the container. There are two formats:

binds:
- /foo/bar -> /bar/baz
- /zomg
- /zomg

The first binds /foo/bar to /bar/baz, and the second binds host /zomg to
container /zomg.
The first format binds `/foo/bar` to `/bar/baz`, and the second binds host `/zomg` to container `/zomg`.

At this time there is no awareness of change for any of these bind mounts, so
`--no-cache` should be used to re-build if the content of the bind mount has
changed.
At this time there is no awareness of change for any of these bind mounts, so `--no-cache` should be used to re-build if the content of the bind mount has changed.

### `config`

The `config` key is a special type of entry in the root of the `stacker.yaml` file.
It cannot contain an image definition; it is used to provide configuration
applicable for building all the images defined in this file. For example,
The `config` key is a special type of entry in the root of the `stacker.yaml` file. It cannot contain an image definition; its purpose is to provide configuration applicable for building all the images defined in this file. For example:

config:
prerequisites:
Expand All @@ -213,33 +189,21 @@ applicable for building all the images defined in this file. For example,

#### `prerequisites`

If the `prerequisites` list is present under the `config` key, stacker will
make sure to build all the images in the stacker.yaml files found at the paths
contained in the list. In this way, stacker supports building multiple
stacker.yaml files in the correct order.
If the `prerequisites` list is present under the `config` key, stacker will make sure to build all the images in the `stacker.yaml` files found at the paths contained in the list. In this way, stacker supports building multiple `stacker.yaml` files in the correct order.

In this particular case, the parent folder of the current folder, let's call it
`parent`, has 3 subfolders (`folder1`, `folder2`, and `folder3`), each containing a
`stacker.yaml` file. The example `config` above is in `parent/folder1/stacker.yaml`.
In this particular case, the parent folder of the current folder (let's call it `parent`) has 3 subfolders (`folder1`, `folder2`, and `folder3`), each containing a `stacker.yaml` file. The `config` example above is in `parent/folder1/stacker.yaml`.

When `stacker build -f parent/folder1/stacker.yaml` is invoked, stacker searches
for the other two stacker.yaml files and builds them first, before building
the stacker.yaml file specified in the command line.
When `stacker build -f parent/folder1/stacker.yaml` is invoked, stacker searches for the other two `stacker.yaml` files and builds them first before building the `stacker.yaml` file specified in the command line.

#### `annotations`

`annotations` is a user-specified key value map that will be included in the
final OCI image. Note that these annotations are included in the image manifest
itself and not as part of the index.json.
`annotations` is a user-specified key value map that will be included in the final OCI image. Note that these annotations are included in the image manifest itself and not as part of the index.json.

annotations:
a.b.c.key: abc_val
p.q.r.key: pqr_val

While the `config` section supports a similar `labels`, it is more pertinent to the
image runtime. On the other hand, `annotations` is intended to be
image-specific metadata aligned with the
[annotations in the image spec](https://github.com/opencontainers/image-spec/blob/main/annotations.md).
While the `config` section supports a similar `labels`, it is more pertinent to the image runtime. On the other hand, `annotations` is intended to be image-specific metadata aligned with the [annotations in the image spec](https://github.com/opencontainers/image-spec/blob/main/annotations.md).

### os

Expand Down
2 changes: 1 addition & 1 deletion docs/whats-new.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

### Publish specific images

- By default, the [`stacker publish`](reference/stacker_cli.md#stacker-publish) command pushes all images in a stacker.yaml file instead of only the required images. Using a new command option, `--image <value>`, you can explicitly specify which images are to be published. This command option can be specified multiple times, selecting each image to be included.
- By default, the [`stacker publish`](reference/stacker_cli.md#stacker-publish) command pushes all images in a stacker.yaml file. Using a new command option, `--image <value>`, you can explicitly specify which images are to be published. This command option can be specified multiple times, selecting each image to be included. In either case, images configured with `build-only: true` are not published.

### Specify a single working directory

Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,12 +98,12 @@ markdown_extensions:

nav:
- Home: index.md
- What's New: whats-new.md
- Get Started:
- Get Stacker: get_started/get_stacker.md
- Hello Stacker Image: get_started/hello_stacker_image.md
- Stacker Tutorial: get_started/tutorial.md
- Features: features.md
- What's New: whats-new.md
- Concepts:
- About: concepts/about.md
- OCI Image Layout: concepts/oci_image_layout.md
Expand Down
Loading