diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index e92b7c7e3b..9475e2bd59 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -11,16 +11,16 @@ Documentation inspired from https://github.com/tektoncd/pipeline/blob/ce7591acec ## Getting started -1. [Ramp up on kubernetes and CRDs](#ramp-up) -1. Create [a GitHub account](https://github.com/join) -1. Setup +1. [Ramp up on kubernetes and CRDs](#ramp-up) +2. Create [a GitHub account](https://github.com/join) +3. Setup [GitHub access via SSH](https://help.github.com/articles/connecting-to-github-with-ssh/) -1. [Create and checkout a repo fork](#checkout-your-fork) -1. Install [requirements](#requirements) -1. [Set up a Kubernetes cluster](#create-a-cluster-and-a-repo) -1. Set up your [shell environment](#environment-setup) -1. [Configure kubectl to use your cluster](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) -1. [Install Shipwright Build in your cluster](#install-shipwright-build) +4. [Create and checkout a repo fork](#checkout-your-fork) +5. Install [requirements](#requirements) +6. [Set up a Kubernetes cluster](#create-a-cluster-and-a-repo) +7. Set up your [shell environment](#environment-setup) +8. [Configure kubectl to use your cluster](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) +9. [Install Shipwright Build in your cluster](#install-shipwright-build) ### Ramp up @@ -58,9 +58,9 @@ The Go tools require that you clone the repository to the To check out this repository: -1. Create your own +1. Create your own [fork of this repo](https://help.github.com/articles/fork-a-repo/) -1. Clone it to your machine: +2. Clone it to your machine: ```shell mkdir -p ${GOPATH}/src/github.com/shipwright-io @@ -78,17 +78,17 @@ _Adding the `upstream` remote sets you up nicely for regularly You must install these tools: -1. [`go`](https://golang.org/doc/install): The language Shipwright Build is +1. [`go`](https://golang.org/doc/install): The language Shipwright Build is built in -1. [`git`](https://help.github.com/articles/set-up-git/): For source control -1 [`ko`](https://github.com/ko-build/ko): To build and deploy changes. -1. [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/): For +2. [`git`](https://help.github.com/articles/set-up-git/): For source control +3. [`ko`](https://github.com/ko-build/ko): To build and deploy changes. +4. [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/): For interacting with your kube cluster ### Create a cluster and a repo 1. Follow the instructions in the Kubernetes doc to [Set up a kubernetes cluster](https://kubernetes.io/docs/setup/) -1. Set up a container image repository for pushing images. Any container image registry that is accessible to your cluster can be used for your repository. This can be a public registry like [Docker Hub](https://docs.docker.com/docker-hub/), [quay.io](https://quay.io), or a container registry runs by your cloud provider +2. Set up a container image repository for pushing images. Any container image registry that is accessible to your cluster can be used for your repository. This can be a public registry like [Docker Hub](https://docs.docker.com/docker-hub/), [quay.io](https://quay.io), or a container registry runs by your cloud provider **Note**: We support Kubernetes version 1.27 to 1.29. 1 cluster worker node for basic usage, 2+ cluster worker nodes for HA @@ -96,9 +96,9 @@ You must install these tools: To run your controller, you'll need to set these environment variables (we recommend adding them to your `.bashrc`): -1. `GOPATH`: If you don't have one, simply pick a directory and add `export +1. `GOPATH`: If you don't have one, simply pick a directory and add `export GOPATH=...` -1. `$GOPATH/bin` on `PATH`: This is so that tooling installed via `go get` will +2. `$GOPATH/bin` on `PATH`: This is so that tooling installed via `go get` will work properly. `.bashrc` example: @@ -126,11 +126,11 @@ The following set of steps highlight how to deploy a Build controller pod into a ./hack/install-tekton.sh ``` -1. Set your `KO_DOCKER_REPO` environment variable. This will be the container +2. Set your `KO_DOCKER_REPO` environment variable. This will be the container image registry you push to, or `kind.local` if you're using [KinD](https://kind.sigs.k8s.io). -1. Build and deploy the controller from source, from within the root of the repo: +3. Build and deploy the controller from source, from within the root of the repo: ```sh ko apply -P -R -f deploy/ diff --git a/README.md b/README.md index 1751ff4da1..d8f6417824 100644 --- a/README.md +++ b/README.md @@ -166,28 +166,28 @@ To find out more on what's the best strategy or what else can Shipwright do for ### Read the Docs -| Version | Docs | Examples | -| ------- | ------------------------------ | --------------------------- | -| HEAD | [Docs @ HEAD](docs/README.md) | [Examples @ HEAD](samples) | -| [v0.12.0](https://github.com/shipwright-io/build/releases/tag/v0.12.0) | [Docs @ v0.12.0](https://github.com/shipwright-io/build/tree/v0.12.0/docs) | [Examples @ v0.12.0](https://github.com/shipwright-io/build/tree/v0.12.0/samples) | -| [v0.11.0](https://github.com/shipwright-io/build/releases/tag/v0.11.0) | [Docs @ v0.11.0](https://github.com/shipwright-io/build/tree/v0.11.0/docs) | [Examples @ v0.11.0](https://github.com/shipwright-io/build/tree/v0.11.0/samples) | -| [v0.10.0](https://github.com/shipwright-io/build/releases/tag/v0.10.0) | [Docs @ v0.10.0](https://github.com/shipwright-io/build/tree/v0.10.0/docs) | [Examples @ v0.10.0](https://github.com/shipwright-io/build/tree/v0.10.0/samples) | -| [v0.9.0](https://github.com/shipwright-io/build/releases/tag/v0.9.0) | [Docs @ v0.9.0](https://github.com/shipwright-io/build/tree/v0.9.0/docs) | [Examples @ v0.9.0](https://github.com/shipwright-io/build/tree/v0.9.0/samples) | -| [v0.8.0](https://github.com/shipwright-io/build/releases/tag/v0.8.0) | [Docs @ v0.8.0](https://github.com/shipwright-io/build/tree/v0.8.0/docs) | [Examples @ v0.8.0](https://github.com/shipwright-io/build/tree/v0.8.0/samples) | -| [v0.7.0](https://github.com/shipwright-io/build/releases/tag/v0.7.0) | [Docs @ v0.7.0](https://github.com/shipwright-io/build/tree/v0.7.0/docs) | [Examples @ v0.7.0](https://github.com/shipwright-io/build/tree/v0.7.0/samples) | -| [v0.6.0](https://github.com/shipwright-io/build/releases/tag/v0.6.0) | [Docs @ v0.6.0](https://github.com/shipwright-io/build/tree/v0.6.0/docs) | [Examples @ v0.6.0](https://github.com/shipwright-io/build/tree/v0.6.0/samples) | -| [v0.5.1](https://github.com/shipwright-io/build/releases/tag/v0.5.1) | [Docs @ v0.5.1](https://github.com/shipwright-io/build/tree/v0.5.1/docs) | [Examples @ v0.5.1](https://github.com/shipwright-io/build/tree/v0.5.1/samples) | -| [v0.5.0](https://github.com/shipwright-io/build/releases/tag/v0.5.0) | [Docs @ v0.5.0](https://github.com/shipwright-io/build/tree/v0.5.0/docs) | [Examples @ v0.5.0](https://github.com/shipwright-io/build/tree/v0.5.0/samples) | -| [v0.4.0](https://github.com/shipwright-io/build/releases/tag/v0.4.0) | [Docs @ v0.4.0](https://github.com/shipwright-io/build/tree/v0.4.0/docs) | [Examples @ v0.4.0](https://github.com/shipwright-io/build/tree/v0.4.0/samples) | -| [v0.3.0](https://github.com/shipwright-io/build/releases/tag/v0.3.0) | [Docs @ v0.3.0](https://github.com/shipwright-io/build/tree/v0.3.0/docs) | [Examples @ v0.3.0](https://github.com/shipwright-io/build/tree/v0.3.0/samples) | -| [v0.2.0](https://github.com/shipwright-io/build/releases/tag/v0.2.0) | [Docs @ v0.2.0](https://github.com/shipwright-io/build/tree/v0.2.0/docs) | [Examples @ v0.2.0](https://github.com/shipwright-io/build/tree/v0.2.0/samples) | -| [v0.1.1](https://github.com/shipwright-io/build/releases/tag/v0.1.1) | [Docs @ v0.1.1](https://github.com/shipwright-io/build/tree/v0.1.1/docs) | [Examples @ v0.1.1](https://github.com/shipwright-io/build/tree/v0.1.1/samples) | -| [v0.1.0](https://github.com/shipwright-io/build/releases/tag/v0.1.0) | [Docs @ v0.1.0](https://github.com/shipwright-io/build/tree/v0.1.0/docs) | [Examples @ v0.1.0](https://github.com/shipwright-io/build/tree/v0.1.0/samples) | +| Version | Docs | Examples | +|------------------------------------------------------------------------|----------------------------------------------------------------------------|-----------------------------------------------------------------------------------| +| HEAD | [Docs @ HEAD](docs/README.md) | [Examples @ HEAD](samples) | +| [v0.12.0](https://github.com/shipwright-io/build/releases/tag/v0.12.0) | [Docs @ v0.12.0](https://github.com/shipwright-io/build/tree/v0.12.0/docs) | [Examples @ v0.12.0](https://github.com/shipwright-io/build/tree/v0.12.0/samples) | +| [v0.11.0](https://github.com/shipwright-io/build/releases/tag/v0.11.0) | [Docs @ v0.11.0](https://github.com/shipwright-io/build/tree/v0.11.0/docs) | [Examples @ v0.11.0](https://github.com/shipwright-io/build/tree/v0.11.0/samples) | +| [v0.10.0](https://github.com/shipwright-io/build/releases/tag/v0.10.0) | [Docs @ v0.10.0](https://github.com/shipwright-io/build/tree/v0.10.0/docs) | [Examples @ v0.10.0](https://github.com/shipwright-io/build/tree/v0.10.0/samples) | +| [v0.9.0](https://github.com/shipwright-io/build/releases/tag/v0.9.0) | [Docs @ v0.9.0](https://github.com/shipwright-io/build/tree/v0.9.0/docs) | [Examples @ v0.9.0](https://github.com/shipwright-io/build/tree/v0.9.0/samples) | +| [v0.8.0](https://github.com/shipwright-io/build/releases/tag/v0.8.0) | [Docs @ v0.8.0](https://github.com/shipwright-io/build/tree/v0.8.0/docs) | [Examples @ v0.8.0](https://github.com/shipwright-io/build/tree/v0.8.0/samples) | +| [v0.7.0](https://github.com/shipwright-io/build/releases/tag/v0.7.0) | [Docs @ v0.7.0](https://github.com/shipwright-io/build/tree/v0.7.0/docs) | [Examples @ v0.7.0](https://github.com/shipwright-io/build/tree/v0.7.0/samples) | +| [v0.6.0](https://github.com/shipwright-io/build/releases/tag/v0.6.0) | [Docs @ v0.6.0](https://github.com/shipwright-io/build/tree/v0.6.0/docs) | [Examples @ v0.6.0](https://github.com/shipwright-io/build/tree/v0.6.0/samples) | +| [v0.5.1](https://github.com/shipwright-io/build/releases/tag/v0.5.1) | [Docs @ v0.5.1](https://github.com/shipwright-io/build/tree/v0.5.1/docs) | [Examples @ v0.5.1](https://github.com/shipwright-io/build/tree/v0.5.1/samples) | +| [v0.5.0](https://github.com/shipwright-io/build/releases/tag/v0.5.0) | [Docs @ v0.5.0](https://github.com/shipwright-io/build/tree/v0.5.0/docs) | [Examples @ v0.5.0](https://github.com/shipwright-io/build/tree/v0.5.0/samples) | +| [v0.4.0](https://github.com/shipwright-io/build/releases/tag/v0.4.0) | [Docs @ v0.4.0](https://github.com/shipwright-io/build/tree/v0.4.0/docs) | [Examples @ v0.4.0](https://github.com/shipwright-io/build/tree/v0.4.0/samples) | +| [v0.3.0](https://github.com/shipwright-io/build/releases/tag/v0.3.0) | [Docs @ v0.3.0](https://github.com/shipwright-io/build/tree/v0.3.0/docs) | [Examples @ v0.3.0](https://github.com/shipwright-io/build/tree/v0.3.0/samples) | +| [v0.2.0](https://github.com/shipwright-io/build/releases/tag/v0.2.0) | [Docs @ v0.2.0](https://github.com/shipwright-io/build/tree/v0.2.0/docs) | [Examples @ v0.2.0](https://github.com/shipwright-io/build/tree/v0.2.0/samples) | +| [v0.1.1](https://github.com/shipwright-io/build/releases/tag/v0.1.1) | [Docs @ v0.1.1](https://github.com/shipwright-io/build/tree/v0.1.1/docs) | [Examples @ v0.1.1](https://github.com/shipwright-io/build/tree/v0.1.1/samples) | +| [v0.1.0](https://github.com/shipwright-io/build/releases/tag/v0.1.0) | [Docs @ v0.1.0](https://github.com/shipwright-io/build/tree/v0.1.0/docs) | [Examples @ v0.1.0](https://github.com/shipwright-io/build/tree/v0.1.0/samples) | ### Dependencies | Dependency | Supported versions | -| -------------------------------------| ---------------------------- | +|--------------------------------------|------------------------------| | [Kubernetes](https://kubernetes.io/) | v1.27.\*, v1.28.\*, v1.29.\* | | [Tekton](https://tekton.dev) | v0.50.\*, v0.53.\*, v0.56.\* | diff --git a/docs/build.md b/docs/build.md index 987f4178ec..5db3b05278 100644 --- a/docs/build.md +++ b/docs/build.md @@ -126,7 +126,7 @@ A `Build` resource can specify a source type, such as a Git repository or an OCI - `source.type` - Specify the type of the data-source. Currently, the supported types are "Git", "OCIArtifact", and "Local". - `source.git.url` - Specify the source location using a Git repository. - `source.git.cloneSecret` - For private repositories or registries, the name references a secret in the namespace that contains the SSH private key or Docker access credentials, respectively. -- `source.git.revision` - A specific revision to select from the source repository, this can be a commit, tag or branch name. If not defined, it will fallback to the Git repository default branch. +- `source.git.revision` - A specific revision to select from the source repository, this can be a commit, tag or branch name. If not defined, it will fall back to the Git repository default branch. - `source.contextDir` - For repositories where the source code is not located at the root folder, you can specify this path here. By default, the Build controller does not validate that the Git repository exists. If the validation is desired, users can explicitly define the `build.shipwright.io/verify.repository` annotation with `true`. For example: @@ -283,7 +283,7 @@ spec: ### Defining ParamValues -A `Build` resource can specify _paramValues_ for parameters that are defined in the referenced `BuildStrategy`. You specify these parameter values to control how the steps of the build strategy behave. You can overwrite values in the `BuildRun` resource. See the related [documentation](./buildrun.md#defining-params) for more information. +A `Build` resource can specify _paramValues_ for parameters that are defined in the referenced `BuildStrategy`. You specify these parameter values to control how the steps of the build strategy behave. You can overwrite values in the `BuildRun` resource. See the related [documentation](buildrun.md#defining-paramvalues) for more information. The build strategy author can define a parameter as either a simple string or an array. Depending on that, you must specify the value accordingly. The build strategy parameter can be specified with a default value. You must specify a value in the `Build` or `BuildRun` for parameters without a default. diff --git a/docs/buildrun.md b/docs/buildrun.md index 841f81afac..4a78b80bbe 100644 --- a/docs/buildrun.md +++ b/docs/buildrun.md @@ -392,7 +392,7 @@ The following table illustrates the different states a BuildRun can have under i | False | BuildRunAmbiguousBuild | Yes | The defined `BuildRun` uses both `spec.build.name` and `spec.build.spec`. Only one of them is allowed at the same time. | | False | BuildRunBuildFieldOverrideForbidden | Yes | The defined `BuildRun` uses an override (e.g. `timeout`, `paramValues`, `output`, or `env`) in combination with `spec.build.spec`, which is not allowed. Use the `spec.build.spec` to directly specify the respective value. | | False | PodEvicted | Yes | The BuildRun Pod was evicted from the node it was running on. See [API-initiated Eviction](https://kubernetes.io/docs/concepts/scheduling-eviction/api-eviction/) and [Node-pressure Eviction](https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/) for more information. | -| False | StepOutOfMemory | Yes | The BuildRun Pod failed because a step went out of memory. | +| False | StepOutOfMemory | Yes | The BuildRun Pod failed because a step went out of memory. | **Note**: We heavily rely on the Tekton TaskRun [Conditions](https://github.com/tektoncd/pipeline/blob/main/docs/taskruns.md#monitoring-execution-status) for populating the BuildRun ones, with some exceptions. @@ -524,4 +524,4 @@ For every BuildRun controller reconciliation, the `buildSpec` in the status of t The `BuildRun` resource abstracts the image construction by delegating this work to the Tekton Pipeline [TaskRun](https://github.com/tektoncd/pipeline/blob/main/docs/taskruns.md). Compared to a Tekton Pipeline [Task](https://github.com/tektoncd/pipeline/blob/main/docs/tasks.md), a `TaskRun` runs all `steps` until completion of the `Task` or until a failure occurs in the `Task`. -During the Reconcile, the `BuildRun` controller will generate a new `TaskRun`. The controller will embed in the `TaskRun` `Task` definition the requires `steps` to execute during the execution. These `steps` are defined in the strategy defined in the `Build` resource, either a `ClusterBuildStrategy` or a `BuildStrategy`. +During the Reconcile, the `BuildRun` controller will generate a new `TaskRun`. The controller will embed in the `TaskRun` `Task` definition the required `steps` to execute during the execution. These `steps` are defined in the strategy defined in the `Build` resource, either a `ClusterBuildStrategy` or a `BuildStrategy`. diff --git a/docs/buildstrategies.md b/docs/buildstrategies.md index 183d7b5ac8..84c7ec95e9 100644 --- a/docs/buildstrategies.md +++ b/docs/buildstrategies.md @@ -54,24 +54,24 @@ A `ClusterBuildStrategy` is available cluster-wide, while a `BuildStrategy` is a Well-known strategies can be bootstrapped from [here](../samples/v1beta1/buildstrategy). The currently supported Cluster BuildStrategy are: -| Name | Supported platforms | -| ---- | ------------------- | -| [buildah](../samples/v1beta1/buildstrategy/buildah) | all | -| [BuildKit](../samples/v1beta1/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml) | all | -| [buildpacks-v3-heroku](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_cr.yaml) | linux/amd64 only | -| [buildpacks-v3](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml) | linux/amd64 only | -| [kaniko](../samples/v1beta1/buildstrategy/kaniko/buildstrategy_kaniko_cr.yaml) | all | -| [ko](../samples/v1beta1/buildstrategy/ko/buildstrategy_ko_cr.yaml) | all | -| [source-to-image](../samples/v1beta1/buildstrategy/source-to-image/buildstrategy_source-to-image_cr.yaml) | linux/amd64 only | +| Name | Supported platforms | +|-------------------------------------------------------------------------------------------------------------------|---------------------| +| [buildah](../samples/v1beta1/buildstrategy/buildah) | all | +| [BuildKit](../samples/v1beta1/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml) | all | +| [buildpacks-v3-heroku](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_cr.yaml) | linux/amd64 only | +| [buildpacks-v3](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml) | linux/amd64 only | +| [kaniko](../samples/v1beta1/buildstrategy/kaniko/buildstrategy_kaniko_cr.yaml) | all | +| [ko](../samples/v1beta1/buildstrategy/ko/buildstrategy_ko_cr.yaml) | all | +| [source-to-image](../samples/v1beta1/buildstrategy/source-to-image/buildstrategy_source-to-image_cr.yaml) | linux/amd64 only | ## Available BuildStrategies The current supported namespaces BuildStrategy are: -| Name | Supported platforms | -| ---- | ------------------- | -| [buildpacks-v3-heroku](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_namespaced_cr.yaml) | linux/amd64 only | -| [buildpacks-v3](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_namespaced_cr.yaml) | linux/amd64 only | +| Name | Supported platforms | +|------------------------------------------------------------------------------------------------------------------------------|---------------------| +| [buildpacks-v3-heroku](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_namespaced_cr.yaml) | linux/amd64 only | +| [buildpacks-v3](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_namespaced_cr.yaml) | linux/amd64 only | --- @@ -81,7 +81,7 @@ The `buildah` ClusterBuildStrategy uses [`buildah`](https://github.com/container The strategy is available in two formats: -- [`buildah-shipwright-managed-push`](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_shipwright_managed_push%20copy_cr.yaml) +- [`buildah-shipwright-managed-push`](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_shipwright_managed_push_cr.yaml) - [`buildah-strategy-managed-push`](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_strategy_managed_push_cr.yaml) Learn more about the differences of [shipwright-, or strategy-managed push](#output-directory-vs-output-image) @@ -105,7 +105,7 @@ The [buildpacks-v3][buildpacks] BuildStrategy/ClusterBuildStrategy uses a Cloud You can install the `BuildStrategy` in your namespace or install the `ClusterBuildStrategy` at cluster scope so that it can be shared across namespaces. -To install the cluster scope strategy, you can chose between the Paketo and Heroku buildpacks family: +To install the cluster scope strategy, you can choose between the Paketo and Heroku buildpacks family: ```sh # Paketo @@ -115,7 +115,7 @@ kubectl apply -f samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_build kubectl apply -f samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_cr.yaml ``` -To install the namespaced scope strategy, you can chose between the Paketo and Heroku buildpacks family: +To install the namespaced scope strategy, you can choose between the Paketo and Heroku buildpacks family: ```sh # Paketo @@ -173,11 +173,11 @@ The sample build strategy contains a `platforms` array parameter that you can se The `buildkit` ClusterBuildStrategy currently locks the following parameters: -- To allow running rootless, it requires both [AppArmor](https://kubernetes.io/docs/tutorials/clusters/apparmor/) as well as [SecComp](https://kubernetes.io/docs/tutorials/clusters/seccomp/) to be disabled using the `unconfined` profile. +- To allow running rootless, it requires both [AppArmor](https://kubernetes.io/docs/tutorials/clusters/apparmor/) and [SecComp](https://kubernetes.io/docs/tutorials/clusters/seccomp/) to be disabled using the `unconfined` profile. ### Usage in Clusters with Pod Security Standards -The BuildKit strategy contains fields with regards to security settings. It therefore depends on the respective cluster setup and administrative configuration. These settings are: +The BuildKit strategy contains fields with regard to security settings. It therefore depends on the respective cluster setup and administrative configuration. These settings are: - Defining the `unconfined` profile for both AppArmor and seccomp as required by the underlying `rootlesskit`. - The `allowPrivilegeEscalation` settings is set to `true` to be able to use binaries that have the `setuid` bit set in order to run with "root" level privileges. In case of BuildKit, this is required by `rootlesskit` in order to set the user namespace mapping file `/proc//uid_map`. @@ -213,18 +213,18 @@ kubectl apply -f samples/v1beta1/buildstrategy/ko/buildstrategy_ko_cr.yaml The build strategy provides the following parameters that you can set in a Build or BuildRun to control its behavior: -| Parameter | Description | Default | -| -- | -- | -- | -| `go-flags` | Value for the GOFLAGS environment variable. | Empty | -| `go-version` | Version of Go, must match a tag from [the golang image](https://hub.docker.com/_/golang?tab=tags) | `1.21` | -| `ko-version` | Version of ko, must be either `latest` for the newest release, or a [ko release name](https://github.com/ko-build/ko/releases) | `latest` | -| `package-directory` | The directory inside the context directory containing the main package. | `.` | -| `target-platform` | Target platform to be built. For example: `linux/arm64`. Multiple platforms can be provided separated by comma, for example: `linux/arm64,linux/amd64`. The value `all` will build all platforms supported by the base image. The value `current` will build the platform on which the build runs. | `current` | +| Parameter | Description | Default | +|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| +| `go-flags` | Value for the GOFLAGS environment variable. | Empty | +| `go-version` | Version of Go, must match a tag from [the golang image](https://hub.docker.com/_/golang?tab=tags) | `1.21` | +| `ko-version` | Version of ko, must be either `latest` for the newest release, or a [ko release name](https://github.com/ko-build/ko/releases) | `latest` | +| `package-directory` | The directory inside the context directory containing the main package. | `.` | +| `target-platform` | Target platform to be built. For example: `linux/arm64`. Multiple platforms can be provided separated by comma, for example: `linux/arm64,linux/amd64`. The value `all` will build all platforms supported by the base image. The value `current` will build the platform on which the build runs. | `current` | ### Volumes -| Volume | Description | -| ------- | ----------- | +| Volume | Description | +|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | gocache | Volume to contain the GOCACHE. Can be set to a persistent volume to optimize compilation performance for rebuilds. The default is an emptyDir volume which means that the cached data is discarded at the end of a BuildRun. | ## Source to Image @@ -263,7 +263,7 @@ Users defining _parameters_ under their strategies require to understand the fol - **Definition**: A list of parameters should be defined under `spec.parameters`. Each list item should consist of a _name_, a _description_, a _type_ (either `"array"` or `"string"`) and optionally a _default_ value (for type=string), or _defaults_ values (for type=array). If no default(s) are provided, then the user must define a value in the Build or BuildRun. - **Usage**: In order to use a parameter in the strategy steps, use the following syntax for type=string: `$(params.your-parameter-name)`. String parameters can be used in all places in the `buildSteps`. Some example scenarios are: - - `image`: to use a custom tag, for example `golang:$(params.go-version)` as it is done in the [ko sample build strategy](../samples/v1beta1/buildstrategy/ko/buildstrategy_ko_cr.yaml)) + - `image`: to use a custom tag, for example `golang:$(params.go-version)` as it is done in the [ko sample build strategy](../samples/v1beta1/buildstrategy/ko/buildstrategy_ko_cr.yaml) - `args`: to pass data into your builder command - `env`: to force a user to provide a value for an environment variable. @@ -297,7 +297,7 @@ Users defining _parameters_ under their strategies require to understand the fol - **Parameterize**: Any `Build` or `BuildRun` referencing your strategy, can set a value for _your-parameter-name_ parameter if needed. -**Note**: Users can provide parameter values as simple strings or as references to keys in [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) and [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/). If they use a ConfigMap or Secret, then the value can only be used if the parameter is used in the `command`, `args`, or `env` section of the `buildSteps`. For example, the above mentioned scenario to set a step's `image` to `golang:$(params.go-version)` does not allow the usage of ConfigMaps or Secrets. +**Note**: Users can provide parameter values as simple strings or as references to keys in [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) and [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/). If they use a ConfigMap or Secret, then the value can only be used if the parameter is used in the `command`, `args`, or `env` section of the `buildSteps`. For example, the above-mentioned scenario to set a step's `image` to `golang:$(params.go-version)` does not allow the usage of ConfigMaps or Secrets. The following example is from the [BuildKit sample build strategy](../samples/v1beta1/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml). It defines and uses several parameters: @@ -425,13 +425,13 @@ See more information on how to use these parameters in a `Build` or `BuildRun` i Contrary to the strategy `spec.parameters`, you can use system parameters and their values defined at runtime when defining the steps of a build strategy to access system information as well as information provided by the user in their Build or BuildRun. The following parameters are available: -| Parameter | Description | -| ------------------------------ | ----------- | -| `$(params.shp-source-root)` | The absolute path to the directory that contains the user's sources. | -| `$(params.shp-source-context)` | The absolute path to the context directory of the user's sources. If the user specified no value for `spec.source.contextDir` in their `Build`, then this value will equal the value for `$(params.shp-source-root)`. Note that this directory is not guaranteed to exist at the time the container for your step is started, you can therefore not use this parameter as a step's working directory. | -| `$(params.shp-output-directory)` | The absolute path to a directory that the build strategy should store the image in. You can store a single tarball containing a single image, or an OCI image layout. | -| `$(params.shp-output-image)` | The URL of the image that the user wants to push, as specified in the Build's `spec.output.image` or as an override from the BuildRun's `spec.output.image`. | -| `$(params.shp-output-insecure)` | A flag that indicates the output image's registry location is insecure because it uses a certificate not signed by a certificate authority, or uses HTTP. | +| Parameter | Description | +|----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `$(params.shp-source-root)` | The absolute path to the directory that contains the user's sources. | +| `$(params.shp-source-context)` | The absolute path to the context directory of the user's sources. If the user specified no value for `spec.source.contextDir` in their `Build`, then this value will equal the value for `$(params.shp-source-root)`. Note that this directory is not guaranteed to exist at the time the container for your step is started, you can therefore not use this parameter as a step's working directory. | +| `$(params.shp-output-directory)` | The absolute path to a directory that the build strategy should store the image in. You can store a single tarball containing a single image, or an OCI image layout. | +| `$(params.shp-output-image)` | The URL of the image that the user wants to push, as specified in the Build's `spec.output.image` or as an override from the BuildRun's `spec.output.image`. | +| `$(params.shp-output-insecure)` | A flag that indicates the output image's registry location is insecure because it uses a certificate not signed by a certificate authority, or uses HTTP. | ### Output directory vs. output image @@ -447,16 +447,16 @@ If you are uncertain about how to implement your build strategy, then follow thi 1. If your build strategy tool cannot locally store an image but always pushes it, then you must do the push operation. An example is the [Buildpacks strategy](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml). You SHOULD respect the `$(params.shp-output-insecure)` parameter. 2. If your build strategy tool can locally store an image, then the choice depends on how you expect your build users to make use of your strategy, and the nature of your strategy. 1. Some build strategies do not produce all layers of an image, but use a common base image and put one or more layers on top with the application. An example is `ko`. Such base image layers are often already present in the destination registry (like in rebuilds). If the strategy can perform the push operation, then it can optimize the process and can omit the download of the base image when it is not required to push it. In the case of a shipwright-managed push, the complete image must be locally stored in `$(params.shp-output-directory)`, which implies that a base image must always be downloaded. - 2. Some build strategy tools do not make it easy to determine the digest or size of the image, which can make it complex for your to set the [strategy results](#system-results). In the case of a shipwright-managed push, Shipwright has the responsibility to set them. + 2. Some build strategy tools do not make it easy to determine the digest or size of the image, which can make it complex for you to set the [strategy results](#system-results). In the case of a shipwright-managed push, Shipwright has the responsibility to set them. 3. Build users can configure the build to amend additional annotations, or labels to the final image. In the case of a shipwright-managed push, these can be set directly and the image will only be pushed once. In a strategy-managed push scenario, your build strategy will push the first version of the image without those annotations and labels. Shipwright will then mutate the image and push it again with the updated annotations and labels. Such a duplicate push can cause unexpected behavior with registries that trigger other actions when an image gets pushed, or that do not allow overwriting a tag. 4. The Shipwright maintainers plan to provide more capabilities in the future that need the image locally, such as vulnerability scanning, or software bill of material (SBOM) creation. These capabilities may be only fully supported with shipwright-managed push. ## System parameters vs Strategy Parameters Comparison -| Parameter Type | User Configurable | Definition | -| ------------------ | ------------ | ------------- | -| System Parameter | No | At run-time, by the `BuildRun` controller. | -| Strategy Parameter | Yes | At build-time, during the `BuildStrategy` creation. | +| Parameter Type | User Configurable | Definition | +|--------------------|-------------------|-----------------------------------------------------| +| System Parameter | No | At run-time, by the `BuildRun` controller. | +| Strategy Parameter | Yes | At build-time, during the `BuildStrategy` creation. | ## Securely referencing string parameters @@ -489,7 +489,7 @@ This opens the door to script injection, for example if the user sets the `sampl some-tool --sample-argument "argument-value" && malicious-command && echo "" ``` -To securely pass a parameter value into a script-style argument, you can chose between these two approaches: +To securely pass a parameter value into a script-style argument, you can choose between these two approaches: 1. Using environment variables. This is used in some of our sample strategies, for example [ko](../samples/v1beta1/buildstrategy/ko/buildstrategy_ko_cr.yaml), or [buildpacks](../samples/v1beta1/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml). Basically, instead of directly using the parameter inside the script, you pass it via environment variable. Using quoting, shells ensure that no command injection is possible: @@ -514,7 +514,7 @@ To securely pass a parameter value into a script-style argument, you can chose b some-tool --sample-argument "${PARAM_SAMPLE_PARAMETER}" ``` -2. Using arguments. This is used in some of our sample build strategies, for example [buildah](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_cr.yaml). Here, you use arguments to your own inline script. Appropriate shell quoting guards against command injection. +2. Using arguments. This is used in some of our sample build strategies, for example [buildah](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_shipwright_managed_push_cr.yaml). Here, you use arguments to your own inline script. Appropriate shell quoting guards against command injection. ```yaml spec: @@ -543,7 +543,7 @@ To securely pass a parameter value into a script-style argument, you can chose b If you are using a strategy-managed push, see [output directory vs output image](#output-directory-vs-output-image), you can optionally store the size and digest of the image your build strategy created to a set of files. | Result file | Description | -| ---------------------------------- | ----------------------------------------------- | +|------------------------------------|-------------------------------------------------| | `$(results.shp-image-digest.path)` | File to store the digest of the image. | | `$(results.shp-image-size.path)` | File to store the compressed size of the image. | @@ -565,9 +565,9 @@ status: Additionally, you can store error details for debugging purposes when a BuildRun fails using your strategy. -| Result file | Description | -| ---------------------------------- | ----------------------------------------------- | -| `$(results.shp-error-reason.path)` | File to store the error reason. | +| Result file | Description | +|-------------------------------------|----------------------------------| +| `$(results.shp-error-reason.path)` | File to store the error reason. | | `$(results.shp-error-message.path)` | File to store the error message. | Reason is intended to be a one-word CamelCase classification of the error source, with the first letter capitalized. @@ -612,7 +612,7 @@ All strategies steps can include a definition of resources(_limits and requests_ ### Strategies with different resources -If the strategy admins would require to have multiple flavours of the same strategy, where one strategy has more resources that the other. Then, multiple strategies for the same type should be defined on the cluster. In the following example, we use Kaniko as the type: +If the strategy admins required to have multiple flavours of the same strategy, where one strategy has more resources that the other. Then, multiple strategies for the same type should be defined on the cluster. In the following example, we use Kaniko as the type: ```yaml --- @@ -752,15 +752,15 @@ For a more concrete example, let´s take a look on the following scenarios: **Scenario 1.** Namespace without `LimitRange`, both steps with the same resource values. -If we will apply the following resources: +If we apply the following resources: -- [buildahBuild](../samples/build/build_buildah_cr.yaml) -- [buildahBuildRun](../samples/buildrun/buildrun_buildah_cr.yaml) -- [buildahClusterBuildStrategy](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_cr.yaml) +- [buildahBuild](../samples/v1beta1/build/build_buildah_shipwright_managed_push_cr.yaml) +- [buildahBuildRun](../samples/v1beta1/buildrun/buildrun_buildah_cr.yaml) +- [buildahClusterBuildStrategy](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_shipwright_managed_push_cr.yaml) We will see some differences between the `TaskRun` definition and the `pod` definition. -For the `TaskRun`, as expected we can see the resources on each `step`, as we previously define on our [strategy](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_cr.yaml). +For the `TaskRun`, as expected we can see the resources on each `step`, as we previously define on our [strategy](../samples/v1beta1/buildstrategy/buildah/buildstrategy_buildah_shipwright_managed_push_cr.yaml). ```sh $ kubectl -n test-build get tr buildah-golang-buildrun-9gmcx-pod-lhzbc -o json | jq '.spec.taskSpec.steps[] | select(.name == "step-buildah-bud" ) | .resources' @@ -824,10 +824,10 @@ In this scenario, only one container can have the `spec.resources.requests` defi **Scenario 2.** Namespace without `LimitRange`, steps with different resources: -If we will apply the following resources: +If we apply the following resources: -- [buildahBuild](../samples/build/build_buildah_cr.yaml) -- [buildahBuildRun](../samples/buildrun/buildrun_buildah_cr.yaml) +- [buildahBuild](../samples/v1beta1/build/build_buildah_shipwright_managed_push_cr.yaml) +- [buildahBuildRun](../samples/v1beta1/buildrun/buildrun_buildah_cr.yaml) - We will use a modified buildah strategy, with the following steps resources: ```yaml diff --git a/docs/configuration.md b/docs/configuration.md index 46f6c465c6..30a64cbc8a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -12,31 +12,31 @@ The controller is installed into Kubernetes with reasonable defaults. However, t The following environment variables are available: -| Environment Variable | Description | -| --- | --- | -| `CTX_TIMEOUT` | Override the default context timeout used for all Custom Resource Definition reconciliation operations. Default is 5 (seconds). | -| `REMOTE_ARTIFACTS_CONTAINER_IMAGE` | Specify the container image used for the `.spec.sources` remote artifacts download, by default it uses `quay.io/quay/busybox:latest`. | -| `TERMINATION_LOG_PATH` | Path of the termination log. This is where controller application will write the reason of its termination. Default value is `/dev/termination-log`. | -| `GIT_ENABLE_REWRITE_RULE` | Enable Git wrapper to setup a URL `insteadOf` Git config rewrite rule for the respective source URL hostname. Default is `false`. | -| `GIT_CONTAINER_TEMPLATE` | JSON representation of a [Container] template that is used for steps that clone a Git repository. Default is `{"image": "ghcr.io/shipwright-io/build/git:latest", "command": ["/ko-app/git"], "env": [{"name": "HOME", "value": "/shared-home"}], "securityContext":{"allowPrivilegeEscalation": false, "capabilities": {"drop": ["ALL"]}, "runAsUser": 1000,"runAsGroup": 1000}}` [^1]. The following properties are ignored as they are set by the controller: `args`, `name`. | -| `GIT_CONTAINER_IMAGE` | Custom container image for Git clone steps. If `GIT_CONTAINER_TEMPLATE` is also specifying an image, then the value for `GIT_CONTAINER_IMAGE` has precedence. | -| `BUNDLE_IMAGE_CONTAINER_TEMPLATE` | JSON representation of a [Container] template that is used for steps that pulls a bundle image to obtain the packaged source code. Default is `{"image": "ghcr.io/shipwright-io/build/bundle:latest", "command": ["/ko-app/bundle"], "env": [{"name": "HOME","value": "/shared-home"}], "securityContext":{"allowPrivilegeEscalation": false, "capabilities": {"drop": ["ALL"]}, "runAsUser":1000,"runAsGroup":1000}}` [^1]. The following properties are ignored as they are set by the controller: `args`, `name`. | -| `BUNDLE_IMAGE_CONTAINER_IMAGE` | Custom container image that pulls a bundle image to obtain the packaged source code. If `BUNDLE_IMAGE_CONTAINER_TEMPLATE` is also specifying an image, then the value for `BUNDLE_IMAGE_CONTAINER_IMAGE` has precedence. | -| `IMAGE_PROCESSING_CONTAINER_TEMPLATE` | JSON representation of a [Container](https://pkg.go.dev/k8s.io/api/core/v1#Container) template that is used for steps that processes the image. Default is `{"image": "ghcr.io/shipwright-io/build/image-processing:latest", "command": ["/ko-app/image-processing"], "env": [{"name": "HOME","value": "/shared-home"}], "securityContext": {"allowPrivilegeEscalation": false, "capabilities": {"add": ["DAC_OVERRIDE"], "drop": ["ALL"]}, "runAsUser": 0, "runAsgGroup": 0}}`. The following properties are ignored as they are set by the controller: `args`, `name`. | -| `IMAGE_PROCESSING_CONTAINER_IMAGE` | Custom container image that is used for steps that processes the image. If `IMAGE_PROCESSING_CONTAINER_TEMPLATE` is also specifying an image, then the value for `IMAGE_PROCESSING_CONTAINER_IMAGE` has precedence. | -| `WAITER_IMAGE_CONTAINER_TEMPLATE` | JSON representation of a [Container] template that waits for local source code to be uploaded to it. Default is `{"image":"ghcr.io/shipwright-io/build/waiter:latest", "command": ["/ko-app/waiter"], "args": ["start"], "env": [{"name": "HOME","value": "/shared-home"}], "securityContext":{"allowPrivilegeEscalation": false, "capabilities": {"drop": ["ALL"]}, "runAsUser":1000,"runAsGroup":1000}}`. The following properties are ignored as they are set by the controller: `args`, `name`. | -| `WAITER_IMAGE_CONTAINER_IMAGE` | Custom container image that waits for local source code to be uploaded to it. If `WAITER_IMAGE_CONTAINER_TEMPLATE` is also specifying an image, then the value for `WAITER_IMAGE_CONTAINER_IMAGE` has precedence. | -| `BUILD_CONTROLLER_LEADER_ELECTION_NAMESPACE` | Set the namespace to be used to store the `shipwright-build-controller` lock, by default it is in the same namespace as the controller itself. | -| `BUILD_CONTROLLER_LEASE_DURATION` | Override the `LeaseDuration`, which is the duration that non-leader candidates will wait to force acquire leadership. | -| `BUILD_CONTROLLER_RENEW_DEADLINE` | Override the `RenewDeadline`, which is the duration that the acting leader will retry refreshing leadership before giving up. | -| `BUILD_CONTROLLER_RETRY_PERIOD` | Override the `RetryPeriod`, which is the duration the LeaderElector clients should wait between tries of actions. | -| `BUILD_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the build controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | -| `BUILDRUN_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the BuildRun controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | -| `BUILDSTRATEGY_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the BuildStrategy controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | -| `CLUSTERBUILDSTRATEGY_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the ClusterBuildStrategy controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | -| `KUBE_API_BURST` | Burst to use for the Kubernetes API client. See [Config.Burst]. A value of 0 or lower will use the default from client-go, which currently is 10. Default is 0. | -| `KUBE_API_QPS` | QPS to use for the Kubernetes API client. See [Config.QPS]. A value of 0 or lower will use the default from client-go, which currently is 5. Default is 0. | -| `VULNERABILITY_COUNT_LIMIT` | holds vulnerability count limit if vulnerability scan is enabled for the output image. If it is defined as 10, then it will output only 10 vulnerabilities sorted by severity in the buildrun status.Output. Default is 50. | +| Environment Variable | Description | +|--------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `CTX_TIMEOUT` | Override the default context timeout used for all Custom Resource Definition reconciliation operations. Default is 5 (seconds). | +| `REMOTE_ARTIFACTS_CONTAINER_IMAGE` | Specify the container image used for the `.spec.sources` remote artifacts download, by default it uses `quay.io/quay/busybox:latest`. | +| `TERMINATION_LOG_PATH` | Path of the termination log. This is where controller application will write the reason of its termination. Default value is `/dev/termination-log`. | +| `GIT_ENABLE_REWRITE_RULE` | Enable Git wrapper to setup a URL `insteadOf` Git config rewrite rule for the respective source URL hostname. Default is `false`. | +| `GIT_CONTAINER_TEMPLATE` | JSON representation of a [Container] template that is used for steps that clone a Git repository. Default is `{"image": "ghcr.io/shipwright-io/build/git:latest", "command": ["/ko-app/git"], "env": [{"name": "HOME", "value": "/shared-home"}], "securityContext":{"allowPrivilegeEscalation": false, "capabilities": {"drop": ["ALL"]}, "runAsUser": 1000,"runAsGroup": 1000}}` [^1]. The following properties are ignored as they are set by the controller: `args`, `name`. | +| `GIT_CONTAINER_IMAGE` | Custom container image for Git clone steps. If `GIT_CONTAINER_TEMPLATE` is also specifying an image, then the value for `GIT_CONTAINER_IMAGE` has precedence. | +| `BUNDLE_IMAGE_CONTAINER_TEMPLATE` | JSON representation of a [Container] template that is used for steps that pulls a bundle image to obtain the packaged source code. Default is `{"image": "ghcr.io/shipwright-io/build/bundle:latest", "command": ["/ko-app/bundle"], "env": [{"name": "HOME","value": "/shared-home"}], "securityContext":{"allowPrivilegeEscalation": false, "capabilities": {"drop": ["ALL"]}, "runAsUser":1000,"runAsGroup":1000}}` [^1]. The following properties are ignored as they are set by the controller: `args`, `name`. | +| `BUNDLE_IMAGE_CONTAINER_IMAGE` | Custom container image that pulls a bundle image to obtain the packaged source code. If `BUNDLE_IMAGE_CONTAINER_TEMPLATE` is also specifying an image, then the value for `BUNDLE_IMAGE_CONTAINER_IMAGE` has precedence. | +| `IMAGE_PROCESSING_CONTAINER_TEMPLATE` | JSON representation of a [Container](https://pkg.go.dev/k8s.io/api/core/v1#Container) template that is used for steps that processes the image. Default is `{"image": "ghcr.io/shipwright-io/build/image-processing:latest", "command": ["/ko-app/image-processing"], "env": [{"name": "HOME","value": "/shared-home"}], "securityContext": {"allowPrivilegeEscalation": false, "capabilities": {"add": ["DAC_OVERRIDE"], "drop": ["ALL"]}, "runAsUser": 0, "runAsgGroup": 0}}`. The following properties are ignored as they are set by the controller: `args`, `name`. | +| `IMAGE_PROCESSING_CONTAINER_IMAGE` | Custom container image that is used for steps that processes the image. If `IMAGE_PROCESSING_CONTAINER_TEMPLATE` is also specifying an image, then the value for `IMAGE_PROCESSING_CONTAINER_IMAGE` has precedence. | +| `WAITER_IMAGE_CONTAINER_TEMPLATE` | JSON representation of a [Container] template that waits for local source code to be uploaded to it. Default is `{"image":"ghcr.io/shipwright-io/build/waiter:latest", "command": ["/ko-app/waiter"], "args": ["start"], "env": [{"name": "HOME","value": "/shared-home"}], "securityContext":{"allowPrivilegeEscalation": false, "capabilities": {"drop": ["ALL"]}, "runAsUser":1000,"runAsGroup":1000}}`. The following properties are ignored as they are set by the controller: `args`, `name`. | +| `WAITER_IMAGE_CONTAINER_IMAGE` | Custom container image that waits for local source code to be uploaded to it. If `WAITER_IMAGE_CONTAINER_TEMPLATE` is also specifying an image, then the value for `WAITER_IMAGE_CONTAINER_IMAGE` has precedence. | +| `BUILD_CONTROLLER_LEADER_ELECTION_NAMESPACE` | Set the namespace to be used to store the `shipwright-build-controller` lock, by default it is in the same namespace as the controller itself. | +| `BUILD_CONTROLLER_LEASE_DURATION` | Override the `LeaseDuration`, which is the duration that non-leader candidates will wait to force acquire leadership. | +| `BUILD_CONTROLLER_RENEW_DEADLINE` | Override the `RenewDeadline`, which is the duration that the acting leader will retry refreshing leadership before giving up. | +| `BUILD_CONTROLLER_RETRY_PERIOD` | Override the `RetryPeriod`, which is the duration the LeaderElector clients should wait between tries of actions. | +| `BUILD_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the build controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | +| `BUILDRUN_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the BuildRun controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | +| `BUILDSTRATEGY_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the BuildStrategy controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | +| `CLUSTERBUILDSTRATEGY_MAX_CONCURRENT_RECONCILES` | The number of concurrent reconciles by the ClusterBuildStrategy controller. A value of 0 or lower will use the default from the [controller-runtime controller Options]. Default is 0. | +| `KUBE_API_BURST` | Burst to use for the Kubernetes API client. See [Config.Burst]. A value of 0 or lower will use the default from client-go, which currently is 10. Default is 0. | +| `KUBE_API_QPS` | QPS to use for the Kubernetes API client. See [Config.QPS]. A value of 0 or lower will use the default from client-go, which currently is 5. Default is 0. | +| `VULNERABILITY_COUNT_LIMIT` | holds vulnerability count limit if vulnerability scan is enabled for the output image. If it is defined as 10, then it will output only 10 vulnerabilities sorted by severity in the buildrun status.Output. Default is 50. | [^1]: The `runAsUser` and `runAsGroup` are dynamically overwritten depending on the build strategy that is used. See [Security Contexts](buildstrategies.md#security-contexts) for more information. diff --git a/docs/development/authentication.md b/docs/development/authentication.md index 3913a50c24..a3b198a3bb 100644 --- a/docs/development/authentication.md +++ b/docs/development/authentication.md @@ -21,7 +21,7 @@ The following document provides an introduction around the different authenticat ## Overview -There are two places where users might need to define authentication when building images. Authentication to a container registry is the most common one, but also users might have the need to define authentications for pulling source-code from Git. Overall, the authentication is done via the definition of [secrets](https://kubernetes.io/docs/concepts/configuration/secret/) in which the require sensitive data will be stored. +There are two places where users might need to define authentication when building images. Authentication to a container registry is the most common one, but also users might have the need to define authentications for pulling source-code from Git. Overall, the authentication is done via the definition of [secrets](https://kubernetes.io/docs/concepts/configuration/secret/) in which the required sensitive data will be stored. ## Build Secrets Annotation diff --git a/docs/development/testing.md b/docs/development/testing.md index 740b4aa693..f8c75c83c1 100644 --- a/docs/development/testing.md +++ b/docs/development/testing.md @@ -34,14 +34,14 @@ For all the testing levels, we rely on Gingko as the testing framework for defin ## Verifying your code -Our CI builds verify that your code conforms to project standards and that all generated code is up to date. +Our CI builds verify that your code conforms to project standards and that all generated code is up-to-date. When adding files or updating APIs, please run `make generate` before submitting your code. ### Counterfeiter Counterfeiter is used to generate and update fake implementations of objects. Currently only used for the `manager` and `client` package interface of the `sigs.k8s.io/controller-runtime`. -This allow us to use test doubles in the unit tests, from where we can instantiate the fakes and then stub return values. This is very useful, for example we can mock all **client** calls that happened during the k8s controllers reconciliation and stub the result. Same case for the **manager** when creating controllers. +This allows us to use test doubles in the unit tests, from where we can instantiate the fakes and then stub return values. This is very useful, for example we can mock all **client** calls that happened during the k8s controllers reconciliation and stub the result. Same case for the **manager** when creating controllers. Counterfeiter is required by the code generator scripts. Run `make install-counterfeiter` to add counterfeiter to your `GOPATH`. @@ -67,11 +67,11 @@ Unit tests are designed based on the following: - Unit tests must pass in different OS distributions( e.g. linux, macOS ). - Unit tests should be run in parallel. -Because we use Ginkgo for this, each controller [package](https://github.com/shipwright-io/build/tree/main/pkg/reconciler) requires a `suite_test.go` file and a relative controller test file. You can generate a suite by running `ginkgo bootstrap` under the package directory. For testing an specific controller class, you can generate the testing class by running `ginkgo generate` under the package directory. +Because we use Ginkgo for this, each controller [package](https://github.com/shipwright-io/build/tree/main/pkg/reconciler) requires a `suite_test.go` file and a relative controller test file. You can generate a suite by running `ginkgo bootstrap` under the package directory. For testing a specific controller class, you can generate the testing class by running `ginkgo generate` under the package directory. When building unit-tests, try to follow: -- Test DRY. Therefore we use the `catalog.go` helper class under the `test` directory, to avoid code repetition. +- Test DRY. Therefore, we use the `catalog.go` helper class under the `test` directory, to avoid code repetition. - Use counterfeiter to generate fakes. - Tests happen on a separate `_test` file. - Assert all errors. @@ -114,12 +114,12 @@ We use e2e tests as the last signal to ensure the controllers behaviour in the c The following table contains a set of environment variables that control the behavior of the e2e tests. -| Environment Variable | Default | Description | -|---------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------| -| `TEST_NAMESPACE` | `default` | Target namespace to execute tests upon, default is `default`. | +| Environment Variable | Default | Description | +|---------------------------------|--------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `TEST_NAMESPACE` | `default` | Target namespace to execute tests upon, default is `default`. | | `TEST_E2E_FLAGS` | `-failFast -flakeAttempts=2 -p -randomizeAllSpecs -slowSpecThreshold=300 -timeout=20m -trace -v` | Ginkgo flags. See all Ginkgo flags here: [The Ginkgo CLI](https://onsi.github.io/ginkgo/#the-ginkgo-cli). Especially of interest are `--focus` and `--skip` to run selective tests. | -| `TEST_E2E_TIMEOUT_MULTIPLIER` | `1` | Multiplier for timeouts in the e2e tests to run them on slower systems. | -| `TEST_E2E_VERIFY_TEKTONOBJECTS` | `true` | Boolean, if false, the verification code will not try to verify the TaskRun object status | +| `TEST_E2E_TIMEOUT_MULTIPLIER` | `1` | Multiplier for timeouts in the e2e tests to run them on slower systems. | +| `TEST_E2E_VERIFY_TEKTONOBJECTS` | `true` | Boolean, if false, the verification code will not try to verify the TaskRun object status | ### Build override parameters @@ -144,20 +144,20 @@ When both `TEST_IMAGE_REPO_SECRET` and `TEST_IMAGE_REPO_DOCKERCONFIGJSON` are in The following table contains a list of environment variables that will override specific paths under the **BuildRun** CRD. -| Environment Variable | Path | Description | -|--------------------------------|-----------------------|-------------| +| Environment Variable | Path | Description | +|--------------------------------|-----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `TEST_E2E_SERVICEACCOUNT_NAME` | `spec.serviceAccount` | The name of the service account used by the build runs, the code will try to create the service account but not fail if it already exists. Special value is `generated`, which will lead to using the auto-generation feature for each build run. | ### Private Git override parameters End-to-end tests can also be executed with the context of private Git repositories, using the following environment variables to configure it. -| Environment Variable | Path | Description | -|-----------------------|--------------------------------|---------------------------------------| -| `TEST_PRIVATE_REPO` | _none_ | Enable private repository e2e tests | -| `TEST_PRIVATE_GITHUB` | `spec.source.git.url` | Private URL, like `git@github.com` | -| `TEST_PRIVATE_GITLAB` | `spec.source.git.url` | Private URL, like `git@gitlab.com` | -| `TEST_SOURCE_SECRET` | `spec.source.git.cloneSecret` | Private repository credentials | +| Environment Variable | Path | Description | +|-----------------------|-------------------------------|-------------------------------------| +| `TEST_PRIVATE_REPO` | _none_ | Enable private repository e2e tests | +| `TEST_PRIVATE_GITHUB` | `spec.source.git.url` | Private URL, like `git@github.com` | +| `TEST_PRIVATE_GITLAB` | `spec.source.git.url` | Private URL, like `git@gitlab.com` | +| `TEST_SOURCE_SECRET` | `spec.source.git.cloneSecret` | Private repository credentials | On using `TEST_SOURCE_SECRET`, the environment variable must contain the name of the Kubernetes Secret containing SSH private key, for given private Git repository. See the [docs](authentication.md) for more information about authentication methods in the Build. diff --git a/docs/metrics.md b/docs/metrics.md index b4506504ed..649fc81e9d 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -27,7 +27,7 @@ Following build metrics are exposed on port `8383`. Environment variables can be set to use custom buckets for the histogram metrics: | Metric | Environment variable | Default | -| ---------------------------------------------------- | ---------------------------------- | ---------------------------------------- | +|------------------------------------------------------|------------------------------------|------------------------------------------| | `build_buildrun_establish_duration_seconds` | `PROMETHEUS_BR_EST_DUR_BUCKETS` | `0,1,2,3,5,7,10,15,20,30` | | `build_buildrun_completion_duration_seconds` | `PROMETHEUS_BR_COMP_DUR_BUCKETS` | `50,100,150,200,250,300,350,400,450,500` | | `build_buildrun_rampup_duration_seconds` | `PROMETHEUS_BR_RAMPUP_DUR_BUCKETS` | `0,1,2,3,4,5,6,7,8,9,10` | @@ -41,7 +41,7 @@ export PROMETHEUS_BR_COMP_DUR_BUCKETS=30,60,90,120,180,240,300,360,420,480 make local ``` -When you deploy the build controller in a Kubernetes cluster, you need to extend the `spec.containers[0].spec.env` section of the sample deployment file, [controller.yaml](../deploy/500-controller.yaml). Add an additional entry: +When you deploy the build controller in a Kubernetes cluster, you need to extend the `spec.containers[0].spec.env` section of the sample deployment file, [controller.yaml](../deploy/500-controller.yaml). Add another entry: ```yaml [...] @@ -74,7 +74,7 @@ export PROMETHEUS_ENABLED_LABELS=buildstrategy,namespace,build make local ``` -When you deploy the build controller in a Kubernetes cluster, you need to extend the `spec.containers[0].spec.env` section of the sample deployment file, [controller.yaml](../deploy/controller.yaml). Add an additional entry: +When you deploy the build controller in a Kubernetes cluster, you need to extend the `spec.containers[0].spec.env` section of the sample deployment file, [controller.yaml](../deploy/500-controller.yaml). Add another entry: ```yaml [...] diff --git a/docs/profiling.md b/docs/profiling.md index 9be18d3213..e1f48dba05 100644 --- a/docs/profiling.md +++ b/docs/profiling.md @@ -20,7 +20,7 @@ kubectl --namespace set image \ ## Connect `go pprof` to build controller -Depending on the respective setup, there could be multiple build controller pods for high availability reasons. In this case, you have to look-up the current leader first. The following command can be used to verify the currently active leader: +Depending on the respective setup, there could be multiple build controller pods for high availability reasons. In this case, you have to look up the current leader first. The following command can be used to verify the currently active leader: ```sh kubectl --namespace get configmap shipwright-build-controller-lock --output json \ @@ -34,7 +34,7 @@ The `pprof` endpoint is not exposed in the cluster and can only be used from ins kubectl --namespace port-forward 8383:8383 ``` -Now, you can setup a local webserver to browse through the profiling data. +Now, you can set up a local webserver to browse through the profiling data. ```sh go tool pprof -http localhost:8080 http://localhost:8383/debug/pprof/heap diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md index cb93a3261c..cecf87f182 100644 --- a/docs/tutorials/README.md +++ b/docs/tutorials/README.md @@ -5,7 +5,7 @@ SPDX-License-Identifier: Apache-2.0 --> # Shipwright Build Tutorial -So you just successfully built a container image via the [Try It!](../../README.md#try-it) section and you want to know more? +So you just successfully built a container image via the [Try It!](../../README.md#try-it) section, and you want to know more? At Shipwright, we've spent a lot of time trying to figure out the best ways to simplify the experience when building container images. Shipwright provides an alternative for securely building container images in your Kubernetes cluster. @@ -20,14 +20,14 @@ What if we could ## Concepts -| Concept | Description | -| ----------- | ----------- | -| **`Strategy`** | Refers to a particular tool that will be used when building a container image, such as Kaniko, Buildah, ko, etc. | -| **`Build`** | Resource used to define a build configuration. | -| **`BuildRun`** | Resource used to start the image build mechanism. | -| **`BuildStrategy` / `ClusterBuildStrategy`** | Resource that holds a template that dictates how to build via a particular strategy. | -| **`Dockerfile-less strategy`** | Is a category given to strategies that can build container images from source code, without the notion of a Dockerfile. | -| **`Dockerfile-based strategy`** | Is a category given to strategies that can build container images from source code, with a reference to a Dockerfile. | +| Concept | Description | +|----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------| +| **`Strategy`** | Refers to a particular tool that will be used when building a container image, such as Kaniko, Buildah, ko, etc. | +| **`Build`** | Resource used to define a build configuration. | +| **`BuildRun`** | Resource used to start the image build mechanism. | +| **`BuildStrategy` / `ClusterBuildStrategy`** | Resource that holds a template that dictates how to build via a particular strategy. | +| **`Dockerfile-less strategy`** | Is a category given to strategies that can build container images from source code, without the notion of a Dockerfile. | +| **`Dockerfile-based strategy`** | Is a category given to strategies that can build container images from source code, with a reference to a Dockerfile. | With the above concepts in mind, lets see how they all play together. @@ -55,13 +55,13 @@ For more information about strategies see the related [docs](../buildstrategies. Depending on your source code you might want to try a specific example. The following table serves as a guide to help you understand which strategy to choose: -| Sample code | Repository | ContextDir | Strategy Type | Strategy to use | -| ----------- | ----------- | ------------- | ------------- | ------------- | -| A go app with a Dockerfile | [shipwright-io/sample-go](https://github.com/shipwright-io/sample-go) | `/docker-build` | Dockerfile-based | Kaniko, BuildKit, Buildah | -| A go app | [shipwright-io/sample-go](https://github.com/shipwright-io/sample-go) | `/source-build` | Dockerfile-less | buildpacks-v3, buildpacks-v3-heroku | -| A ruby app | [shipwright-io/sample-ruby](https://github.com/shipwright-io/sample-ruby) | `/source-build` | Dockerfile-less | buildpacks-v3, buildpacks-v3-heroku | -| A java app with a Dockerfile | [shipwright-io/sample-java](https://github.com/shipwright-io/sample-java) | `/docker-build` | Dockerfile-based | Kaniko, BuildKit, Buildah | -| Shipwright/Build | [shipwright-io/build](https://github.com/shipwright-io/build) | `/cmd/shipwright-build-controller` | Dockerfile-less | ko | +| Sample code | Repository | ContextDir | Strategy Type | Strategy to use | +|------------------------------|---------------------------------------------------------------------------|------------------------------------|------------------|-------------------------------------| +| A go app with a Dockerfile | [shipwright-io/sample-go](https://github.com/shipwright-io/sample-go) | `/docker-build` | Dockerfile-based | Kaniko, BuildKit, Buildah | +| A go app | [shipwright-io/sample-go](https://github.com/shipwright-io/sample-go) | `/source-build` | Dockerfile-less | buildpacks-v3, buildpacks-v3-heroku | +| A ruby app | [shipwright-io/sample-ruby](https://github.com/shipwright-io/sample-ruby) | `/source-build` | Dockerfile-less | buildpacks-v3, buildpacks-v3-heroku | +| A java app with a Dockerfile | [shipwright-io/sample-java](https://github.com/shipwright-io/sample-java) | `/docker-build` | Dockerfile-based | Kaniko, BuildKit, Buildah | +| Shipwright/Build | [shipwright-io/build](https://github.com/shipwright-io/build) | `/cmd/shipwright-build-controller` | Dockerfile-less | ko | _Note_: `ContextDir` is the path under the repository where the source code is located.