Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

standarize images location #12129

Closed
wants to merge 27 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
4276edc
standarize images location
akiioto Oct 11, 2024
f5ffadc
gomod(deps): bump github.com/zricethezav/gitleaks/v8 (#12096)
dependabot[bot] Oct 7, 2024
f269c19
tf(deps): bump hashicorp/google in /configs/terraform/core (#12097)
dependabot[bot] Oct 7, 2024
5811af2
Bumping test-infra and testimages (#12098)
kyma-bot Oct 7, 2024
b538b92
Remove gcr-cleaner prow jobs (#12074)
KacperMalachowski Oct 8, 2024
518f069
Fix dead links in the documentation (#12095)
Sawthis Oct 8, 2024
3aaec90
Bumping sec-scanners-config.yaml (#12099)
kyma-bot Oct 8, 2024
c23ed86
Bumping test-infra and testimages (#12101)
kyma-bot Oct 8, 2024
a864c7f
Hello world (#12103)
szumejker Oct 8, 2024
535f556
slack-msg-sender(deps): bump python (#12106)
dependabot[bot] Oct 8, 2024
99f5ab8
alpine-git-gke-aws-auth(deps): bump google/cloud-sdk (#12105)
dependabot[bot] Oct 8, 2024
2224609
Bumping test-infra and testimages (#12107)
kyma-bot Oct 8, 2024
901a854
Bumping sec-scanners-config.yaml (#12108)
kyma-bot Oct 8, 2024
04bfded
Bumping test-infra and testimages (#12109)
kyma-bot Oct 8, 2024
a6e7d14
Add autobump docs github action (#12112)
szumejker Oct 9, 2024
c084ce3
Add GitHub action md index (#12113)
szumejker Oct 9, 2024
ba7b525
Bumping index.md (#12114)
kyma-bot Oct 9, 2024
d923e16
Add GitHub action md index (#12115)
szumejker Oct 9, 2024
dcef5b6
gomod(deps): bump cloud.google.com/go/pubsub from 1.43.0 to 1.44.0 (#…
dependabot[bot] Oct 9, 2024
5263270
Bumping test-infra and testimages (#12119)
kyma-bot Oct 9, 2024
061ff9b
Bumping sec-scanners-config.yaml (#12120)
kyma-bot Oct 9, 2024
2855bd9
Add schedule to supported signing events in image builder README file…
KacperMalachowski Oct 10, 2024
5e6017d
gomod(deps): bump google.golang.org/api from 0.199.0 to 0.200.0 (#12124)
dependabot[bot] Oct 10, 2024
5325867
Bumping test-infra and testimages (#12125)
kyma-bot Oct 10, 2024
e48dc99
Bumping sec-scanners-config.yaml (#12126)
kyma-bot Oct 10, 2024
6c9d4af
Add image autobumper to the github workflows (#12127)
KacperMalachowski Oct 11, 2024
7f216b2
Rename autobump token secret for github (#12128)
KacperMalachowski Oct 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/autobump-docs-index-md.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: autobump-docs-index-md.yml

on:
push:
branches:
- main
paths:
- '**/*.md'
workflow_dispatch: {}

env:
AUTOBUMP_CONFIG_PATH: configs/autobump-config/test-infra-markdown-index-autobump-config.yaml

jobs:
autobump:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
concurrency:
group: post-test-infra-markdown-index-autobump
cancel-in-progress: false

steps:
- uses: actions/checkout@v4

- name: Setup git config
run: |
GIT_USERNAME=$(grep "gitName" ${{ env.AUTOBUMP_CONFIG_PATH }} | cut -d '"' -f 2)
GIT_EMAIL=$(grep "gitEmail" ${{ env.AUTOBUMP_CONFIG_PATH }} | cut -d '"' -f 2)
git config user.name "$GIT_USERNAME"
git config user.email "$GIT_EMAIL"

git config --unset-all http.https://github.com/.extraheader

- name: Authenticate in GCP
id: 'auth'
uses: 'google-github-actions/auth@v2'
with:
project_id: ${{ vars.GCP_KYMA_PROJECT_PROJECT_ID }}
workload_identity_provider: ${{ vars.GH_COM_KYMA_PROJECT_GCP_WORKLOAD_IDENTITY_FEDERATION_PROVIDER }}

- name: Get kyma bot token from Secret Manager
id: 'secrets'
uses: 'google-github-actions/get-secretmanager-secrets@v2'
with:
secrets: |-
kyma-autobump-token:${{ vars.GCP_KYMA_PROJECT_PROJECT_ID }}/${{ vars.KYMA_AUTOBUMP_BOT_GITHUB_SECRET_NAME }}

- name: Store Github Token for autobumper
run: |
echo "${{ steps.secrets.outputs.kyma-autobump-token }}" > ~/token

- name: Run markdown index autobump
run: |
docker run --rm \
-v ~/token:/etc/github/token:ro \
-v ${{ github.workspace }}:/github/test-infra \
--workdir /github/test-infra \
--user $UID \
europe-docker.pkg.dev/kyma-project/prod/markdown-index:v20241007-aa6bbc21 \
--config=${{ env.AUTOBUMP_CONFIG_PATH }} \
--labels-override=kind/chore,area/documentation
10 changes: 10 additions & 0 deletions .github/workflows/autobump-images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
name: autobump-images

on:
schedule:
- cron: 45 * * * 1-5
workflow_dispatch:

jobs:
autobump:
uses: kyma-project/test-infra/.github/workflows/reusable-image-autobumper.yml@main
24 changes: 12 additions & 12 deletions .github/workflows/buildx-images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ on:
pull_request_target:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- "images/**"
- "cmd/images/**"
- ".github/workflows/image-builder.yml"
push:
branches:
- main
paths:
- "images/**"
- "cmd/images/**"
- ".github/workflows/image-builder.yml"
workflow_dispatch: {}

Expand All @@ -20,7 +20,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: alpine
dockerfile: images/alpine/Dockerfile
dockerfile: cmd/images/alpine/Dockerfile
context: .

unpack-alpine:
Expand All @@ -42,7 +42,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: alpine-git
dockerfile: images/alpine/git/Dockerfile
dockerfile: cmd/images/alpine/git/Dockerfile
context: .
build-args: BASE_ALPINE_IMAGE=${{ needs.unpack-alpine.outputs.clean_image }}

Expand All @@ -52,7 +52,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: alpine-git-gke-aws-auth
dockerfile: images/alpine/git/gke-aws-auth/Dockerfile
dockerfile: cmd/images/alpine/git/gke-aws-auth/Dockerfile
context: .
build-args: BASE_ALPINE_IMAGE=${{ needs.unpack-alpine.outputs.clean_image }}

Expand All @@ -61,7 +61,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: buildpack
dockerfile: images/buildpack/Dockerfile
dockerfile: cmd/images/buildpack/Dockerfile
context: .

unpack-buildpack:
Expand All @@ -83,7 +83,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: buildpack-go
dockerfile: images/buildpack/go/Dockerfile
dockerfile: cmd/images/buildpack/go/Dockerfile
context: .
build-args: BASE_BUILDPACK_IMAGE=${{ needs.unpack-buildpack.outputs.clean_image }}

Expand All @@ -92,7 +92,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: unified-agent
dockerfile: images/unified-agent/Dockerfile
dockerfile: cmd/images/unified-agent/Dockerfile
context: .

unpack-unified-agent:
Expand All @@ -114,7 +114,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: unified-agent-go
dockerfile: images/unified-agent/go/Dockerfile
dockerfile: cmd/images/unified-agent/go/Dockerfile
context: .
build-args: BASE_UNIFIED_AGENT_IMAGE=${{ needs.unpack-unified-agent.outputs.clean_image }}

Expand All @@ -124,7 +124,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: unified-agent-nodejs
dockerfile: images/unified-agent/nodejs/Dockerfile
dockerfile: cmd/images/unified-agent/nodejs/Dockerfile
context: .
build-args: BASE_UNIFIED_AGENT_IMAGE=${{ needs.unpack-unified-agent.outputs.clean_image }}

Expand All @@ -134,7 +134,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: unified-agent-python
dockerfile: images/unified-agent/python/Dockerfile
dockerfile: cmd/images/unified-agent/python/Dockerfile
context: .
build-args: BASE_UNIFIED_AGENT_IMAGE=${{ needs.unpack-unified-agent.outputs.clean_image }}

Expand All @@ -143,5 +143,5 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: e2e-gcloud
dockerfile: images/e2e-gcloud/Dockerfile
dockerfile: cmd/images/e2e-gcloud/Dockerfile
context: .
52 changes: 52 additions & 0 deletions .github/workflows/hello-world.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: hello-world
on: workflow_dispatch

env:
AUTOBUMP_CONFIG_PATH: configs/autobump-config/test-infra-markdown-index-autobump-config.yaml

jobs:
autobump:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
concurrency:
group: post-test-infra-markdown-index-autobump
cancel-in-progress: false

steps:
- uses: actions/checkout@v4

- name: Setup git config
run: |
GIT_USERNAME=$(grep "gitName" ${{ env.AUTOBUMP_CONFIG_PATH }} | cut -d '"' -f 2)
GIT_EMAIL=$(grep "gitEmail" ${{ env.AUTOBUMP_CONFIG_PATH }} | cut -d '"' -f 2)
git config user.name "$GIT_USERNAME"
git config user.email "$GIT_EMAIL"
git config --unset-all http.https://github.com/.extraheader
- name: Authenticate in GCP
id: 'auth'
uses: 'google-github-actions/auth@v2'
with:
project_id: ${{ vars.GCP_KYMA_PROJECT_PROJECT_ID }}
workload_identity_provider: ${{ vars.GH_COM_KYMA_PROJECT_GCP_WORKLOAD_IDENTITY_FEDERATION_PROVIDER }}

- name: Get kyma bot token from Secret Manager
id: 'secrets'
uses: 'google-github-actions/get-secretmanager-secrets@v2'
with:
secrets: |-
kyma-autobump-token:${{ vars.GCP_KYMA_PROJECT_PROJECT_ID }}/${{ vars.KYMA_AUTOBUMP_BOT_GITHUB_SECRET_NAME }}
- name: Store Github Token for autobumper
run: |
echo "${{ steps.secrets.outputs.kyma-autobump-token }}" > ~/token
- name: Run markdown index autobump
run: |
docker run --rm \
-v ~/token:/etc/github/token:ro \
-v ${{ github.workspace }}:/github/workspace \
--workdir /github/workspace \
--user $UID \
europe-docker.pkg.dev/kyma-project/prod/markdown-index:v20241007-aa6bbc21 \
--config=${{ env.AUTOBUMP_CONFIG_PATH }} \
--labels-override=kind/chore,area/documentation
2 changes: 1 addition & 1 deletion .github/workflows/image-builder.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
description: Path to the dockerfile used to build docker image
required: false
type: string
default: "prow/images/ginkgo/Dockerfile"
default: "cmd/images/ginkgo/Dockerfile"
context:
description: Build context to build container from
required: false
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pull-image-builder-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
uses: ./.github/workflows/image-builder.yml
with:
name: test-infra/ginkgo
dockerfile: prow/images/ginkgo/Dockerfile
dockerfile: cmd/images/ginkgo/Dockerfile
context: .
env-file: "envs"
tags: ${{ needs.compute-tag.outputs.tag }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pull-validate-kaniko-build-config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,4 @@ jobs:
- uses: docker://europe-docker.pkg.dev/kyma-project/prod/image-builder:v20240621-63f4f2b1
id: build
with:
args: --name=test-infra/ginkgo --context=. --dockerfile=prow/images/ginkgo/Dockerfile --azure-access-token=${{ steps.secrets.outputs.ado-pat }} --oidc-token=${{ steps.get_oidc.outputs.jwt }} --env-file='envs' --build-in-ado=true --test-kaniko-build-config=true --config="./configs/image-builder-client-config.yaml"
args: --name=test-infra/ginkgo --context=. --dockerfile=cmd/images/ginkgo/Dockerfile --azure-access-token=${{ steps.secrets.outputs.ado-pat }} --oidc-token=${{ steps.get_oidc.outputs.jwt }} --env-file='envs' --build-in-ado=true --test-kaniko-build-config=true --config="./configs/image-builder-client-config.yaml"
2 changes: 1 addition & 1 deletion .github/workflows/push-update-security-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ jobs:
--workdir /github/workspace \
--rm \
--user $UID \
europe-docker.pkg.dev/kyma-project/prod/image-detector:v20241007-36d14dd7 \
europe-docker.pkg.dev/kyma-project/prod/image-detector:v20241010-d3f08490 \
--terraform-dir=${{ env.TERRAFORM_CONFIGS_DIR }} \
--sec-scanner-config=${{ env.SEC_SCANNERS_CONFIG_PATH }} \
--autobump-config=${{ env.AUTOBUMP_CONFIG_PATH }}
66 changes: 66 additions & 0 deletions .github/workflows/reusable-image-autobumper.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: reusable-image-autobumper
on:
workflow_call:
inputs:
docker-image:
description: 'Docker image with tag to be used'
default: 'europe-docker.pkg.dev/kyma-project/prod/image-autobumper:v20240927-bc42bcd3'
type: string

env:
AUTOBUMP_CONFIG_PATH: configs/image-autobumper-config/image-autobumper-config.yaml

jobs:
autobump:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Configure Workload Identity Federation
id: auth
uses: google-github-actions/auth@v2
with:
project_id: ${{ vars.GCP_KYMA_PROJECT_PROJECT_ID }}
workload_identity_provider: ${{ vars.GH_COM_KYMA_PROJECT_GCP_WORKLOAD_IDENTITY_FEDERATION_PROVIDER }}

- name: Access Google Cloud Secret
id: access-secret
uses: google-github-actions/get-secretmanager-secrets@v2
with:
secrets: |
kyma-autobump-token:${{ vars.GCP_KYMA_PROJECT_PROJECT_ID }}/${{ vars.KYMA_AUTOBUMP_BOT_GITHUB_SECRET_NAME }}

- name: Store Github Token for autobumper
run: |
echo ${{ steps.secrets.outputs.kyma-autobump-token }} > ~/token
chmod 644 ~/token

- name: Checkout repository
with:
token: ${{ steps.secrets.outputs.kyma-autobump-token }}
uses: actions/checkout@v4

# Setup git config with commiter data from config
# Prevent silent passing github token
# see https://stackoverflow.com/a/69979203/23148781
- name: Setup git config
run: |
GIT_USERNAME=$(grep "gitName" ${{ env.AUTOBUMP_CONFIG_PATH }} | cut -d '"' -f 2)
GIT_EMAIL=$(grep "gitEmail" ${{ env.AUTOBUMP_CONFIG_PATH }} | cut -d '"' -f 2)
git config user.name $GIT_USERNAME
git config user.email $GIT_EMAIL

git config --unset-all http.https://github.com/.extraheader

- name: Run Docker container
run: |
docker run \
--rm \
--cap-drop=ALL \
--privileged \
-v "${{ github.workspace }}:/workspace" \
-v "~/token:/tmp/github_token:ro" \
-w /workspace \
${{ inputs.docker-image }} \
--autobump-config=${{ env.AUTOBUMP_CONFIG_PATH }}
2 changes: 1 addition & 1 deletion cmd/cloud-run/slack-message-sender/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.12.7-alpine3.20
FROM python:3.13.0-alpine3.20

# Allow statements and log messages to immediately appear in the Knative logs
ENV PYTHONUNBUFFERED True
Expand Down
5 changes: 3 additions & 2 deletions cmd/image-builder/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
uses: kyma-project/test-infra/.github/workflows/image-builder.yml@main # Usage: kyma-project/test-infra/.github/workflows/image-builder.yml@main
with:
name: test-infra/ginkgo
dockerfile: prow/images/ginkgo/Dockerfile
dockerfile: cmd/images/ginkgo/Dockerfile
context: .
env-file: "envs"
tags: ${{ needs.compute-tag.outputs.tag }}
Expand Down Expand Up @@ -95,6 +95,7 @@ The Image Builder reusable workflow supports the following GitHub events to trig
* `push` - to build images on push to the specified branch.
* `pull_request_target` - to build images on pull requests.
* `workflow_dispatch` - to manually trigger the workflow.
* `schedule` - to build images on a regular basis.

## Reusable Workflow Reference

Expand Down Expand Up @@ -146,7 +147,7 @@ By default, Image Builder signs images with the production signify service.
Image signing allows verification that the image comes from a trusted repository and has not been altered in the meantime.

> [!NOTE]
> Image Builder signs images built on the push and workflow_dispatch events only. Images built on the pull_request_target event are not signed.
> Image Builder only signs images built on the `push`, `schedule`, and `workflow_dispatch` events. Images built on the `pull_request_target` event are not signed.

## Image Signing with Signify

Expand Down
19 changes: 0 additions & 19 deletions images/README.md → cmd/images/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,3 @@ To add additional applications into the images, open a pull request (PR) with ch
* Always build from a source to ensure compiler vulnerabilities do not affect the resulting binary.
* Link the binary to a specific version so that it's easier to update when necessary.
* Build binaries in a separate stage, then copy the resulting binary into the final image to ensure images are small and contain the least number of layers.

## Write Image Tests

To write simple smoke tests with your image, add an executable file called `test.sh`.
The scripts should contain all steps that perform basic or advanced test operations against the image.
The test script must exit with a non-zero number if any steps have failed.

By default, current context of a test script is always Docker build context. Image name is passed as a variable `IMG`.

### Example

The example below showcases the example definition of the `test.sh` script.
```shell
#!/usr/bin/env bash
set -e
echo "$IMG"
docker run --rm $IMG -- some-command
test $? -eq 0 || exit 1
```
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM google/cloud-sdk:495.0.0 AS base
FROM google/cloud-sdk:496.0.0 AS base

ARG AWS_IAM_AUTHENTICATOR_VERSION="0.6.11"
RUN curl -fsSL \
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loading
Loading