Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add DockerHub and ScaleWay registries #85

Closed
wants to merge 14 commits into from
Closed

feat: add DockerHub and ScaleWay registries #85

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
de274f5
feat: add 2 new registries
Bidon15 Oct 17, 2023
1c3bee6
chore: add .gitignore
Bidon15 Oct 19, 2023
2822ece
chore: no more ide folders
Bidon15 Oct 19, 2023
60b5c53
chore: fix no cache for security
Bidon15 Oct 19, 2023
ea3b0cd
chore: add more checkers for package namings. Remove cache from amd64…
Bidon15 Oct 20, 2023
c0a3358
Update .github/workflows/reusable_dockerfile_pipeline.yml
Bidon15 Oct 20, 2023
68f6dbc
Update .github/workflows/reusable_dockerfile_pipeline.yml
Bidon15 Oct 20, 2023
e7285a6
feat: add a simple DockerFile and runner test
Bidon15 Oct 20, 2023
4a3a7c8
Update Dockerfile new line
Bidon15 Oct 20, 2023
14d1f7f
Revert "feat: add a simple DockerFile and runner test"
Bidon15 Nov 14, 2023
1fb6926
fix: amend the DockerFile addition
Bidon15 Nov 14, 2023
d2f7700
fix: remove redundant checks
Bidon15 Nov 14, 2023
60892e7
Merge branch 'main' into three-registries
Bidon15 Nov 14, 2023
4a35997
fix: protect from repoonwer+reponame as an input.packageName
Bidon15 Nov 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 99 additions & 56 deletions .github/workflows/reusable_dockerfile_pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,15 @@ on:
default: "${{ github.repository }}"

env:
REGISTRY: ghcr.io
GITHUB_REG: ghcr.io
MAINTAINER: ${{ github.repository_owner }}
DESCRIPTION: "${{ github.repository_owner }} repository ${{ github.repository }}"

jobs:
prepare-env:
runs-on: "ubuntu-latest"
outputs:
repo_owner: ${{ steps.setting_env.outputs.repo_owner }}
output_short_sha: ${{ steps.setting_env.outputs.short_sha }}
output_image_name: ${{ steps.setting_env.outputs.image_name }}
build_for_pr: ${{ steps.setting_logic.outputs.build_for_pr }}
Expand All @@ -35,25 +36,53 @@ jobs:
- name: Add vars to ENV
id: setting_env
run: |
echo "SHORT_SHA=`echo ${GITHUB_SHA} | cut -c1-8`" >> $GITHUB_ENV
echo "short_sha=`echo ${GITHUB_SHA} | cut -c1-8`" >> "$GITHUB_OUTPUT"
# yamllint disable
echo "IMAGE_NAME=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
echo "image_name=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
# here we validate if we have specified a different package name in
# the inputs, if so, we change the package to it.
if [[ ${{ inputs.packageName }} != ${{ github.repository}} ]];then
# validate the input package name characters
if [[ ! "${{ inputs.packageName }}" =~ ^[A-Za-z0-9\-]+$ ]]; then
echo "------------------------------------------------------------"
echo "ERROR: Package name not valid! => [ ${{ inputs.packageName }} ]"
echo "ONLY can use: A-Za-z0-9\-"
echo "------------------------------------------------------------"
exit 1
# Extract both the repository owner and repository name
# According to docs, github.repository is in the format of owner/repo, not just repo
# https://docs.github.com/en/actions/learn-github-actions/contexts#github-context
REPO_OWNER=$(echo "${{ github.repository }}" | awk -F'/' '{print $1}' | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | awk -F'/' '{print $2}' | tr '[:upper:]' '[:lower:]')
MSevey marked this conversation as resolved.
Show resolved Hide resolved

echo "Repository Owner: $REPO_OWNER"
echo "Repository Name: $REPO_NAME"

echo "REPO_OWNER=$REPO_OWNER" >> $GITHUB_ENV
echo "repo_owner=$REPO_OWNER" >> "$GITHUB_OUTPUT"

echo "SHORT_SHA=`echo ${GITHUB_SHA} | cut -c1-8`" >> $GITHUB_ENV
echo "short_sha=`echo ${GITHUB_SHA} | cut -c1-8`" >> "$GITHUB_OUTPUT"
# yamllint disable

# Set the default package name to the repository name
PACKAGE_NAME=$REPO_NAME
MSevey marked this conversation as resolved.
Show resolved Hide resolved

# If a user submitted package name that has the naming containing
# both the repository owner and repository name, we fail
# e.g: inputs.packageName = "celestiaorg/celestiaorg" is not allowed
if [[ ! "${{ inputs.packageName }}" == "${{ github.repository }}" ]]; then
echo "------------------------------------------------------------"
echo "ERROR: Package name not valid! => [ $PACKAGE_NAME} ]"
echo "Don't use the repository owner and repository name in the package name."
echo "------------------------------------------------------------"
exit 1
fi
MSevey marked this conversation as resolved.
Show resolved Hide resolved

# If there is a user submitted package name, use it
if [[ ! "${{ inputs.packageName }}" != "" ]]; then
PACKAGE_NAME=$(echo "${{ inputs.packageName }}" | tr '[:upper:]' '[:lower:]')
fi

# validate the package name characters
if [[ ! $PACKAGE_NAME =~ ^[A-Za-z0-9\-]+$ ]]; then
echo "------------------------------------------------------------"
echo "ERROR: Package name not valid! => [ $PACKAGE_NAME} ]"
echo "ONLY can use: A-Za-z0-9\-"
echo "------------------------------------------------------------"
exit 1
fi

echo "IMAGE_NAME=$PACKAGE_NAME" >> $GITHUB_ENV
echo "image_name=$PACKAGE_NAME" >> "$GITHUB_OUTPUT"
fi
Bidon15 marked this conversation as resolved.
Show resolved Hide resolved
echo "IMAGE_NAME=$(echo ${{ github.repository_owner }}/${{ inputs.packageName }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
echo "image_name=$(echo ${{ github.repository_owner }}/${{ inputs.packageName }} | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
fi
# yamllint enable

# The key logic that we want to determine is whether or not we are working
Expand Down Expand Up @@ -135,23 +164,42 @@ jobs:
severity: "CRITICAL,HIGH"

docker-build:
name: docker-build (${{ matrix.registry.name }}; ${{ matrix.registry.registry-url }}/${{ matrix.registry.registry-owner }}/${{ needs.prepare-env.outputs.output_image_name }}:${{ needs.prepare-env.outputs.output_short_sha }})
runs-on: "ubuntu-latest"
# wait until the jobs are finished.
needs: ["prepare-env", "logic-check", "docker-security"]
permissions:
contents: write
packages: write

strategy:
matrix:
registry:
- name: DockerHub
user-secret: DOCKERHUB_USERNAME
token-secret: DOCKERHUB_TOKEN
registry-url: docker.io
registry-owner: celestiaorg
- name: GHCR
user-secret: ${{ github.repository_owner }}
token-secret: GITHUB_TOKEN
registry-url: ${{ env.GITHUB_REG }}
registry-owner: ${{ needs.prepare-env.outputs.repo_owner }}
- name: ScaleWay
user-secret: SCALEWAY_USERNAME
token-secret: SCW_SECRET_KEY
registry-url: rg.fr-par.scw.cloud
registry-owner: celestiaorg
fail-fast: false
steps:
- name: Checkout
uses: "actions/checkout@v4"

- name: Login to GHCR
- name: Login to ${{ matrix.registry.name }}
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: ${{ matrix.registry.registry-url }}
username: ${{ matrix.registry.registry-url == env.GITHUB_REG && matrix.registry.user-secret || secrets[matrix.registry.user-secret] }}
password: ${{ secrets[matrix.registry.token-secret] }}

- name: Extract Docker Metadata
id: meta
Expand All @@ -160,12 +208,12 @@ jobs:
OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
with:
images: ${{ env.REGISTRY }}/${{ env.OUTPUT_IMAGE_NAME }}
images: ${{ env.OUTPUT_IMAGE_NAME }}
# yamllint disable
labels: |
maintainer=${{ env.MAINTAINER }}
commitUrl=https://github.com/${{ github.repository }}/commit/${{ github.sha }}
dockerPull=docker pull ${{ env.REGISTRY }}/${{ github.repository }}:${{ env.OUTPUT_SHORT_SHA }}
dockerPull=docker pull ${{ env.GITHUB_REG }}/${{ github.repository }}:${{ env.OUTPUT_SHORT_SHA }}
org.opencontainers.image.description=${{ env.DESCRIPTION }}
tags: |
# output minimal (short sha)
Expand All @@ -176,55 +224,50 @@ jobs:
type=ref,enable=true,prefix=pr-,suffix=,event=pr
# yamllint enable

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# Build amd64 images always, and publish when it is not a fork. The Github
# security model prevents forks from pushing to the registry so we can
# only push if the branch/PR is not generated from a fork. Even though
# forks can't push, we still want to try and build the image to catch
# bugs. For testing purposes we only need an amd64 image.
- name: "Pull Request Trigger: Build and Push amd64 Docker Image"
if: ${{ needs.prepare-env.outputs.build_for_pr == 'true' }}
# Build and Publish images on main, master, and versioned branches.
#
# The reason we split out these steps into 2 is for better handling of
# forks when building amd64 images and to enable faster availability of
# the amd64 image since building the arm64 image takes significantly
# longer.
- name: "Merge on Main Trigger: Build and Push All Docker Images"
MSevey marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ needs.prepare-env.outputs.build_for_merge == 'true' }}
uses: docker/build-push-action@v5
env:
OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
with:
context: .
# We don't use the cache to reduce complexity. We've seen issues of
# the same commit from a PR and on main causing incorrect images being
# built.
no-cache: true
platforms: linux/arm64,linux/amd64
provenance: false
platforms: linux/amd64
# Only push if the head and base repos match, meaning it is not a fork
push: ${{ needs.prepare-env.outputs.not_a_fork == 'true' }}
tags: ${{ steps.meta.outputs.tags }}
push: true
tags: ${{ matrix.registry.registry-url }}/${{ matrix.registry.registry-owner }}/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
file: ${{ inputs.dockerfile }}

# Build and Publish images on main, master, and versioned branches.
#
# The reason we split out these steps into 2 is for better handling of
# forks when building amd64 images and to enable faster availability of
# the amd64 image since building the arm64 image takes significantly
# longer.
- name: "Merge on Main Trigger: Build and Push All Docker Images"
if: ${{ needs.prepare-env.outputs.build_for_merge == 'true' }}
# Build amd64 images always, and publish when it is not a fork. The Github
# security model prevents forks from pushing to the registry so we can
# only push if the branch/PR is not generated from a fork. Even though
# forks can't push, we still want to try and build the image to catch
# bugs. For testing purposes we only need an amd64 image.
- name: "Pull Request Trigger: Build and Push amd64 Docker Image"
if: ${{ needs.prepare-env.outputs.build_for_pr == 'true' }}
uses: docker/build-push-action@v5
env:
OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
with:
context: .
# We don't use the cache to reduce complexity. We've seen issues of
# the same commit from a PR and on main causing incorrect images being
# built.
no-cache: true
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
provenance: false
push: true
tags: ${{ steps.meta.outputs.tags }}
# Only push if the head and base repos match, meaning it is not a fork
push: ${{ needs.prepare-env.outputs.not_a_fork == 'true' }}
tags: ${{ matrix.registry.registry-url }}/${{ matrix.registry.registry-owner }}/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
file: ${{ inputs.dockerfile }}