Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add DockerHub and ScaleWay registries #85

Closed
wants to merge 14 commits into from
Closed
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 88 additions & 56 deletions .github/workflows/reusable_dockerfile_pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,15 @@ on:
default: "${{ github.repository }}"

env:
REGISTRY: ghcr.io
GITHUB_REG: ghcr.io
MAINTAINER: ${{ github.repository_owner }}
DESCRIPTION: "${{ github.repository_owner }} repository ${{ github.repository }}"

jobs:
prepare-env:
runs-on: "ubuntu-latest"
outputs:
repo_owner: ${{ steps.setting_env.outputs.repo_owner }}
output_short_sha: ${{ steps.setting_env.outputs.short_sha }}
output_image_name: ${{ steps.setting_env.outputs.image_name }}
build_for_pr: ${{ steps.setting_logic.outputs.build_for_pr }}
Expand All @@ -35,25 +36,42 @@ jobs:
- name: Add vars to ENV
id: setting_env
run: |
echo "SHORT_SHA=`echo ${GITHUB_SHA} | cut -c1-8`" >> $GITHUB_ENV
echo "short_sha=`echo ${GITHUB_SHA} | cut -c1-8`" >> "$GITHUB_OUTPUT"
# yamllint disable
echo "IMAGE_NAME=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
echo "image_name=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
# here we validate if we have specified a different package name in
# the inputs, if so, we change the package to it.
if [[ ${{ inputs.packageName }} != ${{ github.repository}} ]];then
# validate the input package name characters
if [[ ! "${{ inputs.packageName }}" =~ ^[A-Za-z0-9\-]+$ ]]; then
echo "------------------------------------------------------------"
echo "ERROR: Package name not valid! => [ ${{ inputs.packageName }} ]"
echo "ONLY can use: A-Za-z0-9\-"
echo "------------------------------------------------------------"
exit 1
# Extract both the repository owner and repository name
# According to docs, github.repository is in the format of owner/repo, not just repo
# https://docs.github.com/en/actions/learn-github-actions/contexts#github-context
REPO_OWNER=$(echo "${{ github.repository }}" | awk -F'/' '{print $1}' | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | awk -F'/' '{print $2}' | tr '[:upper:]' '[:lower:]')
MSevey marked this conversation as resolved.
Show resolved Hide resolved

echo "Repository Owner: $REPO_OWNER"
echo "Repository Name: $REPO_NAME"

echo "REPO_OWNER=$REPO_OWNER" >> $GITHUB_ENV
echo "repo_owner=$REPO_OWNER" >> "$GITHUB_OUTPUT"

echo "SHORT_SHA=`echo ${GITHUB_SHA} | cut -c1-8`" >> $GITHUB_ENV
echo "short_sha=`echo ${GITHUB_SHA} | cut -c1-8`" >> "$GITHUB_OUTPUT"
# yamllint disable

# Set the default package name to the repository name
PACKAGE_NAME=$REPO_NAME
MSevey marked this conversation as resolved.
Show resolved Hide resolved

# If there is a user submitted package name, use it
if [[ ! "${{ inputs.packageName }}" != "" ]]; then
PACKAGE_NAME=$(echo "${{ inputs.packageName }}" | tr '[:upper:]' '[:lower:]')
fi

# validate the package name characters
if [[ ! $PACKAGE_NAME =~ ^[A-Za-z0-9\-]+$ ]]; then
echo "------------------------------------------------------------"
echo "ERROR: Package name not valid! => [ $PACKAGE_NAME} ]"
echo "ONLY can use: A-Za-z0-9\-"
echo "------------------------------------------------------------"
exit 1
fi

echo "IMAGE_NAME=$PACKAGE_NAME" >> $GITHUB_ENV
echo "image_name=$PACKAGE_NAME" >> "$GITHUB_OUTPUT"
fi
Bidon15 marked this conversation as resolved.
Show resolved Hide resolved
echo "IMAGE_NAME=$(echo ${{ github.repository_owner }}/${{ inputs.packageName }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
echo "image_name=$(echo ${{ github.repository_owner }}/${{ inputs.packageName }} | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
fi
# yamllint enable

# The key logic that we want to determine is whether or not we are working
Expand Down Expand Up @@ -135,23 +153,42 @@ jobs:
severity: "CRITICAL,HIGH"

docker-build:
name: docker-build (${{ matrix.registry.name }}; ${{ matrix.registry.registry-url }}/${{ matrix.registry.registry-owner }}/${{ needs.prepare-env.outputs.output_image_name }}:${{ needs.prepare-env.outputs.output_short_sha }})
runs-on: "ubuntu-latest"
# wait until the jobs are finished.
needs: ["prepare-env", "logic-check", "docker-security"]
permissions:
contents: write
packages: write

strategy:
matrix:
registry:
- name: DockerHub
user-secret: DOCKERHUB_USERNAME
token-secret: DOCKERHUB_TOKEN
registry-url: docker.io
registry-owner: celestiaorg
- name: GHCR
user-secret: ${{ github.repository_owner }}
token-secret: GITHUB_TOKEN
registry-url: ${{ env.GITHUB_REG }}
registry-owner: ${{ needs.prepare-env.outputs.repo_owner }}
- name: ScaleWay
user-secret: SCALEWAY_USERNAME
token-secret: SCW_SECRET_KEY
registry-url: rg.fr-par.scw.cloud
registry-owner: celestiaorg
fail-fast: false
steps:
- name: Checkout
uses: "actions/checkout@v4"

- name: Login to GHCR
- name: Login to ${{ matrix.registry.name }}
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: ${{ matrix.registry.registry-url }}
username: ${{ matrix.registry.registry-url == env.GITHUB_REG && matrix.registry.user-secret || secrets[matrix.registry.user-secret] }}
password: ${{ secrets[matrix.registry.token-secret] }}

- name: Extract Docker Metadata
id: meta
Expand All @@ -160,12 +197,12 @@ jobs:
OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
with:
images: ${{ env.REGISTRY }}/${{ env.OUTPUT_IMAGE_NAME }}
images: ${{ env.OUTPUT_IMAGE_NAME }}
# yamllint disable
labels: |
maintainer=${{ env.MAINTAINER }}
commitUrl=https://github.com/${{ github.repository }}/commit/${{ github.sha }}
dockerPull=docker pull ${{ env.REGISTRY }}/${{ github.repository }}:${{ env.OUTPUT_SHORT_SHA }}
dockerPull=docker pull ${{ env.GITHUB_REG }}/${{ github.repository }}:${{ env.OUTPUT_SHORT_SHA }}
org.opencontainers.image.description=${{ env.DESCRIPTION }}
tags: |
# output minimal (short sha)
Expand All @@ -176,55 +213,50 @@ jobs:
type=ref,enable=true,prefix=pr-,suffix=,event=pr
# yamllint enable

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# Build amd64 images always, and publish when it is not a fork. The Github
# security model prevents forks from pushing to the registry so we can
# only push if the branch/PR is not generated from a fork. Even though
# forks can't push, we still want to try and build the image to catch
# bugs. For testing purposes we only need an amd64 image.
- name: "Pull Request Trigger: Build and Push amd64 Docker Image"
if: ${{ needs.prepare-env.outputs.build_for_pr == 'true' }}
# Build and Publish images on main, master, and versioned branches.
#
# The reason we split out these steps into 2 is for better handling of
# forks when building amd64 images and to enable faster availability of
# the amd64 image since building the arm64 image takes significantly
# longer.
- name: "Merge on Main Trigger: Build and Push All Docker Images"
MSevey marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ needs.prepare-env.outputs.build_for_merge == 'true' }}
uses: docker/build-push-action@v5
env:
OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
with:
context: .
# We don't use the cache to reduce complexity. We've seen issues of
# the same commit from a PR and on main causing incorrect images being
# built.
no-cache: true
platforms: linux/arm64,linux/amd64
provenance: false
platforms: linux/amd64
# Only push if the head and base repos match, meaning it is not a fork
push: ${{ needs.prepare-env.outputs.not_a_fork == 'true' }}
tags: ${{ steps.meta.outputs.tags }}
push: true
tags: ${{ matrix.registry.registry-url }}/${{ matrix.registry.registry-owner }}/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
file: ${{ inputs.dockerfile }}

# Build and Publish images on main, master, and versioned branches.
#
# The reason we split out these steps into 2 is for better handling of
# forks when building amd64 images and to enable faster availability of
# the amd64 image since building the arm64 image takes significantly
# longer.
- name: "Merge on Main Trigger: Build and Push All Docker Images"
if: ${{ needs.prepare-env.outputs.build_for_merge == 'true' }}
# Build amd64 images always, and publish when it is not a fork. The Github
# security model prevents forks from pushing to the registry so we can
# only push if the branch/PR is not generated from a fork. Even though
# forks can't push, we still want to try and build the image to catch
# bugs. For testing purposes we only need an amd64 image.
- name: "Pull Request Trigger: Build and Push amd64 Docker Image"
if: ${{ needs.prepare-env.outputs.build_for_pr == 'true' }}
uses: docker/build-push-action@v5
env:
OUTPUT_SHORT_SHA: ${{ needs.prepare-env.outputs.output_short_sha }}
OUTPUT_IMAGE_NAME: ${{ needs.prepare-env.outputs.output_image_name }}
with:
context: .
# We don't use the cache to reduce complexity. We've seen issues of
# the same commit from a PR and on main causing incorrect images being
# built.
no-cache: true
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
provenance: false
push: true
tags: ${{ steps.meta.outputs.tags }}
# Only push if the head and base repos match, meaning it is not a fork
push: ${{ needs.prepare-env.outputs.not_a_fork == 'true' }}
tags: ${{ matrix.registry.registry-url }}/${{ matrix.registry.registry-owner }}/${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
file: ${{ inputs.dockerfile }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.idea/*
.vscode/*