Skip to content
This repository has been archived by the owner on Jan 4, 2025. It is now read-only.

Commit

Permalink
Merge Mailu#2403
Browse files Browse the repository at this point in the history
2403: Feature: switch CI/CD from build to buildx r=mergify[bot] a=Diman0

## What type of PR?

Feature and enhancement

## What does this PR do?

Switch from docker build to buildx for CI/CD.
    - The main workflow file has been optimised and simplified.
    - Images are built in parallel when building locally resulting in much faster build times.
    - The github action workflow is about 50% faster.
    - Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it. The arm images are built on merge on master and release branch (x.y). They do not influence the normal CI/CD workflow used for bors (for PR) and real releases (merge on master and branch x.y for x86_64). 
    - Arm images (and normal x86_64 images) can also be built locally.
    - Reusable workflow is introduced for building, testing and deploying the images. This allows the workflow to be reused for other purposes in the future.
    - Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.

The main workflow makes use of github actions cache to store the cache layer. This layer is used to quickly rebuilt the images in the testing step and deploy step.

Unfortunately the building the arm images fails sometimes due to timeouts. Sometimes the connection to github actions cache is very slow. Restarting the workflow from the last failed step resolves this. I have not observed this with the normal build.

Just as previous time, you can use a forked project for testing the changes (https://github.com/Diman0/Mailu_Fork). You should still have owner access. I have created branch 1.11 for testing. You can see I already push 4 times to branch 1.11 (current version is 1.11.3).

### Related issue(s)
- Mention an issue like: #1
- closes Mailu#2383 
- closes Mailu#1830
- closes Mailu#1200

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <[email protected]>
  • Loading branch information
bors[bot] and Diman0 authored Aug 19, 2022
2 parents 3327500 + ae18217 commit 53de7b7
Show file tree
Hide file tree
Showing 25 changed files with 918 additions and 943 deletions.
678 changes: 0 additions & 678 deletions .github/workflows/CI.yml

This file was deleted.

101 changes: 101 additions & 0 deletions .github/workflows/arm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: start-linux-arm
on:
push:
branches:
- '1.9'
- master

concurrency: ci-arm-${{ github.ref }}

###############################################
# REQUIRED secrets
# ${{ secrets.Docker_Login }}
# Username of docker login for pushing the images to repo env.DOCKER_ORG and env.DOCKER_ORG_TESTS
# ${{ secrets.Docker_Password }}
# Password of docker login for pushing the images to repo env.DOCKER_ORG and env.DOCKER_ORG_TESTS
# Add the above secrets to your github repo to determine where the images will be pushed.
################################################
# REQUIRED global variables
# DOCKER_ORG, docker org used for pushing release images (branch x.y and master)
# DOCKER_ORG_TEST, docker org used for pushing images for testing (branch testing).
env:
DOCKER_ORG: mailu
DOCKER_ORG_TEST: mailuci

jobs:
# This job calculates all global job variables that are required by all the subsequent jobs.
# All subsequent jobs will retrieve and use these variables. This way the variables only have to be derived once.
derive-variables:
name: derive variables
runs-on: ubuntu-latest
outputs:
MAILU_VERSION: ${{ env.MAILU_VERSION }}
PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
DOCKER_ORG: ${{ env.DOCKER_ORG_DERIVED }}
BRANCH: ${{ env.BRANCH }}
DEPLOY: ${{ env.DEPLOY }}
RELEASE: ${{ env.RELEASE }}
steps:
- uses: actions/checkout@v3
with:
# fetch-depth 0 is required to also retrieve all tags.
fetch-depth: 0
- name: Extract branch name
shell: bash
run: |
echo "BRANCH=${GITHUB_REF#refs/heads/}" >> $GITHUB_ENV
#For branch TESTING, we set the image tag to pr-xxxx
- name: Derive MAILU_VERSION and DEPLOY/RELEASE for other branches than testing
if: env.BRANCH != 'testing'
shell: bash
run: |
echo "MAILU_VERSION=${{ env.BRANCH }}" >> $GITHUB_ENV
echo "DOCKER_ORG_DERIVED=${{ env.DOCKER_ORG }}" >> $GITHUB_ENV
echo "DEPLOY=true" >> $GITHUB_ENV
echo "RELEASE=false" >> $GITHUB_ENV
- name: Derive PINNED_MAILU_VERSION and DEPLOY/RELEASE for normal release x.y
if: env.BRANCH != 'testing' && env.BRANCH != 'staging' && env.BRANCH != 'master'
shell: bash
run: |
version=$( git tag --sort=version:refname --list "${{ env.MAILU_VERSION }}.*" | tail -1 );root_version=${version%.*};patch_version=${version##*.};if [ "$patch_version" == "" ]; then pinned_version=${{ env.MAILU_VERSION }}.0; else pinned_version=$root_version.$(expr $patch_version + 1); fi;echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV
echo "RELEASE=true" >> $GITHUB_ENV
echo "DEPLOY=true" >> $GITHUB_ENV
echo "RELEASE=true" >> $GITHUB_ENV
- name: Derive PINNED_MAILU_VERSION for staging for master
if: env.BRANCH == 'master'
shell: bash
env:
GITHUB_SHA: ${{ env.GITHUB_SHA }}
run: |
echo "PINNED_MAILU_VERSION=$GITHUB_SHA" >> $GITHUB_ENV
echo "DEPLOY=true" >> $GITHUB_ENV
echo "RELEASE=false" >> $GITHUB_ENV
build-test-deploy:
needs:
- derive-variables
uses: ./.github/workflows/build_test_deploy.yml
with:
architecture: 'linux/arm64,linux/arm/v7'
mailu_version: ${{needs.derive-variables.outputs.MAILU_VERSION}}-arm
pinned_mailu_version: ${{needs.derive-variables.outputs.PINNED_MAILU_VERSION}}-arm
docker_org: ${{needs.derive-variables.outputs.DOCKER_ORG}}
branch: ${{needs.derive-variables.outputs.BRANCH}}
deploy: ${{needs.derive-variables.outputs.DEPLOY}}
release: ${{needs.derive-variables.outputs.RELEASE}}
secrets: inherit

################################################
# Code block that is used as one liner for the step:
# Derive PINNED_MAILU_VERSION and DEPLOY/RELEASE for normal release x.y
##!/bin/bash
#version=$( git tag --sort=version:refname --list "{{ env.MAILU_VERSION }}.*" | tail -1 )
#root_version=${version%.*}
#patch_version=${version##*.}
#if [ "$patch_version" == "" ]
#then
# pinned_version={{ env.MAILU_VERSION }}.0
#else
# pinned_version=$root_version.$(expr $patch_version + 1)
#fi
#echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV
Loading

0 comments on commit 53de7b7

Please sign in to comment.