Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build vttestserver in GHA and send Slack message on Docker Build failure #16963

Merged
merged 5 commits into from
Oct 23, 2024
Merged

Build vttestserver in GHA and send Slack message on Docker Build failure #16963

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
50e83c8
Add documentation on how to build and push vttestserver manually
frouioui Oct 15, 2024
654e41d
Add vttestserver docker build to gha and send slack notification on b…
frouioui Oct 15, 2024
1c14d44
wip - test slack notification
frouioui Oct 15, 2024
e25cbd3
Remove WIP and fix Slack notification
frouioui Oct 15, 2024
1e950a0
Make docs more clear
frouioui Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 94 additions & 21 deletions .github/workflows/docker_build_images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Docker Build Images
name: Build Docker Images
on:
push:
branches:
Expand All @@ -8,14 +8,68 @@ on:
workflow_dispatch:

concurrency:
group: format('{0}-{1}', ${{ github.ref }}, 'Docker Build Images (v20+)')
group: format('{0}-{1}', ${{ github.ref }}, 'Build Docker Images')
cancel-in-progress: true

permissions: read-all

jobs:
build_and_push_vttestserver:
name: Build and push vttestserver
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'

strategy:
fail-fast: true
matrix:
branch: [ mysql80 ]

steps:
- name: Check out code
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Login to Docker Hub
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set Dockerfile path
run: |
echo "DOCKERFILE=./docker/vttestserver/Dockerfile.${{ matrix.branch }}" >> $GITHUB_ENV

- name: Build and push on main
if: startsWith(github.ref, 'refs/tags/') == false
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: vitess/vttestserver:${{ matrix.branch }}

######
# All code below only applies to new tags
######
- name: Get the Git tag
if: startsWith(github.ref, 'refs/tags/')
run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV

- name: Set Docker tag name
if: startsWith(github.ref, 'refs/tags/')
run: |
echo "DOCKER_TAG=vitess/vttestserver:${TAG_NAME}-${{ matrix.branch }}" >> $GITHUB_ENV

- name: Build and push on tags
if: startsWith(github.ref, 'refs/tags/')
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: ${{ env.DOCKER_TAG }}

build_and_push_lite:
name: Build and push vitess/lite Docker images
name: Build and push lite
runs-on: ubuntu-latest
if: github.repository == 'vitessio/vitess'

Expand All @@ -26,10 +80,10 @@ jobs:

steps:
- name: Check out code
uses: actions/checkout@v4
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason to use the SHA here instead of a version?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the SHA instead of the version is part of the OpenSSF best practices. We started doing this some time ago through #16612. This is to comply with #16607.


- name: Login to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
Expand All @@ -43,8 +97,8 @@ jobs:
fi

- name: Build and push on main
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
if: startsWith(github.ref, 'refs/tags/') == false
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: .
file: ${{ env.DOCKERFILE }}
Expand All @@ -69,31 +123,32 @@ jobs:

- name: Build and push on tags
if: startsWith(github.ref, 'refs/tags/')
uses: docker/build-push-action@v5
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: ${{ env.DOCKER_TAG }}

build_and_push_components:
name: Build and push vitess components Docker images
needs: build_and_push_lite
name: Build and push
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'
if: github.repository == 'vitessio/vitess' && needs.build_and_push_lite.result == 'success'
needs:
- build_and_push_lite

strategy:
fail-fast: true
matrix:
debian: [ bullseye, bookworm ]
component: [ vtadmin, vtorc, vtgate, vttablet, mysqlctld, mysqlctl, vtctl, vtctlclient, vtctld, logrotate, logtail, vtbackup, vtexplain ]
component: [ vtadmin, vtorc, vtgate, vttablet, mysqlctld, mysqlctl, vtctl, vtctlclient, vtctld, vtctldclient, logrotate, logtail, vtbackup, vtexplain ]
frouioui marked this conversation as resolved.
Show resolved Hide resolved

steps:
- name: Check out code
uses: actions/checkout@v4
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Login to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
Expand All @@ -103,8 +158,8 @@ jobs:
echo "DOCKER_CTX=./docker/binaries/${{ matrix.component }}" >> $GITHUB_ENV

- name: Build and push on main latest tag
if: github.ref == 'refs/heads/main' && matrix.debian == 'bookworm'
uses: docker/build-push-action@v5
if: startsWith(github.ref, 'refs/tags/') == false && matrix.debian == 'bookworm'
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: ${{ env.DOCKER_CTX }}
push: true
Expand All @@ -114,8 +169,8 @@ jobs:
DEBIAN_VER=${{ matrix.debian }}-slim

- name: Build and push on main debian specific tag
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
if: startsWith(github.ref, 'refs/tags/') == false
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: ${{ env.DOCKER_CTX }}
push: true
Expand Down Expand Up @@ -147,7 +202,7 @@ jobs:
# Build and Push component image to DOCKER_TAG, applies to both debian version
- name: Build and push on tags using Debian extension
if: startsWith(github.ref, 'refs/tags/')
uses: docker/build-push-action@v5
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: ${{ env.DOCKER_CTX }}
push: true
Expand All @@ -160,11 +215,29 @@ jobs:
# It is fine to build a second time here when "matrix.debian == 'bookworm'" as we have cached the first build already
- name: Build and push on tags without Debian extension
if: startsWith(github.ref, 'refs/tags/') && matrix.debian == 'bookworm'
uses: docker/build-push-action@v5
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: ${{ env.DOCKER_TAG_DEFAULT_DEBIAN }}
build-args: |
VT_BASE_VER=${{ env.TAG_NAME }}
DEBIAN_VER=${{ matrix.debian }}-slim
DEBIAN_VER=${{ matrix.debian }}-slim

slack_notification:
name: Slack Notification if failed
runs-on: ubuntu-latest
needs:
- build_and_push_vttestserver
- build_and_push_lite
- build_and_push_components
if: ${{ failure() }}
steps:
- name: Slack Workflow Notification
uses: Gamesight/slack-workflow-status@68bf00d0dbdbcb206c278399aa1ef6c14f74347a # v1.3.0
with:
repo_token: ${{secrets.GITHUB_TOKEN}}
slack_webhook_url: ${{secrets.SLACK_WEBHOOK_URL}}
channel: '#docker-build-notifications'
name: 'Docker Build Notification'
icon_url: https://avatars.githubusercontent.com/u/33043890?s=96&v=4
17 changes: 17 additions & 0 deletions docker/vttestserver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
## vttestserver docker image

### How to build manually during a release

If for whatever reason the automatic build did not happen for `vttestserver` during a release, or if it has failed,
there is a way of building it by hand. Here is how it goes:

```bash
docker login

# we first checkout to the git tag of the release we want to build
# we assume in this example that we are releasing v21.0.0-rc1, but replace this by any other tag
git checkout v21.0.0-rc1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove the -rc1 right, since we no longer use that for the branch names? Or are we using tag names?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to checkout the git tag directly, not the release branch. I updated the comment to be more understandable in 1e950a0.


docker build -f docker/vtttestserver/Dockerfile.mysql80 -t vitess/vttestserver:v21.0.0-rc1-mysql80 .
docker push vitess/vttestserver:v21.0.0-rc1-mysql80
```