Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reusable workflows for release #63

Merged
37 changes: 18 additions & 19 deletions .github/workflows/bump-sec-scanners-config-reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
# Setting a secret for a repo: https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions
#
# If changes were done by the script, the workflow will create a PR and wait for it to be merged.
# The waiting will happen with a timeout that can be set via the input of `timeout`. The units are seconds.
# It has a default value of 3600 (seconds (= 1 hour)). Note that GitHub Action jobs will automatically fail after 6 hours:
# Further reads:
# Default limits for GitHub Actions: https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits
# The waiting will happen with a timeout that can be set via the input of `TIMEOUT`. The units are seconds.
#
# Examples of using this workflow:
# 1. Set all awailable inputs and secrets.
Expand All @@ -26,7 +26,7 @@
# uses: kyma-project/eventing-tools/.github/workflows/bump-sec-scanners-config-reusable.yml@main
# with:
# version_tag: 2.3.4
# timeout: 3600 # 1 hour
# TIMEOUT: 3600 # 1 hour
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lowercase or Uppercase version_tag and TIMEOUT, please keep this consistent.

# secrets:
# BOT_PAT: ${{ secrets.my_pat }}
#
Expand All @@ -45,11 +45,11 @@ name: bump sec-scanners-config.yaml (reusable)
on:
workflow_call:
inputs:
version_tag:
VERSION:
required: true
type: string
description: The semantic version number, that will be used to tag the main image in the sec scanner config.
timeout:
TIMEOUT:
required: false
type: number
description: The time in seconds this workflow will wait for a resulting PR to be merged.
Expand All @@ -62,22 +62,19 @@ jobs:
bump:
name: Bump sec-scanners-config.yaml
runs-on: ubuntu-latest
env:
REPO: ${{ github.repository }}

steps:
- name: Checkout Code
- name: Checkout code
uses: actions/checkout@v4

- name: Render sec-scanners-config.yaml
env:
VERSION_TAG: ${{ inputs.version_tag }}
shell: bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why shell: bash is removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that was a mishap and I will revert it.

VERSION: ${{ inputs.VERSION }}
# Where ever you use this workflow, the script hack/scripts/render-sec-scanners-config.sh must exist.
run: ./hack/scripts/render-sec-scanners-config.sh "${VERSION_TAG}"
run: ./hack/scripts/render-sec-scanners-config.sh "${VERSION}"

# Check if there are changes so we can determin if all following steps can be skipped.
- name: Check For Changes
- name: Check for changes
shell: bash
run: |
if [ -z "$(git status --porcelain)" ]; then
Expand All @@ -87,29 +84,30 @@ jobs:
echo "CREATE_PR=true" >> $GITHUB_ENV
fi

- name: Print Content of sec-scanners-config.yaml
if: ${{ always() }}
Copy link
Contributor

@marcobebway marcobebway Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need if always? Can we remove it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

always() will make sure this will get executed even if an earlier step fails. This is particularly useful for debugging say in case you want to know if at least rendering the file went all right.

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

always() will ensure this will get executed even if an earlier step fails. This is particularly useful for debugging say in case you want to know if at least rendering the file went all right.

- name: Print out sec-scanners-config.yaml
shell: bash
run: |
FILE="sec-scanners-config.yaml"
[ -f "${FILE}" ] && cat "${FILE}" || echo "${FILE} not found."

- name: Set Up Git
- name: Set up git
if: ${{ env.CREATE_PR == 'true' }}
env:
GH_TOKEN: ${{ secrets.BOT_PAT }}
REPO: ${{ github.repository }}
shell: bash
run: |
# set git username
ghusername=$(curl -H "Authorization: token ${GH_TOKEN}" https://api.github.com/user)
ghusername=$(curl -s -H "Authorization: token ${GH_TOKEN}" https://api.github.com/user | jq '.login')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a step to install jq or is it available out of the box?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jq is available on the ubuntu-latest image used by GitHub.

git config user.name "${ghusername}"
# set git mail address
ghmailaddress=$(curl -H "Authorization: token ${GH_TOKEN}" https://api.github.com/email)
ghmailaddress="${ghusername}@users.noreply.github.com"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you want no reply email thing?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because the kyma-eventing-bot will not answer emails.

git config user.email "${ghmailaddress}"
# set remote url
git remote set-url origin "https://x-access-token:${GH_TOKEN}@github.com/${REPO}.git"

- name: Set All Variables
- name: Set all variables
if: ${{ env.CREATE_PR == 'true' }}
shell: bash
run: |
Expand All @@ -125,9 +123,10 @@ jobs:
echo "name of the new branch: ${BRANCH_NAME}"
echo "BRANCH_NAME=${BRANCH_NAME}" >> $GITHUB_ENV

- name: Create a Pull Request
- name: Create a pull request
if: ${{ env.CREATE_PR == 'true' }}
env:
REPO: ${{ github.repository }}
CURRENT_BRANCH: ${{ env.CURRENT_BRANCH }}
PR_DATE: ${{ env.PR_DATE }}
BRANCH_NAME: ${{ env.BRANCH_NAME }}
Expand Down Expand Up @@ -157,11 +156,11 @@ jobs:
run: |
echo "please review ${PR_URL}"

- name: Wait for PR to be Merged
if: ${{ env.CREATE_PR == 'true' }}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested this line if it worked as before? I think, it doesn't work as if is outside the job.

Copy link
Contributor Author

@friedrichwilken friedrichwilken Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not touch this line in this PR.

But yes, in the PR that created this, I mention:

tested by creating a carbon copy of it here and using it here. The workflow was triggered here, resulting in this PR. After merging the PR, the started workflow finished successfully.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo "PR_URL=${PR_URL}" >> $GITHUB_ENV
is like defining a env var in the job; it will become available in all the steps.

- name: Wait for PR to be merged
shell: bash
env:
TIMEOUT: ${{ inputs.timeout }}
TIMEOUT: ${{ inputs.TIMEOUT }}
PR_URL: ${{ env.PR_URL }}
GH_TOKEN: ${{ secrets.BOT_PAT }}
run: |
Expand Down
43 changes: 43 additions & 0 deletions .github/workflows/create-draft-release-reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Create draft release (reusable)

on:
workflow_call:
inputs:
VERSION:
required: true
type: string
description: The semantic version number.
secrets:
BOT_PAT:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) for readability, does it make sense to rename BOT_PAT to BOT_TOKEN?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhh, I am indifferent to the wording. My argument for PAT is that the personal access token is a special kind of token. But BOT_TOKEN would be fine for me too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you say?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a nitpick, let's keep the pat!

required: true
description: The github personal access token of your bot.
GH_TOKEN:
required: true

jobs:
create-draft-release:
name: Create a draft release
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Write changelog to file
env:
VERSION: ${{ inputs.VERSION }}
shell: bash
run: |
# Note: your repository needs to have this script.
Copy link
Contributor

@marcobebway marcobebway Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this script does not exist, will there be an error message indicating that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

# Running this script should result in a file named CHANGELOG.md, located in the base directory.
./hack/scripts/create_changelog.sh "${VERSION}"

- name: Print out changelog
run: cat CHANGELOG.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are no changes, what will be printed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That depends on the script. With what we have it will be only headlines and a link to the full changelogs.

Copy link
Contributor

@marcobebway marcobebway Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense here to print a message before and after the cat indicating the beginning and end of the change log?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the reason why I placed this in its own step. I makes it easy to find and easy to read. It will look like this

|> Run cat CHANGELOG.md
Changelog blah blah blah
Changelog blah 
Changelog blah-blah

So to answer the question, no, I think it is good already.


- name: Create the draft release
env:
VERSION: ${{ inputs.VERSION }}
GH_TOKEN: ${{ secrets.BOT_PAT }}
shell: bash
run: |
gh release create "${VERSION}" --draft --notes-file CHANGELOG.md
45 changes: 45 additions & 0 deletions .github/workflows/get-version-from-release-branch-reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Get version from release branch (reusable)

on:
workflow_call:
outputs:
VERSION:
description: "The semantic version x.y.z, e.g.: 1.7.4"
value: ${{ jobs.create-version.outputs.VERSION }}
Copy link
Contributor

@marcobebway marcobebway Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the name create-version under our control? If yes, please rename it to create_version to be consistent with similar names in other places like ${{ github.repository_owner }}.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In github actions it is customary practice to name jobs and step IDs with a dash.

my-job:
  steps: 
     - name: "my step"
        id: my-step

while env vars and secrets spelled most of the time with an underscore e.g. MY_VAR and MY_SECRET


jobs:
create-version:
name: generate version number
runs-on: ubuntu-latest
outputs:
VERSION: ${{ steps.generate.outputs.VERSION }}

steps:
- name: checkout code
uses: actions/checkout@v4

- name: Verify that the current is branch is a release branch
shell: bash
run: |
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
[[ $CURRENT_BRANCH =~ ^release-([0-9]+)\.([0-9]+)$ ]] || exit 1
echo "MAJOR=${BASH_REMATCH[1]}" >> $GITHUB_ENV
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the value of BASH_REMATCH? Which part sets that value?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$BASH_REMATCH is an array and contains the matches of a regular expression.
doc
A very handy little tool.

echo "MINOR=${BASH_REMATCH[2]}" >> $GITHUB_ENV
exit 0

- name: Generate version
id: generate
shell: bash
env:
MAJOR: ${{ env.MAJOR }}
MINOR: ${{ env.MINOR }}
run: |
TAGS=$(git tag -l "$MAJOR.$MINOR.*")
if [[ -z $TAGS ]]; then
PATCH=0
else
PATCH=$(( $(echo $TAGS | cut -d '.' -f 3 | sort -n | tail -n 1) + 1))
fi
VERSION="${MAJOR}.${MINOR}.${PATCH:-0}"
echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT
exit 0
29 changes: 29 additions & 0 deletions .github/workflows/publish-release-reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Publish release (reusable)

on:
workflow_call:
inputs:
VERSION:
required: true
type: string
description: The semantic version number.
secrets:
BOT_PAT:
required: true
description: The github personal access token of your bot.

jobs:
publish-release:
name: Publish release
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Publish
env:
VERSION: ${{ inputs.VERSION }}
GH_TOKEN: ${{ secrets.BOT_PAT }}
shell: bash
run: |
gh release edit "${VERSION}" --draft=false --latest
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you have --latest flag because if set the latest release is edited. We want to edit the VERSION. Should we delete the latest flag?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have set the --latest flag because it will mark the release -identified as VERSION- as latest.

--latest                       Explicitly mark the release as "Latest"

In this step we edit the draft release and turn it into a real release. We also set it to be the latest release.
I think we should leave this flag.

60 changes: 60 additions & 0 deletions .github/workflows/render-and-upload-manifests-reusbale.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Render and upload manifests (reusable)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filename is misspelled:
render-and-upload-manifests-reusbale.yml -> render-and-upload-manifests-reusable.yml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! Resolved.


on:
workflow_call:
inputs:
VERSION:
required: true
type: string
description: The semantic version number.
CR_FILE:
type: string
required: true
description: The file name of the CR.
CRD_FILE:
type: string
required: true
description: The file name of the CRD.
secrets:
BOT_PAT:
required: true
description: The github personal access token of your bot.

jobs:
render-and-upload-manifests:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Render CRD
env:
VERSION: ${{ inputs.VERSION }}
CRD_FILE: ${{ inputs.CRD_FILE }}
shell: bash
# Note: your repository needs to have this script.
run: ./hack/scripts/render_crd.sh "${VERSION}" "${CRD_FILE}"

- name: Print out CR file
env:
CR_FILE: ${{ inputs.CR_FILE }}
shell: bash
run: cat "${CR_FILE}"

- name: Print out CRD file
env:
CRD_FILE: ${{ inputs.CRD_FILE }}
shell: bash
run: cat "${CRD_FILE}"

- name: Upload manifests
env:
VERSION: ${{ inputs.VERSION }}
GH_TOKEN: ${{ secrets.BOT_PAT }}
CRD_FILE: ${{ inputs.CRD_FILE }}
CR_FILE: ${{ inputs.CR_FILE }}
shell: bash
run: |
gh release upload "${VERSION}" "${CR_FILE}"
gh release upload "${VERSION}" "${CRD_FILE}"
72 changes: 72 additions & 0 deletions .github/workflows/trigger-prow-build-job-reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Trigger prow build job (reusable)

on:
workflow_call:
inputs:
VERSION:
required: true
type: string
description: The semantic version number.
TIMEOUT:
Copy link

@muralov muralov Jan 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see TIMEOUT is not used anywhere.

type: number
default: 60000 # 10 minutes in miliseconds
INTERVAL:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see INTERVAL is not used anywhere.

type: number
default: 60000 # 1 minute in miliseconds
CONTEXT:
required: true
type: string
description: The context is the name of the prow job we are waiting for.
secrets:
BOT_PAT:
required: true
GH_TOKEN:
required: true

jobs:
trigger-prow-build-job:
name: Trigger prow build job
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up git
if: ${{ env.CREATE_PR == 'true' }}
env:
GH_TOKEN: ${{ secrets.BOT_PAT }}
REPO: ${{ github.repository }}
shell: bash
run: |
# set git username
ghusername=$(curl -s -H "Authorization: token ${GH_TOKEN}" https://api.github.com/user | jq '.login')
git config user.name "${ghusername}"
# set git mail address
ghmailaddress="${ghusername}@users.noreply.github.com"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) mail -> email

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasonable. I changed it.

git config user.email "${ghmailaddress}"
# set remote url
git remote set-url origin "https://x-access-token:${GH_TOKEN}@github.com/${REPO}.git"

- name: Push git tag to trigger the prow build job
env:
VERSION: ${{ inputs.VERSION }}
run: |
git tag "${VERSION}"
git push origin "${VERSION}"

- name: Wait for the build job to succeed
id: wait-build
uses: kyma-project/wait-for-commit-status-action@2b3ffe09af8b6f40e1213d5fb7f91a7bd41ffb20
env:
GITHUB_TOKEN: "${{ secrets.GH_TOKEN }}"
GITHUB_OWNER: "${{ github.repository_owner }}"
GITHUB_REPO: ${{ github.event.repository.name }}
VERSION: "${{ inputs.VERSION }}"
with:
context: "${{ inputs.CONTEXT }}"
commit_ref: "release-${VERSION}" # the name of the release branch.
timeout:
600000 # 10 minutes in milliseconds
# The check interval is kept long otherwise it will exhaust the GitHub rate limit (More info: https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limiting)
check_interval: 60000 # 1 minute in milliseconds
Loading