Skip to content

Commit

Permalink
Add release changes (kyma-project#34)
Browse files Browse the repository at this point in the history
* Setup E2E test using NATS Backend (kyma-project#30)

* Setup E2E github action flow

* Move hack Makefile

* Move ci folder outside of e2e

* Correct makefile path

* Add missing step to e2e flow to checkout repo

* Correct path in Makefile

* Add kustomize to Makefile

* Add kustomization

* Add LOCALBIN variable

* Add k3d cluster create step

* Remove go step

* Add missing variables to Makefile

* Add kyma script

* Make script executable

* Checkout EM repositry

* Correct repository checkout

* Use EM repo for e2e tests

* Remove hack/Makefile

* Replace make command with local one

* Try adding sudo for `permission denied` problem

* Try using `kyma` instead of variable

* Try using explicit path

* Use different make targets to deploy EM

* Update image

* Fix reference of variable

* Add step to create namespace

* Fix Github token

* Fix make calls in workflow

* Add step to install NATS

* Try to use Github variable

* Deploy nats manager via makefile

* Add step to install NATS CR

* Fix reference of variable

* Try using `create-k3d` from `eventing-manager` repo

* Change KYMA_CLI path

* Undo variable changes

* Remove unused files

* Add clean-up step
Fix capitalisation

* Improve patch command by using `jq`

* Add missing path to cleanup command

* Try using `MODULE_VERSION` from EM Repo

* Echo Module Version
Use rel version for nats manager

* Print deployment image

* Fix kubectl command

* Try to use string

* Try using `yg` to extract version

* Fix command

* Use `grep` instead of `yq`

* Use curl to get latest release of NM

* Use `curl` to get EM latest release
Use `latest` for NM

* Use NM image used in release

* Use `kubectl apply` to deploy latest NATS and Eventing Manager

* Remove step to test EM setup

* Remove unused variable

* Check for pod rollout

* Add missing `$` to variable

* Wait for rollout to complete

* Add step to check for success of EPP image update

* Replace `k` with `kubectl`

* Release workflow (kyma-project#32)

* Release workflow

* Rename to `sec-scan`

* Fix all variables to `eventing-publisher-proxy`

* Fix output of next release version

* Fix run name to branch name
Use correct script in `Get the next release version` step
Add lines between steps

* Use correct image in `sec-scanners-config`

* Use env variable for using job output
Move `outputs` at beginning of step for readability

* Rename scripts to be more descriptive
Rename steps to be more clear

* Use script with more comments

* Add release flow (kyma-project#31)

* Add release flow

* Release using branch name as input

* Fix `needs` input

* Fix `needs` input

* Change workflow to be triggered manually

* No inputs for manual trigger

* Release workflow

* Remove unused scripts
  • Loading branch information
grischperl authored Jan 5, 2024
1 parent 4005ced commit 3e70eb2
Show file tree
Hide file tree
Showing 11 changed files with 192 additions and 37 deletions.
41 changes: 25 additions & 16 deletions .github/workflows/create-release.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Create Release
run-name: Create Release ${{ inputs.name }}
run-name: Create Release ${{ github.ref_name }}

env:
IMAGE_REPO: europe-docker.pkg.dev/kyma-project/prod/eventing-publisher-proxy
Expand All @@ -10,11 +10,14 @@ jobs:
verify-release:
name: Verify image version
runs-on: ubuntu-latest
outputs:
release_version: ${{ steps.release-version.outputs.release_version }}
steps:
- name: Checkout EPP repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Verify that the current branch has a name that starts with 'release-'
run: |
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
Expand All @@ -25,47 +28,52 @@ jobs:
echo "Branch name does not start with 'release-'."
exit 1
fi
- name: Get the next release version
id: release-version
run: |
RELEASE_VERSION=$(./scripts/check_artifacts_existence.sh "$current_branch")
RELEASE_VERSION=$(./scripts/get_next_release_version.sh "$current_branch")
echo "release_version=$RELEASE_VERSION" >> $GITHUB_OUTPUT
- name: Check image tag
- name: Check if tags in sec_scanners_config.yaml match the release version
env:
RELEASE_VERSION: ${{ steps.release-version.outputs.RELEASE_VERSION }}
run: ./scripts/check_tag_info.sh $RELEASE_VERSION
outputs:
release_version: ${{ steps.release-version.outputs.release_version }}
RELEASE_VERSION: ${{ steps.release-version.outputs.release_version }}
run: ./scripts/check_tags_in_sec_scanners_config.sh $RELEASE_VERSION

create-draft:
name: Create the draft release
needs: verify-release
runs-on: ubuntu-latest
env:
RELEASE_VERSION: ${{ needs.release-version.outputs.release_version }}
outputs:
release_id: ${{ steps.create-draft.outputs.release_id }}
steps:
- name: Checkout EPP repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Create changelog
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./scripts/create_changelog.sh ${{ needs.release-version.outputs.release_version }}
run: ./scripts/create_changelog.sh $RELEASE_VERSION

- name: Create the draft release
id: create-draft
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
RELEASE_ID=$(./scripts/create_draft_release.sh ${{ needs.release-version.outputs.release_version }})
RELEASE_ID=$(./scripts/create_draft_release.sh $RELEASE_VERSION)
echo "release_id=$RELEASE_ID" >> $GITHUB_OUTPUT
- name: Add lightweight tag
- name: Add lightweight tag to trigger release EPP build job
run: |
git tag ${{ needs.release-version.outputs.release_version }}
git push origin ${{ needs.release-version.outputs.release_version }}
- name: Verify job status
run: ./scripts/verify_status.sh ${{ github.ref_name }} 600 10 30
git tag $RELEASE_VERSION
git push origin $RELEASE_VERSION
outputs:
release_id: ${{ steps.create-draft.outputs.release_id }}
- name: Verify build job status
run: ./scripts/verify_prow_post_job_status.sh ${{ github.ref_name }} 600 10 30

publish-release:
name: Publish release
Expand All @@ -76,6 +84,7 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Publish release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
121 changes: 121 additions & 0 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
name: E2E Test

on:
pull_request:
branches:
- main
- "release-*"
paths-ignore:
- "docs/**"
- "**.md"
- "sec-scanners-config.yaml"

env:
EPP_IMAGE: europe-docker.pkg.dev/kyma-project/dev/eventing-publisher-proxy:PR-${{ github.event.number }}

jobs:
e2e-nats:
runs-on: ubuntu-latest
steps:
- name: Checkout EPP repository
uses: actions/checkout@v4
with:
path: main

- name: Checkout EM repository
uses: actions/checkout@v4
with:
repository: kyma-project/eventing-manager
path: eventing-manager

- name: Cache binaries
id: cache-binaries
uses: actions/cache@v3
with:
path: bin
key: ${{ runner.os }}-bin

- name: Install k3d tools
run: |
make -C eventing-manager/hack/ci/ install-k3d-tools
- name: Install Kyma CLI & setup k3d cluster using Kyma CLI
run: |
make -C eventing-manager kyma
make -C main/hack/ci/ create-k3d
kubectl version
kubectl cluster-info
- name: Create Namespace
run: |
kubectl create ns kyma-system
- name: Deploy NATS Manager
run: |
kubectl apply -f https://github.com/kyma-project/nats-manager/releases/latest/download/nats-manager.yaml
kubectl apply -f https://github.com/kyma-project/nats-manager/releases/latest/download/nats_default_cr.yaml
echo "Using NATS Manager image:"
kubectl get -n kyma-system deployment nats-manager -o=jsonpath='{$.spec.template.spec.containers[:1].image}'
- name: Deploy Eventing Manager
run: |
kubectl apply -f https://github.com/kyma-project/eventing-manager/releases/latest/download/eventing-manager.yaml
kubectl apply -f https://github.com/kyma-project/eventing-manager/releases/latest/download/eventing_default_cr.yaml
echo "Using Eventing Manager image:"
kubectl get -n kyma-system deployment eventing-manager -o=jsonpath='{$.spec.template.spec.containers[:1].image}'
- name: Wait for build job to succeed
uses: kyma-project/wait-for-commit-status-action@2b3ffe09af8b6f40e1213d5fb7f91a7bd41ffb20
with:
context: "pull-eventing-publisher-proxy-build"
commit_ref: "${{ github.event.pull_request.head.sha }}" # Note: 'github.event.pull_request.head.sha' is not same as 'github.sha' on pull requests.
timeout: 600000 # 10 minutes in milliseconds
# The check interval is kept long otherwise it will exhaust the GitHub rate limit (More info: https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limiting)
check_interval: 60000 # 1 minute in milliseconds
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
GITHUB_OWNER: "${{ github.repository_owner }}"
GITHUB_REPO: "eventing-publisher-proxy"

- name: Setup Eventing Manager with new EPP image
run: |
kubectl get deployment eventing-manager -n kyma-system -o=json |
jq --arg new_image "$EPP_IMAGE" \
'.spec.template.spec.containers[0].env |= map(if .name == "PUBLISHER_IMAGE" then .value = $new_image else . end)' |
kubectl apply -f -
kubectl rollout status deployment/eventing-manager -n kyma-system --timeout=120s
kubectl rollout status deployment/eventing-publisher-proxy -n kyma-system --timeout=120s
- name: Check if EPP image successfully applied
run: |
DEPLOYMENT_IMAGE=$(kubectl get deployment -n kyma-system eventing-publisher-proxy -o=jsonpath='{$.spec.template.spec.containers[:1].image}')
if [ "$DEPLOYMENT_IMAGE" != "$EPP_IMAGE" ]; then
echo "EPP images do not match."
echo "Desired EPP image: $EPP_IMAGE. Image in EPP deployment: $DEPLOYMENT_IMAGE"
echo "Updating Eventing Manager with new image failed."
exit 1
else
echo "EPP image update successful."
fi
- name: Setup subscriptions for testing
run: |
make -C eventing-manager e2e-eventing-setup
- name: Test Eventing
run: |
make -C eventing-manager e2e-eventing
- name: Cleanup test resources
run: |
make -C eventing-manager e2e-cleanup
- name: On error get NATS CR
if: failure()
run: |
kubectl get nats -n kyma-system -o yaml
- name: On error get Eventing CR
if: failure()
run: |
kubectl get eventing -n kyma-system -o yaml
7 changes: 7 additions & 0 deletions hack/ci/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
KYMA_CLI ?= "/home/runner/work/eventing-publisher-proxy/eventing-publisher-proxy/eventing-manager/bin/kyma-unstable"
CLUSTER_NAME ?= kyma
REGISTRY_PORT ?= 5001

.PHONY: create-k3d
create-k3d: ## Create k3d with kyma CRDs.
"${KYMA_CLI}" provision k3d -p 8081:80@loadbalancer -p 8443:443@loadbalancer --registry-port ${REGISTRY_PORT} --name ${CLUSTER_NAME} --ci
15 changes: 15 additions & 0 deletions hack/get_kyma_file_name.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

function get_kyma_file_name () {

local _OS_TYPE=$1
local _OS_ARCH=$2

[ "$_OS_TYPE" == "Linux" ] && [ "$_OS_ARCH" == "x86_64" ] && echo "kyma-linux" ||
[ "$_OS_TYPE" == "Linux" ] && [ "$_OS_ARCH" == "arm64" ] && echo "kyma-linux-arm" ||
[ "$_OS_TYPE" == "Windows" ] && [ "$_OS_ARCH" == "x86_64" ] && echo "kyma.exe" ||
[ "$_OS_TYPE" == "Windows" ] && [ "$_OS_ARCH" == "arm64" ] && echo "kyma-arm.exe" ||
[ "$_OS_TYPE" == "Darwin" ] && [ "$_OS_ARCH" == "x86_64" ] && echo "kyma-darwin"
}

get_kyma_file_name "$@"
4 changes: 2 additions & 2 deletions scripts/check_tag_info.sh → scripts/check_tags_in_sec_scanners_config.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash

##############################
# Check tags in security-scan-config.yaml
# Check tags in sec-scanners-config.yaml
# Image Tag, rc-tag
##############################

Expand All @@ -22,7 +22,7 @@ if [[ "$IMAGE_TAG" != "$DESIRED_TAG" ]] || [[ "$RC_TAG" != "$DESIRED_TAG" ]]; th
# ERROR: Tag issue
echo "Tags are not correct:
- wanted: $DESIRED_TAG
- security-scanner image tag: $IMAGE_TAG
- sec-scanner image tag: $IMAGE_TAG
- rc-tag: $RC_TAG"
exit 1
fi
Expand Down
2 changes: 1 addition & 1 deletion scripts/create_changelog.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ set -o pipefail # prevents errors in a pipeline from being masked

RELEASE_TAG=$1

REPOSITORY=${REPOSITORY:-kyma-project/eventing-manager}
REPOSITORY=${REPOSITORY:-kyma-project/eventing-publisher-proxy}
GITHUB_URL=https://api.github.com/repos/${REPOSITORY}
GITHUB_AUTH_HEADER="Authorization: token ${GITHUB_TOKEN}"
CHANGELOG_FILE="CHANGELOG.md"
Expand Down
29 changes: 16 additions & 13 deletions scripts/create_draft_release.sh
Original file line number Diff line number Diff line change
@@ -1,37 +1,40 @@
#!/usr/bin/env bash

# This script returns the id of the draft release
# This script creates a draft release and returns its id .

# standard bash error handling
# Error handling:
set -o nounset # treat unset variables as an error and exit immediately.
set -o errexit # exit immediately when a command fails.
set -E # needs to be set if we want the ERR trap
set -o pipefail # prevents errors in a pipeline from being masked

RELEASE_TAG=$1

REPOSITORY=${REPOSITORY:-kyma-project/eventing-manager}
REPOSITORY=${REPOSITORY:-kyma-project/eventing-publisher-proxy}
GITHUB_URL=https://api.github.com/repos/${REPOSITORY}
GITHUB_AUTH_HEADER="Authorization: Bearer ${GITHUB_TOKEN}"
CHANGELOG_FILE=$(cat CHANGELOG.md)

# Create the json payload to create a draft release.
JSON_PAYLOAD=$(jq -n \
--arg tag_name "$RELEASE_TAG" \
--arg name "$RELEASE_TAG" \
--arg body "$CHANGELOG_FILE" \
'{
--arg tag_name "$RELEASE_TAG" \
--arg name "$RELEASE_TAG" \
--arg body "$CHANGELOG_FILE" \
'{
"tag_name": $tag_name,
"name": $name,
"body": $body,
"draft": true
}')

# Send the payload to github to create the draft release. The response contains the id of the release.
CURL_RESPONSE=$(curl -L \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "${GITHUB_AUTH_HEADER}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
${GITHUB_URL}/releases \
-d "$JSON_PAYLOAD")
-X POST \
-H "Accept: application/vnd.github+json" \
-H "${GITHUB_AUTH_HEADER}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
${GITHUB_URL}/releases \
-d "$JSON_PAYLOAD")

# Return the draft release id.
echo "$(echo $CURL_RESPONSE | jq -r ".id")"
2 changes: 1 addition & 1 deletion scripts/get_next_release_version.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ NEXT_PATCH_VERSION=$(echo "$LATEST_PATCH_VERSION" | awk -F'.' '{print ($3 == ""

# Print the next release version
NEXT_RELEASE_VERSION="$MAJOR_MINOR_VERSION.$NEXT_PATCH_VERSION"
echo "Next release version: $NEXT_RELEASE_VERSION"
echo $NEXT_RELEASE_VERSION
2 changes: 1 addition & 1 deletion scripts/publish_release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ set -o pipefail # prevents errors in a pipeline from being masked

RELEASE_ID=$1

REPOSITORY=${REPOSITORY:-kyma-project/eventing-manager}
REPOSITORY=${REPOSITORY:-kyma-project/eventing-publisher-proxy}
GITHUB_URL=https://api.github.com/repos/${REPOSITORY}
GITHUB_AUTH_HEADER="Authorization: Bearer ${GITHUB_TOKEN}"

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
#!/usr/bin/env bash

echo "Checking status of POST Jobs for Eventing-Manager"
echo "Checking status of POST Jobs for Eventing-Publisher-Proxy"

REF_NAME="${1:-"main"}"
TIMEOUT_TIME="${2:-600}"
INTERVAL_TIME="${3:-3}"
INITIAL_WAIT_TIME="${4:-30}"

# Generate job Status URL
STATUS_URL="https://api.github.com/repos/kyma-project/eventing-manager/commits/${REF_NAME}/status"
STATUS_URL="https://api.github.com/repos/kyma-project/eventing-publisher-proxy/commits/${REF_NAME}/status"

# Dates
START_TIME=$(date +%s)
Expand Down
2 changes: 1 addition & 1 deletion sec-scanners-config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
module-name: eventing-publisher-proxy
rc-tag: 0.0.0
protecode:
- europe-docker.pkg.dev/kyma-project/prod/event-publisher-proxy:v20231025-3f5d1600
- europe-docker.pkg.dev/kyma-project/prod/eventing-publisher-proxy:0.0.0
whitesource:
language: golang-mod
subprojects: false
Expand Down

0 comments on commit 3e70eb2

Please sign in to comment.