Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved workflows for CI and releases #343

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/actions/repo-state/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: 'Repository state'
description: 'Get the latest tag, the latest process-note, the number of commits since the last tag and the version numbers from the last tag.'
outputs:
tag:
description: "Last tagged version"
value: ${{ steps.tag.outputs.value }}
note:
description: "Note on the last tag"
value: ${{ steps.note.outputs.value }}
commits:
description: "Number of commits since the last tag"
value: ${{ steps.commits.outputs.value }}
version_major:
description: "Major version number"
value: ${{ steps.version.outputs.major }}
version_minor:
description: "Minor version number"
value: ${{ steps.version.outputs.minor }}
version_revision:
description: "Revision number"
value: ${{ steps.version.outputs.revision }}
runs:
using: "composite"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: 'master'
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does ref: master do here? Just guarantee that the were are not looking for tags on branches?

Copy link
Contributor Author

@Masgalor Masgalor Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow runs per default with the commit that was in HEAD when it started, so checkout will always refer to this commit. This is bad for us, because we change some git-data or add another commit in the process, so we have to tell checkout not to use the context it got from the workflow but to work with the actual HEAD of master.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I totally follow yet. Isn't this workflow just searching back through the history to find tags and notes? What git data is being changed in this workflow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This action does not add anything, it is read-only, that is correct.
The workflow Release Bugfix might add a new commit during job: Check and add a new release-tag to it if there was no commit after the last tag. If we get to job: Publish we use this action to get the tag that was created earlier. But by default we would neither see the commit nor the tag because they did not exist before the workflow was started. ref: master makes sure we can reach commits that were added during one of our workflows.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found another discussion that might explain this better than I can.
https://github.com/orgs/community/discussions/25985

They solve the problem slightly different, but it will help to understand the underlying problem.

- name: Get the last release-tag
id: tag
AsherGlick marked this conversation as resolved.
Show resolved Hide resolved
shell: bash
run: echo "value=$(git describe --tags --abbrev=0)" >> $GITHUB_OUTPUT
- name: Get the note on the last release-tag
id: note
shell: bash
run: |
git fetch origin refs/notes/*:refs/notes/*
echo 'value<<EOF' >> $GITHUB_OUTPUT
(git notes show $(git describe --tags --abbrev=0) | cat || true) >> $GITHUB_OUTPUT
AsherGlick marked this conversation as resolved.
Show resolved Hide resolved
echo 'EOF' >> $GITHUB_OUTPUT
- name: Get the commits since the last tag
id: commits
shell: bash
run: echo "value=$(git rev-list --count $(git describe --tags --abbrev=0)..HEAD)" >> $GITHUB_OUTPUT
- name: Get the version numbers
id: version
shell: bash
run: |
echo "major=$(git describe --tags --abbrev=0 | sed 's@^[^0-9]*\([0-9]\+\).*@\1@;tx;s/.*/0/;:x')" >> $GITHUB_OUTPUT
echo "minor=$(git describe --tags --abbrev=0 | sed 's@^[^0-9]*[0-9]*[.]\([0-9]\+\).*@\1@;tx;s/.*/0/;:x')" >> $GITHUB_OUTPUT
echo "revision=$(git describe --tags --abbrev=0 | sed 's@^[^0-9]*[0-9]*[.][0-9]*[.]\([0-9]\+\).*@\1@;tx;s/.*/0/;:x')" >> $GITHUB_OUTPUT
103 changes: 103 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: Build

on:
workflow_call:
inputs:
godot-version:
required: true
type: string
ubuntu-version:
required: true
type: string
Masgalor marked this conversation as resolved.
Show resolved Hide resolved
outputs:
checksum:
description: "md5 sums of the generated output files"
value: ${{ jobs.build.outputs.checksum }}

jobs:
build:
runs-on: ubuntu-${{ inputs.ubuntu-version }}
outputs:
checksum: ${{ steps.final.outputs.checksum }}
steps:
- uses: actions/checkout@v4
- name: Restore cached dependencies
uses: actions/cache/restore@v4
id: cache-godot
with:
path: |
Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip
Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz
key: ${{ runner.os }}-${{ inputs.ubuntu-version }}-${{ inputs.godot-version }}
- name: Download Godot and its Export Templates
if: steps.cache-godot.outputs.cache-hit != 'true'
run: |
wget -q https://downloads.tuxfamily.org/godotengine/${{ inputs.godot-version }}/Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip
wget -q https://downloads.tuxfamily.org/godotengine/${{ inputs.godot-version }}/Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz
- name: Add dependencies to the cache if necessary
if: steps.cache-godot.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: |
Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip
Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz
key: ${{ steps.cache-godot.outputs.cache-primary-key }}
- name: Install Godot and the export templates
run: |
unzip Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip
rm Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip
unzip Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz
rm Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz
mkdir -v -p ~/.local/share/godot/templates/${{ inputs.godot-version }}.stable/
mv templates/* ~/.local/share/godot/templates/${{ inputs.godot-version }}.stable/
ls ~/.local/share/godot/templates/${{ inputs.godot-version }}.stable/
AsherGlick marked this conversation as resolved.
Show resolved Hide resolved
- name: Install mingw
run: sudo apt-get install gcc-mingw-w64
- name: Create the output directories
run: |
mkdir output
mkdir output/burrito_link
- name: Build X11_FG
run: |
cd burrito-fg
cargo build --release
- name: Build taco_parser
run: |
cd taco_parser
cargo build --release
- name: Build Burrito Link
run: |
mkdir burrito_link/build
cd burrito_link/build
cmake ..
make
- name: Build Burrito
run: |
mkdir build
./Godot_v${{ inputs.godot-version }}-stable_linux_headless.64 --export "Linux/X11"
chmod +x build/burrito.x86_64
- name: Fill the output folder
run: |
mv burrito_link/build/burrito_link.exe output/burrito_link
mv burrito_link/build/d3d11.dll output/burrito_link
mv build/burrito.x86_64 output/
mv build/libburrito_fg.so output/
mv build/libgw2_taco_parser.so output/
- name: Upload output folder as an artifact
uses: actions/upload-artifact@v4
id: artifact-upload
with:
name: burrito-binaries
path: output
if-no-files-found: error
overwrite: true
- name: Strip the burrito_link binaries for reproducibility
run: strip output/burrito_link/*
Masgalor marked this conversation as resolved.
Show resolved Hide resolved
- name: Get the md5 sums for the produced output files
id: final
run: |
mv output/burrito_link/* output/
rm -rf output/burrito_link
echo 'checksum<<EOF' >> $GITHUB_OUTPUT
md5sum output/* >> $GITHUB_OUTPUT
Masgalor marked this conversation as resolved.
Show resolved Hide resolved
echo 'EOF' >> $GITHUB_OUTPUT
12 changes: 12 additions & 0 deletions .github/workflows/ci-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: CI

on:
pull_request:
branches: [ "*" ]

jobs:
Build:
uses: ./.github/workflows/build.yml
with:
godot-version: 3.3.2
ubuntu-version: 20.04
122 changes: 0 additions & 122 deletions .github/workflows/main.yml

This file was deleted.

57 changes: 57 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Publish

on:
workflow_call:
inputs:
artifact-name:
required: true
type: string
release-message:
type: string
add-commits:
type: string
checksum:
type: string

jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: .github
- name: Get the repository state
uses: ./.github/actions/repo-state
id: repo-state
- name: Download all available artifacts
uses: actions/download-artifact@v4
- name: Zip the output files for deployment
run: (cd burrito-binaries && zip -r ../${{ inputs.artifact-name }} .)
- name: Handle the manually entered release message if it exists
if: inputs.release-message != ''
run: |
echo "${{ inputs.release-message }}" >> release-notes-tmp
echo "" >> release-notes-tmp
- name: Create an automatic release message if requested or necessary
id: release-message-auto
if: inputs.release-message == '' || inputs.add-commits == 'true'
uses: simbo/changes-between-tags-action@v1
with:
tag-pattern: '^([a-z]+)?-[0-9]+\.[0-9]+\.?([0-9]+)?$'
validate-tag: 'false'
- name: Handle the automatically generated release message if requested or necessary
if: inputs.release-message == '' || inputs.add-commits == 'true'
run: echo "${{ steps.release-message-auto.outputs.changes }}" >> release-notes-tmp
- name: Create and publish the final release
uses: ncipollo/release-action@v1
with:
tag: ${{ steps.repo-state.outputs.tag }}
artifacts: ${{ inputs.artifact-name }}
bodyFile: release-notes-tmp
- name: Write a git-note with all checksums to the latest release
run: |
git fetch origin refs/notes/*:refs/notes/*
git config user.email 'github-actions[bot]@users.noreply.github.com'
git config user.name 'github-actions[bot]'
git notes add -f -m "${{ inputs.checksum }}" $(git describe --tags --abbrev=0)
git push origin refs/notes/*
Loading