-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved workflows for CI and releases #343
Open
Masgalor
wants to merge
4
commits into
AsherGlick:master
Choose a base branch
from
Masgalor:workflows
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
name: 'Repository state' | ||
description: 'Get the latest tag, the latest process-note, the number of commits since the last tag and the version numbers from the last tag.' | ||
outputs: | ||
tag: | ||
description: "Last tagged version" | ||
value: ${{ steps.tag.outputs.value }} | ||
note: | ||
description: "Note on the last tag" | ||
value: ${{ steps.note.outputs.value }} | ||
commits: | ||
description: "Number of commits since the last tag" | ||
value: ${{ steps.commits.outputs.value }} | ||
version_major: | ||
description: "Major version number" | ||
value: ${{ steps.version.outputs.major }} | ||
version_minor: | ||
description: "Minor version number" | ||
value: ${{ steps.version.outputs.minor }} | ||
version_revision: | ||
description: "Revision number" | ||
value: ${{ steps.version.outputs.revision }} | ||
runs: | ||
using: "composite" | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
ref: 'master' | ||
- name: Get the last release-tag | ||
id: tag | ||
AsherGlick marked this conversation as resolved.
Show resolved
Hide resolved
|
||
shell: bash | ||
run: echo "value=$(git describe --tags --abbrev=0)" >> $GITHUB_OUTPUT | ||
- name: Get the note on the last release-tag | ||
id: note | ||
shell: bash | ||
run: | | ||
git fetch origin refs/notes/*:refs/notes/* | ||
echo 'value<<EOF' >> $GITHUB_OUTPUT | ||
(git notes show $(git describe --tags --abbrev=0) | cat || true) >> $GITHUB_OUTPUT | ||
AsherGlick marked this conversation as resolved.
Show resolved
Hide resolved
|
||
echo 'EOF' >> $GITHUB_OUTPUT | ||
- name: Get the commits since the last tag | ||
id: commits | ||
shell: bash | ||
run: echo "value=$(git rev-list --count $(git describe --tags --abbrev=0)..HEAD)" >> $GITHUB_OUTPUT | ||
- name: Get the version numbers | ||
id: version | ||
shell: bash | ||
run: | | ||
echo "major=$(git describe --tags --abbrev=0 | sed 's@^[^0-9]*\([0-9]\+\).*@\1@;tx;s/.*/0/;:x')" >> $GITHUB_OUTPUT | ||
echo "minor=$(git describe --tags --abbrev=0 | sed 's@^[^0-9]*[0-9]*[.]\([0-9]\+\).*@\1@;tx;s/.*/0/;:x')" >> $GITHUB_OUTPUT | ||
echo "revision=$(git describe --tags --abbrev=0 | sed 's@^[^0-9]*[0-9]*[.][0-9]*[.]\([0-9]\+\).*@\1@;tx;s/.*/0/;:x')" >> $GITHUB_OUTPUT |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
name: Build | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
godot-version: | ||
required: true | ||
type: string | ||
ubuntu-version: | ||
required: true | ||
type: string | ||
Masgalor marked this conversation as resolved.
Show resolved
Hide resolved
|
||
outputs: | ||
checksum: | ||
description: "md5 sums of the generated output files" | ||
value: ${{ jobs.build.outputs.checksum }} | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-${{ inputs.ubuntu-version }} | ||
outputs: | ||
checksum: ${{ steps.final.outputs.checksum }} | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Restore cached dependencies | ||
uses: actions/cache/restore@v4 | ||
id: cache-godot | ||
with: | ||
path: | | ||
Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip | ||
Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz | ||
key: ${{ runner.os }}-${{ inputs.ubuntu-version }}-${{ inputs.godot-version }} | ||
- name: Download Godot and its Export Templates | ||
if: steps.cache-godot.outputs.cache-hit != 'true' | ||
run: | | ||
wget -q https://downloads.tuxfamily.org/godotengine/${{ inputs.godot-version }}/Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip | ||
wget -q https://downloads.tuxfamily.org/godotengine/${{ inputs.godot-version }}/Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz | ||
- name: Add dependencies to the cache if necessary | ||
if: steps.cache-godot.outputs.cache-hit != 'true' | ||
uses: actions/cache/save@v4 | ||
with: | ||
path: | | ||
Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip | ||
Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz | ||
key: ${{ steps.cache-godot.outputs.cache-primary-key }} | ||
- name: Install Godot and the export templates | ||
run: | | ||
unzip Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip | ||
rm Godot_v${{ inputs.godot-version }}-stable_linux_headless.64.zip | ||
unzip Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz | ||
rm Godot_v${{ inputs.godot-version }}-stable_export_templates.tpz | ||
mkdir -v -p ~/.local/share/godot/templates/${{ inputs.godot-version }}.stable/ | ||
mv templates/* ~/.local/share/godot/templates/${{ inputs.godot-version }}.stable/ | ||
ls ~/.local/share/godot/templates/${{ inputs.godot-version }}.stable/ | ||
AsherGlick marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- name: Install mingw | ||
run: sudo apt-get install gcc-mingw-w64 | ||
- name: Create the output directories | ||
run: | | ||
mkdir output | ||
mkdir output/burrito_link | ||
- name: Build X11_FG | ||
run: | | ||
cd burrito-fg | ||
cargo build --release | ||
- name: Build taco_parser | ||
run: | | ||
cd taco_parser | ||
cargo build --release | ||
- name: Build Burrito Link | ||
run: | | ||
mkdir burrito_link/build | ||
cd burrito_link/build | ||
cmake .. | ||
make | ||
- name: Build Burrito | ||
run: | | ||
mkdir build | ||
./Godot_v${{ inputs.godot-version }}-stable_linux_headless.64 --export "Linux/X11" | ||
chmod +x build/burrito.x86_64 | ||
- name: Fill the output folder | ||
run: | | ||
mv burrito_link/build/burrito_link.exe output/burrito_link | ||
mv burrito_link/build/d3d11.dll output/burrito_link | ||
mv build/burrito.x86_64 output/ | ||
mv build/libburrito_fg.so output/ | ||
mv build/libgw2_taco_parser.so output/ | ||
- name: Upload output folder as an artifact | ||
uses: actions/upload-artifact@v4 | ||
id: artifact-upload | ||
with: | ||
name: burrito-binaries | ||
path: output | ||
if-no-files-found: error | ||
overwrite: true | ||
- name: Strip the burrito_link binaries for reproducibility | ||
run: strip output/burrito_link/* | ||
Masgalor marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- name: Get the md5 sums for the produced output files | ||
id: final | ||
run: | | ||
mv output/burrito_link/* output/ | ||
rm -rf output/burrito_link | ||
echo 'checksum<<EOF' >> $GITHUB_OUTPUT | ||
md5sum output/* >> $GITHUB_OUTPUT | ||
Masgalor marked this conversation as resolved.
Show resolved
Hide resolved
|
||
echo 'EOF' >> $GITHUB_OUTPUT |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
name: CI | ||
|
||
on: | ||
pull_request: | ||
branches: [ "*" ] | ||
|
||
jobs: | ||
Build: | ||
uses: ./.github/workflows/build.yml | ||
with: | ||
godot-version: 3.3.2 | ||
ubuntu-version: 20.04 |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
name: Publish | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
artifact-name: | ||
required: true | ||
type: string | ||
release-message: | ||
type: string | ||
add-commits: | ||
type: string | ||
checksum: | ||
type: string | ||
|
||
jobs: | ||
publish: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
sparse-checkout: .github | ||
- name: Get the repository state | ||
uses: ./.github/actions/repo-state | ||
id: repo-state | ||
- name: Download all available artifacts | ||
uses: actions/download-artifact@v4 | ||
- name: Zip the output files for deployment | ||
run: (cd burrito-binaries && zip -r ../${{ inputs.artifact-name }} .) | ||
- name: Handle the manually entered release message if it exists | ||
if: inputs.release-message != '' | ||
run: | | ||
echo "${{ inputs.release-message }}" >> release-notes-tmp | ||
echo "" >> release-notes-tmp | ||
- name: Create an automatic release message if requested or necessary | ||
id: release-message-auto | ||
if: inputs.release-message == '' || inputs.add-commits == 'true' | ||
uses: simbo/changes-between-tags-action@v1 | ||
with: | ||
tag-pattern: '^([a-z]+)?-[0-9]+\.[0-9]+\.?([0-9]+)?$' | ||
validate-tag: 'false' | ||
- name: Handle the automatically generated release message if requested or necessary | ||
if: inputs.release-message == '' || inputs.add-commits == 'true' | ||
run: echo "${{ steps.release-message-auto.outputs.changes }}" >> release-notes-tmp | ||
- name: Create and publish the final release | ||
uses: ncipollo/release-action@v1 | ||
with: | ||
tag: ${{ steps.repo-state.outputs.tag }} | ||
artifacts: ${{ inputs.artifact-name }} | ||
bodyFile: release-notes-tmp | ||
- name: Write a git-note with all checksums to the latest release | ||
run: | | ||
git fetch origin refs/notes/*:refs/notes/* | ||
git config user.email 'github-actions[bot]@users.noreply.github.com' | ||
git config user.name 'github-actions[bot]' | ||
git notes add -f -m "${{ inputs.checksum }}" $(git describe --tags --abbrev=0) | ||
git push origin refs/notes/* |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does
ref: master
do here? Just guarantee that the were are not looking for tags on branches?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The workflow runs per default with the commit that was in
HEAD
when it started, socheckout
will always refer to this commit. This is bad for us, because we change some git-data or add another commit in the process, so we have to tellcheckout
not to use the context it got from the workflow but to work with the actualHEAD
ofmaster
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think I totally follow yet. Isn't this workflow just searching back through the history to find tags and notes? What git data is being changed in this workflow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This action does not add anything, it is read-only, that is correct.
The workflow
Release Bugfix
might add a new commit duringjob: Check
and add a new release-tag to it if there was no commit after the last tag. If we get tojob: Publish
we use this action to get the tag that was created earlier. But by default we would neither see the commit nor the tag because they did not exist before the workflow was started.ref: master
makes sure we can reach commits that were added during one of our workflows.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found another discussion that might explain this better than I can.
https://github.com/orgs/community/discussions/25985
They solve the problem slightly different, but it will help to understand the underlying problem.