Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved workflows for CI and releases #343

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

Masgalor
Copy link
Contributor

As discussed here #328 and in accordance with #333 this PR fundamentally reworks the workflows used for CI and release-management.

This PR covers every aspect of the meta-issue for this change.
I am aware that this is not how it was outlined in the action plan, but I found it difficult to develop this in individual steps.
My idea for testing this right now is, to create a fork of AsherGlick/Burrito and merge #342 and this PR into it so every scenario imaginable can be tested. That is what I did during development.

We can split this into some smaller PRs, but since I am working with reusable workflows and actions I see this more as a one or nothing change.

That's it for the intro, lets talk about the details.


What is the general concept?

First step is to build the git repository as is job: Pre-Build, this is just to get the checksums of the binaries when they are built from the latest commit.
The second step is job: Check, we want to check here if the binaries are different from the last ones, or if we have an untagged commit to create a new release. I know it would be more efficient to check for this before building the reference binaries, but this increases complexity while having no real benefit. If we are sure there are changes warranting a new release we will create a new tag for it.
The third step job: Build will build the binaries again, this not important right now as those would not differ from the Pre-Buildones. But this is necessary to use the new tag to set the right version when building the binaries (in the future).
The final step job: Publish will create a release and write the checksum for every binary to a git-note at the current tag.


What individual parts do we have?

Composite action to analyze our git repository

We need some information from our git data all the time, so we use the custom action `repo-state' to generate an overview for us...

  • Last tagged version
  • Note on the last tag
  • Number of commits since the last tag
  • Major version number
  • Minor version number
  • Revision number

Reusable build-job

The build-process is always exactly the same, otherwise we were not able to compare the checksums of the produced binaries.
This job takes care of...

  • Downloading and caching the godot installer and templates
  • Installing build-dependencies as mingw
  • Building everything
  • Uploading the built binaries as an artifact
  • Creating checksums of everything

Reusable publish-job

If we made it to the publish-stage it is always the same...

  • Download the artifacts and package them
  • Create a release note
  • Create a release
  • Write the checksums of the released files to the last tag's note

Create a major release

This is a manually called workflow that creates a major release using the reusable jobs above.
It requires a release-note when triggered by hand and can optionally add the commit messages of the relevant commits.
This workflow can only be triggered by hand, it will abort if there are no commits without tag or if the binaries are the same as with the last release.

Create a minor release

This is a automatically called workflow that creates a minor release using the reusable jobs above.
It is automatically triggered every Tuesday 13h, it does not depend on any inputs, the commit message will be automatically created from the relevant commits.
This workflow will abort if there are no commits without tag or if the binaries are the same as with the last release.

Create a bugfix release

This is a manually called workflow that creates a bugfix release using the reusable jobs above.
It requires a release-note when triggered by hand and can optionally add the commit messages of the relevant commits.
This workflow can only be triggered by hand, it will do anything necessary to make sure a new release is created, no matter if the reference-binaries are the same as with the last release or if there is no commit without a tag. If there is no commit without tag it will create one to make sure there is no commit with two tags at the same time.


Notes

We can discuss changes or how to split this in the appropriate issues.
I do not like dependencies to random libraries therefore the only external resources are ncipollo/release-action@v1 and simbo/changes-between-tags-action@v1.

Copy link
Owner

@AsherGlick AsherGlick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think we should still split this up into smaller PRs. I want to get this running in production in the order I set out the original issues so we can spend some time making sure everything is working as intended.

Your plan to make a fork that runs these workflows is a good idea and will greatly help with the testing, but I still want to go with my release schedule.

This is likely going to be a giant merge conflict with the workflow changes we made in the xml_converter branch. Those changes split up the compile steps to run in parallel. I'm ok with resolving that conflict so long as we can mostly overwrite the "build" step and apply some of the changes you made, such as caching.

- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: 'master'
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does ref: master do here? Just guarantee that the were are not looking for tags on branches?

Copy link
Contributor Author

@Masgalor Masgalor Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow runs per default with the commit that was in HEAD when it started, so checkout will always refer to this commit. This is bad for us, because we change some git-data or add another commit in the process, so we have to tell checkout not to use the context it got from the workflow but to work with the actual HEAD of master.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I totally follow yet. Isn't this workflow just searching back through the history to find tags and notes? What git data is being changed in this workflow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This action does not add anything, it is read-only, that is correct.
The workflow Release Bugfix might add a new commit during job: Check and add a new release-tag to it if there was no commit after the last tag. If we get to job: Publish we use this action to get the tag that was created earlier. But by default we would neither see the commit nor the tag because they did not exist before the workflow was started. ref: master makes sure we can reach commits that were added during one of our workflows.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found another discussion that might explain this better than I can.
https://github.com/orgs/community/discussions/25985

They solve the problem slightly different, but it will help to understand the underlying problem.

.github/actions/repo-state/action.yml Show resolved Hide resolved
.github/actions/repo-state/action.yml Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/release_major.yml Outdated Show resolved Hide resolved
@Masgalor Masgalor mentioned this pull request Aug 23, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants