Release Automation

[s373nZ]

This PR fixes #7776 by adding a new CI action called "Release" which is triggered when a release tag is pushed, or via manual dispatch. The action:

1. Does some sanity checking of the release tag against the branch and Git branch to short-circuit prior to lengthy build time.
2. Prepares the build environments, builds the release targets and uploads release artifacts in parallel, as a matrix strategy.
3. Merges the independent build artifacts into a final release artifact, deleting the intermediaries.
4. Signs the releases using a PGP key configured in the repository secret configuration.
5. Creates a new release in the repository and attaches the artifacts as assets.

Requires the addition of repository variables named GPG_PRIVATE_KEY and GPG_PASSPHRASE which represent a CI signing key. The release captain and signers would be expected to perform a local reproducible build and add their key to SHA256SUMS.asc as a manual process and re-upload that asset to the release.

Release documentation has been updated as a best guess for rc1 instructions.

Fixes #7776
Fixes #6228

Notes

• Modified the existing tools/build-release.sh script to allow for distribution-specific targets for the Ubuntu builds, for parallelization.
• Also added a --without-zip option to tools/build-release.sh script to skip creating a default archive when one just wants to sign the artifacts from Github.

Testing

It's a little hard to test, as I think tag pushes and workflow_dispatches need to be in master to trigger. However, I have another branch with some testing configuration setup here and have been running the CI via a branch push (ex https://github.com/s373nZ/lightning/actions/runs/11728964685).

In order to test, one must:

1. Update the CHANGELOG.md file with a dummy release title line that will map to a release tag.
2. Tag the local branch with that release tag and push it.
3. Push the testing branch which is set to run by branch configuration.

See this commit for an example.

Possible improvements or TODOs

• Audit and remove unused Dockerfiles and release tooling.
• Add a PGP signing key to as a Github secret variable and do initial release signing as part of the CI as well.
• Create the draft release object in CI and assign the release artifacts to it.
• Try to address Releases don't include architecture type #6228 as well, by changing the release file names to include amd64 in the Ubuntu target versions.

Checklist

Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked:

• The changelog has been updated in the relevant commit(s) according to the guidelines.
• Tests have been added or modified to reflect the changes.
• Documentation has been reviewed and updated as needed.
• Related issues have been listed and linked, including any that this PR closes.

[s373nZ]

• Made a few tweaks to the shell script changes to satisfy shellcheck.
• Added a commit to remove the following seemingly old Dockerfiles:
  • action.yml
    • Seems to try and declare a reusable Github Action?
    • Reference to contrib/Dockerfile.tester is broken.
  • contrib/docker/Dockerfile.alpine
  • contrib/docker/Dockerfile.builder
  • contrib/docker/Dockerfile.tester
  • contrib/docker/Dockerfile.ubuntu
  • contrib/docker/scripts/*
  • I was tempted to move contrib/docker/Dockerfile.builder.fedora to contrib/reprobuild/, but it's technically non-reproducible. LMK if we think it should go there anyway.
• Changed the tools/check-release.sh to require a --version argument and changed the comments to make more sense after updating the script from reading $GITHUB_* variable to passing the version in as an argument.
• Removed an echo statement from debugging Ubuntu $distribution parsing.

[s373nZ]

Rebased against master and added a commit to change the Ubuntu release artifact filenames to include the architecture (amd64). Can move this to a different PR if you prefer.

Fixes #6228

[s373nZ]

One possible improvement, or point of discussion, would be to have manually triggered workflows just check out the tag that is specified by user input, instead of expecting the tag they enter to match that of HEAD.

Advantages are that one could manually build releases for arbitrary tags, avoids somewhat flaky expectation that tag == HEAD, and possibly more intuitive.

Disadvantage is that we might need duplicate checkout actions for each trigger, and less enforcement that releases are on the cutting edge of the master branch.

[s373nZ]

From @ShahanaFarooqui on Discord:

Regarding your question about configuring the signing key (#7776 (comment)), we could proceed on the assumption that team members won’t need to upload their keys. Instead, we can create a dedicated signing key for the 'core lightning' account to sign the release. Then team members would be able to verify the release locally.

Latest commits add a new release job which signs the artifacts using a GPG key and passphrase stored as Github secrets (I used repository-level) GPG_PRIVATE_KEY and GPG_PASSPHRASE. This step generates the SHA256SUMS and SHA256SUMS.asc files, then prepares a Github draft release from some metadata from the CHANGELOG.md and attaches the release artifacts.

Also removed the workflow_dispatch input to specify a release tag, as when you kick off a manual workflow, it is assumed you choose the branch or tag to run from. I also added a create_release input choice to yes or no in case a release captain wishes to generate release assets for a release without creating the Release draft.

[s373nZ]

Rebased against master, fixed a typo in the documentation and updated the PR description with extra feature functionality and a note about the required secrets configuration for Github Actions.