Improve release automation

changelog/+newsfrag.added.md

@@ -0,0 +1 @@
+Added `breaking` news fragment type to towncrier

changelog/18.added.md

@@ -0,0 +1 @@
+Improved release automation: Added workflow that builds the changelog and creates/updates a PR on pushes to the default branch. Added trigger for release workflow when this PR is merged.

data/versions.yaml

@@ -72,6 +72,15 @@ dorny/paths-filter: 'de90cc6fb38fc0963ad72b210f1f284cd68cea36  # v3.0.2'
 # renovate: datasource=git-tags depName=https://github.com/geekyeggo/delete-artifact depType=action
 geekyeggo/delete-artifact: '7ee91e82b4a7f3339cd8b14beace3d826a2aac39  # v5.1.0'
 
+# renovate: datasource=git-tags depName=https://github.com/juliangruber/find-pull-request-action depType=action
+juliangruber/find-pull-request-action: '2f36c5fe1abfda4745dfab4f38217ebad8ded4eb  # v1.9.0'
+
+# renovate: datasource=git-tags depName=https://github.com/mathieudutour/github-tag-action depType=action
+mathieudutour/github-tag-action: 'd28fa2ccfbd16e871a4bdf35e11b3ad1bd56c0c1  # v6.2'
+
+# renovate: datasource=git-tags depName=https://github.com/peter-evans/create-pull-request depType=action
+peter-evans/create-pull-request: '5e914681df9dc83aa4e4905692ca88beb2f9e91f  # v7.0.5'
+
 # renovate: datasource=git-tags depName=https://github.com/pypa/gh-action-pypi-publish depType=action
 pypa/gh-action-pypi-publish: 'c44d2f0e52f028349e3ecafbf7f32561da677277  # v1.10.3'
 

docs/topics/documenting/changelog.md

@@ -1,3 +1,4 @@
+(changelog-target)=
 # Keeping a changelog
 
 Your Saltext project uses [towncrier](https://towncrier.readthedocs.io/en/stable/) to manage and render its {path}`CHANGELOG.md` file, which is included in the rendered documentation as well.
@@ -20,10 +21,15 @@ For every user-facing change, ensure your patch includes a corresponding news fr
     * `changed`
     * `removed`
     * `deprecated`
+    * `breaking`
     * `security`
 
 4. The file contents should be written in Markdown.
 
+:::{hint}
+It's possible to create a news fragment that does not reference an issue by prefixing the file name with a `+`, e.g. `+foo.changed.md`.
+:::
+
 ## Example
 
 Suppose a PR fixes a crash when the `foo.bar` configuration value is missing. The news fragment can be created as follows:
@@ -36,4 +42,4 @@ Include this file in the PR.
 
 ## Building the changelog
 
-Before tagging a release, the individual `changelog/*.md` files need to be compiled into the actual changelog. Refer to [Building the changelog](changelog-build-target) for instructions on how to do this.
+Before tagging a release, the individual `changelog/*.md` files need to be compiled into the actual changelog. This is taken care of by the [release automation](release-automated-target). For [manual releases](release-manual-target), refer to [Building the changelog](changelog-build-target) for instructions on how to do this.

docs/topics/publishing.md

@@ -9,11 +9,32 @@ There are currently no included workflows for other Git hosting providers or CI
 
 Once your Salt extension is ready, you can submit it to PyPI.
 
-## 0: Prerequisites
+Ensure you meet the following prerequisites:
 
 * Your project is hosted on GitHub.
 * It is either in the `salt-extensions` organization or you have set up the [required secrets](required-secrets-target).
 * You have commit rights to the repository.
+
+(release-automated-target)=
+## Automated
+Generated projects include a workflow that automatically detects the next version bump based on [news fragments](changelog-target) in {path}`changelog`, builds the changelog and submits a PR with these changes. Once you are ready to release a new version, simply merge this PR, which creates a new git tag and triggers the release workflow.
+
+:::{important}
+Before merging, ensure the PR is based on the current default branch HEAD.
+:::
+
+:::{hint}
+To force a custom version or manually trigger an update to the release PR (e.g. to adjust the release date), go to `Actions` > `Prepare Release PR` > `Run workflow`.
+:::
+
+:::{note}
+The generated PR is only created automatically if there is at least one news fragment to render. You can still trigger a manual run as described above.
+:::
+
+(release-manual-target)=
+## Manual
+### 0: Prerequisites
+
 * You have added a git remote `upstream` to your local repository, pointing to the official repository via **SSH**.
 * You have executed the [first steps](first-steps-target) to setup your repository and virtual environment in some way.
 * You have activated your virtual environment.
@@ -25,7 +46,7 @@ git switch main && git fetch upstream && git rebase upstream/main
 ```
 
 (changelog-build-target)=
-## 1: Build the changelog
+### 1: Build the changelog
 
 Create and switch to a new branch:
 
@@ -41,15 +62,15 @@ towncrier build --yes --version v1.0.0
 
 This command combines all news fragments into {path}`CHANGELOG.md` and clears them. Commit the change.
 
-## 2: Submit the changelog
+### 2: Submit the changelog
 
 Submit this commit as a PR and merge it into the default branch on `upstream`.
 
 :::{tip}
 Squash-merging this PR results in a cleaner tag target.
 :::
 
-## 3: Tag a release
+### 3: Tag a release
 
 Ensure your `main` branch is up to date (again):
 
@@ -67,14 +88,14 @@ git tag v1.0.0
 The tag must start with `v` for the default publishing workflows to work correctly.
 :::
 
-## 4: Push the tag
+### 4: Push the tag
 
 Push the new tag upstream to trigger the publishing workflow:
 
 ```bash
 git push upstream v1.0.0
 ```
 
-## 5: Check the result
+### 5: Check the result
 
 If CI passes, a new release should be available on both PyPI and your GitHub repository.

docs/topics/workflows.md

@@ -10,6 +10,7 @@ The workflows currently:
 * Ensure `pre-commit` checks pass
 * Run the test suite and upload code coverage reports
 * Build the documentation
+* Build the changelog and submit a PR that triggers a release when merged
 * Optionally deploy built documentation to GitHub Pages
 * Optionally build and release your project to PyPI
 

project/pyproject.toml.j2

@@ -132,6 +132,11 @@ title_format = "## {version} ({project_date})"
 issue_format = "[#{issue}]({{ tracker_url }}/{issue})"
 {%- endif %}
 
+[[tool.towncrier.type]]
+directory = "breaking"
+name = "Breaking changes"
+showcontent = true
+
 [[tool.towncrier.type]]
 directory = "removed"
 name = "Removed"

project/tools/version.py

@@ -0,0 +1,92 @@
+"""
+Very simple heuristic to generate the next version number
+based on the current changelog news fragments.
+
+This looks for the most recent version by parsing the
+CHANGELOG.md file and increments a specific part,
+depending on the fragment types present and their contents.
+
+Major bumps are caused by:
+  * files named `.removed.md`
+  * files named `.breaking.md`
+  * files containing `BREAKING:`
+
+Minor bumps are caused by:
+  * files named `.added.md`
+
+Otherwise, only the patch version is bumped.
+"""
+
+import re
+import sys
+from pathlib import Path
+
+PROJECT_ROOT = Path(__file__).parent.parent.resolve()
+CHANGELOG_DIR = PROJECT_ROOT / "changelog"
+CHANGELOG_FILE = PROJECT_ROOT / "CHANGELOG.md"
+
+
+class Version:
+    def __init__(self, version):
+        match = re.search(r"v?(?P<release>[0-9]+(?:\.[0-9]+)*)", version)
+        if not match:
+            raise ValueError(f"Invalid version: '{version}'")
+        self.release = tuple(int(i) for i in match.group("release").split("."))
+
+    @property
+    def major(self):
+        return self._ret(0)
+
+    @property
+    def minor(self):
+        return self._ret(1)
+
+    @property
+    def patch(self):
+        return self._ret(2)
+
+    def __str__(self):
+        return ".".join(str(i) for i in self.release)
+
+    def _ret(self, cnt):
+        try:
+            return self.release[cnt]
+        except IndexError:
+            return 0
+
+
+def last_release():
+    for line in CHANGELOG_FILE.read_text(encoding="utf-8").splitlines():
+        if line.startswith("## "):
+            return Version(line.split(" ")[1])
+    return Version("0.0.0")
+
+
+def get_next_version(last):
+    major = minor = False
+
+    for fragment in CHANGELOG_DIR.glob("[!.]*"):
+        name = fragment.name.lower()
+        if ".added" in name:
+            minor = True
+        elif ".breaking" in name or ".removed" in name:
+            major = True
+            break
+        if "breaking:" in fragment.read_text(encoding="utf-8").lower():
+            major = True
+            break
+    if major:
+        return Version(f"{last.major + 1}.0.0")
+    if minor:
+        return Version(f"{last.major}.{last.minor + 1}.0")
+    return Version(f"{last.major}.{last.minor}.{last.patch + 1}")
+
+
+if __name__ == "__main__":
+    try:
+        if sys.argv[1] == "next":
+            print(get_next_version(last_release()))
+            raise SystemExit(0)
+    except IndexError:
+        pass
+    print(last_release())

project/{% if 'github.com' in source_url %}.github{% endif %}/workflows/ci.yml.j2

@@ -48,6 +48,46 @@ jobs:
       - pre-commit
     uses: ./.github/workflows/docs-action.yml
 
+  check-prepare-release:
+    name: Check if we can prepare release PR
+    if: >-
+      github.event_name == 'push' &&
+      github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
+    needs:
+      - docs
+      - test
+{%- endraw %}
+    runs-on: ubuntu-{{ versions["ubuntu"] }}
+{%- raw %}
+    outputs:
+      news-fragments-available: ${{ steps.check-available.outputs.available }}
+
+    steps:
+{%- endraw %}
+      - uses: actions/checkout@{{ versions["actions/checkout"] }}
+{%- raw %}
+
+      - name: Check if news fragments are available
+        id: check-available
+        run: |
+          if [ -n "$(find changelog -type f -not -name '.*' -print -quit)" ]; then
+            echo "available=1" >> "$GITHUB_OUTPUT"
+          else
+            echo "available=0" >> "$GITHUB_OUTPUT"
+          fi
+
+  prepare-release:
+    name: Prepare Release PR
+    if: ${{ needs.check-prepare-release.outputs.news-fragments-available == '1' }}
+    needs:
+      - check-prepare-release
+      - docs
+      - test
+    permissions:
+      contents: write
+      pull-requests: write
+    uses: ./.github/workflows/prepare-release-action.yml
+
   deploy-docs:
     name: Deploy Docs
     uses: ./.github/workflows/deploy-docs-action.yml

project/{% if 'github.com' in source_url %}.github{% endif %}/workflows/pr.yml.j2

@@ -22,5 +22,5 @@ jobs:
       contents: write
       id-token: write
       pages: write
-      pull-requests: read
+      pull-requests: write
 {%- endraw %}

project/{% if 'github.com' in source_url %}.github{% endif %}/workflows/prepare-release-action.yml.j2

@@ -0,0 +1,71 @@
+{%- raw -%}
+---
+name: Prepare Release PR
+
+on:
+  workflow_call:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: Override the autogenerated version.
+        required: false
+        default: ''
+        type: string
+
+jobs:
+  update-release:
+    name: Render changelog and create/update PR
+{%- endraw %}
+    runs-on: ubuntu-{{ versions["ubuntu"] }}
+{%- raw %}
+    if: github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
+    permissions:
+      contents: write
+      pull-requests: write
+
+    steps:
+      - name: Checkout code
+{%- endraw %}
+        uses: actions/checkout@{{ versions["actions/checkout"] }}
+{%- raw %}
+
+      - name: Set up Python 3.10
+{%- endraw %}
+        uses: actions/setup-python@{{ versions["actions/setup-python"] }}
+{%- raw %}
+        with:
+          python-version: '3.10'
+
+      - name: Install project
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install '.[changelog]'
+
+      - name: Get next version
+        if: github.event_name == 'push' || inputs.version == ''
+        id: next-version
+        run: echo "version=$(python tools/version.py next)" >> "$GITHUB_OUTPUT"
+
+      - name: Update CHANGELOG.md and push to release PR
+        env:
+          NEXT_VERSION: ${{ (github.event_name == 'workflow_dispatch' && inputs.version != '') && inputs.version || steps.next-version.outputs.version }}
+        run: towncrier build --yes --version "${NEXT_VERSION}"
+
+      - name: Create/update release PR
+{%- endraw %}
+        uses: peter-evans/create-pull-request@{{ versions["peter-evans/create-pull-request"] }}
+{%- raw %}
+        with:
+          commit-message: Release v${{ (github.event_name == 'workflow_dispatch' && inputs.version != '') && inputs.version || steps.next-version.outputs.version }}
+          branch: release/auto
+          sign-commits: true
+          title: Release v${{ (github.event_name == 'workflow_dispatch' && inputs.version != '') && inputs.version || steps.next-version.outputs.version }}
+          body: |
+            This automated PR builds the latest changelog. When merged, a new release is published automatically.
+
+            Before merging, please ensure it's based on the most recent default branch HEAD.
+
+            If you want to rebuild this PR with a custom version or the current date, you can also trigger the corresponding workflow manually in `Actions` > `Prepare Release PR` > `Run workflow`.
+
+            You can still follow the manual release procedure outlined in: https://salt-extensions.github.io/salt-extension-copier/topics/publishing.html
+{%- endraw %}