Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More build options: docs, versioning #19

Merged
merged 22 commits into from
Jul 31, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
0308157
docs template generation made optional
ds-jakub-cierocki Jul 8, 2024
4d72023
removed docs generation from CI/CD if disabled during project setup
ds-jakub-cierocki Jul 8, 2024
72c4341
refacotred post-gen hooks
ds-jakub-cierocki Jul 9, 2024
35f381f
python-semantic-release partial support: pyproject.toml + GitHub CI
ds-jakub-cierocki Jul 9, 2024
2972384
jinja2 typo fix
ds-jakub-cierocki Jul 23, 2024
1548f5b
fix: escpaed semantic-relase controls including jinja2 reserved syntax
ds-jakub-cierocki Jul 23, 2024
f7ba881
refactor: better var naming in hooks
ds-jakub-cierocki Jul 23, 2024
cee6968
skipping docs fix
ds-jakub-cierocki Jul 23, 2024
2daf1f3
fix: missing contents read permission in GH actions (#18)
mhordynski Jul 9, 2024
9a3f742
chore: add issue templates (#20)
mhordynski Jul 23, 2024
838df60
versioning fixes
ds-jakub-cierocki Jul 24, 2024
e5031ac
removed remaninings of PyPI publishing from GitHub CI/CD
ds-jakub-cierocki Jul 24, 2024
938d960
removed package building from pyproject.toml
ds-jakub-cierocki Jul 24, 2024
516387b
added Semantic Release part + made docs and bumpversion optional
ds-jakub-cierocki Jul 24, 2024
8593552
added validation to ensure it's not possible to build project with Gi…
ds-jakub-cierocki Jul 24, 2024
aa5030b
added TODO to make installing bump2version optional in Gitlab CI
ds-jakub-cierocki Jul 24, 2024
7def7e6
implemented code review suggestions regarding cookiecutter.json config
ds-jakub-cierocki Jul 25, 2024
21ffd22
code review fixes: made GitLab CI docs generation conditional using G…
ds-jakub-cierocki Jul 29, 2024
00f362a
code review fixes - removed Python Semantic Release (it will be provi…
ds-jakub-cierocki Jul 29, 2024
f036ca3
Merge branch 'main' into jc/more-build-options
Sahcim Jul 30, 2024
a326651
Remove Python Semantic Versioning from readme
Sahcim Jul 30, 2024
b91a046
Remove build docs step in gitlab ci when is set to True
Jul 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/ISSUE_TEMPLATE/01_feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: 🚀 Feature Request
Sahcim marked this conversation as resolved.
Show resolved Hide resolved
description: Submit a proposal/request for a new ds-template feature.
title: "feat: <short_name>"
labels: ["feature"]
body:
- type: markdown
attributes:
value: |
Thanks for contributing to ds-template!
- type: textarea
id: feature-description
attributes:
label: Feature description
description: A clear and concise description of the feature proposal
placeholder: Tell us what you want!
validations:
required: true
- type: textarea
id: feature-motivation
attributes:
label: Motivation
description: A clear and concise description of what the problem is, e.g., I'm always frustrated when [...]
placeholder: Why do you need this feature?
validations:
required: true
- type: textarea
id: feature-context
attributes:
label: Additional context
description: Add any other context or screenshots about the feature request here.
placeholder: Screenshots, code snippets, etc.
31 changes: 31 additions & 0 deletions .github/ISSUE_TEMPLATE/02_bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: 🐞 Bug Report
Sahcim marked this conversation as resolved.
Show resolved Hide resolved
description: File a bug report
title: "bug: <short_name>"
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: XYZ doesn't work as expected...
value: "A bug happened!"
validations:
required: true
- type: textarea
id : how-to-reproduce
attributes:
label: How can we reproduce it?
description: Please provide steps (commands) that led you to the bug.
placeholder: cookiecutter [email protected]:deepsense-ai/ds-template.git
render: bash
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
33 changes: 13 additions & 20 deletions cookiecutter.json
Original file line number Diff line number Diff line change
@@ -1,21 +1,14 @@
{
"client_name": "ds",
"project_name": "default",
"__project_name_slug": "{{cookiecutter.project_name | slugify}}",
"__client_name_slug": "{{cookiecutter.client_name | slugify}}",
"repo_name": "{{cookiecutter.__client_name_slug}}-{{cookiecutter.__project_name_slug}}",
"ci": [
"GitLab",
"Github",
"None"
],
"jupytext": [
"No",
"Yes"
],
"python_package_name": "{{ cookiecutter.__project_name_slug.replace('-', '_') }}",
"__package_name": "{{ cookiecutter.python_package_name }}",
"_copy_without_render": [
".github/workflows/*.yml"
]
}
"client_name": "ds",
"project_name": "default",
"__project_name_slug": "{{cookiecutter.project_name | slugify}}",
"__client_name_slug": "{{cookiecutter.client_name | slugify}}",
"repo_name": "{{cookiecutter.__client_name_slug}}-{{cookiecutter.__project_name_slug}}",
"ci": ["Github", "GitLab", "None"],
jcierocki marked this conversation as resolved.
Show resolved Hide resolved
"jupytext": ["No", "Yes"],
"docs": ["Sphinx", "No docs"],
Sahcim marked this conversation as resolved.
Show resolved Hide resolved
"versioning": ["Python Semantic Release", "Bumpversion", "None"],
"python_package_name": "{{ cookiecutter.__project_name_slug.replace('-', '_') }}",
"__package_name": "{{ cookiecutter.python_package_name }}",
"_copy_without_render": [".github/workflows/*.yml", ".github/workflows/*.yaml"]
}
46 changes: 36 additions & 10 deletions hooks/post_gen_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,59 @@

print("Running post generation...")

ci = "{{ cookiecutter.ci }}"
files_to_be_removed = []
Sahcim marked this conversation as resolved.
Show resolved Hide resolved

REMOVE_PATHS = []

gitlab_files = [
GITLAB_FILES = [
".gitlab-ci.yml",
"docker/precommit"
]

github_files = [
GITHUB_FILES = [
".github/",
]

DOCS_FILES = [
"docs/",
"build_docs.sh",
".github/workflows/documentation.yml"
]

BUMPVERSION_FILES = [
".bumpversion.cfg",
"bump_version.sh"
]

SEMANTIC_RELEASE_FILES = [
".github/workflows/check_future_version.yml",
".github/workflows/release.yml",
]

{% if cookiecutter.ci != "GitLab" %}
REMOVE_PATHS.extend(gitlab_files)
files_to_be_removed.extend(GITLAB_FILES)
{% endif %}

{% if cookiecutter.ci != "Github" %}
REMOVE_PATHS.extend(github_files)
files_to_be_removed.extend(GITHUB_FILES)
{% endif %}

{% if cookiecutter.jupytext != "Yes" %}
REMOVE_PATHS.extend(["notebooks/example.py"])
files_to_be_removed.append("notebooks/example.py")
{% endif %}

{% if cookiecutter.docs == "No docs" %}
files_to_be_removed.extend(DOCS_FILES)
{% endif %}

{% if cookiecutter.versioning != "Bumpversion" %}
files_to_be_removed.extend(BUMPVERSION_FILES)
{% endif %}

{% if cookiecutter.versioning != "Python Semantic Release" %}
files_to_be_removed.extend(SEMANTIC_RELEASE_FILES)
{% endif %}

print("Cleaning files... 🌀")
for path in REMOVE_PATHS:
for path in files_to_be_removed:
path = Path(path)
if path.exists() and path.is_file():
print(f"Clean up file: '{path}'")
Expand All @@ -44,4 +70,4 @@
for path in Path("").rglob("*.sh"):
path.chmod(path.stat().st_mode | stat.S_IXUSR)

print("DONE 🎆")
print("DONE 🎆")
7 changes: 7 additions & 0 deletions hooks/pre_gen_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,11 @@
print(f"ERROR: '{package_name}' is not a valid Python module name.")
sys.exit(1)

# TODO temporary, until GitLab CI support for Semantic Release will be added
ci_type = '{{ cookiecutter.ci }}'
versioning_approach = '{{ cookiecutter.versioning }}'
if ci_type == 'GitLab' and versioning_approach == "Python Semantic Release":
print(f"ERROR: 'Python Semantic Release' is currently not supported for projects with GitLab CI.")
sys.exit(1)
jcierocki marked this conversation as resolved.
Show resolved Hide resolved

print("Template looks ok.")
4 changes: 3 additions & 1 deletion {{ cookiecutter.repo_name }}/.bumpversion.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,6 @@ values =
dev
prod

[bumpversion:file:./src/{{cookiecutter.__package_name}}/VERSION]
[bumpversion:file:./src/{{cookiecutter.__package_name}}/__version__.py]
search = __version__ = '{current_version}'
replace = __version__ = '{new_version}'
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Check New Version

on: workflow_dispatch

jobs:
release:
runs-on: ubuntu-latest
concurrency: release
permissions:
id-token: write
contents: write

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Python Semantic Release
uses: python-semantic-release/python-semantic-release@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
root_options: "-v --noop"
jcierocki marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions {{ cookiecutter.repo_name }}/.github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
checks: write
pull-requests: write
steps:
Expand Down
45 changes: 45 additions & 0 deletions {{ cookiecutter.repo_name }}/.github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Semantic Release
Sahcim marked this conversation as resolved.
Show resolved Hide resolved

on:
workflow_dispatch:
inputs:
releaseType:
description: "version update type"
required: true
type: choice
default: "automatic"
options:
- "automatic"
- "major"
- "minor"
- "patch"

jobs:
release:
runs-on: ubuntu-latest
concurrency: release
permissions:
id-token: write
contents: write

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Python Semantic Release Manual
id: release_manual
if: ${{ github.event.inputs.releaseType != 'automatic' }}
uses: python-semantic-release/python-semantic-release@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
force: ${{ github.event.inputs.releaseType }}
changelog: false

- name: Python Semantic Release Automatic
id: release_automatic
if: ${{ github.event.inputs.releaseType == 'automatic' }}
uses: python-semantic-release/python-semantic-release@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
changelog: false
15 changes: 8 additions & 7 deletions {{ cookiecutter.repo_name }}/.gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,11 @@ variables:
DOCKER_REGISTRY: $CI_REGISTRY/$CI_PROJECT_PATH
PRECOMMIT_IMAGE: $DOCKER_REGISTRY/precommit


# run CI on default branch or MR only
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_MERGE_REQUEST_IID'
- if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
- if: "$CI_MERGE_REQUEST_IID"

stages:
- preparation
Expand Down Expand Up @@ -58,7 +57,7 @@ stages:
stage: preparation
image: docker:23
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
changes:
- docker/precommit/Dockerfile
- .pre-commit-config.yaml
Expand Down Expand Up @@ -87,7 +86,6 @@ stages:
# Consider to use stable commit SHA from main branch instead
# and manual changes to reduce surprises.


# To overcome issue that latest might not be present
# and assume that this either new project / done in
# atomic merge request and is not main branch
Expand Down Expand Up @@ -120,7 +118,7 @@ lint-precommit-changed:
- docker/precommit/Dockerfile
- .pre-commit-config.yaml
when: always
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
when: never

#############################################
Expand All @@ -140,6 +138,7 @@ lint-precommit-changed:
###########################################

# Job to run pytest with code coverage + license check
# TODO make installing bump2version below conditional on cookiecutter parameter choice
tests:
image: $PYTHON_DOCKER_IMAGE
stage: tests
Expand Down Expand Up @@ -236,6 +235,7 @@ tests:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

{% if cookiecutter.docs != "No docs" %}
jcierocki marked this conversation as resolved.
Show resolved Hide resolved
##############################################

# Job to build Sphinx docs and host it on GitLab Pages
Expand All @@ -249,6 +249,7 @@ pages:
- ./build_docs.sh
artifacts:
paths:
- public
- public
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
{% endif %}
22 changes: 21 additions & 1 deletion {{ cookiecutter.repo_name }}/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ Read more about different modes in [documentation](https://ipython.org/ipython-d
All code should be in `src/` to make reusability and review straightforward, keep notebooks simple for exploratory data analysis.
See also [Cookiecutter Data Science opinion](https://drivendata.github.io/cookiecutter-data-science/#notebooks-are-for-exploration-and-communication).

{%- if cookiecutter.docs != "No docs" %}
# Project documentation

In `docs/` directory are Sphinx RST/Markdown files.
Expand All @@ -62,6 +63,7 @@ $ ./build_docs.sh
Then open `public/index.html` file.

Please read the official [Sphinx documentation](https://www.sphinx-doc.org/en/master/) for more details.
{% endif -%}

{% if cookiecutter.ci == "GitLab" %}

Expand Down Expand Up @@ -104,6 +106,7 @@ Treat them as read-only files and edit only notebooks.

{%- endif -%}

{%- if cookiecutter.versioning == "Bumpversion" %}
# Semantic version bump

To bump version of the library please use `bump2version` which will update all version strings.
Expand All @@ -118,11 +121,28 @@ $ ./bump_version.sh minor
$ ./bump_version.sh major
$ ./bump_version.sh patch
# to see what is going to change run:
$ ./bump_version.sh --dry-run major
$ ./bump_versemantic-release version --patch --printsion.sh --dry-run major
```
Script updates **VERSION** file and setup.cfg automatically uses that version.

You can configure it to update version string in other files as well - please check out the bump2version configuration file.
{% endif -%}

{%- if cookiecutter.versioning == "Python Semantic Release" %}
jcierocki marked this conversation as resolved.
Show resolved Hide resolved
If using GitHub, you can manually trigger version check and release in GitHub Actions web GUI. It will also automatically trigger based on your commits.

You can also trigger it manually using:
```bash
# dry-run to preview changes
$ semantic-release --noop version
# dry-run to preview only the new version
$ semantic-release version --print
# to apply new version and without pushing
$ semantic-release version --no-push
# to apply new version and push
$ semantic-release version
```
{% endif -%}

{% if cookiecutter.ci == "GitLab" %}

Expand Down
Loading
Loading