Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use uv project interface in template #36

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 20 additions & 28 deletions {{cookiecutter.plugin_name}}/.github/workflows/actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,42 +3,34 @@ on: [push]
jobs:
install-and-test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python_version: ["3.9", "3.10", "3.11"]
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.9

- name: Set up Python {% raw -%}${{ matrix.python_version }}{% endraw %}
uses: actions/setup-python@v5
with:
python-version: 3.9
- name: Install dependencies
python-version: {% raw -%}${{ matrix.python_version }}{% endraw %}

- name: Install uv
uses: astral-sh/setup-uv@v2

- name: Install the project
run: uv sync --all-extras

- name: Run tests
run: |
pip install --upgrade pip
pip install '.[dev]' --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
pip install coverage coveralls
- name: Test with pytest
run: |
python -m coverage run -m pytest -sv
uv run pytest --cov

- name: Submit to coveralls
continue-on-error: true
env:
GITHUB_TOKEN: {% raw -%}${{ secrets.GITHUB_TOKEN }}{% endraw %}
run: |
coveralls --service=github
build-and-install:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: 3.9
- name: Build the package
run: |
pip install --upgrade pip
pip install build
python -m build --sdist
- name: Install the package
run: |
pip install dist/*.tar.gz --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
run: uv run --with coveralls coveralls --service=github

ruff-linting:
runs-on: ubuntu-latest
steps:
Expand All @@ -53,4 +45,4 @@ jobs:
# - uses: actions/checkout@v4
# - uses: chartboost/ruff-action@v1
# with:
# args: "format . --check"
# args: "format . --check"
113 changes: 49 additions & 64 deletions {{cookiecutter.plugin_name}}/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,87 +4,70 @@

This `nomad` plugin was generated with `Cookiecutter` along with `@nomad`'s [`cookiecutter-nomad-plugin`](https://github.com/FAIRmat-NFDI/cookiecutter-nomad-plugin) template.


## Development

If you want to develop locally this plugin, clone the project and in the plugin folder, create a virtual environment (you can use Python 3.9, 3.10, or 3.11):
If you want to locally develop this plugin, clone the project.

```sh
git clone https://github.com/{{cookiecutter.github_username}}/{{cookiecutter.plugin_name}}.git
cd {{cookiecutter.plugin_name}}
python3.11 -m venv .pyenv
. .pyenv/bin/activate
```

Make sure to have `pip` upgraded:
```sh
pip install --upgrade pip
```
We highly recommend using `uv` to manage the project. Consider installing `uv` by following the official installation [guide](https://docs.astral.sh/uv/getting-started/installation/#installing-uv).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, do not send people people to read some other documentation in another package management. Can you add a better explanation + step-by-step as it was before? Think that people is not going to open that uv guide (not even me), so is uv replacing the virtual environment? Can you summarize how uv manages projects in 1 or 2 lines?


We recommend installing `uv` for fast pip installation of the packages:
```sh
pip install uv
```
Read the short [guide](https://docs.astral.sh/uv/guides/projects/) on how to manage projects.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment.


Install the `nomad-lab` package:
```sh
uv pip install '.[dev]' --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
```
Install all the dependencies:

**Note!**
Until we have an official pypi NOMAD release with the plugins functionality make
sure to include NOMAD's internal package registry (via `--index-url` in the above command).

The plugin is still under development. If you would like to contribute, install the package in editable mode (with the added `-e` flag):
```sh
uv pip install -e '.[dev]' --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
uv sync --extra-all
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this is new and I wrote a comment before regarding the tool.uv part with the index-url. So maybe you can ask Bernadette if this works for her? And maybe others in the team?

```

**Note!**
If you're not using `uv`, setup a `venv` and pip install the package using `pip install -e '.[dev]' --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple`.
Until we have an official pypi NOMAD release with the plugins functionality make sure to include NOMAD's internal package registry (via --extra-index-url in the above command).

### Run the tests

You can run locally the tests:
You can run the tests using:

```sh
python -m pytest -sv tests
uv run pytest
```

where the `-s` and `-v` options toggle the output verbosity.
Or generate a test coverage report:

Our CI/CD pipeline produces a more comprehensive test report using the `pytest-cov` package. You can generate a local coverage report:
```sh
uv pip install pytest-cov
python -m pytest --cov=src tests
uv run pytest --cov
```

### Run linting and auto-formatting

We use [Ruff](https://docs.astral.sh/ruff/) for linting and formatting the code. Ruff auto-formatting is also a part of the GitHub workflow actions. You can run locally:
We use [Ruff](https://docs.astral.sh/ruff/) for linting and formatting the code. Ruff auto-formatting is also a part of the GitHub workflow actions. You can check linting and formatting using:

```sh
ruff check .
ruff format . --check
uv run ruff check .
uv run ruff format . --check
```


### Debugging

For interactive debugging of the tests, use `pytest` with the `--pdb` flag. We recommend using an IDE for debugging, e.g., _VSCode_. If that is the case, add the following snippet to your `.vscode/launch.json`:

```json
{
"configurations": [
{
"name": "<descriptive tag>",
"type": "debugpy",
"request": "launch",
"cwd": "${workspaceFolder}",
"program": "${workspaceFolder}/.pyenv/bin/pytest",
"justMyCode": true,
"env": {
"_PYTEST_RAISE": "1"
},
"args": [
"-sv",
"--pdb",
"<path-to-plugin-tests>",
]
{
"name": "<descriptive tag>",
"type": "debugpy",
"request": "launch",
"cwd": "${workspaceFolder}",
"program": "${workspaceFolder}/.venv/bin/pytest",
"justMyCode": true,
"env": {
"_PYTEST_RAISE": "1"
},
"args": ["-sv", "--pdb", "<path-to-plugin-tests>"]
}
]
}
Expand All @@ -94,23 +77,18 @@ where `<path-to-plugin-tests>` must be changed to the local path to the test mod

The settings configuration file `.vscode/settings.json` automatically applies the linting and formatting upon saving the modified file.


### Documentation on Github pages

To view the documentation locally, install the related packages using:
```sh
uv pip install -r requirements_docs.txt
```
To view the documentation locally, run the docs server:

Run the documentation server:
```sh
mkdocs serve
uv run mkdocs serve
```


## Adding this plugin to NOMAD

Currently, NOMAD has two distinct flavors that are relevant depending on your role as an user:

1. [A NOMAD Oasis](#adding-this-plugin-in-your-nomad-oasis): any user with a NOMAD Oasis instance.
2. [Local NOMAD installation and the source code of NOMAD](#adding-this-plugin-in-your-local-nomad-installation-and-the-source-code-of-nomad): internal developers.

Expand All @@ -121,13 +99,16 @@ Read the [NOMAD plugin documentation](https://nomad-lab.eu/prod/v1/staging/docs/
### Adding this plugin in your local NOMAD installation and the source code of NOMAD

Modify the text file under `/nomad/default_plugins.txt` and add:

```sh
<other-content-in-default_plugins.txt>
{{cookiecutter.plugin_name}}==x.y.z
```

where `x.y.z` represents the released version of this plugin.

Then, go to your NOMAD folder, activate your NOMAD virtual environment and run:

```sh
deactivate
cd <route-to-NOMAD-folder>/nomad
Expand All @@ -137,15 +118,13 @@ source .pyenv/bin/activate

Alternatively and only valid for your local NOMAD installation, you can modify `nomad.yaml` to include this plugin, see [NOMAD Oasis - Install plugins](https://nomad-lab.eu/prod/v1/staging/docs/howto/oasis/plugins_install.html).


### Build the python package

The `pyproject.toml` file contains everything that is necessary to turn the project
into a pip installable python package. Run the python build tool to create a package distribution:

```sh
pip install build
python -m build --sdist
uv build
```

You can install the package with pip:
Expand All @@ -157,15 +136,21 @@ pip install dist/{{cookiecutter.plugin_name}}-{{cookiecutter.version}}
Read more about python packages, `pyproject.toml`, and how to upload packages to PyPI
on the [PyPI documentation](https://packaging.python.org/en/latest/tutorials/packaging-projects/).


### Template update

We use cruft to update the project based on template changes. A `cruft-update.yml` is included in Github workflows to automatically check for updates and create pull requests to apply updates. Follow the [instructions](https://github.blog/changelog/2022-05-03-github-actions-prevent-github-actions-from-creating-and-approving-pull-requests/) on how to enable Github Actions to create pull requests.
We use cruft to update the project based on template changes. A `cruft-update.yml` is included in Github workflows to automatically check for updates and create pull requests to apply updates. Follow the [instructions](https://github.blog/changelog/2022-05-03-github-actions-prevent-github-actions-from-creating-and-approving-pull-requests/) on how to enable Github Actions to create pull requests.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cruft update is never working (as the changes are a bit too hard to track), so I doubt anyone is ever going to accept these changes, and the email notifications are a bit annoying.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, maybe we can just keep the local command and remove the CI step.


To run the check for updates locally, follow the instructions on [`cruft` website](https://cruft.github.io/cruft/#updating-a-project).
To check for updates locally, run:

```sh
uvx cruft update
```

**Note!**
`uvx` is included in `uv` installation.

## Main contributors
| Name | E-mail |
|------|------------|
| {{cookiecutter.full_name}} | [{{cookiecutter.email}}](mailto:{{cookiecutter.email}})

| Name | E-mail |
| -------------------------- | ------------------------------------------------------- |
| {{cookiecutter.full_name}} | [{{cookiecutter.email}}](mailto:{{cookiecutter.email}}) |
34 changes: 33 additions & 1 deletion {{cookiecutter.plugin_name}}/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,34 @@ dependencies = ["nomad-lab>=1.3.0"]
Repository = "https://github.com/{{cookiecutter.github_username}}/{{cookiecutter.plugin_name}}"

[project.optional-dependencies]
dev = ["ruff", "pytest", "structlog"]
dev = [
"ruff",
"pytest>=6.0",
"structlog",
"pytest-cov"
]

docs = [
"mkdocs",
"mkdocs-material==8.1.1",
"pymdown-extensions",
"mkdocs-click"
]

[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-q -s -v"
testpaths = ["tests"]

[tool.coverage.run]
branch = true

[tool.coverage.paths]
source = ["src"]

[tool.coverage.report]
show_missing = true
skip_covered = true

[tool.ruff]
# Exclude a variety of commonly ignored directories.
Expand Down Expand Up @@ -111,6 +138,11 @@ skip-magic-trailing-comma = false
# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"

[tool.uv]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not always working in MacOS (I think depending on the version) and Windows, and in Ubuntu Bernadette was also having the same issue. I saw all of this during the hackathon last week

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure they were not using just pip here? This is just the equivalent to passing the argument in the CLI.

extra-index-url = [
"https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple",
]

[tool.setuptools]
package-dir = { "" = "src" }

Expand Down
4 changes: 0 additions & 4 deletions {{cookiecutter.plugin_name}}/requirements_docs.txt

This file was deleted.

Loading