Prepare python publish (#27)

* Add python server test + python coverage + mypy + pyproject meta added + dynamic version + fixes return types of py server * Make CI greener * Added cff * Added code of conduct * Update python/pyproject.toml Co-authored-by: Bart Schilperoort <[email protected]> * Add extra explanation on max_keepalive_connections kwarg. * Reference issue in todo comment * Make pytest workflow run when pyproject.toml is modified * Set mypy to check with minimal py version * Add test matrix for python * Add test for py3.13, only test min and max versions * Install package before type checking (so dependencies are available) * Also install dev packages for mypy --------- Co-authored-by: Bart Schilperoort <[email protected]>
eWaterCycle · Nov 15, 2024 · d6a7125 · d6a7125
1 parent ea6d377
commit d6a7125
Show file tree

Hide file tree

Showing 12 changed files with 505 additions and 60 deletions.
diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml
@@ -0,0 +1,33 @@
+name: Publish Python 🐍 distribution 📦 to PyPI
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  publish-to-pypi:
+    name: >-
+      Publish Python 🐍 distribution 📦 to PyPI
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/remotebmi
+    permissions:
+      id-token: write # IMPORTANT: mandatory for trusted publishing
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: "pip"
+      - name: Install build
+        run: pip install build
+      - name: Build package
+        run: python3 -m build
+      - name: Publish distribution 📦 to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml
@@ -1,30 +1,36 @@
-name: Python
+name: Python 🐍
 
 on:
   push:
     paths:
       - "openapi.yaml"
       - "python/remotebmi/**"
       - "python/tests/**"
+      - "python/pyproject.toml"
       - .github/workflows/python.yml
   pull_request:
     paths:
       - "openapi.yaml"
       - "python/remotebmi/**"
       - "python/tests/**"
+      - "python/pyproject.toml"
       - .github/workflows/python.yml
     types: [opened, reopened]
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
+permissions:
+  id-token: write # This is required for codecov
+
 defaults:
   run:
     working-directory: python
 
 jobs:
   lint-format:
+    name: formatting and type checking 🔍️
     runs-on: ubuntu-latest
 
     steps:
@@ -33,37 +39,80 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
+          python-version: "3.13"
           cache: "pip"
       - name: Install ruff
         run: pip install ruff
       - name: Run ruff check
         run: ruff check
       - name: Run ruff format
         run: ruff format --check
+  typing:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+          cache: "pip"
+      - name: Install remotebmi
+        run: pip install .[dev]  # dev too so tests can be type checked
+      - name: Install mypy
+        run: pip install mypy numpy
+      - name: Run mypy
+        run: mypy .
   test:
+    name: run test suite 🐛
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: true
+      matrix:
+        os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
+        python-version: ['3.10', '3.13']
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: "pip"
+      - name: Install dependencies
+        run: pip install -e .[dev]
+      - name: Run tests
+        run: pytest
+  cov:
+    name: compute and upload coverage
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
       - name: Setup Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
+          python-version: "3.13"
           cache: "pip"
       - name: Install dependencies
         run: pip install -e .[dev]
       - name: Run tests
-        run: pytest tests
+        run: pytest --cov --cov-report=xml
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v4
+        with:
+          file: coverage.xml
+          use_oidc: true
   build:
+    name: build the package 🛠️
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
       - name: Setup Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
+          python-version: "3.13"
           cache: "pip"
       - name: Install build
         run: pip install build

diff --git a/CITATION.cff b/CITATION.cff
@@ -0,0 +1,26 @@
+# YAML 1.2
+---
+authors:
+  -
+    affiliation: "Netherlands eScience Center"
+    family-names: Verhoeven
+    given-names: Stefan
+    orcid: "https://orcid.org/0000-0002-5821-2060"
+  -
+    affiliation: "Netherlands eScience Center"
+    family-names: Schilperoort
+    given-names: Bart
+    orcid: "https://orcid.org/0000-0003-4487-9822"
+
+cff-version: "1.2.0"
+keywords:
+  - bmi
+license: "Apache-2.0"
+message: "If you use this software, please cite it using these metadata."
+repository-code: "https://github.com/eWaterCycle/remotebmi"
+title: remotebmi Python/Julia/R packages
+identifiers:
+  - description: Latest version of software
+    type: doi
+    value: "TODO fill with concept DOI after fisrt release"
+...
diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md
@@ -0,0 +1,78 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our
+project and our community a harassment-free experience for everyone,
+regardless of age, body size, disability, ethnicity, gender identity and
+expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+- The use of sexualized language or imagery and unwelcome sexual
+    attention or advances
+- Trolling, insulting/derogatory comments, and personal or political
+    attacks
+- Public or private harassment
+- Publishing others\' private information, such as a physical or
+    electronic address, without explicit permission
+- Other conduct which could reasonably be considered inappropriate in
+    a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of
+acceptable behavior and are expected to take appropriate and fair
+corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit,
+or reject comments, commits, code, wiki edits, issues, and other
+contributions that are not aligned to this Code of Conduct, or to ban
+temporarily or permanently any contributor for other behaviors that they
+deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public
+spaces when an individual is representing the project or its community.
+Examples of representing a project or community include using an
+official project e-mail address, posting via an official social media
+account, or acting as an appointed representative at an online or
+offline event. Representation of a project may be further defined and
+clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may
+be reported by contacting the project team at
+<[email protected]>. All complaints will be reviewed and
+investigated and will result in a response that is deemed necessary and
+appropriate to the circumstances. The project team is obligated to
+maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted
+separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in
+good faith may face temporary or permanent repercussions as determined
+by other members of the project\'s leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor
+Covenant](https://www.contributor-covenant.org), version 1.4, available
+at
+<https://www.contributor-covenant.org/version/1/4/code-of-conduct.html>
diff --git a/python/README.md b/python/README.md
@@ -0,0 +1,88 @@
+# Remote BMI
+
+[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)
+[![Codecov test coverage](https://codecov.io/gh/eWaterCycle/remotebmi/graph/badge.svg)](https://app.codecov.io/gh/eWaterCycle/remotebmi)
+
+The [Basic Model Interface (BMI)](https://bmi.readthedocs.io/en/stable/) is a standard interface for models. 
+The interface is available in different languages and a [language agnosting version in SIDL](https://github.com/csdms/bmi/blob/stable/bmi.sidl).
+
+To have a consumer of the model and the provider of the model seperated you can use [grpc4bmi](https://grpc4bmi.readthedocs.io/), but this only works on languages that have a grpc implementation.
+This Python package replaced the gRPC protocol with an REST API.
+The [REST API specification](https://github.com/eWaterCycle/remotebmi/blob/main/openapi.yaml) is in the [OpenAPI](https://swagger.io/specification/) format.
+
+Remotebmi is available in other languages see [here](https://github.com/eWaterCycle/remotebmi?tab=readme-ov-file#structure).
+
+## Difference from BMI
+
+- Request body and response body are in JSON format
+- On errors you get 4xx and 5xx responses with [Problem Details](https://tools.ietf.org/html/rfc7807) as response body
+- Variable names must be URL safe
+- Variable type must be in enum.
+- get_value_ptr function is not available
+
+## Consumer
+
+Installation
+
+```shell
+pip install remotebmi
+```
+
+A client can connect to a running server with the following code.
+
+```python
+from remotebmi.client.client import RemoteBmiClient
+
+model = RemoteBmiClient('http://localhost:50051')
+# Now you can use the BMI methods on model
+# for example
+model.initialize('config.file')
+model.update()
+model.get_value('var_name')
+```
+
+A client can also start a [Apptainer](https://apptainer.org) container containing the model and the server.
+
+```python
+from remotebmi.client.apptainer import BmiClientApptainer
+
+model = BmiClientApptainer('my_model.sif', work_dir='/tmp')
+```
+
+The client picks a random port and expects the container to run the BMI web server on that port.
+The port is passed to the container using the `BMI_PORT` environment variable.
+
+A client can also start a [Docker](https://docs.docker.com/engine/) container containing the model and the server.
+
+```python
+from remotebmi.client.docker import BmiClientDocker
+
+model = BmiClientDocker('ewatercycle/wflowjl:0.7.3', work_dir='/tmp')
+```
+
+The BMI web server inside the Docker container should be running on port 50051.
+If the port is different, you can pass the port as the `image_port` argument to the `BmiClientDocker` constructor.
+
+## Provider
+
+Given you have a model class called `MyModel` in a package `mypackage` then the web service can be started with the following command.
+
+```shell
+BMI_MODULE=mypackage BMI_CLASS=MyModel run-bmi-server 
+```
+
+For example [leakybucket](https://github.com/eWaterCycle/leakybucket-bmi):
+
+```shell
+pip install leakybucket
+BMI_MODULE=leakybucket.leakybucket_bmi BMI_CLASS=LeakyBucketBmi run-bmi-server
+```
+
+and the client can connect to it with the following code.
+
+```python
+> from remotebmi.client.client import RemoteBmiClient
+> client = RemoteBmiClient('http://localhost:50051')
+> client.get_component_name()
+leakybucket
+```