Skip to content

Commit

Permalink
Merge pull request #150 from jmccreight/feat_pywatershed
Browse files Browse the repository at this point in the history
Rename to pywatershed
  • Loading branch information
jmccreight authored Apr 27, 2023
2 parents c40e1eb + 2c5ae2d commit 012d9a9
Show file tree
Hide file tree
Showing 139 changed files with 560 additions and 553 deletions.
28 changes: 14 additions & 14 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:

jobs:

pynhm_setup:
pyws_setup:
name: standard installation
runs-on: ubuntu-latest
strategy:
Expand All @@ -23,7 +23,7 @@ jobs:

- name: Set environment variables
run: |
echo "PYNHM_FORTRAN=true" >> $GITHUB_ENV
echo "PYWS_FORTRAN=true" >> $GITHUB_ENV
echo 'SETUPTOOLS_ENABLE_FEATURES="legacy-editable"' >> $GITHUB_ENV
- name: Setup Python
Expand All @@ -42,14 +42,14 @@ jobs:
- name: Print pyhmn version
run: |
python -c "import pynhm; print(pynhm.__version__)"
python -c "import pywatershed; print(pywatershed.__version__)"
- name: Build pynhm, check dist outputs
- name: Build pywatershed, check dist outputs
run: |
python -m build
twine check --strict dist/*
pynhm_lint:
pyws_lint:
name: linting
runs-on: ubuntu-latest
strategy:
Expand Down Expand Up @@ -81,26 +81,26 @@ jobs:
echo "if isort check fails update isort using"
echo " pip install isort --upgrade"
echo "and run"
echo " isort ./pynhm ./autotest"
echo " isort ./pywatershed ./autotest"
echo "and then commit the changes."
isort --check --diff ./pynhm
isort --check --diff ./pywatershed
- name: Run black
run: |
echo "if black check fails update black using"
echo " pip install black --upgrade"
echo "and run"
echo " black ./pynhm ./autotest"
echo " black ./pywatershed ./autotest"
echo "and then commit the changes."
black --check --diff ./pynhm
black --check --diff ./pywatershed
- name: Run flake8
run: |
flake8 --count --show-source --exit-zero ./pynhm ./autotest
flake8 --count --show-source --exit-zero ./pywatershed ./autotest
- name: Run pylint
run: |
pylint --jobs=2 --errors-only --exit-zero ./pynhm ./autotest
pylint --jobs=2 --errors-only --exit-zero ./pywatershed ./autotest
test:
name: ${{ matrix.os}} py${{ matrix.python-version }}
Expand All @@ -120,7 +120,7 @@ jobs:
- name: Set environment variables
run: |
echo "PYTHON_VERSION=${{ matrix.python-version }}" >> $GITHUB_ENV
echo "PYNHM_FORTRAN=true" >> $GITHUB_ENV
echo "PYWS_FORTRAN=true" >> $GITHUB_ENV
echo 'SETUPTOOLS_ENABLE_FEATURES="legacy-editable"' >> $GITHUB_ENV
cat .mf6_ci_ref_remote >> $GITHUB_ENV
Expand Down Expand Up @@ -166,7 +166,7 @@ jobs:
run: |
python update_flopy.py
- name: Install pynhm
- name: Install pywatershed
run: |
pip install .
Expand All @@ -193,7 +193,7 @@ jobs:
-n=auto
--durations=0
--all_domains
--cov=pynhm
--cov=pywatershed
--cov-report=xml
--junitxml=pytest.xml

Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/ci_examples.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
- name: Download GIS files
working-directory: examples
run: |
curl -LO https://github.com/EC-USGS/pynhm/releases/download/v2022.0.1/pynhm_gis.zip
curl -LO https://github.com/EC-USGS/pywatershed/releases/download/v2022.0.1/pynhm_gis.zip
unzip pynhm_gis.zip
- name: Set environment variables
Expand All @@ -47,7 +47,7 @@ jobs:
uses: mamba-org/provision-with-micromamba@main
with:
environment-file: examples/examples_env.yml
environment-name: pynhm_nb
environment-name: pyws_nb
cache-env: true
# cache-env-key: "${{runner.os}}-${{runner.arch}}-py${{matrix.python-version}}-${{env.TODAY}}"

Expand All @@ -70,9 +70,9 @@ jobs:
echo -------
pip list
- name: Import pynhm
- name: Import pywatershed
run: |
python -c "import pynhm"
python -c "import pywatershed"
- name: Run tests
working-directory: autotest_exs
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ generated/

# gis data
examples/pynhm_gis
examples/pywatershed_gis
examples/model_loop_custom_output

# graphics
Expand Down
50 changes: 25 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# pynhm
[![ci-badge](https://github.com/ec-usgs/pynhm/workflows/CI/badge.svg?branch=main)](https://github.com/ec-usgs/pynhm/actions?query=workflow%3ACI)
[![codecov-badge](https://codecov.io/gh/ec-usgs/pynhm/branch/main/graph/badge.svg)](https://codecov.io/gh/ec-usgs/pynhm)
[![Documentation Status](https://readthedocs.org/projects/pynhm/badge/?version=latest)](https://pynhm.readthedocs.io/en/latest/?badge=latest)
# pywatershed
[![ci-badge](https://github.com/ec-usgs/pywatershed/workflows/CI/badge.svg?branch=main)](https://github.com/ec-usgs/pywatershed/actions?query=workflow%3ACI)
[![codecov-badge](https://codecov.io/gh/ec-usgs/pywatershed/branch/main/graph/badge.svg)](https://codecov.io/gh/ec-usgs/pywatershed)
[![Documentation Status](https://readthedocs.org/projects/pywatershed/badge/?version=latest)](https://pywatershed.readthedocs.io/en/latest/?badge=latest)

[//]: # (<img src="https://raw.githubusercontent.com/ec-usgs/pynhm/main/resources/images/prms_flow.png" alt="prms_flow" style="width:50;height:20">)
[//]: # (<img src="https://raw.githubusercontent.com/ec-usgs/pywatershed/main/resources/images/prms_flow.png" alt="prms_flow" style="width:50;height:20">)

Purpose
=========
Expand All @@ -26,7 +26,7 @@ Goals for EC Watershed Modeling:
* Redesign PRMS to be more modern and flexible
* Prioritize process representations in the current National Hydrological Model (NHM) based on PRMS 5.2.1

Prototype an EC watershed model: "pynhm"
Prototype an EC watershed model: "pywatershed"
* Redesign PRMS quickly in python
* Couple to MF6 via BMI/XMI interface (Hughes et al, 2021; Hutton et al, 2020)
* Establish a prototyping ground for EC codes that couples to the compiled framework: low cost proof of concepts (at the price of potentially less computational performance)
Expand All @@ -45,7 +45,7 @@ instructions.

To install the software you will need Python >= 3.8. We recommend installing
the python package dependencies using anaconda or miniconda. Most users will
likely want to create the `pynhm_nb` conda environment by running
likely want to create the `pyws_nb` conda environment by running

```conda env create -f examples/examples_env.yml```.

Expand All @@ -56,15 +56,15 @@ One could also do
but this is not guaranteed.

Once the environment is established, activate the environment and install
pynhm
pywatershed

`conda activate pynhm_nb; cd pynhm; pip install .`
`conda activate pyws_nb; cd pywatershed; pip install .`

If you would like to compile the fortran computational kernels for
certain physical process representations (not required), you'll need a fortran
compiler and you will run

`export PYNHM_FORTRAN=true; cd pynhm; pip install .`
`export PYWS_FORTRAN=true; cd pywatershed; pip install .`

See Developer Requirements below for more details.

Expand All @@ -82,17 +82,17 @@ two applications:

1. Compiling and running C/Fortran PRMS code to generate testing/verification data
2. Compiling (installing) and running fortran backends/kernels for some hydrological
process representations in pynhm
process representations in pywatershed

On Apple Silicon, the PRMS source code is only currently known to compile with intel while
the fortran kernels in pynhm only compile with gnu.
the fortran kernels in pywatershed only compile with gnu.

Python >= 3.8 is required. Three different python environments are specified within the repository.
These are:

* Minimal (for developing/testing), 'pynhm': ci/requirements/environment.yml
* Notebooks (~= minimal + jupyter), 'pynhm_nb': examples/examples_env.yml
* Documentation (only if you want to build the documentation), 'pynhm-docs': ci/requirements/doc.yml
* Minimal (for developing/testing), 'pyws': ci/requirements/environment.yml
* Notebooks (~= minimal + jupyter), 'pyws_nb': examples/examples_env.yml
* Documentation (only if you want to build the documentation), 'pyws-docs': ci/requirements/doc.yml

We recommend (because we test it in CI) using anacoda or miniconda to establish these environments
with the following commands
Expand All @@ -112,9 +112,9 @@ There are also .txt equivalents that can be used for installing from pip, like s
though these are not comprehensive installs as with conda and not tested.

Once the python environment and dependencies are established and activated (`conda activate env_of_choice`),
pynhm is installed for development into that environment with the following command
pywatershed is installed for development into that environment with the following command

`cd pynhm; pip install -e .`
`cd pywatershed; pip install -e .`

The numpy extension F2PY is used to provide fortran compiled kernels of core calculations to boost
performance. F2PY is documented [within numpy](https://numpy.org/doc/stable/f2py/index.html). This
Expand All @@ -126,8 +126,8 @@ installation is achieved by the following code:
export SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
export CC=path/to/gcc # for example
export FC=path/to/gfortran # for example
export PYNHM_FORTRAN=true
cd path/to/pynhm
export PYWS_FORTRAN=true
cd path/to/pywatershed
pip install -e .
```

Expand All @@ -137,15 +137,15 @@ To run the tests, we first need to generate the test data. This consists of runn
and then converting the output to netcdf:

```
cd path/to/pynhm/test_data/scripts
cd path/to/pywatershed/test_data/scripts
pytest -v -n=4 test_run_domains.py
pytest -v -n=8 test_nc_domains.py
```

Finally, run the tests themselves,

```
cd path/to/pynhm/autotest
cd path/to/pywatershed/autotest
pytest -v -n=8
```

Expand All @@ -164,9 +164,9 @@ be verified moving ahead in time.
Example Notebooks
==================
Jupyter notebooks containing examples are found in the
[examples/](https://github.com/EC-USGS/pynhm/tree/main/examples) directory. Numbered notebooks are tested.
[examples/](https://github.com/EC-USGS/pywatershed/tree/main/examples) directory. Numbered notebooks are tested.
Notebooks 00 and 01 walk the user through the setting the python environment and running the software tests.
Notebook 02 demonstrates modeling with pynhm. Non-numbered notebooks cover additional topics. These notebooks
Notebook 02 demonstrates modeling with pywatershed. Non-numbered notebooks cover additional topics. These notebooks
are note yet covered by testing and so may be expected to have some issues until they are added to testing.


Expand All @@ -176,13 +176,13 @@ The contents of directories at this level is described. Therein you may discover

```
.github/ Github actions for deploying continuous integration (CI)
autotest/ pynhm package testing using pytest
autotest/ pywatershed package testing using pytest
bin/ PRMS executables distributed
ci/ Python environments for CI
doc/ Package/code documentation source code
examples/ How to use the package, mostly jupyter notebooks
prms_src/ PRMS source used for generating executables in bin/
pynhm/ Package source
pywatershed/ Package source
reference/ Ancillary materials for development
resources/ Static stuff like images
test_data/ Data used for automated testing
Expand Down
6 changes: 3 additions & 3 deletions autotest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Pytest options can be explored via `pytest --help`.

## Developer

This is how the pynhm package tests itself.
This is how the pywatershed package tests itself.

The test suite consists both
* stand alone tests, and
Expand Down Expand Up @@ -40,9 +40,9 @@ inputs (e.g. CBH forcing files, parameter files), static or reference model
output (from PRMS/NHM), and the answers to domain tests.

Examples of `domain_yaml` files can be found in, for example, in
`pynhm/test_data/drb_2yr/drb_2yr.yaml`
`pywatershed/test_data/drb_2yr/drb_2yr.yaml`
and
`pynhm/test_data/conus_2yr/conus_2yr.yaml`.
`pywatershed/test_data/conus_2yr/conus_2yr.yaml`.


### Domain inputs
Expand Down
12 changes: 6 additions & 6 deletions autotest/test_atmosphere.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
import numpy as np
import pytest

from pynhm.atmosphere.PRMSAtmosphere import PRMSAtmosphere
from pynhm.base.adapter import adapter_factory
from pynhm.base.control import Control
from pynhm.parameters import PrmsParameters
from pywatershed.atmosphere.PRMSAtmosphere import PRMSAtmosphere
from pywatershed.base.adapter import adapter_factory
from pywatershed.base.control import Control
from pywatershed.parameters import PrmsParameters


@pytest.fixture(scope="function")
Expand Down Expand Up @@ -104,7 +104,7 @@ def test_init(self, domain, control, tmp_path):
print(f"time step {istep}")
print(f"output variable {key}")
print(f"prms {a1.min()} {a1.max()}")
print(f"pynhm {a2.min()} {a2.max()}")
print(f"pywatershed {a2.min()} {a2.max()}")
print(f"diff {diffmin} {diffmax}")
print(f"absdiffmax {absdiffmax}")
print(f"wh_absdiffmax {wh_absdiffmax}")
Expand All @@ -113,4 +113,4 @@ def test_init(self, domain, control, tmp_path):
atm.finalize()

if not all_success:
raise Exception("pynhm results do not match prms results")
raise Exception("pywatershed results do not match prms results")
2 changes: 1 addition & 1 deletion autotest/test_base.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import pytest

from pynhm.base.accessor import Accessor
from pywatershed.base.accessor import Accessor


class TestAccessor:
Expand Down
4 changes: 2 additions & 2 deletions autotest/test_budget.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import numpy as np
import pytest

from pynhm.base.budget import Budget
from pynhm.base.control import Control
from pywatershed.base.budget import Budget
from pywatershed.base.control import Control

# TODO
# * Test restart more robustly
Expand Down
Loading

0 comments on commit 012d9a9

Please sign in to comment.