refactor(exes/context): update executables container and test context

.github/workflows/ci.yml

@@ -117,6 +117,9 @@ jobs:
           repository: MODFLOW-USGS/modflow6-largetestmodels
           path: modflow6-largetestmodels
 
+      - name: Install executables
+        uses: modflowpy/install-modflow-action@v1
+
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
@@ -154,6 +157,7 @@ jobs:
       - name: Run tests
         working-directory: modflow-devtools
         env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          BIN_PATH: ~/.local/bin/modflow
           REPOS_PATH: ${{ github.workspace }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: pytest -v -n auto --durations 0

DEVELOPER.md

@@ -31,18 +31,22 @@ This repository's tests use [`pytest`](https://docs.pytest.org/en/latest/) and s
 
 This repository's tests expect a few environment variables:
 
+- `BIN_PATH`: path to MODFLOW 6 and related executables
 - `REPOS_PATH`: the path to MODFLOW 6 example model repositories
 - `GITHUB_TOKEN`: a GitHub authentication token
 
 These may be set manually, but the recommended approach is to configure environment variables in a `.env` file in the project root, for instance:
 
 ```
+BIN_PATH=/path/to/modflow/executables
 REPOS_PATH=/path/to/repos
 GITHUB_TOKEN=yourtoken...
 ```
 
 The tests use [`pytest-dotenv`](https://github.com/quiqua/pytest-dotenv) to detect and load variables from this file.
 
+**Note:** at minimum, the tests require that the `mf6` executable is present in `BIN_PATH`.
+
 ### Running the tests
 
 Tests should be run from the project root. To run the tests in parallel with verbose output:

README.md

@@ -12,11 +12,12 @@ Python tools for MODFLOW development and testing.
 - [Installation](#installation)
 - [Included](#included)
   - [`MFZipFile` class](#mfzipfile-class)
-  - [Keepable temporary directories](#keepable-temporary-directories)
+  - [Keepable temporary directory fixtures](#keepable-temporary-directory-fixtures)
   - [Model-loading fixtures](#model-loading-fixtures)
-    - [Test model fixtures](#test-model-fixtures)
-    - [Example scenario fixtures](#example-scenario-fixtures)
+    - [Test models](#test-models)
+    - [Example scenarios](#example-scenarios)
   - [Reusable test case framework](#reusable-test-case-framework)
+  - [Executables container](#executables-container)
   - [Conditionally skipping tests](#conditionally-skipping-tests)
   - [Miscellaneous](#miscellaneous)
     - [Generating TOCs with `doctoc`](#generating-tocs-with-doctoc)
@@ -59,7 +60,7 @@ Note that `pytest` requires this to be a top-level `conftest.py` living in your
 
 Python's `ZipFile` doesn't preserve execute permissions. The `MFZipFile` subclass modifies `ZipFile.extract()` to do so, as per the recommendation [here](https://stackoverflow.com/questions/39296101/python-zipfile-removes-execute-permissions-from-binaries).
 
-### Keepable temporary directories
+### Keepable temporary directory fixtures
 
 Tests often need to exercise code that reads from and/or writes to disk. The test harness may also need to create test data during setup and clean up the filesystem on teardown. Temporary directories are built into `pytest` via the [`tmp_path`](https://docs.pytest.org/en/latest/how-to/tmp_path.html#the-tmp-path-fixture) and `tmp_path_factory` fixtures.
 
@@ -114,7 +115,7 @@ To use these fixtures, the environment variable `REPOS_PATH` must point to the l
 
 **Note**: example models must be built by running the `ci_build_files.py` script in `modflow6-examples/etc` before running tests using the `example_scenario` fixture.
 
-#### Test model fixtures
+#### Test models
 
 The `test_model_mf5to6`, `test_model_mf6` and `large_test_model` fixtures are each a `Path` to the directory containing the model's namefile. For instance, to load `mf5to6` models from the [`MODFLOW-USGS/modflow6-testmodels`](https://github.com/MODFLOW-USGS/modflow6-testmodels) repository:
 
@@ -125,7 +126,7 @@ def test_mf5to6_model(tmpdir, testmodel_mf5to6):
 
 This test function will be parametrized with all `mf5to6` models found in the `testmodels` repository (likewise for `mf6` models, and for large test models in their own repository).
 
-#### Example scenario fixtures
+#### Example scenarios
 
 The [`MODFLOW-USGS/modflow6-examples`](https://github.com/MODFLOW-USGS/modflow6-examples) repository contains a collection of scenarios, each consisting of 1 or more models. The `example_scenario` fixture is a `Tuple[str, List[Path]]`. The first item is the name of the scenario. The second item is a list of namefile `Path`s, ordered alphabetically by name. Model naming conventions are as follows:
 
@@ -170,6 +171,38 @@ def case_qa(case):
     print(case.name, case.question, case.answer)
 ```
 
+### Executables container
+
+The `Executables` class is just a mapping between executable names and paths on the filesystem. This can be useful to test multiple versions of the same program, and is easily injected into test functions as a fixture:
+
+```python
+from os import environ
+from pathlib import Path
+import subprocess
+import sys
+
+import pytest
+
+from modflow_devtools.misc import get_suffixes
+from modflow_devtools.executables import Executables
+
+_bin_path = Path("~/.local/bin/modflow").expanduser()
+_dev_path = Path(environ.get("BIN_PATH")).absolute()
+_ext, _ = get_suffixes(sys.platform) 
+
+@pytest.fixture
+@pytest.mark.skipif(not (_bin_path.is_dir() and _dev_path.is_dir()))
+def exes():
+    return Executables(
+        mf6_rel=_bin_path / f"mf6{_ext}",
+        mf6_dev=_dev_path / f"mf6{_ext}"
+    )
+
+def test_exes(exes):
+    print(subprocess.check_output([f"{exes.mf6_rel}", "-v"]).decode('utf-8'))
+    print(subprocess.check_output([f"{exes.mf6_dev}", "-v"]).decode('utf-8'))
+```
+
 ### Conditionally skipping tests
 
 Several `pytest` markers are provided to conditionally skip tests based on executable availability, Python package environment or operating system.

modflow_devtools/executables.py

@@ -0,0 +1,96 @@
+import sys
+from os import PathLike
+from pathlib import Path
+from shutil import which
+from types import SimpleNamespace
+from typing import Dict, Optional
+from warnings import warn
+
+from modflow_devtools.misc import get_suffixes, run_cmd
+
+
+class Executables(SimpleNamespace):
+    """
+    Container mapping executable names to their paths.
+    """
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+    @staticmethod
+    def get_version(
+        exe="mf6", path: PathLike = None, flag: str = "-v"
+    ) -> Optional[str]:
+        """Get the version number of an executable."""
+
+        pth = Executables.get_path(exe, path)
+        if not pth:
+            warn(
+                f"Executable {exe} not found"
+                + ("" if not pth else f" at path: {pth}")
+            )
+            return None
+
+        out, err, ret = run_cmd(exe, flag)
+        if ret == 0:
+            out = "".join(out).strip()
+            return out.split(":")[1].strip()
+        else:
+            return None
+
+    @staticmethod
+    def get_path(exe: str = "mf6", path: PathLike = None) -> Optional[Path]:
+        pth = None
+        found = None
+        if path is not None:
+            pth = Path(path)
+            found = which(exe, path=str(pth))
+        if found is None:
+            found = which(exe)
+
+        if found is None:
+            warn(
+                f"Executable {exe} not found"
+                + ("" if not pth else f" at path: {pth}")
+            )
+            return found
+
+        return Path(found)
+
+    def as_dict(self) -> Dict[str, Path]:
+        """
+        Returns a dictionary mapping executable names to paths.
+        """
+
+        return self.__dict__.copy()
+
+
+def build_default_exe_dict(bin_path: PathLike) -> Dict[str, Path]:
+    p = Path(bin_path)
+    d = dict()
+
+    # paths to executables for previous versions of MODFLOW
+    dl_bin = p / "downloaded"
+    rb_bin = p / "rebuilt"
+
+    # get platform-specific filename extensions
+    ext, so = get_suffixes(sys.platform)
+
+    # downloaded executables
+    d["mf2005"] = Executables.get_path(f"mf2005dbl{ext}", dl_bin)
+    d["mfnwt"] = Executables.get_path(f"mfnwtdbl{ext}", dl_bin)
+    d["mfusg"] = Executables.get_path(f"mfusgdbl{ext}", dl_bin)
+    d["mflgr"] = Executables.get_path(f"mflgrdbl{ext}", dl_bin)
+    d["mf2005s"] = Executables.get_path(f"mf2005{ext}", dl_bin)
+    d["mt3dms"] = Executables.get_path(f"mt3dms{ext}", dl_bin)
+
+    # executables rebuilt from last release
+    d["mf6_regression"] = Executables.get_path(f"mf6{ext}", rb_bin)
+
+    # local development version
+    d["mf6"] = p / f"mf6{ext}"
+    d["libmf6"] = p / f"libmf6{so}"
+    d["mf5to6"] = p / f"mf5to6{ext}"
+    d["zbud6"] = p / f"zbud6{ext}"
+
+    return d

modflow_devtools/markers.py

@@ -5,6 +5,7 @@
     get_current_branch,
     has_exe,
     has_pkg,
+    is_connected,
     is_in_ci,
 )
 
@@ -55,3 +56,14 @@ def excludes_branch(branch):
     return pytest.mark.skipif(
         current == branch, reason=f"can't run on branch: {branch}"
     )
+
+
+requires_github = pytest.mark.skipif(
+    not is_connected("github.com"), reason="github.com is required."
+)
+
+
+requires_spatial_reference = pytest.mark.skipif(
+    not is_connected("spatialreference.org"),
+    reason="spatialreference.org is required.",
+)

modflow_devtools/misc.py

@@ -7,7 +7,7 @@
 from pathlib import Path
 from shutil import which
 from subprocess import PIPE, Popen
-from typing import List, Optional
+from typing import List, Optional, Tuple
 from urllib import request
 
 import pkg_resources
@@ -28,6 +28,40 @@ def set_dir(path: PathLike):
         print(f"Returned to previous directory: {origin}")
 
 
+class add_sys_path:
+    """
+    Context manager for temporarily editing the system path
+    (https://stackoverflow.com/a/39855753/6514033)
+    """
+
+    def __init__(self, path):
+        self.path = path
+
+    def __enter__(self):
+        sys.path.insert(0, self.path)
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        try:
+            sys.path.remove(self.path)
+        except ValueError:
+            pass
+
+
+def get_suffixes(ostag) -> Tuple[str, str]:
+    """Returns executable and library suffixes for the given OS (as returned by sys.platform)"""
+
+    tag = ostag.lower()
+
+    if tag in ["win32", "win64"]:
+        return ".exe", ".dll"
+    elif tag == "linux":
+        return "", ".so"
+    elif tag == "mac" or tag == "darwin":
+        return "", ".dylib"
+    else:
+        raise KeyError(f"unrecognized OS tag: {ostag!r}")
+
+
 def run_cmd(*args, verbose=False, **kwargs):
     """Run any command, return tuple (stdout, stderr, returncode)."""
     args = [str(g) for g in args]

modflow_devtools/test/test_download.py

@@ -1,7 +1,9 @@
 import pytest
 from modflow_devtools.download import download_and_unzip
+from modflow_devtools.markers import requires_github
 
 
+@requires_github
 @pytest.mark.parametrize("delete_zip", [True, False])
 def test_download_and_unzip(function_tmpdir, delete_zip):
     zip_name = "mf6.3.0_linux.zip"

modflow_devtools/test/test_executables.py

@@ -0,0 +1,50 @@
+import subprocess
+import sys
+from os import environ
+from pathlib import Path
+
+import pytest
+from modflow_devtools.executables import Executables
+from modflow_devtools.misc import add_sys_path, get_suffixes
+
+_bin_path = Path(environ.get("BIN_PATH"))
+_ext, _ = get_suffixes(sys.platform)
+
+
+@pytest.fixture
+def bin_path(module_tmpdir) -> Path:
+    return _bin_path.absolute()
+
+
+@pytest.mark.skipif(not _bin_path.is_dir(), reason="bin directory not found")
+def test_get_path(bin_path):
+    with add_sys_path(str(_bin_path)):
+        ext, _ = get_suffixes(sys.platform)
+        assert (
+            Executables.get_path("mf6", path=_bin_path)
+            == _bin_path / f"mf6{ext}"
+        )
+
+
+def test_get_version(bin_path):
+    with add_sys_path(str(bin_path)):
+        ver_str = Executables.get_version("mf6", path=bin_path).partition(" ")
+        print(ver_str)
+        version = int(ver_str[0].split(".")[0])
+        assert version >= 6
+
+
+@pytest.fixture
+def exes(bin_path):
+    return Executables(mf6=bin_path / f"mf6{_ext}")
+
+
+def test_executables_mapping(bin_path, exes):
+    print(exes.mf6)
+    assert exes.mf6 == bin_path / f"mf6{_ext}"
+
+
+def test_executables_usage(exes):
+    output = subprocess.check_output([f"{exes.mf6}", "-v"]).decode("utf-8")
+    print(output)
+    assert "mf6: 6" in output