feat(Case): add class for reusable test case data

README.md

@@ -13,9 +13,10 @@ Python tools for MODFLOW development and testing.
 - [Included](#included)
   - [`MFZipFile` class](#mfzipfile-class)
   - [Keepable temporary directories](#keepable-temporary-directories)
-  - [Example model tests](#example-model-tests)
+  - [Model-loading fixtures](#model-loading-fixtures)
     - [Test model fixtures](#test-model-fixtures)
     - [Example scenario fixtures](#example-scenario-fixtures)
+  - [Reusable test case framework](#reusable-test-case-framework)
   - [Conditionally skipping tests](#conditionally-skipping-tests)
   - [Miscellaneous](#miscellaneous)
     - [Generating TOCs with `doctoc`](#generating-tocs-with-doctoc)
@@ -37,14 +38,14 @@ This package is not yet published to PyPI or a Conda channel. To install it plea
 This package contains shared tools for developing and testing MODFLOW 6 and FloPy, including standalone utilities as well as `pytest` fixtures, CLI options, and test parametrizations:
 
 - a `ZipFile` subclass preserving file attributes
-- various `pytest` fixtures and utilities
-  - keepable temporary directories
-  - a smoke testing CLI option
-  - dynamic test parametrization from example repos
-  - markers to conditionally skip test cases based on
-    - operating system
-    - Python packages installed
-    - executables available on the path
+- variably-scoped `pytest` temporary directory fixtures
+- a `pytest` smoke test CLI option (to run a fast subset of cases)
+- a minimal `pytest` framework for reusing test functions and data
+- a `pytest_generate_tests` hook to load example/test model fixtures
+- a set of `pytest` markers to conditionally skip test cases based on
+  - operating system
+  - Python packages installed
+  - executables available on the path
 
 To import `pytest` fixtures in a project consuming `modflow-devtools`, add the following to a `conftest.py` file in the project root:
 
@@ -94,7 +95,7 @@ pytest <test file> --keep temp
 
 There is also a `--keep-failed <path>` variant which only preserves outputs from failing test cases.
 
-### Example model tests
+### Model-loading fixtures
 
 Fixtures are provided to load models from the MODFLOW 6 example and test model repositories and feed them to test functions. Models can be loaded from:
 
@@ -143,6 +144,32 @@ def test_example_scenario(tmpdir, example_scenario):
         # ...
 ```
 
+### Reusable test case framework
+
+A second approach to testing, more flexible than loading pre-existing models from a repository, is to construct test models in code. This typically involves defining variables or `pytest` fixtures in the same test script as the test function. While this pattern is effective for manually defined scenarios, it tightly couples test functions to test cases, prevents easy reuse of the test case by other tests, and tends to lead to duplication, as each test script may reproduce similar test functions and data-generation procedures.
+
+This package provides a minimal framework for self-describing test cases which can be defined once and plugged into arbitrary test functions. At its core is the `Case` class, which is just a `SimpleNamespace` with a few defaults and a `copy_update()` method for easy modification. This pairs nicely with [`pytest-cases`](https://smarie.github.io/python-pytest-cases/), which is recommended but not required.
+
+A `Case` requires only a `name`, and has a single default attribute, `xfail=False`, indicating whether the test case is expected to succeed. (Test functions may of course choose to use or ignore this.)
+
+For instance, to generate a set of similar test cases with `pytest-cases`:
+
+```python
+from pytest_cases import parametrize
+
+from modflow_devtools.case import Case
+
+template = Case(name="QA")
+cases = [
+  template.copy_update(name=template.name + "1", question="What's the meaning of life, the universe, and everything?", answer=42),
+  template.copy_update(name=template.name + "2", question="Is a Case immutable?", answer="No, but it's better not to mutate it.")
+]
+
+@parametrize(data=cases, ids=[c.name for c in cases])
+def case_qa(case):
+    print(case.name, case.question, case.answer)
+```
+
 ### Conditionally skipping tests
 
 Several `pytest` markers are provided to conditionally skip tests based on executable availability, Python package environment or operating system.

modflow_devtools/case.py

@@ -0,0 +1,39 @@
+from types import SimpleNamespace
+
+
+class Case(SimpleNamespace):
+    """
+    Minimal container for a reusable test case.
+    """
+
+    def __init__(self, **kwargs):
+        if "name" not in kwargs:
+            raise ValueError(f"Case name is required")
+
+        # set defaults
+        if "xfail" not in kwargs:
+            kwargs["xfail"] = False
+        # if 'compare' not in kwargs:
+        #     kwargs['compare'] = True
+
+        super().__init__(**kwargs)
+
+    def __repr__(self):
+        return self.name
+
+    def copy(self):
+        """
+        Copies the test case.
+        """
+
+        return SimpleNamespace(**self.__dict__.copy())
+
+    def copy_update(self, **kwargs):
+        """
+        A utility method for copying a test case with changes.
+        Recommended for dynamically generating similar cases.
+        """
+
+        cpy = self.__dict__.copy()
+        cpy.update(kwargs)
+        return SimpleNamespace(**cpy)

modflow_devtools/test/test_case.py

@@ -0,0 +1,33 @@
+import pytest
+from modflow_devtools.case import Case
+
+
+def test_requires_name():
+    with pytest.raises(ValueError):
+        Case()
+
+
+def test_defaults():
+    assert not Case(name="test").xfail
+
+
+def test_copy():
+    case = Case(name="test", foo="bar")
+    copy = case.copy()
+
+    assert case is not copy
+    assert case == copy
+
+
+def test_copy_update():
+    case = Case(name="test", foo="bar")
+    copy = case.copy_update()
+
+    assert case is not copy
+    assert case == copy
+
+    copy2 = case.copy_update(foo="baz")
+
+    assert copy is not copy2
+    assert copy.foo == "bar"
+    assert copy2.foo == "baz"

setup.cfg

@@ -55,6 +55,7 @@ test =
     %(lint)s
     coverage
     flaky
+    pytest-cases
     pytest-cov
     pytest-dotenv
     pytest-xdist