Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: remove reference solution comparison files #10

Merged
merged 1 commit into from
Dec 22, 2023

Conversation

wpbonelli
Copy link
Contributor

@wpbonelli wpbonelli commented Dec 21, 2023

This repository contains reference solution files in compare subdirs of model dirs and have extra suffix .cmp. The MF6 regression test framework has up to now retained the ability to compare with these via the --original-regression command line flag*.

These are omitted from CI in favor of comparing against the latest MF6 release, however. And some comparisons fail, confirmed back to at least 6.4.1

test001e_noUZF_3lay
test028_sfr_rewet_simple
test028_sfr_rewet
test028_sfr_simple
test028_sfr
test034_nwtp2_1d
test045_lake1tr_nr
test045_lake2tr_nr
test045_lake2tr
test020_NT_EI
test020_NevilleTonkinTransient
test034_nwtp2
test051_uzfp3_lakmvr_v2
test051_uzfp3_wellakmvr_v2
test045_lake4ss
test017_Crinkle
test045_lake4ss_nr_embedded
test059_mvlake_lak_ss

Given that some *.cmp files are outdated and they see very limited use, it seems they could be removed.

An alternative to these (and to the pattern where we download the previous MF6 release, rebuild it, and run it alongside the development version) could be to snapshot and version output files with a pytest plugin

*Recent autotest refactoring broke this comparison function — the most recent commit in which it works is ad4acb76dfc2a4885623c806efed69181db117a3

Copy link
Contributor

@langevin-usgs langevin-usgs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it would be good to remove these compare files, especially since they are no longer needed. I'm not sure if we should remove the general capability to use a compare file though, since in some cases a regression comparison with a previous mf6 version may not be possible.

@wpbonelli wpbonelli merged commit c72e1ae into MODFLOW-USGS:master Dec 22, 2023
@wpbonelli wpbonelli deleted the develop branch December 22, 2023 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants