include pytest-cov meta.yaml #126

ilaflott · 2024-07-16T17:08:55Z

Describe your changes

add pytest-cov coverage tool to pytest calls in meta.yaml, see how well it works.

not totally sure I want to merge this yet, but it's easier to run the workflow with a draft PR open.

Issue ticket number and link (if applicable)

Issue #122

Checklist before requesting a review

I ran my code
I tried to make my code readable
N/A ~~I tried to comment my code~~
N/A ~~I wrote a new test, if applicable~~
N/A ~~I wrote new instructions/documentation, if applicable~~
I ran pytest and inspected it's output
N/A ~~I ran pylint and attempted to implement some of it's feedback~~

add `pytest-cov` coverage tool to `pytest` calls in `meta.yaml`, see how well it works.

ilaflott · 2024-07-16T17:09:43Z

Taking out of draft- the workflow will not run for draft PRs

ilaflott · 2024-07-16T17:24:07Z

Seems to work great!

Name                                                                  Stmts   Miss  Cover
-----------------------------------------------------------------------------------------
fre/__init__.py                                                          10     10     0%
fre/app/__init__.py                                                       3      3     0%
fre/app/freapp.py                                                        29     29     0%
fre/app/generate_time_averages/__init__.py                                1      1     0%
fre/app/generate_time_averages/cdoTimeAverager.py                        53     53     0%
fre/app/generate_time_averages/frenctoolsTimeAverager.py                 33     33     0%
fre/app/generate_time_averages/frepytoolsTimeAverager.py                130    130     0%
fre/app/generate_time_averages/generate_time_averages.py                 33     33     0%
fre/app/generate_time_averages/tests/test_generate_time_averages.py     139     55    60%
fre/app/generate_time_averages/timeAverager.py                           32     32     0%
fre/app/maskAtmosPlevel.py                                               77     77     0%
fre/catalog/__init__.py                                                   2      2     0%
fre/catalog/frecatalog.py                                                19     19     0%
fre/catalog/gen_intake_gfdl.py                                           42     42     0%
fre/check/__init__.py                                                     3      3     0%
fre/check/frecheck.py                                                    12     12     0%
fre/check/frecheckexample.py                                              9      9     0%
fre/cmor/CMORmixer.py                                                   199    199     0%
fre/cmor/__init__.py                                                      3      3     0%
fre/cmor/frecmor.py                                                      16     16     0%
fre/fre.py                                                                8      8     0%
fre/lazy_group.py                                                        22     22     0%
fre/list/__init__.py                                                      3      3     0%
fre/list/frelist.py                                                      12     12     0%
fre/list/frelistexample.py                                                9      9     0%
fre/make/__init__.py                                                      7      7     0%
fre/make/createCheckout.py                                               55     55     0%
fre/make/createCompile.py                                                53     53     0%
fre/make/createDocker.py                                                 47     47     0%
fre/make/createMakefile.py                                               43     43     0%
fre/make/fremake.py                                                      66     66     0%
fre/make/runFremake.py                                                   91     91     0%
fre/pp/__init__.py                                                       10     10     0%
fre/pp/checkoutScript.py                                                 32     32     0%
fre/pp/configure_script_xml.py                                          288    288     0%
fre/pp/configure_script_yaml.py                                          87     87     0%
fre/pp/frepp.py                                                          86     86     0%
fre/pp/install.py                                                        10     10     0%
fre/pp/run.py                                                            10     10     0%
fre/pp/status.py                                                         10     10     0%
fre/pp/validate.py                                                       13     13     0%
fre/pp/wrapper.py                                                        55     55     0%
fre/run/__init__.py                                                       3      3     0%
fre/run/frerun.py                                                        12     12     0%
fre/run/frerunexample.py                                                  9      9     0%
fre/test/__init__.py                                                      3      3     0%
fre/test/fretest.py                                                      12     12     0%
fre/test/fretestexample.py                                                9      9     0%
fre/tests/test_fre_cli.py                                                12      0   100%
fre/yamltools/__init__.py                                                 3      3     0%
fre/yamltools/data_table/__init__.py                                      0      0   100%
fre/yamltools/data_table/combine_data_table_yamls.py                     45     45     0%
fre/yamltools/data_table/data_table_to_yaml.py                           72     72     0%
fre/yamltools/data_table/is_valid_data_table_yaml.py                     64     64     0%
fre/yamltools/diag_table/__init__.py                                      0      0   100%
fre/yamltools/diag_table/combine_diag_table_yamls.py                     86     86     0%
+ pylint fre/
fre/yamltools/diag_table/diag_table_to_yaml.py                          250    250     0%
fre/yamltools/diag_table/is_valid_diag_table_yaml.py                    141    141     0%
fre/yamltools/field_table/__init__.py                                     0      0   100%
fre/yamltools/field_table/combine_field_table_yamls.py                   75     75     0%
fre/yamltools/field_table/field_table_to_yaml.py                        177    177     0%
fre/yamltools/freyamltools.py                                            12     12     0%
fre/yamltools/freyamltoolsexample.py                                      9      9     0%
-----------------------------------------------------------------------------------------
TOTAL                                                                  2856   2760     3%

ilaflott · 2024-07-16T17:26:24Z

aacccctuualllllly i don't know why i'm getting coverage metrics for tests/... i assumed pytest scripts would be omitted.

add `omit` field so that `pytest-cov` doesn't include testing scripts (have names matching `*/test_py`) in coverage metrics

for specifying testing paths/directories, etc. directories without testing paths/directories are commented out

ceblanton · 2024-07-16T18:32:32Z

I see what you mean. How does it distinguish the test scripts from the actual code, you're asking.

fre/tests/test_fre_cli.py should be counted as coverage for fre/fre.py right?

ilaflott · 2024-07-16T18:35:08Z

I see what you mean. How does it distinguish the test scripts from the actual code, you're asking.

fre/tests/test_fre_cli.py should be counted as coverage for fre/fre.py right?

latest commit should take care of this! see .coveragerc and .pytest.ini

ceblanton · 2024-07-16T18:53:17Z

can you see the updated coverage output in the CI logs? I looked for it but couldn't see it.

The documentation CI "test" is unhappy now, not related to these updates I think.

…`meta.yaml` this fixes the syntax highlighting locally when viewing the file in emacs.

restrict `pytest` and `pytest-cov` version to be later. make `.pytest.ini` config file reading more explicit in `pytest` call within `meta.yaml`

in github CI/CD, they move stuff to a temporary dir, so .coveragerc and .pytest.ini were not found by github's runners make references to these configs explicit in meta.yaml call now to help avoid confusion

…st-cov at all. last check before asking for review and merge. if this pipeline doesnt work, restrictions to versions will be added back in

ilaflott · 2024-07-16T19:42:49Z

can you see the updated coverage output in the CI logs? I looked for it but couldn't see it.

... no, but i think i see something relevant here... from our CI/CD:

platform linux -- Python 3.9.19, pytest-7.4.4, pluggy-1.0.0
rootdir: $SRC_DIR
plugins: cov-4.1.0, anyio-4.2.0

... from my local copy however:

platform linux -- Python 3.9.19, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/Ian.Laflotte/Working/fre-cli_pytest_cov
configfile: .pytest.ini
plugins: cov-5.0.0, anyio-4.4.0
collected 20 items

The documentation CI "test" is unhappy now, not related to these updates I think.

for doc changes that fix this: #125

…argeted test directories. lets see if that helps the github CI/CD

ilaflott · 2024-07-16T20:05:09Z

looks good enough to me- the new empty __init.py__ files get caught by the coverage tool, but the testing code does not! additionally, the coverage metric isn't affected, since the added files are empty.

merging!

include pytest-cov meta.yaml

caee951

add `pytest-cov` coverage tool to `pytest` calls in `meta.yaml`, see how well it works.

ilaflott marked this pull request as ready for review July 16, 2024 17:09

ilaflott added 2 commits July 16, 2024 14:25

Create .coveragerc

f6384a1

add `omit` field so that `pytest-cov` doesn't include testing scripts (have names matching `*/test_py`) in coverage metrics

Create .pytest.ini

ee1ed0a

for specifying testing paths/directories, etc. directories without testing paths/directories are commented out

ilaflott requested a review from ceblanton July 16, 2024 18:35

ilaflott self-assigned this Jul 16, 2024

change single quotes to double for arguments of environ.get within …

a1f7964

…`meta.yaml` this fixes the syntax highlighting locally when viewing the file in emacs.

ceblanton approved these changes Jul 16, 2024

View reviewed changes

ilaflott and others added 3 commits July 16, 2024 15:16

Update meta.yaml

e211dc0

restrict `pytest` and `pytest-cov` version to be later. make `.pytest.ini` config file reading more explicit in `pytest` call within `meta.yaml`

unhide coverage/testing config files and put them in fre/...

daf7933

in github CI/CD, they move stuff to a temporary dir, so .coveragerc and .pytest.ini were not found by github's runners make references to these configs explicit in meta.yaml call now to help avoid confusion

not sure the problem i had was related to versions of pytest and pyte…

6bd5c03

…st-cov at all. last check before asking for review and merge. if this pipeline doesnt work, restrictions to versions will be added back in

apparently, pytest-cov, which uses coverage, expects __init__.py in t…

29113b3

…argeted test directories. lets see if that helps the github CI/CD

ilaflott merged commit 51a5a91 into main Jul 16, 2024
1 of 2 checks passed

ilaflott deleted the pytest-cov branch July 16, 2024 20:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

include pytest-cov meta.yaml #126

include pytest-cov meta.yaml #126

ilaflott commented Jul 16, 2024 •

edited

Loading

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ceblanton commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ceblanton commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024

include pytest-cov meta.yaml #126

include pytest-cov meta.yaml #126

Conversation

ilaflott commented Jul 16, 2024 • edited Loading

Describe your changes

Issue ticket number and link (if applicable)

Checklist before requesting a review

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ceblanton commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ceblanton commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024

ilaflott commented Jul 16, 2024 •

edited

Loading