Add Makefile

[sckott]

can you please test locally @tefirman and @seankross and let me know any feedback, gracias

[seankross]

@sckott should I just run make or should I try to make all of these targets individually? Or are there certain workflows you have in mind for using this makefile?

[sckott]

there's no all target. yeah individually please. you can just undo any changes to files for the ruff stuff. want to make sure the checks for env vars work and that virtual env is turned on. so do run the test targets without virtual env spun up and make sure you get the error message there

[seankross]

This looks good:

skross wdl-unit-tests % make check_venv 
Virtual env is not activated, run: 'source .venv/bin/activate'
make: *** [check_venv] Error 1

I don't have ruff, should we add it to pyproject.toml?

(wdl-unit-tests) skross wdl-unit-tests % make lint-fix 
uv sync
Resolved 31 packages in 8ms
Audited 28 packages in 0.08ms
ruff check --select I --fix tests/
make: ruff: No such file or directory
make: *** [lint-fix] Error 1

[sckott]

good question about ruff. I have it installed globally, and that's one approach. but let's go with having it in the project in this case

[seankross]

If you commit the changes to the toml in this branch then I'll pull and go back through.

[sckott]

okay, pull down again. changes

• ruff now a dep in the dev group in pyproject toml
• i removed the check of in a venv or not BECAUSE uv run actually runs the code passed to it in the virtual env and then exits for you. And you can check this yourself: if you have a venv folder already, rm -rf it, then run a command and it will be recreated and deps pulled down, then the command runs (or should at least). uv run pulls in the dev dep groups as well as the main one

try running test commands with PATH_ROOTS and PROOF_API_TOKEN_DEV env vars as not set, and try again with them set

let me know how it goes.

[seankross]

idk if you care but when both vars are empty it only reports the first one as being empty.

skross wdl-unit-tests % make check_env_vars 
Makefile:23: *** Undefined PATH_ROOTS (env var for which paths to scrub in vcr cassettes).  Stop.
skross wdl-unit-tests % echo $PATH_ROOTS

skross wdl-unit-tests % echo $PROOF_API_TOKEN_DEV

skross wdl-unit-tests % 

[seankross]

• make test_api_cached works well
• make test_api_rewrite every test fails spectacularly. Here's the summary:

============================================= short test summary info =============================================
ERROR tests/cromwellapi/test-abort.py::test_abort[helloHostname] - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-abort.py::test_abort[helloDockerHostname] - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-abort.py::test_abort[helloModuleHostname] - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-call.py::test_call_initial - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-call.py::test_call_final - tenacity.RetryError: RetryError[<Future at 0x1079520f0 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-failures.py::test_failures_initial - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-failures.py::test_failures_final - tenacity.RetryError: RetryError[<Future at 0x1079520f0 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-labels.py::test_labels - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-metadata.py::test_metadata_initial - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-metadata.py::test_metadata_final - tenacity.RetryError: RetryError[<Future at 0x1079520f0 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-outputs.py::test_outputs - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-search.py::test_search_no_results - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-search.py::test_search_results - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-submit.py::test_submit_works - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-validate.py::test_validate_good_wdl - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-validate.py::test_validate_bad_wdl - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
ERROR tests/cromwellapi/test-version.py::test_version - tenacity.RetryError: RetryError[<Future at 0x10785b610 state=finished raised HTTPStatusError>]
=============================================== 17 errors in 18.37s ===============================================

[sckott]

when both vars are empty it only reports the first one as being empty.

i think that's fine. user can fix the first one missing, then the 2nd one will be surfaced if missing

[sckott]

For the failure on rewrite cmd, HTTPStatusError is pretty general. I was able to reproduce it by being on the VPN but setting PROOF_API_TOKEN_DEV to an invalid value., e.g.

 PROOF_API_TOKEN_DEV=aaa make test_api_rewrite

Was that your issue? I do see some 401 Unauthorized bits above in stack traces when I ran above.

Perhaps we can surface the actual http status problem rather than a general one

[seankross]

How are you running this in the context of using 1pw? op run -- make test_api_rewrite?