[FDS-2238, FDS-2236] Automate CLI test plan #1528
Conversation
andrewelamb
commented
Oct 30, 2024
•
andrewelamb
commented
- Fixes FDS-2238
- Fixes FDS-2236
- Automates the CLI test plan
andrewelamb
requested review from
GiaJordan,
linglp and
BryanFauble
as code owners
tests/integration/test_commands.py
Outdated
| """Tests the schematic/manifest/commands validate manifest command""" | ||
|
|
||
| def test_generate_empty_csv_manifest(self, runner: CliRunner) -> None: | ||
| """Generate two empty csv manifests""" |
There was a problem hiding this comment.
I'll leave it to @BryanFauble for final review, but did we want to follow similar paradigm as the API testing documentation to list exactly what is being tested?
also are there any manual things that need to be checked in the manifests here?
There was a problem hiding this comment.
Yep it was a lot fo C+P and the modifying for the CLI tests hwere needed. If there's anythign I missed that I should include please let me know :)
There was a problem hiding this comment.
C+P? I'll leave it to your responsibility to ensure that you've captured everything that the manual test plan outlined.
As for the documentation, I'd be great if there was heavier documentation in the doc strings like:
Download a manifest from the Schematic API and verify that it is a valid Excel
file. We are validating the following:
- The first row of the Excel file contains the column headers
- The first row is locked on scroll
- Each cell in the first row has a comment/tooltip "TBD"
- The "Family History" cell has a comment/tooltip with the expected message
- The cell corresponding to "Sex" in Sheet1 has a dropdown list with values from Sheet2!B2:B4
- The cell corresponding to "Diagnosis" in Sheet1 has a dropdown list with values from Sheet2!D2:D3
- The cell corresponding to "Cancer Type" in Sheet1 has a dropdown list with values from Sheet2!F2:F6
- The workbook contains two sheets: "Sheet1" and "Sheet2"
- "Sheet2" is hidden
- The values in "Sheet2" are as expected
- The fill colors of the first row cells are as expected
- Conditional formatting is functioning as expected
Manual verification steps:
- Open the Excel file prefixed with TestManifestGeneration_test_single_manifest_generation_excel.xlsx
- When Diagnosis = "Cancer", [Cancer Type, Family History] is Light Blue (Required)
If there are manul verification steps required for the manifest itself it should follow the same paradigm that Bryan set up
There was a problem hiding this comment.
+1 My hope is that the docstrings are the single source of great documentation that we can add to the pdocs site.
thomasyu888
changed the title
|
Converting this to draft, going to work on some more. |
|
The test for the submit command still needs to be done, after it has been fixed, so this is still a draft for now. I can start gettign feedback on the other parts though. |
There was a problem hiding this comment.
I took a look at the test failures and there's something that changed that's also causing the synapse storage tests to fail...
FAILED tests/test_store.py::TestSynapseStorage::test_init - AssertionError: assert 'syn28559058' == 'syn23643253'
Related to the other testing problem:
hmm, would it matter if we ran tests/integration/test_manifest_generation.py outside of all the other tests? (it's not ideal, but maybe the ordering of this DOES matter)?
|
