[New Workflow] Adding the TheiaMeta_Panel_Illumina_PE Workflow

[sage-wright]

This PR closes #553

🗑️ This dev branch should NOT be deleted after merging to main.

🧠 Summary

The Illumina VSP panel is one example of a panel-based sequencing assay. This particular panel contains over 200 target viruses,. This data cannot be analyzed using the traditional TheiaMeta analysis pathway due to the numerous target organisms which makes the assembly first-bin later approach undesirable. TheiaMeta_Panel performs taxonomic binning first with Kraken, then attempts to assemble those bins. If the organism to which the bin belongs is supported in TheiaCoV, we perform additional characterization steps, though the results of those characterizations are not always of high quality. The user must be cautious before reporting on the characterization results.

⚡ Impacted Workflows/Tasks

• TheiaMeta_Illumina_PE

This PR may lead to different results in pre-existing outputs: No

This PR uses an element that could cause duplicate runs to have different results: No

🛠️ Changes

⚙️ Algorithm

TheiaMeta_Panel takes the following approach:

1. Perform general quality control; the minimum read length may need to be lowered to account for the shorter reads generated using this panel.
2. Kraken2 is run using (by default) the Viral database built on RefSeq.
3. After scattering on a list of taxonomic IDs (NCBI), the reads associated with each taxon are extracted
4. The binned reads run through fastq_scan
5. If the number of reads is greater than 1000, assembly is attempted.
6. Assembly is performed first with metaspades -> minimap2 asssembly correction-> samtools sam to bam conversion-> pilon assembly polishing
7. If an assembly was generated, the workflow runs morgana_magic which performs characterization depending on the organism.
8. Results for all shards are gathered and printed to a TSV file.

Assembly is fault resistant, meaning that if one shard fails, the workflow continues
This has impacts for TheiaMeta as workflows will no longer fail if assembly fails in either the metaspades or pilon tasks

➡️ Inputs

Many, please see documentation

⬅️ Outputs

Many, please see documentation

🧪 Testing

Tested regular TheiaMeta on 5 HAV samples here
Tested 21 Illumina VSP samples with TheiaMeta Panel here

Suggested Scenarios for Reviewer to Test

🔬 Final Developer Checklist

• The workflow/task has been tested and results, including file contents, are as anticipated
• The CI/CD has been adjusted and tests are passing (Theiagen developers)
• Code changes follow the style guide
• Documentation and/or workflow diagrams have been updated if applicable (Theiagen developers only)

🎯 Reviewer Checklist

• All changed results have been confirmed
• You have tested the PR appropriately (see the testing guide for more information)
• All code adheres to the style guide
• MD5 sums have been updated
• The PR author has addressed all comments
• The documentation has been updated

[AndrewLangvt]

This some really great work @sage-wright & @cimendes. Just a few points for us to discuss, but I think they're pretty minor overall. I think this workflow has potential to become really widely adopted by labs as multi-pathogen panels become more readily used for "bug screening."

[AndrewLangvt]

does the kraken read extraction retain only mate pairs, or do we have chance for singletons in L & R read datasets? I.e. should this be checking if both read1 and read2 are above the threshold?