Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add CI for a scaled down version of a cancer WES pipeline #499

Merged
merged 12 commits into from
Apr 18, 2024

Conversation

tedil
Copy link
Member

@tedil tedil commented Apr 11, 2024

This PR adds a snakemake workflow that executes a snappy-pipeline (which in turn invokes snakemake, but, well…) to be run as as a CI target. The output is not validated yet, so for now this simply tests whether execution is successful or not.
The snappy-pipeline is a basic cancer_wes one with steps:

  • ngs_mapping
  • somatic_variant_calling
  • somatic_variant_annotation
  • somatic_variant_filtration

where ngs_mapping is done using bwa, variant calling is done with mutect2, variant annotation is done with vep, and filtration is done with bedtools.
The reference is restricted to chr12, the raw reads (from the TCRB dataset) are filtered down to those mapping to a specific region on chr12, and the vep cache is also restricted to the same region.

(It would probably make sense to restrict the trigger such that it only runs when any of the steps/wrappers used are changed, but I'll leave it set to run on PR and on push to main for now)

Copy link

  • Please format your Python code with black: make black
  • Please format your Snakemake code with snakefmt: make snakefmt
  • Please organize your imports isorts: make isort
  • Please ensure that your code passes flake8: make flake8

You can trigger all lints locally by running make lint

@tedil tedil marked this pull request as draft April 11, 2024 15:20
@tedil
Copy link
Member Author

tedil commented Apr 11, 2024

This currently uses a post-deploy script https://github.com/bihealth/snappy-pipeline/pull/499/files#diff-74b65b07424e0be9f2b93fb89d5f2918b63381841c6feb6612a63da6aa20288c to install snappy-pipeline, which is stupid because the whole point is to use the version of the branch that is modified/the PR is based on.
That needs to be changed ^^

Still uses the post deploy script, but uses the snappy-pipeline version of the current state (of the PR/branch)

@coveralls
Copy link

coveralls commented Apr 11, 2024

Coverage Status

coverage: 85.764% (+0.02%) from 85.742%
when pulling 5ce7038 on end-to-end-ci
into 6993233 on main.

@tedil tedil marked this pull request as ready for review April 15, 2024 09:55
@tedil tedil requested a review from ericblanc20 April 16, 2024 12:35
.tests/test-workflow/config/cancer_wes/config.yaml.jinja2 Outdated Show resolved Hide resolved
.tests/test-workflow/config/cancer_wes/config.yaml.jinja2 Outdated Show resolved Hide resolved
.tests/test-workflow/config/cancer_wes/config.yaml.jinja2 Outdated Show resolved Hide resolved
.tests/test-workflow/workflow/envs/snappy.post-deploy.sh Outdated Show resolved Hide resolved
.tests/test-workflow/workflow/rules/reference.smk Outdated Show resolved Hide resolved
.tests/test-workflow/workflow/rules/cancer_wes.smk Outdated Show resolved Hide resolved
.tests/test-workflow/workflow/schemas/config.schema.yaml Outdated Show resolved Hide resolved
.github/workflows/ci-e2e.yml Outdated Show resolved Hide resolved
@ericblanc20 ericblanc20 merged commit b971088 into main Apr 18, 2024
8 checks passed
@ericblanc20 ericblanc20 deleted the end-to-end-ci branch April 18, 2024 16:31
@tedil tedil mentioned this pull request Jun 28, 2024
This was referenced Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants