Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature hic scaffolding #123

Merged
merged 85 commits into from
Nov 25, 2024
Merged

Feature hic scaffolding #123

merged 85 commits into from
Nov 25, 2024

Commits on Sep 10, 2024

  1. Add Pixi dev env

    @mahesh-panchal
    mahesh-panchal committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    d2be70c View commit details
    Browse the repository at this point in the history

  2. Update/install YAHS and BWAMEM2 modules

    @mahesh-panchal
    mahesh-panchal committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    c5bd396 View commit details
    Browse the repository at this point in the history

  3. Add Scaffold skeleton

    @mahesh-panchal
    mahesh-panchal committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    2163263 View commit details
    Browse the repository at this point in the history

  4. Add scaffold subworkflow to main.nf

    @mahesh-panchal
    mahesh-panchal committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    2521f99 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. add pairtools and pairix modules for HiC pipeline

    @MartinPippel
    MartinPippel committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    a680642 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. add more modules for scaffolding workflow

    @MartinPippel
    MartinPippel committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    ee6395a View commit details
    Browse the repository at this point in the history

  2. add .DS_Store to .gitignore

    @MartinPippel
    MartinPippel committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    31c6321 View commit details
    Browse the repository at this point in the history

  3. add pairtools to scaffolding subworkflow

    @MartinPippel
    MartinPippel committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    bb2f9b6 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. add initial yahs scaffolding subworkflow

    @MartinPippel
    MartinPippel committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    f414e95 View commit details
    Browse the repository at this point in the history

  2. fix file names and process argument

    @MartinPippel
    MartinPippel committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    5329c3c View commit details
    Browse the repository at this point in the history

  3. add version file collection

    @MartinPippel
    MartinPippel committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    10843e8 View commit details
    Browse the repository at this point in the history

  4. remove hard-coded ncbi.taxdb path from test profile

    @MartinPippel
    MartinPippel committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    8b7daab View commit details
    Browse the repository at this point in the history

  5. fix typo ch_versions

    @MartinPippel
    MartinPippel committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    fe78d11 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. increase all cpu and memory default values for scaffolding subworkflow

    @MartinPippel
    MartinPippel committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    79db0e0 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. increase time limits for scaffolding processes

    @MartinPippel
    MartinPippel committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    32e7f2c View commit details
    Browse the repository at this point in the history

  2. add publishDir for yahs process

    @MartinPippel
    MartinPippel committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    38540a5 View commit details
    Browse the repository at this point in the history

  3. fix stage name for yahs publishDir

    @MartinPippel
    MartinPippel committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    a053955 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. publish hic alignment statistics and hard code the threads for compre… 

    …ssion and decomression tasks
    @MartinPippel
    MartinPippel committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    03ae6d0 View commit details
    Browse the repository at this point in the history

  2. bump up pairtools version to 1.1.0

    @MartinPippel
    MartinPippel committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    cc2a8ce View commit details
    Browse the repository at this point in the history

  3. add tiny read set: PacBio HiFi and HiC, species: drosophila, 2Mb

    @MartinPippel
    MartinPippel committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    fc2bf4a View commit details
    Browse the repository at this point in the history

  4. Merge pull request #124 from MartinPippel/feature_hic_scaffolding_pai… 

    …rtools

Feature hic scaffolding - pairtools + yahs subworkflow
    @MartinPippel
    MartinPippel authored Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2236d9a View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. Add testing (#126) 

    * Initialise nf-test

* Remove defunct tests

* Add test data provenance README

* Update docker settings

* Docker as default profile

* Add Tiny CI test yaml

* Update test config

* Update pairtools patch

* Update gitignore

* Comment out cpus

* Add Resume to test config

* Fix intermittent null object issue with LazyMap

* Move around resource configurations

* Improve process tags

* Clean up resource allocation

* Add config README

* Fix pairtools parse cpu allocation

* Update tests/data/tiny/README.md

---------

Co-authored-by: Martin Pippel <[email protected]>
    @mahesh-panchal @MartinPippel
    mahesh-panchal and MartinPippel authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    f11fb12 View commit details
    Browse the repository at this point in the history

  2. Fix Scaffolding to work in Multiple HiC

    @mahesh-panchal
    mahesh-panchal committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    b84377e View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. Assembly report (#127) 

    * Update pairtools patch

* Move and clean up resource configurations

* Sketch out prototype

* Move Groovy things inside workflow

* Add mamba for conda development when container doesn't exist

* Add debug profile

* Add GenomeScope2 plots

* Update assets/notebooks/assembly_report.qmd

Co-authored-by: Martin Pippel <[email protected]>

* Add Pixi tasks to toml

* Add container to Quarto notebook

* Update report qmd

* Add dev task

* Add pandas to quarto env

* Change debug flag name to diagnostics

* Update report

* Transpose DToL table

* Add more tables and plots

* Add KatComp

* Add HiFiasm Kmer graph

* Add Quast

* Add Merqury{,FK} figures and tables

* Add Purgedups plot

* Add Pairtools plots

---------

Co-authored-by: Martin Pippel <[email protected]>
    @mahesh-panchal @MartinPippel
    mahesh-panchal and MartinPippel authored Sep 30, 2024
    Configuration menu
    Copy the full SHA
    2aaeeb7 View commit details
    Browse the repository at this point in the history

  2. Remove custom purging cutoff

    @mahesh-panchal
    mahesh-panchal committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    0207444 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. copy scaffolding subworkflow to new curation subworkflow

    @MartinPippel
    MartinPippel committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    2084623 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. add initial curation subworkflow

    @MartinPippel
    MartinPippel committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    2060a54 View commit details
    Browse the repository at this point in the history

  2. cleanup view calls and add tool versions of curation subworkflow

    @MartinPippel
    MartinPippel committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    2fbcdcd View commit details
    Browse the repository at this point in the history

  3. bugfix renamed process name aliases: SAMTOOLS_MERGE_HIC and SAMTOOLS_… 

    …MERGE_HIFI
    @MartinPippel
    MartinPippel committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    9b49ecc View commit details
    Browse the repository at this point in the history

  4. scaffolding: add dedup stats file to output directory

    @MartinPippel
    MartinPippel committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    665c1fc View commit details
    Browse the repository at this point in the history

  5. write final higlass and pretext results to output dir

    @MartinPippel
    MartinPippel committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    fca2762 View commit details
    Browse the repository at this point in the history

  6. switch from combine to join channels

    @MartinPippel
    MartinPippel committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    805451a View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. cleanup code

    @MartinPippel
    MartinPippel committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    d9c1e64 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. add default runtime limits: cpus,memory,time for some processes

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    493ae17 View commit details
    Browse the repository at this point in the history

  2. cite sanger and arima

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    61d5416 View commit details
    Browse the repository at this point in the history

  3. Update configs/modules.config 

    Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
    @MartinPippel @mahesh-panchal
    MartinPippel and mahesh-panchal authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    2776cbf View commit details
    Browse the repository at this point in the history

  4. Update configs/modules.config 

    Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
    @MartinPippel @mahesh-panchal
    MartinPippel and mahesh-panchal authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    2bcce86 View commit details
    Browse the repository at this point in the history

  5. remove hardcoded ncbi.taxdb path

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    988877c View commit details
    Browse the repository at this point in the history

  6. add curly brackets

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    f268513 View commit details
    Browse the repository at this point in the history

  7. add resourceLimits to test.config and cleanup

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    9e1a69c View commit details
    Browse the repository at this point in the history

  8. Update modules/local/hic_curation/bam2bed_sort.nf 

    Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
    @MartinPippel @mahesh-panchal
    MartinPippel and mahesh-panchal authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    629c3f5 View commit details
    Browse the repository at this point in the history

  9. Update modules/local/hic_curation/bam2coverageTracks.nf 

    Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
    @MartinPippel @mahesh-panchal
    MartinPippel and mahesh-panchal authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    8f3bec9 View commit details
    Browse the repository at this point in the history

  10. cite Sanger and DAMARVEL repo for BAM2BED_SORT module

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    7ca522d View commit details
    Browse the repository at this point in the history

  11. replace container = <CONT> with container <CONT>

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    876b22d View commit details
    Browse the repository at this point in the history

  12. Update modules/local/hic_curation/create_telomer_hitile_track.nf 

    Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
    @MartinPippel @mahesh-panchal
    MartinPippel and mahesh-panchal authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    c559cb1 View commit details
    Browse the repository at this point in the history

  13. Update modules/local/hic_curation/pretext_tracks_ingestion.nf 

    Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
    @MartinPippel @mahesh-panchal
    MartinPippel and mahesh-panchal authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    b04ee59 View commit details
    Browse the repository at this point in the history

  14. add comment to transpose(by:1)

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    1a574b7 View commit details
    Browse the repository at this point in the history

  15. add comments about joining

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    815c8a6 View commit details
    Browse the repository at this point in the history

  16. merge conflict

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    29e4ffc View commit details
    Browse the repository at this point in the history

  17. Merge branch 'feature_hic_scaffolding_pretext' of github.com:MartinPi… 

    …ppel/Earth-Biogenome-Project-pilot into feature_hic_scaffolding_pretext
    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    172e14f View commit details
    Browse the repository at this point in the history

  18. bugfix add missing mapQV value two two_read_combiner

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    3d0520e View commit details
    Browse the repository at this point in the history

  19. bugfix - groupTuples on bam files

    @MartinPippel
    MartinPippel committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    1a92177 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Add test resource limits (#132) 

    * Update test profile limits

* Add resource limits to Gitpod
    @mahesh-panchal
    mahesh-panchal authored Oct 29, 2024
    Configuration menu
    Copy the full SHA
    fe33b60 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. small bugfixed

    @MartinPippel
    MartinPippel committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    66a2a63 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature_hic_scaffolding' into test_scaff_defResources

    @MartinPippel
    MartinPippel committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    87fb393 View commit details
    Browse the repository at this point in the history

  3. create new test branch including HiC and default-resources

    @MartinPippel
    MartinPippel committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    5c45fc1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'feature_hic_scaffolding_pretext' into test_scaff_defRes… 

    …ources
    @MartinPippel
    MartinPippel committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    12bf6b2 View commit details
    Browse the repository at this point in the history

  5. bugfix convert string to integer

    @MartinPippel
    MartinPippel committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    80841a4 View commit details
    Browse the repository at this point in the history

  6. increase memory for FASTK

    @MartinPippel
    MartinPippel committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    a241bc8 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2024

  1. bugfix FASTK resources: issue compressed reads are written out uncomp… 

    …ressed to disk
    @MartinPippel
    MartinPippel committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    8a1e96f View commit details
    Browse the repository at this point in the history

  2. reduce cpus for meryl count process

    @MartinPippel
    MartinPippel committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    cd19f75 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2024

  1. reduce memory limit for bwa index process

    @MartinPippel
    MartinPippel committed Nov 3, 2024
    Configuration menu
    Copy the full SHA
    be11616 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. fix read variable and reduce memory in meryl count module

    @MartinPippel
    MartinPippel committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    d3304fa View commit details
    Browse the repository at this point in the history

  2. fix MemoryUnit multiplication

    @MartinPippel
    MartinPippel committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    6ac35a9 View commit details
    Browse the repository at this point in the history

  3. fix MemoryUnit multiplication - again

    @MartinPippel
    MartinPippel committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    bef40e9 View commit details
    Browse the repository at this point in the history

  4. create bigwig files instead of hitile file - solve issue #133

    @MartinPippel
    MartinPippel committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    d0bd8f2 View commit details
    Browse the repository at this point in the history

  5. cleanup

    @MartinPippel
    MartinPippel committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    ed6f221 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2024

  1. add Maheshs Quarto fix from #aa25974

    @MartinPippel
    MartinPippel committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    1d825a1 View commit details
    Browse the repository at this point in the history

  2. add exception for FCSGX resource to all profiles

    @MartinPippel
    MartinPippel committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    1a45957 View commit details
    Browse the repository at this point in the history

  3. bugfix: add commas to configs

    @MartinPippel
    MartinPippel committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    ca8ce5d View commit details
    Browse the repository at this point in the history

  4. bugfix missing closing bracket

    @MartinPippel
    MartinPippel committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    7f56d22 View commit details
    Browse the repository at this point in the history

  5. another bugfix trial

    @MartinPippel
    MartinPippel committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    e0e13cb View commit details
    Browse the repository at this point in the history

  6. increase memory limit for MINIMAP2_ALIGN_ASSEMBLY_PRIMARY

    @MartinPippel
    MartinPippel committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    1cbb56e View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2024

  1. bugfix FCSGX_RUNGX runtime limits

    @MartinPippel
    MartinPippel committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    d125f48 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. skip hitile creation, use bigwig insteadf

    @MartinPippel
    MartinPippel committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    6a61242 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. cleanup uppmax.config, force FCSGX_RUNGX jobs to run on node partitio… 

    …n on rackham
    @MartinPippel
    MartinPippel committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    b2c31d9 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. Follow dardel config update from nf-core/configs#803

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    78f5954 View commit details
    Browse the repository at this point in the history

  2. Move task.cpus from config to process

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    406eb8c View commit details
    Browse the repository at this point in the history

  3. Remove extra resourceLimits

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    5a4b9be View commit details
    Browse the repository at this point in the history

  4. Move params.hifi_coverage_cap to process input

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    6755a0f View commit details
    Browse the repository at this point in the history

  5. Move params.hic_map_sort_by to process input:

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    1c2cf57 View commit details
    Browse the repository at this point in the history

  6. Fix report

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    7dd6f20 View commit details
    Browse the repository at this point in the history

  7. Fix decontaminate subworkflow

    @mahesh-panchal
    mahesh-panchal committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    2fca1ac View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. cleanup, remove -5 bwa default argument, remove samtools fixmate step… 

    …, add ugly grep hack to avoid samtools header issue
    @MartinPippel
    MartinPippel committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    f89c113 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2024

  1. bugfix args3

    @MartinPippel
    MartinPippel committed Nov 23, 2024
    Configuration menu
    Copy the full SHA
    cc31858 View commit details
    Browse the repository at this point in the history