New nf-core role to pull methylseq, rnaseq, atacseq etc and other changes #332
Conversation
aanil
commented
Jan 27, 2020
aanil
commented
- New nf-core role to pull
- methylseq v1.4 (Updated version as well)
- rnaseq v1.3 (Updated version as well)
- atacseq
- ampliseq
- Deleted old roles for methylseq, rnaseq, atacseq and ampliseq
- Update Sarek to pull from nf-core
- Updated ngi reports
- Updated taca
- Updated neutronstar to 1.0.0
- Updated multiqc to v1.8
roles/neutronstar/tasks/main.yml
Outdated
| @@ -39,14 +52,19 @@ | |||
| lineinfile: | |||
| dest: "{{ ngi_pipeline_conf }}/{{ bash_env_sthlm_script }}" | |||
| line: > | |||
| alias neutronstar='nextflow run {{ neutronstar_dest }}/main.nf -profile standard,uppmax | |||
| -c {{ ngi_pipeline_conf }}/neutronstar_{{ item.site }}.config' | |||
| alias neutronstar='nextflow run {{ neutronstar_dest }}/main.nf -profile standard,uppmax \ | |||
There was a problem hiding this comment.
Does this use the nf-core/configs repo? You should drop-profile standard,uppmax for -profile singularity
There are still some more options that is more suited and/or needed for irma. As a config file load with -c:
params {
busco_folder = '/sw/apps/bioinfo/BUSCO/v2_lineage_sets'
busco_data = 'eukaryota_odb9'
clusterOptions = '-A ngi2016003'
max_memory = 250.GB
max_cpus = 16
max_time = 240.h
}
process {
executor = 'slurm'
}Or alternatively add them as parameters to that alias with -process.executor='slurm'
There was a problem hiding this comment.
I'm a bit unsure if to keep busco_data = 'eukaryota_odb9' - I think it's a reasonable default and does make the pipeline more foolproof.
There was a problem hiding this comment.
I've updated the parameters in this config file. Also changed the profile to singularity.
There was a problem hiding this comment.
@remiolsen - you should add this config as an uppmax + pipeline specific config in the nf-core/configs repo.
See https://github.com/nf-core/configs/tree/master/conf/pipeline and https://github.com/nf-core/configs/blob/master/pipeline/ and ask @maxulysse for help if you need it 👍
There was a problem hiding this comment.
Note that it's only params.busco_folder and params.busco_data that looks unique to me. I think everything else is already set up in the uppmax profile.
|
I'm happy for this to be merged / deployed now 👍 If we go ahead now, could we please make issues to address the following in the future though, so that they're not forgotten:
|
roles/nf-core/tasks/main.yml
Outdated
| dest: "{{ ngi_pipeline_conf }}/{{ item.1.script }}" | ||
| line: > | ||
| alias {{ item.0.name }}='nextflow run {{ sw_path }}{{ item.0.name }}/workflow/ \ | ||
| --custom_config_base {{ sw_path }}{{ item.0.name }}/configs \ |
There was a problem hiding this comment.
If we are using nf-core download then I'm pretty sure that we don't need this line.
There was a problem hiding this comment.
Will nf-core download link the general site config that we create in the previous step?
There was a problem hiding this comment.
@ewels wont the pipeline start look online for configs if you skip this line?
There was a problem hiding this comment.
No - nf-core download actually edits the pipeline config files to set params.custom_config_base to point to the parent directory (see code). So as long as you leave the directory structure in tact then it should just work..
There was a problem hiding this comment.
I've removed the custom_config_base option.
| -c {{ ngi_pipeline_conf }}/neutronstar_{{ item.site }}.config' | ||
| alias neutronstar='nextflow run {{ neutronstar_dest }}/main.nf -profile uppmax \ | ||
| -c {{ ngi_pipeline_conf }}/nextflow_irma_{{ item.site }}.config \ | ||
| -c {{ ngi_pipeline_conf }}/neutronstar_{{ item.site }}.config --supernova_container {{ supernova_container_path }}' |
There was a problem hiding this comment.
Cool 👍
We could move supernova_container in to the neutronstar config file, but seeing as we will hopefully get rid of this config file before long, I think it's better how you currently have it ✅
roles/nf-core/defaults/main.yml
Outdated
| @@ -0,0 +1,13 @@ | |||
| nextflow_java: "/sw/comp/java/x86_64/sun_jdk1.8.0_151" | |||
| nextflow_version_tag: "v19.10.0" | |||
There was a problem hiding this comment.
A new version of nextflow came out a few days ago, so you can grab that now
| - name: Create directories for singularity images | ||
| file: | ||
| path: "{{ ngi_containers }}/{{ item.name }}" | ||
| state: directory | ||
| mode: g+s | ||
| with_items: "{{ pipelines }}" | ||
|
|
||
| - name: Move singularity images | ||
| shell: "mv {{ sw_path }}{{ item.name }}/singularity-images/nf-core-{{ item.name }}-{{ item.release }}.simg {{ ngi_containers }}/{{ item.name }}/" | ||
| with_items: | ||
| - "{{ pipelines }}" |
There was a problem hiding this comment.
Didn't we say we were going to remove this, and leave the nf-core download dir structure?
There was a problem hiding this comment.
I've created #336 for it as its dependent on some pending PRs in nf-core.
|
Merging with verbal approval from Phil! |
