Merge pull request #294 from mskcc/release/0.13.0

Release/0.13.0

README.md

@@ -8,7 +8,7 @@ https://www.nextflow.io/
 Inspiration from `Sarek` at the NBIS and SciLifeLab in Sweden:
 https://github.com/SciLifeLab/Sarek
 
-Vaporware contains several pipeline scripts (`pipeline.nf`, `make_bam_and_qc.nf`, `somatic.nf`, `germline.nf`). A detalied explanation for each step is provided below.
+Vaporware contains several pipeline `*.nf` scripts but all functionality is contained in `pipeline.nf`.
 
 ## Setup instructions
 
@@ -29,6 +29,9 @@ This installs Nextflow within the subdirectory `vaporware`.
 
 Optionally, move the `nextflow` file to a directory accessible by your `$PATH` variable in order to avoid remembering and typing the full path to `nextflow` each time.
 
+## Description of pipeline.nf
+
+The `pipeline.nf` script takes a mapping and pairing file to perform BAM preprocessing, somatic analysis, and germline analysis. BAM Preprocessing always occurs; to activate the somatic and germline pipelines, use the `--somatic` and `--germline`, respectively, at the command line. See the sections below for command examples.
 
 ## For submitting via LSF on juno
 
@@ -71,7 +74,7 @@ nextflow run pipeline.nf --somatic --germline --mapping test_inputs/aws/test_mak
 
 **NOTE:** To being able to run locally you need to provide reference files from `conf/references.config` and create `samples.tsv` as described in the wiki page [Bioinformatic components](https://github.com/mskcc/vaporware/wiki/Bioinformatic-Components)
 
-The default parameters are for local-use *WITHOUT* containers
+The default parameters are for local-use *WITHOUT* containers.
 
 * For Docker use, do the following:
 
@@ -85,6 +88,16 @@ nextflow run pipeline.nf --somatic --germline --mapping mapping.tsv --pairing pa
 nextflow run pipeline.nf --somatic --germline --mapping mapping.tsv --pairing pairing.tsv -profile singularity
 ```
 
+## Useful Tips
+
+* __Number of dashes matter__: Nextflow has a quirk where Nextflow executor-specific, built-in flags are initiated with a single dash, whereas parameters we define require two. For example, to define which run profile to use, we call Nextflow's built-in feature `-profile`; to do resume, we use `-resume`. To provide mapping or pairing input file paths, which call functions the development team at MSKCC had to write, we have to call it with `--mapping` and `--pairing`, respectively.
+
+* __Modify and Resume__: Nextflow supports [modify and resume](https://www.nextflow.io/docs/latest/getstarted.html?#modify-and-resume); to activate this feature, use `-resume` (note the single dash) at the command line to use Nextflow's cache history and continue a job from where it left off. This even lets you make changes to values in the script and continue from there - Nextflow will use the cached information from the unchanged sections while running only the modified processes.
+
+* __View the Nextflow Log__: You can access the Nextflow cache metadata by running `nextflow log`. It contains information like TIMESTAMP, DURATION, and RUN_NAME, and a STATUS indicating a failed or successful run, among others. The values under RUN_NAME can also be submitted following the `-resume` flag to resume previously-run Nextflow jobs.
+
+* __Run/Skip specific tools:__ `pipeline.nf` lets you run tools you want and skip others. The aforementioned `--somatic` and `--germline` flags already have a preset of tools to include during a run, but you can limit this further by providing the `--tools` flag, followed by a comma-delimited string. For example, to use only DELLY for your somatic/germline runs, do `--somatic --germline --tools delly`; to use MuTect2, Manta, and Strelka2, do `--somatic --tools mutect2,manta,strelka2`.
+
 ## Components 
 
 The following links to the wiki provide greater details:

containers/delly/Dockerfile

@@ -1,32 +1,48 @@
-FROM frolvlad/alpine-glibc:alpine-3.9
+FROM ubuntu:18.04
 
-LABEL maintainer="Nikhil Kumar ([email protected])" \
-      version.image="1.0.0" \
-      version.delly="0.7.7" \
-      version.htslib="1.6" \
-      version.bcftools="1.6" \
-      version.samtools="1.6" \
-      version.alpine="3.8" \
-      source.delly="https://github.com/dellytools/delly/releases/tag/v0.7.7" \
-      source.htslib="https://github.com/samtools/htslib/releases/tag/1.6" \
-      source.bcftools="https://github.com/samtools/bcftools/releases/tag/1.6" \
-      source.samtools="https://github.com/samtools/samtools/releases/tag/1.6"
+LABEL maintainer="Sinisa Ivkovic ([email protected])" \
+      version.ubuntu="18.04" \
+      version.delly="0.8.1" \
+      version.htslib="1.9"
 
-ENV DELLY_VERSION v0.8.1
+# set the environment variables
+ENV delly_version 0.8.1
+ENV htslib_version 1.9
 
+# run update
+RUN apt-get update -y && apt-get install -y \
+    libnss-sss \
+    curl \
+    less \
+    vim  \
+    wget \
+    unzip \
+    build-essential \
+    libboost-all-dev \
+    zlib1g-dev \
+    libncurses5-dev \
+    libncursesw5-dev \
+    libnss-sss \
+    libbz2-dev \
+    liblzma-dev \
+    bzip2 \
+    libcurl4-openssl-dev
 
-# set environment
-ENV BOOST_ROOT /usr
-ENV SEQTK_ROOT /opt/htslib
-
-RUN apk add --update --no-cache ncurses \
-    && apk add --virtual=deps --update --no-cache ncurses-dev musl-dev git g++ make zlib-dev \
-    && apk add bash build-base asciidoc bzip2-dev xz-dev boost-dev \
-    && cd /opt \
-    # install delly
-    && cd /opt \
-    && git clone --recursive https://github.com/dellytools/delly.git -b ${DELLY_VERSION} \
-    && cd /opt/delly/ \
-    && make all \
-    && install -p /opt/delly/src/delly /usr/local/bin/ 
+# install htslib
+WORKDIR /usr/local/bin/
+RUN curl -SL https://github.com/samtools/htslib/releases/download/${htslib_version}/htslib-${htslib_version}.tar.bz2 \
+    > /usr/local/bin/htslib-${htslib_version}.tar.bz2
+RUN tar -xjf /usr/local/bin/htslib-${htslib_version}.tar.bz2 -C /usr/local/bin/
+RUN cd /usr/local/bin/htslib-${htslib_version}/ && ./configure
+RUN cd /usr/local/bin/htslib-${htslib_version}/ && make
+RUN cd /usr/local/bin/htslib-${htslib_version}/ && make install
+ENV LD_LIBRARY_PATH /usr/local/bin/htslib-${htslib_version}/
 
+# install delly
+WORKDIR /usr/local/bin
+RUN wget https://github.com/dellytools/delly/archive/v${delly_version}.zip
+RUN unzip v${delly_version}.zip
+WORKDIR /usr/local/bin/delly-${delly_version}
+RUN make all
+WORKDIR /usr/local/bin
+RUN ln -s /usr/local/bin/delly-0.8.1/src/delly /usr/local/bin/delly

containers/facets-suite/Dockerfile

@@ -2,16 +2,16 @@ FROM frolvlad/alpine-glibc:alpine-3.9
 
 LABEL maintainer="Nikhil Kumar ([email protected])" \
       version.image="1.0.0" \
-      version.facets_suite="1.5.6" \
+      version.facets_suite="1.5.7" \
       version.facets="0.5.14"\
       version.alpine="3.8" \
       version.r="3.5.1" \
       version.pctGCdata="0.2.0" \
-      source.facets_suite="https://github.com/mskcc/facets-suite/releases/tag/1.5.6" \
+      source.facets_suite="https://github.com/mskcc/facets-suite/releases/tag/1.5.7" \
       source.facets="https://github.com/mskcc/facets/archive/v0.5.14.tar.gz"\
       source.r="https://pkgs.alpinelinux.org/package/edge/community/x86/R"
 
-ENV FACETS_SUITE_VERSION 1.5.6
+ENV FACETS_SUITE_VERSION 1.5.7
 ENV FACETS_VERSION 0.5.14
 ENV PCTGCDATA 0.2.0
 
@@ -24,6 +24,8 @@ RUN apk add --update \
         && apk add cairo cairo-dev libxt-dev libxml2-dev font-xfree86-type1 \
     # download and install R
         && apk add R R-dev \
+    # change permissions for /tmp; still doesn't resolve R warnings
+        && chmod  777 -R /tmp \
     # download and unzip facets, facets-suite, pctGCdata
         && cd /tmp \
 	    && wget https://github.com/mskcc/facets-suite/archive/${FACETS_SUITE_VERSION}.tar.gz -O facets-suite-${FACETS_SUITE_VERSION}.tar.gz \
@@ -53,7 +55,7 @@ RUN apk add --update \
                 && sed -i "s/opt\/common\/CentOS_6-dev\/R\/R-3.2.2\//usr\//g" *.R \
             # copy execs to /usr/bin/facets-suite
                 && mkdir -p /usr/bin/facets-suite/ \
-                && cp /tmp/facets-suite-${FACETS_SUITE_VERSION}/* /usr/bin/facets-suite/ \
+                && cp -r /tmp/facets-suite-${FACETS_SUITE_VERSION}/* /usr/bin/facets-suite/ \
     # clean up
         && rm -rf /var/cache/apk/* /tmp/*
 

nextflow.config

@@ -18,7 +18,7 @@ manifest {
   mainScript = 'main_align_markDups_BaseRecal.nf'
   name = 'Vaporware'
   nextflowVersion = '>=0.32.0'
-  version = '0.12.0'
+  version = '0.13.0'
 }
 
 // Provides extra options for the Java and Nextflow runtime.