RedDog V1beta.11 260719 ("StopBreakingIt")
====== 
Copyright (c) 2016 David Edwards, Bernie Pope, Kat Holt
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, 
are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, 
this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, 
this list of conditions and the following disclaimer in the documentation 
and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors 
may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Description: 
This program implements a workflow pipeline for short read length
sequencing analysis, including the read mapping task, through to variant
detection, followed by analyses (SNPs only).

It uses Rubra (https://github.com/bjpop/rubra) based on the
Ruffus library.

It supports parallel evaluation of independent pipeline stages,
and can run stages on a cluster environment.

Note: for Illumina paired-end or single reads, or Ion Torrent single reads.
IMPORTANT: See config file/instructions for input options/requirements

current version:
V1beta.11   ("StopBreakingIt")
            Added check for ambiguous base calls in reference
            Added ‘no_check’ option for running as single job on cluster
           
previous versions:
V1beta.10.3.1 (Calico Cat)
            fix for parseSNPtabe
V1beta.10.3 (Calico Cat)
            fix for SE and IT reads (no stand bias calling atm - need to validate b4 adding)
            update to many programs used (local Helix installation)
            User Manual update
V1beta.10.2 minor corrections for local install and R scripts
V1beta.10.1 fix for minor error in q30VarFilter
V1beta.10   added strand bias test
V0.1        converted to vcf output via mpileup instead of depreciated pileup 
..
V0.2        tested version V0.1.1 
V0.2.1      adding statistic reporting and fixed "success" handling 
V0.3        tested version of V0.2.1 
V0.3.1      added alternative output paths to aid clean up 
            new name: pipe_VariantDiscovery.py 
            added q20 and q30 mpileups and associated stats collection 
            various cleanup of code/naming conventions used in pipeline 
V0.3.2      added alternative path for IT or PE data analysis 
V0.3.2.1    added alternative path for SE data analysis 
            added minimum read of depths option for variant filtering 
V0.3.2.2    update in various tools (bwa, bamtools and tmap) 
            changes only affects config file
V0.3.3      tested version of V0.3.2.2 
V0.3.4      added options for qc of IT reads 
V0.3.4.1    added size option for qc of IT reads 
            updated statistics reporting - minimum reads 
            version numbers change (1.x to 0.x) 
V0.3.4.2    added pass/fail to stats reporting 
            added outgroup/ingroup to stats reporting 
V0.3.5      tested version of V0.3.4.2 
V0.3.5.1    update to various tools (BWA and tmap) 
            ouput folders now created within the pipeline 
            added separate folder for success files within the temp folder 
            added two new output folders (bam and vcf) 
V0.3.5.2    tested version of V0.3.4.1 
V0.3.5.3    changed to pipe_vda including:
                allow for merging of new sets of reads into a prior run 
                inclusion of analysis pipelets 
                    - pipe_VCFAnalysis and pipe_AllGeneCover
                clean up of temp directory and/or output directory (if merging) 
                changed "type" to "readType" 
                slight change to pipeline order 
V0.4        tested version of V0.3.5.3 
V0.4.0.1    merging of bams from different read sets of same strain 
                (either during "new run" or "merge run")
            fixed bug in gene cover and depth matrices script 
V0.4.0.2    tested version of V0.4.0.1 
V0.4.0.3    removal of QC from within pipeline (and testing) 
V0.4.0.4    replace filter.awk with python-based filtering of all hets from Q30 vcfs 
            includes counting removed het SNPS and reporting same in stat.tab (and testing) 
V0.4.0.5    inclusion of parseSNPtable script (alignment, SNP consequences) (KH, DE)
            and tree generation 
V0.4.0.6    corrections to many scripts used by pipeline, including allele matrix calling 
                and downstream effects to pipeline 
            allele matrix calling now uses consensus sequences 
            addition of differences of SNPs as distance matrix in NEXUS format 
            gene cover and depth matrices no longer contain "fails" 
            addition of parseGeneContent script (KH, DE)
            removal of q20 vcfs (and reporting) - not required 
V0.4.0.7    removed duplicated stages from config file 
V0.4.0.7.1  fix to allow sequences from different folders to be analysed in the same run 
V0.4.0.7.2  fix to deriveStats that let some failed reads pass on depth 
V0.4.1      handling of reference with multiple "chromosomes": pangenome mapping 
                - simplest case: new run (no merging of runs or samples)
                - up to stats collection ('collateRepStats', no post-stats analyses)
            add final '/' to output path(s) if missing 
V0.4.2      handling of reference with multiple "chromosomes": phylogenetic mapping 
                - simplest case: new run (no merging of runs or samples)
                - up to stats collection ('collateRepStats', no post-stats analyses)
            added start-up message 
            changed reference entry from GenBank and Fasta formats to GenBank or Fasta format 
                - fasta reference generated from user GenBank reference
            added pre-run checks including 
                - pairs of reads exist before starting PE analysis 
                - check for 'sequence' option - bad pattern entry 
                - valid run and read types are entered 
            zeroing of SAM files when no longer needed 
V0.4.3      added 'post-stats' analyses - pangenome and phylogeny - no genbank 
            added pre-run reporting and run start confirmation 
V0.4.4      added 'post-stats' analyses - pangenome and phylogeny - with genbank 
V0.4.4.1    various small fixes 
V0.4.4.2    more various small fixes 
V0.4.4.3    increased speed of deriveAllRepGeneCover and getCoverByRep 
V0.4.4.4    conversion of pipeline to use SLURMed Rubra 
V0.4.4.5    fix for bug in BWA sampe/samse v0.7.5 
V0.4.5      renamed pipeline 
            add merging of runs for pangenome and phylogenetic mapping 
            remove single replicon run 
            added bowtie2 to mapping options (all read types) 
            removal of tmap 
            removal and replacement of bamtools (pileup for coverage instead) 
            cleanup of pipeline scripting (amalgamation of repeated stages) 
            converted emboss call to a biopython script 
            add 'check_reads_mapped' variable for multiple replicon runs 
V0.4.5.1    fix for replicon statistics generation for pangenome runs 
V0.4.5.1.1  fix for all statistics generation when no reads map 
V0.4.5.2    check that replicons all have unique names 
            check that output and out_merge_target folders are different 
            check that output folder is not empty string 
            splitting of getRepAlleleMatrix to improve performance 
                includes sequence list generation (start of post-run report)
V0.4.6      update to newer version of parseSNPtable.py 
                - generation of variable and conserved SNP tables 
                - includes of additional option of setting conservation level 
            further early checks that include:
                - name of reference/replicons/isolates won't confuse post-NEXUS analysis (i.e. no '+')
            fix for when a replicon consensus fasta is missing 
                includes new 'warning' file 
            change behaviour of outgroups - reported (also in outgroups.txt fle) but not removed 
            Editing and reorder of options in config file 
V0.4.7      changed 'sequence_list.txt' generation to function 
            added stage counts and check before firing last stage deleteDir 
            added check for isolates/reads with identical names 
            fixes for errors in mergeRepStats and parseSNPtable 
                - latter includes fixes in script to improve performance 
V0.4.8      include post-run report file function 
            test for 'output' folder prior to run 
            default conservation changed to 0.95 
            consolidated chrom_info functions into pipe_utils 
            further replicon name checking 
            fix for pipe-generated gene 'tags' when missing 
            write cns warning files to outMerge on merge run 
V0.4.9      splitting location of intermediate files in temp folder to improve stability for large runs 
            changed -X switch in bowtie2 PE mapping from 500 (default) to 1500 
            removal of some redundant scripts and stages in config file 
V0.5        added check for deletion of previous run success file on merge run 
            added checkpoints for better pipeline running - will halt on errors as expected 
            includes changes to complex stages - flagFiles behaviour 
V0.5.1      added -X option for bowtie2 mapping 
            fixed parseGeneContent output 
            updated to fixed and extended parseSNPtable 
            added scripts for tutorial (filterCoords.py, get_cover.py, getRecomb.R, plotTree.R) 
            changed getDifferenceMatrix to optional output in pipe, changed script to take options 
            added option for VCF output of filtered hets 
            implemented changes to run report from user feedback 
V0.5.2      run report provides settings for merge runs (continuity checks) 
                includes more robust 'check_reads_mapped' 
            update to use SAMtools v1+ 
                includes (limited) addition of multiallelic SNP calling option - bcftools 
            added checkpoint_getSamStats to capture failure during initial BAM construction 
            changed bams from glob call to list call 
            various small fixes to some default values 
            fixed getRecomb.R 
V1.0b       Any fixes from final testing 
                includes handling of "." in replicon name 
            Add licensing information to all scripts 
            one more post analysis script added [for Gubbins recombination analysis] 
            update parseGeneContent.py (P/A matrix based on cover and depth) 
V1beta.1    fix for parseSNPTable - reported position of snp in non-coding feature 
V1beta.2    fix for parseSNPTable - improved speed of reading and parsing snp table 
V1beta.3    changed fasttree to raxml 
            added option to stop tree generation,
            or force tree if > 200 isolates 
V1beta.4    added simple check for correct BAM generation 
            added checkpoint for consensus calling 
V1beta.5    added further filter of SNPS in finalFilter 
V1beta.6    changed back to FastTree - precision error in RAxML -m ASC_GTRCAT 
            changed maximum isolates for tree to 500 
            changed checkBam to pass BAMs from simulated reads 
V1beta.7 (BlackCat)
            fixed bug in quality filtering of variant calls 
V1beta.8    tutorial update 
V1beta.9    local system update 
            manual update