6.0.0

NOTE!

We now have a config file! https://toil.readthedocs.io/en/latest/running/cliOptions.html#the-config-file

Breaking Changes

• Removed the parasol batch system
• Removed the TES batch system (this is now a plugin)
• Removed our WDL compiler in favor of an interpreter (we still support WDL, we just do it differently now)
• We no longer support python3.7

CWL

• Support CWL 1.2.1 (#4682)
• CWL Pipefish compatibility (#4636)
• Support per-task preemptibility in CWL (#4551)
• Fix configargparse in CWL (#4618)
• cwl: use the latest commit from the proposed CWL v1.2.1 branch (#4565)
• Upgrade cwltool to avoid broken galaxy-tool-util release. (#4639)
• Implement a better config file system for CWL/WDL options (#4666)
• Allow working with remote files in CWL and WDL workflows (#4690)
• Make cwl mutually exclusive groups exist only when cwl is not suppressed (#4725)
• Log more usefully for CWL workflows (#4736)

WDL

• Simplify WDL Toil job graphs (#4524)
• More WDL and Slurm documentation (#4558)
• Improve WDL documentation (#4732)
• Add String to File functionality into toil-wdl-runner (#4589)
• Run WDL output through Toil export system to support URIs (#4579)
• Allow the WDL output section to reference itself (#4592)
• Ensure sibling files in toil-wdl-runner (#4610)
• Make WDLOutputJob collect all task outputs (#4602)
• Report errors in WDL using MiniWDL's error location printer (#4637)
• Remove the WDL compiler. (#4679)
• Implement a better config file system for CWL/WDL options (#4666)
• Allow working with remote files in CWL and WDL workflows (#4690)
• Strip leading whitespace from WDL commands (#4720)

Misc

• Add config file support (#4569)
• Support Python3.11 and drop Python 3.7 (#4646)
• Move TES batch system to a plugin (#4650)
• Turn batch system tests back on (#4649)
• Separate out integration tests to run on a schedule (#4612)
• Avoid concurrent modification in cluster scaler tests (#4600)
• Remove old buckets from AWS (#4588)
• Tests: only request a single core (#4572)
• Reduce the number of assert statements (#4590)
• take any nvidia-smi exception as not having gpu (#4611)
• More resiliancy (#4395)
• Remove useage of the deprecated pkg_resources (#4701)
• Make sure cwltool always knows we have an outdir to fix #4698 (#4699)
• AWS jobStoreTest: re-use delete_s3_bucket from toil.lib.aws (#4700)
• Only count output file usage when using the file store (#4692)
• Remove the parasol batch system. (#4678)
• Move around reqs and move aws dev libraries to aws (#4664)
• Make sure the --batchLogsDir exists if it is set (#4635)
• Update EC2 instances and EC2 update script. (#4745)
• remove extraneous dependency on old 'mock' (#4739)
• Point CI at the new public URLs for stuff we host
• Add init.py to options folder (#4723)

Bug Fixes

• Lower redirect log level to fix #4526 (#4578)
• Fix mypy from being broken by new boto types (#4577)
• Fix CI on local Gitlab runners (#4571)
• Banish ghost jobs (#4563)
• Stop deleting chained-to jobs which fail as orphaned jobs (#4557)
• Fix pickling error when jobstate file doesnt exist and fix threading error when lock file exists then disappears (#4575)
• Fix #3867 and try to explain but not crash when bad things happen to our mutex file (#4656)
• Fix CI Appliance Builds (#4655)
• Tolerate a failed AMI polling attempt (#4727)* Add pure Python fallback for getDirSizeRecursively() (#4753)
• Don't mark inputs (or outputs) executable for no reason (#4728)
• Fix scheduled CI tests (#4742)
• Fix --printJobInfo (#4709)

Thank you to our contributors: @stxue1 , @w-gao, @DailyDreaming , @mr-c , @adamnovak , @glennhickey, @misterbrandonwalker, and @a-detiste !