2_running_nextflow_pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9c5526cf",
   "metadata": {},
   "source": [
    "# Running the pipeline\n",
    "Here, we will run the pipeline. In notebook #1, you should have generated a `metadata.tsv` file now that looks like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "979d7b5a-ed81-4200-aa7d-cb15bd88ba07",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:24:19.398556Z",
     "iopub.status.busy": "2023-09-15T12:24:19.398296Z",
     "iopub.status.idle": "2023-09-15T12:24:19.505154Z",
     "shell.execute_reply": "2023-09-15T12:24:19.504606Z",
     "shell.execute_reply.started": "2023-09-15T12:24:19.398539Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sample_name\ttechnology\tfastq_PE1_path\tfastq_barcode_path\tfastq_PE2_path\n",
      "BIO_ddseq_4\tbiorad\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/BIO_ddseq_4__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/BIO_ddseq_4__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/BIO_ddseq_4__R1.LIBDS.fastq.gz\n",
      "EPF_hydrop_1\thydrop_2x384\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/EPF_hydrop_1__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/EPF_hydrop_1__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/EPF_hydrop_1__R1.LIBDS.fastq.gz\n",
      "OHS_s3atac_1\ts3atac_1\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/OHS_s3atac_1__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/OHS_s3atac_1__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/OHS_s3atac_1__R1.LIBDS.fastq.gz\n",
      "VIB_10xv2_1\tatac_revcomp\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/VIB_10xv2_1__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/VIB_10xv2_1__R1.LIBDS.fastq.gz\t/lustre1/project/stg_00002/lcb/fderop/data/00000000_PUMATAC_example_fastq/VIB_10xv2_1__R1.LIBDS.fastq.gz\n"
     ]
    }
   ],
   "source": [
    "cat metadata.tsv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00dd2821-82a4-4386-9206-6f931abcc46b",
   "metadata": {},
   "source": [
    "Make sure PUMATAC is up to date:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "89920182-7fc2-41f5-a85e-0df1845732c0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:24:20.628891Z",
     "iopub.status.busy": "2023-09-15T12:24:20.628635Z",
     "iopub.status.idle": "2023-09-15T12:24:21.134239Z",
     "shell.execute_reply": "2023-09-15T12:24:21.133684Z",
     "shell.execute_reply.started": "2023-09-15T12:24:20.628875Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Already up to date.\n"
     ]
    }
   ],
   "source": [
    "cd PUMATAC && git pull && cd .."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d1debfd",
   "metadata": {},
   "source": [
    "### 1. Generate a .config file\n",
    "The config file contains a number of variables and paths that are needed by the pipeline. `$NXF_WORK` is a temporary `work` directory, which will contain intermediate files used by the pipeline. The final output files will be written in a directory which we aribtrarily name `PUMATAC_tutorial_preprocessing_out`. A finished config should look like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c34356d6-1b1a-4878-81fc-a62d9c6115a6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:24:23.019455Z",
     "iopub.status.busy": "2023-09-15T12:24:23.019215Z",
     "iopub.status.idle": "2023-09-15T12:24:24.015771Z",
     "shell.execute_reply": "2023-09-15T12:24:24.015240Z",
     "shell.execute_reply.started": "2023-09-15T12:24:23.019439Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "manifest\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39mname\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvib-singlecell-nf/vsn-pipelines\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mdescription\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mA repository of pipelines for single-cell data in Nextflow DSL2\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mhomePage\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mhttps://github.com/vib-singlecell-nf/vsn-pipelines\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mversion\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m0.27.0\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mmainScript\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmain.nf\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mdefaultBranch\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmaster\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mnextflowVersion\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m!21.04.3\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "params\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39mglobal\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mproject_name\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m10x_PBMC\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39moutdir\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mmisc\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mtest\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mutils\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/utils:0.4.0\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mfile_converter\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39moff\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mh5ad\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mtagCellWithSampleId\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mremove10xGEMWell\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39museFilteredMatrix\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mmakeVarIndexUnique\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mpublish\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcompressionLevel\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m6\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mannotateWithBatchVariableName\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mmode\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcopy\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39matac_preprocess_tools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mmark_duplicates_method\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mMarkDuplicates\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39madapter_trimming_method\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTrim_Galore\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mdata\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39matac_preprocess\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mmetadata\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmetadata.tsv\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mtools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39msinglecelltoolkit\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/singlecelltoolkit:2022-04-15-16314db\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mbarcode_correction\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39mmax_mismatches\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39mmin_frac_bcs_to_find\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m0.5\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39mwhitelist\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m               \u001b[39matac\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-cratac-v1.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39matac_revcomp\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-cratac-v1.REVCOMP.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mmultiome\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-arc-v1.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mmultiome_revcomp\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-arc-v1.REVCOMP.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mhydrop_2x384\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/hydrop_384x384.REVCOMP.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mhydrop_3x96_short\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/20230120_hydrop-atac_ligation_all_revcomp.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39ms3_atac_1\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/s3_atac_1.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39m}\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m         \u001b[39mbarcode_10x_scatac_fastqs\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39muncorrected_bc_tag\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mCR\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39mbarcode_quality_tag\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mCY\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mtrimgalore\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/trimgalore:0.6.6\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mtrim\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39mpaired\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mtrue\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mbwamaptools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/bwamaptools:bwa-mem2-2.2.1-zlibng-2.0.6\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mbwa_fasta\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/genomes/hg38_bwamem2/genome.fa\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mbwa_version\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbwa-mem2\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39msamtools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/samtools:0.3-1.16.1\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mbarcard\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mbarcode_multiplet\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39mreport_ipynb\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m/src/barcard/bin/barcard_otsu_filtering.ipynb\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "}\n",
      "\n",
      "process\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39mexecutor\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mlocal\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m2\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m60 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mclusterOptions\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-A cluster_account\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__default\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mtime\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m1h\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__sctk_barcode\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m2\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m20 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__barcode_10x_scatac_fastq_5cpus\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m40 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__trimgalore__trim_5cpus\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m20 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__picard__merge_sam_files_and_sort\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m100 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__picard__mark_duplicates_and_sort\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m100 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__sinto__fragments\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m40 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__sinto__sort_fragments\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m40 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__bap_barcode_multiplet_pipeline_8cpus\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m80 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m3\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__minimal\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m1 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__mem\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m160 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__cpu\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m20\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m80 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__report\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m2\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m160 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__24hqueue\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mtime\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m24h\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__bwa_mem\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mexecutor\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mlocal\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m6\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m60 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mtime\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m24h\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "}\n",
      "\n",
      "timeline\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/execution_timeline.html\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "report\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/execution_report.html\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "trace\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/execution_trace.txt\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "dag\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/pipeline_dag.svg\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "\u001b[38;5;28mmin\u001b[39m\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "}\n",
      "\n",
      "vsc\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "}\n",
      "\n",
      "singularity\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mautoMounts\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mrunOptions\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m--cleanenv -H $PWD -B /lustre1,/staging,/data,${VSC_SCRATCH},${VSC_SCRATCH}/tmp:/tmp,${HOME}/.nextflow/assets/\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mcacheDir\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m/staging/leuven/res_00001/software/vsn_containers/\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "pygmentize -g atac_preprocess_rapid.config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1ea89d9-4267-46b6-8fda-4aec14d22be0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:08:34.368773Z",
     "iopub.status.busy": "2023-09-15T12:08:34.368529Z",
     "iopub.status.idle": "2023-09-15T12:08:34.476304Z",
     "shell.execute_reply": "2023-09-15T12:08:34.475647Z",
     "shell.execute_reply.started": "2023-09-15T12:08:34.368756Z"
    }
   },
   "source": [
    "We generate the config file as such:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "360516e5-1104-4731-abbf-da65f68d0f01",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:25:19.312279Z",
     "iopub.status.busy": "2023-09-15T12:25:19.312008Z",
     "iopub.status.idle": "2023-09-15T12:25:19.462735Z",
     "shell.execute_reply": "2023-09-15T12:25:19.461756Z",
     "shell.execute_reply.started": "2023-09-15T12:25:19.312262Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "chmod 755 PUMATAC_dependencies/nextflow/nextflow-22.10.7-all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e82549eb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:25:19.679960Z",
     "iopub.status.busy": "2023-09-15T12:25:19.679752Z",
     "iopub.status.idle": "2023-09-15T12:25:22.422020Z",
     "shell.execute_reply": "2023-09-15T12:25:22.421204Z",
     "shell.execute_reply.started": "2023-09-15T12:25:19.679944Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "module load Java/17.0.2\n",
    "NXF_WORK=./work\n",
    "[ ! -d $NXF_WORK ] && mkdir $NXF_WORK\n",
    "PUMATAC_dependencies/nextflow/nextflow-22.10.7-all config ./PUMATAC/main_atac.nf \\\n",
    "    -profile atac_preprocess_rapid,vsc \\\n",
    "    > atac_preprocess_rapid.config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a50476d-e498-420d-9ff1-648e1aa6181c",
   "metadata": {},
   "source": [
    "### 2. Edit the .config file\n",
    "Now, you must modify `atac_preprocess_rapid_template.config` to fit your use case. I recommend saving a new file named `atac_preprocess_rapid.config`.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e8248bb-6c90-4606-b071-964a371efbb8",
   "metadata": {},
   "source": [
    "Rename the project and the output directory to your liking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd16c9e1-d38d-4319-a346-1043ee73c12c",
   "metadata": {},
   "outputs": [],
   "source": [
    "params {\n",
    "   global {\n",
    "      project_name = 'PUMATAC_tutorial'\n",
    "      outdir = 'PUMATAC_tutorial_preprocessing_out'\n",
    "   }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3adf2889-15dc-461a-83b4-a3f2cae3cee6",
   "metadata": {},
   "source": [
    "Redirect variable `metadata` to your newly made `metadata.tsv` file. `metadata.tsv` is the standard name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8354d18e-e568-4e6d-86f2-8d811c26f801",
   "metadata": {},
   "outputs": [],
   "source": [
    "params {\n",
    "   data {\n",
    "      atac_preprocess {\n",
    "         metadata = 'metadata.tsv'\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4a60e46-344d-4ba1-9679-61e4790750dc",
   "metadata": {},
   "source": [
    "Add correct whitelist for each technology called in `metadata.tsv`. For example, we included a technology named `atac`. We therefore add `atac = 'resources/whitelists/737K-cratac-v1.txt.gz'`. `atac` is not one of the 5 basic options (`standard`, `hydrop_3x96`, `hydrop_2x384`, `multiome` and `biorad`.), so the pipeline will treat it as a `standard` barcode correction case, but with a custom whitelist.  \n",
    "  \n",
    "`PUMATAC_dependencies` already contains whitelists for 10x and HyDrop. If you have a custom technique, such as `s3_atac`, you can add your own whitelist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "136c9498-9b40-46d4-a858-34a78011106a",
   "metadata": {},
   "outputs": [],
   "source": [
    "params {\n",
    "    tools {\n",
    "       singlecelltoolkit {\n",
    "         container = 'vibsinglecellnf/singlecelltoolkit:2022-04-15-16314db'\n",
    "         barcode_correction {\n",
    "            max_mismatches = 1\n",
    "            min_frac_bcs_to_find = 0.5\n",
    "            whitelist {\n",
    "               atac = 'PUMATAC_dependencies/whitelists/737K-cratac-v1.txt.gz'\n",
    "               atac_revcomp = 'PUMATAC_dependencies/whitelists/737K-cratac-v1.REVCOMP.txt.gz'\n",
    "               multiome = 'PUMATAC_dependencies/whitelists/737K-arc-v1.txt.gz'\n",
    "               multiome_revcomp = 'PUMATAC_dependencies/whitelists/737K-arc-v1.REVCOMP.txt.gz'\n",
    "               hydrop_2x384 = 'PUMATAC_dependencies/whitelists/hydrop_384x384.REVCOMP.txt.gz'\n",
    "               hydrop_3x96_short = 'PUMATAC_dependencies/whitelists/20230120_hydrop-atac_ligation_all_revcomp.txt.gz'\n",
    "               s3_atac_1 = 'PUMATAC_dependencies/whitelists/s3_atac_1.txt.gz'\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}\n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7086c4d1-c536-499d-88c9-582211686d9c",
   "metadata": {},
   "source": [
    "Change `bwa_fasta = '/path/to/bwa_index/hg38.fa'` to your genome of choice. This needs to be the .fa file associated with a bwa-mem2 genome index in the same directory. Generating such an index is a simple as running `bwa-mem2 index <in.fa>`. You can use the container `PUMATAC_dependencies/cache/vibsinglecellnf-bwamaptools-bwa-mem2-2.2.1-zlibng-2.0.6.img` to run bwa-mem2. Please see https://github.com/bwa-mem2/bwa-mem2 for more information about bwa-mem2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1225eb4-ff85-429e-834f-94a7e4c70eb4",
   "metadata": {},
   "outputs": [],
   "source": [
    "params {\n",
    "    tools {\n",
    "         bwamaptools {\n",
    "         bwa_fasta = 'PUMATAC_dependencies/genomes/hg38_bwamem2/genome.fa'\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "889f0439-6bb0-4bed-a4dd-f9cec9df6c8e",
   "metadata": {},
   "source": [
    "Change the `withLabel:compute_resources__bwa_mem` parameters, e.g. change `executor` to `local` to run on current node, instead of submitting a job with the PBS system. You can also change the number of CPUs per sample (i.e. per line in the `metadata.tsv`) here. If the runtime of `bwa-mem2` exceeds the time defined here (24 hours), the pipeline will stop. You can thus increase this time limit if necessary. maxForks increases the maximum number of samples that you allow to be run in parallel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d1d8139-cf50-4092-9fb8-85486a4f76c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "process {\n",
    "   withLabel:compute_resources__bwa_mem {\n",
    "      executor = 'local'\n",
    "      cpus = 20\n",
    "      memory = '120 GB'\n",
    "      time = '24h'\n",
    "      maxForks = 2\n",
    "   }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1b7ac4c-916f-457a-bf4f-996ee499f1a1",
   "metadata": {},
   "source": [
    "Edit the singularity run options to include all mounts in which files that you require or want to write are located. The following line is valid for our computing environment, but this will be different for you:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ffea4968-6392-40e9-86f7-e37c9eef834b",
   "metadata": {},
   "outputs": [],
   "source": [
    "runOptions = '--cleanenv -H $PWD -B /lustre1,/staging,/data,${VSC_SCRATCH},${VSC_SCRATCH}/tmp:/tmp,${HOME}/.nextflow/assets/'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcdc347d-034a-4391-a764-f3b0704a2f49",
   "metadata": {},
   "source": [
    "Give an arbitrary location to the `cache` directory. If you have not downloaded `PUMATAC_dependencies`, then PUMATAC will automatically download the containers it needs to run code to your defined cache directory. If you have downloaded `PUMATAC_dependencies`, then these containers will be present there. If you want to re-run some of the tools that the pipeline calls, without re-running the entire pipeline, you can simply open the relevant singularity container, and call the tool from within the singularity container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfde01ab-3616-4e27-b5f5-c11d96304cd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "cacheDir = 'PUMATAC_dependencies/cache'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "312f9d53-c8c4-4665-a1e2-5cbb2c8b5f47",
   "metadata": {},
   "source": [
    "That should be everything. Here is an example of a functional `.config`. I suggest you run a difftool (e.g. https://text-compare.com/) on this to compare this to your own `.config` if you run into problems:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b261007f-a3bb-4267-bc50-e2b9c59713f1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-15T12:25:37.422349Z",
     "iopub.status.busy": "2023-09-15T12:25:37.422100Z",
     "iopub.status.idle": "2023-09-15T12:25:39.165425Z",
     "shell.execute_reply": "2023-09-15T12:25:39.164879Z",
     "shell.execute_reply.started": "2023-09-15T12:25:37.422332Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "manifest\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39mname\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvib-singlecell-nf/vsn-pipelines\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mdescription\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mA repository of pipelines for single-cell data in Nextflow DSL2\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mhomePage\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mhttps://github.com/vib-singlecell-nf/vsn-pipelines\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mversion\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m0.27.0\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mmainScript\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmain.nf\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mdefaultBranch\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmaster\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mnextflowVersion\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m!21.04.3\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "params\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39mglobal\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mproject_name\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_tutorial_out\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39moutdir\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_out\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mmisc\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mtest\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mutils\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/utils:0.4.0\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mfile_converter\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39moff\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mh5ad\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mtagCellWithSampleId\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mremove10xGEMWell\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39museFilteredMatrix\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mmakeVarIndexUnique\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mpublish\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcompressionLevel\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m6\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mannotateWithBatchVariableName\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mmode\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcopy\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39matac_preprocess_tools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mmark_duplicates_method\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mMarkDuplicates\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39madapter_trimming_method\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTrim_Galore\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mdata\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39matac_preprocess\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mmetadata\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mmetadata.tsv\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mtools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39msinglecelltoolkit\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/singlecelltoolkit:2022-04-15-16314db\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mbarcode_correction\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39mmax_mismatches\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39mmin_frac_bcs_to_find\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m0.5\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39mwhitelist\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m               \u001b[39matac\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-cratac-v1.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39matac_revcomp\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-cratac-v1.REVCOMP.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mmultiome\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-arc-v1.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mmultiome_revcomp\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/737K-arc-v1.REVCOMP.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mhydrop_2x384\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/hydrop_384x384.REVCOMP.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39mhydrop_3x96_short\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/20230120_hydrop-atac_ligation_all_revcomp.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m               \u001b[39ms3_atac_1\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPUMATAC_dependencies/whitelists/s3_atac_1.txt.gz\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39m}\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m         \u001b[39mbarcode_10x_scatac_fastqs\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39muncorrected_bc_tag\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mCR\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m            \u001b[39mbarcode_quality_tag\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mCY\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mtrimgalore\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/trimgalore:0.6.6\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mtrim\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39mpaired\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mtrue\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mbwamaptools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/bwamaptools:bwa-mem2-2.2.1-zlibng-2.0.6\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mbwa_fasta\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m/lustre1/project/stg_00002/lcb/fderop/data/00000000_genomes/GRCh38_mm10_bwa_mem2/genome.fa\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39mbwa_version\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbwa-mem2\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39msamtools\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mcontainer\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mvibsinglecellnf/samtools:0.3-1.16.1\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39mbarcard\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m         \u001b[39mbarcode_multiplet\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m            \u001b[39mreport_ipynb\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m/src/barcard/bin/barcard_otsu_filtering.ipynb\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m         \u001b[39m}\n",
      "\u001b[38;5;250m      \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "}\n",
      "\n",
      "process\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39mexecutor\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mlocal\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m2\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m60 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mclusterOptions\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-A cluster_account\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__default\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mtime\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m1h\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__sctk_barcode\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m2\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m20 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__barcode_10x_scatac_fastq_5cpus\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m40 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__trimgalore__trim_5cpus\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m20 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m5\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__picard__merge_sam_files_and_sort\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m100 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__picard__mark_duplicates_and_sort\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m100 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__sinto__fragments\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m40 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__sinto__sort_fragments\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m40 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__bap_barcode_multiplet_pipeline_8cpus\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m8\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m80 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m3\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__minimal\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m1 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__mem\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m160 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__cpu\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m20\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m80 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__report\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m2\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m1\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m160 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__24hqueue\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mtime\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m24h\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "\u001b[38;5;250m   \u001b[39mwithLabel:compute_resources__bwa_mem\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m      \u001b[39mexecutor\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mlocal\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mcpus\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m6\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmemory\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m60 GB\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mtime\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m24h\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m      \u001b[39mmaxForks\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;241m4\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39m}\n",
      "}\n",
      "\n",
      "timeline\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/execution_timeline.html\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "report\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/execution_report.html\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "trace\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/execution_trace.txt\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "dag\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mfile\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mout/nextflow_reports/pipeline_dag.svg\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n",
      "\n",
      "\u001b[38;5;28mmin\u001b[39m\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mfalse\u001b[39m\n",
      "}\n",
      "\n",
      "vsc\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "}\n",
      "\n",
      "singularity\u001b[38;5;250m \u001b[39m{\n",
      "\u001b[38;5;250m   \u001b[39menabled\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mautoMounts\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;28mtrue\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mrunOptions\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m--cleanenv -H $PWD -B /lustre1,/staging,/data,${VSC_SCRATCH},${VSC_SCRATCH}/tmp:/tmp,${HOME}/.nextflow/assets/\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "\u001b[38;5;250m   \u001b[39mcacheDir\u001b[38;5;250m \u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;250m \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m/staging/leuven/res_00001/software/vsn_containers/\u001b[39m\u001b[38;5;124m'\u001b[39m\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "pygmentize -g atac_preprocess_rapid.config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "702c8f47",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-09T08:56:14.239433Z",
     "iopub.status.busy": "2022-06-09T08:56:14.239085Z",
     "iopub.status.idle": "2022-06-09T08:56:14.493186Z",
     "shell.execute_reply": "2022-06-09T08:56:14.490236Z",
     "shell.execute_reply.started": "2022-06-09T08:56:14.239394Z"
    }
   },
   "source": [
    "### 3. Run the pipeline\n",
    "We now call the right version of NextFlow to run our pipeline `main_atac.nf` on our `atac_preprocess_rapid.config`. I usually do this in a tmux session, or as a submitted job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2dd9e5a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "PUMATAC_dependencies/nextflow/nextflow-21.04.3-all -C atac_preprocess_rapid.config run PUMATAC/main_atac.nf -entry atac_preprocess_rapid"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fec3fcab-f0e6-4eda-9c3b-460f970a401c",
   "metadata": {},
   "source": [
    "The output will look like so:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b41bd777-5bf0-4280-8aee-03ac5b7283a3",
   "metadata": {},
   "source": [
    "```\n",
    "N E X T F L O W  ~  version 21.04.3                                                                                                                                                                                                          Launching `PUMATAC/main_atac.nf` [festering_easley] - revision: c18ea24e2f                                                                                                                                                                  executor >  local (35)                                                                                                                                                                                                                       [03/1df167] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_standard:SCTK__BARCODE_CORRECTION (2)                         [100%] 2 of 2 ✔                                                                                   executor >  local (35)                                                                                                                                                                                                                       [03/1df167] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_standard:SCTK__BARCODE_CORRECTION (2)                         [100%] 2 of 2 ✔                                                                                   executor >  local (35)                                                                                                                                                                                                                       [03/1df167] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_standard:SCTK__BARCODE_CORRECTION (2)                         [100%] 2 of 2 ✔                                                                                   executor >  local (51)                                                                                                                                                                                                                       [03/1df167] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_standard:SCTK__BARCODE_CORRECTION (2)                         [100%] 2 of 2 ✔                                                                                   [68/016dd5] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_standard:PUBLISH_BC_STATS (2)                                 [100%] 2 of 2 ✔                                                                                   [17/c1a9fc] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_standard:SCTK__BARCODE_10X_SCATAC_FASTQ (1)                   [100%] 2 of 2 ✔                                                                                   [-        ] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:SCTK__EXTRACT_HYDROP_ATAC_BARCODE_3x96                                   -                                                                                                 [-        ] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_hydrop_3x96:SCTK__BARCODE_CORRECTION                          -                                                                                                 [-        ] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_hydrop_3x96:PUBLISH_BC_STATS                                  -                                                                                                 [-        ] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_hydrop_3x96:SCTK__BARCODE_10X_SCATAC_FASTQ                    -                                                                                                 [a3/c8a3a3] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:SCTK__EXTRACT_HYDROP_ATAC_BARCODE_2x384 (1)                              [100%] 1 of 1, cached: 1 ✔                                                                        [17/7f7075] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_hydrop_2x384:SCTK__BARCODE_CORRECTION (1)                     [100%] 1 of 1 ✔                                                                                   [90/d88953] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_hydrop_2x384:PUBLISH_BC_STATS (1)                             [100%] 1 of 1 ✔                                                                                   [38/9b45b9] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_hydrop_2x384:SCTK__BARCODE_10X_SCATAC_FASTQ (1)               [100%] 1 of 1 ✔                                                                                   [e8/f70d41] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_biorad:SCTK__EXTRACT_AND_CORRECT_BIORAD_BARCODE (1)           [100%] 1 of 1, cached: 1 ✔                                                                        [73/069f44] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:bc_correct_biorad:PUBLISH_BR_BC_STATS (1)                                [100%] 1 of 1, cached: 1 ✔                                                                        [ba/727e0e] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:adapter_trimming:TRIMGALORE__TRIM (3)                                    [100%] 4 of 4, cached: 1 ✔                                                                        [53/e25895] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:adapter_trimming:PUBLISH_FASTQS_TRIMLOG_PE1 (4)                          [100%] 4 of 4, cached: 1 ✔                                                                        [7e/b3bbae] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:adapter_trimming:PUBLISH_FASTQS_TRIMLOG_PE2 (4)                          [100%] 4 of 4, cached: 1 ✔                                                                        [66/e06b36] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:BWA_MAPPING_PE:BWA_MEM_PE (4)                                    [100%] 4 of 4, cached: 1 ✔                                                                        [c2/40a212] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:BWA_MAPPING_PE:PUBLISH_BAM (4)                                   [100%] 4 of 4, cached: 1 ✔                                                                        [b0/24e40b] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:BWA_MAPPING_PE:PUBLISH_BAM_INDEX (4)                             [100%] 4 of 4, cached: 1 ✔                                                                        [e6/7a5bf4] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:BWA_MAPPING_PE:MAPPING_SUMMARY (4)                               [100%] 4 of 4, cached: 1 ✔                                                                        [99/3a3557] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:BWA_MAPPING_PE:PUBLISH_MAPPING_SUMMARY (4)                       [100%] 4 of 4, cached: 1 ✔                                                                        [-        ] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:PICARD__MERGE_SAM_FILES_AND_SORT                                 -                                                                                                 [0e/2213fe] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:BAM_TO_FRAGMENTS:BARCARD__CREATE_FRAGMENTS_FROM_BAM (4)          [100%] 4 of 4, cached: 1 ✔                                                                        [26/ba7d27] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:DETECT_BARCODE_MULTIPLETS:BARCARD__DETECT_BARCODE_MULTIPLETS (4) [100%] 4 of 4, cached: 1 ✔                                                                        [4b/4b05bd] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:DETECT_BARCODE_MULTIPLETS:GENERATE_REPORT (4)                    [100%] 4 of 4, cached: 1 ✔                                                                        [5d/9aa30c] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:DETECT_BARCODE_MULTIPLETS:REPORT_TO_HTML (4)                     [100%] 4 of 4, cached: 1 ✔                                                                        [4f/0355ac] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:PUBLISH_FRAGMENTS (4)                                            [100%] 4 of 4, cached: 1 ✔                                                                        [78/2a227d] process > atac_preprocess_rapid:ATAC_PREPROCESS_RAPID:mapping:PUBLISH_FRAGMENTS_INDEX (4)                                      [100%] 4 of 4, cached: 1 ✔                                                                        Completed at: 19-Apr-2023 14:12:58                                                                                                                                                                                                           Duration    : 2h 45m 24s                                                                                                                                                                                                                     CPU hours   : 59.7 (21.3% cached)                                                                                                                                                                                                            Succeeded   : 51                                                                                                                                                                                                                             Cached      : 17                                                                                          \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cb8c089-1e4d-49f1-946a-1a78af811f84",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}