Add celltyping to external instructions

examples/README.md

@@ -3,6 +3,7 @@
 ## Example files
 
 This directory contains an example [metadata file](../external-data-instructions.md#prepare-the-metadata-file) and [configuration file](../external-data-instructions.md#configuration-files) for the `scpca-nf` workflow.
+There is also an example [cell type annotation metadata file](../external-data-instructions.md#performing-cell-type-annotation).
 These files should be used as an example of formats and content, but note that the values in these files may not be applicable or sufficient to allow running `scpca-nf` to be used directly on your system.
 
 ## Testing your setup with example data

examples/example_project_celltype_metadata.tsv

@@ -0,0 +1,4 @@
+scpca_project_id	singler_ref_name	singler_ref_file	cellassign_ref_name	cellassign_ref_file
+project01	BlueprintEncodeData	BlueprintEncodeData_model.rds	blood	PanglaoDB-blood.tsv
+project02	HumanPrimaryCellAtlasData	HumanPrimaryCellAtlasData_model.rds	NA	NA
+project03	NA	NA	blood	PanglaoDB-blood.tsv

external-instructions.md

@@ -19,6 +19,10 @@
   - [Libraries with additional feature data (ADT or cellhash)](#libraries-with-additional-feature-data-adt-or-cellhash)
   - [Multiplexed (cellhash) libraries](#multiplexed-cellhash-libraries)
   - [Spatial transcriptomics libraries](#spatial-transcriptomics-libraries)
+- [Cell type annotation](#cell-type-annotation)
+  - [Providing existing cell type labels](#providing-existing-cell-type-labels)
+  - [Performing cell type annotation](#performing-cell-type-annotation)
+    - [Repeating cell type annotation](#repeating-cell-type-annotation)
 - [Output files](#output-files)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -126,16 +130,18 @@ To run the workflow, you will need to create a tab separated values (TSV) metada
 | `assay_ontology_term_id` | [Experimental Factor Ontology](https://www.ebi.ac.uk/ols/ontologies/efo) term id associated with the `tech_version` |
 | `seq_unit`        | Sequencing unit (one of: `cell`, `nucleus`, `bulk`, or `spot`)|
 | `sample_reference`| The name of the reference to use for mapping, available references include: `Homo_sapiens.GRCh38.104` and `Mus_musculus.GRCm39.104` |
-| `files_directory` | path/uri to directory containing fastq files (unique per run) |
+| `files_directory` | The full path/uri to directory containing fastq files (unique per run) |
 
-The following columns may be necessary for running other data modalities (CITE-seq, spatial trancriptomics) or are optional and can be included in the metadata file if desired:
+The following columns may be necessary for running other data modalities (CITE-seq, spatial trancriptomics) or including existing cell type labels.
 
 | column_id       | contents                                                       |
 |-----------------|----------------------------------------------------------------|
-| `feature_barcode_file` | path/uri to file containing the feature barcode sequences (only required for ADT and cellhash samples); for samples with ADT tags, this file can optionally indicate whether antibodies are targets or controls.  |
+| `feature_barcode_file` | The full path/uri to TSV file containing the feature barcode sequences (only required for ADT and cellhash samples); for samples with ADT tags, this file can optionally indicate whether antibodies are targets or controls |
 | `feature_barcode_geom` | A salmon `--read-geometry` layout string. <br> See https://github.com/COMBINE-lab/salmon/releases/tag/v1.4.0 for details (only required for ADT and cellhash samples) |
 | `slide_section`   | The slide section for spatial transcriptomics samples (only required for spatial transcriptomics) |
 | `slide_serial_number`| The slide serial number for spatial transcriptomics samples (only required for spatial transcriptomics)   |
+| `submitter_cell_types_file` | The full path/uri to TSV file containing cell labels if you have cell type annotations results to include. See [instructions below](#providing-existing-cell-type-labels) for more information about preparing this file |
+
 
 We have provided an example run metadata file for reference.
 
@@ -408,7 +414,7 @@ This file will contain one row for each library-sample pair (i.e. a library cont
 
 | column_id       | contents                                                       |
 |-----------------|----------------------------------------------------------------|
-| `scpca_library_id`| Multiplexed library ID matching values in the metadata file. |
+| `scpca_library_id`| Multiplexed library ID matching values in the metadata file |
 | `scpca_sample_id` | Sample ID for a sample contained in the listed multiplexed library |
 | `barcode_id`      | The barcode ID used for the sample within the library, as defined in `feature_barcode_file` |
 
@@ -428,7 +434,70 @@ As an example, the Dockerfile that we used to build Space Ranger can be found [h
 After building the docker image, you will need to push it to a [private docker registry](https://www.docker.com/blog/how-to-use-your-own-registry/) and set `params.SPACERANGER_CONTAINER` to the registry location and image id in the `user_template.config` file.
 *Note: The workflow is currently set up to work only with spatial transcriptomic libraries produced from the [Visium Spatial Gene Expression protocol](https://www.10xgenomics.com/products/spatial-gene-expression) and has not been tested using output from other spatial transcriptomics methods.*
 
+## Cell type annotation
+
+### Providing existing cell type labels
+
+If you have already performed cell type annotation and wish to include these labels in the final workflow results, you can include the column `submitter_cell_types_file` in your run metadata file ([see example here](examples/example_run_metadata.tsv)).
+This column should be filled with the path or uri to a TSV file with existing cell type labels.
+
+This file _must_ include the following columns:
+
+| column_id       | contents                                                       |
+|-----------------|----------------------------------------------------------------|
+| `scpca_library_id`| Multiplexed library ID matching values in the metadata file |
+| `cell_barcode`   | The cell id with the given annotation label |
+| `cell_type_assignment` |  The annotation label for that cell |
+
+Optionally, you can also include a column `cell_type_ontology` with ontology labels corresponding to the given annotation label.
+
+### Performing cell type annotation
+
+`scpca-nf` can perform cell type annotation using two complementary methods: the reference-based method [`SingleR`](https://bioconductor.org/packages/release/bioc/html/SingleR.html) and the marker-gene based method [`CellAssign`](https://github.com/Irrationone/cellassign).
+
+You can turn on cell type annotation by using the `--perform_celltyping` flag.
+You will also need to provide an additional workflow parameter `celltype_project_metafile` containing the path/uri to a TSV file with information about which references to use for cell typing, at a project level, which can be specified at the command line (shown below) or defined in your configuration file.
+
+For example, you would run from the command line as:
+
+```sh
+nextflow run AlexsLemonade/scpca-nf \
+  --perform_celltyping \
+  --celltype_project_metafile = examples/example_project_celltype_metadata.tsv
+```
+
+References for use in cell type annotation have been pre-compiled as follows:
+
++ `SingleR` annotation uses references from the [`celldex` package](https://bioconductor.org/packages/release/data/experiment/html/celldex.html).
+Available reference options include `BlueprintEncodeData`, `DatabaseImmuneCellExpressionData`, `HumanPrimaryCellAtlasData`, and `MonacoImmuneData`.
+  + Please consult the [`celldex` documentation](https://bioconductor.org/packages/release/data/experiment/vignettes/celldex/inst/doc/userguide.html) to determine which of these references, if any, is most suitable for your dataset.
++ `CellAssign` annotation uses marker gene set references from [PanglaoDB](https://panglaodb.se/).
+Available organ-based references include `blood`, `brain`, and `muscle`.
+  + TODO do we want to say this? Please reach out to the Data Lab if you require a different set of marker genes for cell type annotation besides those in organs listed.
+
+This file should contain these five columns with the following information (see the example file in [`examples/example_project_celltype_metadata.tsv`](examples/example_project_celltype_metadata.tsv)):
+
+| column_id       | contents                                                       |
+|-----------------|----------------------------------------------------------------|
+| `scpca_project_id`| Project ID matching values in the metadata file |
+| `singler_ref_name` | Reference name for `SingleR` annotation. Must be one of `BlueprintEncodeData`, `DatabaseImmuneCellExpressionData`, `HumanPrimaryCellAtlasData`, or `MonacoImmuneData`. Use `NA` to skip `SingleR` annotation |
+| `singler_ref_file` | Path to internal `SingleR` reference file. Must be formatted as `<singler_ref_name>_model.rds`, e.g. `BlueprintEncodeData_model.rds`. Use `NA` to skip `SingleR` annotation |
+| `cellassign_ref_name` | Reference name for `CellAssign` annotation. Must be one of `blood`, `brain`, or `muscle`. Use `NA` to skip `CellAssign` annotation |
+| `cellassign_ref_file` | Path to internal `CellAssign` reference file. Must be formatted as `PanglaoDB-<cellassign_ref_name>`, e.g. `PanglaoDB-blood.tsv`. Use `NA` to skip `CellAssign` annotation |
+
+#### Repeating cell type annotation
 
+When cell typing is turned on with `--perform_celltyping`, `scpca-nf` will, by default, skip cell type annotation for any libraries whose cell type annotation results exist in the `checkpoints` folder of the output directory.
+If the cell type annotation reference versions are unchanged, this will save substantial processing time and cost.
+However, you may wish to repeat the cell typing process if there have been other changes to the data.
+
+To force repeating the cell type annotation process, use the `--repeat_celltyping` flag along with the `--perform_celltyping` flag at the command line:
+
+```sh
+nextflow run AlexsLemonade/scpca-nf \
+  --perform_celltyping \
+  --repeat_celltyping \
+```
 
 ## Output files
 
@@ -468,6 +537,8 @@ results
 
 If bulk libraries were processed, a `bulk_quant.tsv` and `bulk_metadata.tsv` summarizing the counts data and metadata across all libraries will also be present in the `results` directory.
 
+If you performed cell type annotation, an additional QC report specific to cell typing results called `library_id_celltype-report.html` will also be present in the `results` directory.
+
 The `checkpoints` folder will contain intermediate files that are produced by individual steps of the workflow, including mapping with `salmon`.
 The contents of this folder are used to allow restarting the workflow from internal checkpoints (in particular so the initial read mapping does not need to be repeated, see [repeating mapping steps](#repeating-mapping-steps)), and may contain log files and other outputs useful for troubleshooting or alternative analysis.
 
@@ -502,5 +573,7 @@ nextflow run AlexsLemonade/scpca-nf \
   --publish_fry_outs
 ```
 
-If genetic demultiplexing was performed, there will also be a folder called `vireo` with the output from running [vireo](https://vireosnp.readthedocs.io/en/latest/index.html) using genotypes identified from the bulk RNA-seq.
+If genetic demultiplexing was performed, there will also be a checkpoints folder called `vireo` with the output from running [vireo](https://vireosnp.readthedocs.io/en/latest/index.html) using genotypes identified from the bulk RNA-seq.
 Note that we do not output the genotype calls themselves for each sample or cell, as these may contain identifying information.
+
+If cell type annotation was performed, there will also be a checkpoints folder called `celltype` with the output from running `SingleR` and `CellAssign`.