Working to the next release (#30)

This is a major refactor of the FIRE pipeline and the naming of its outputs and will represent the v0.1.0 of the software.

There are no changes to the underlying methods or ideas, but I have made significant efforts to speed up the software and reduce dependencies while increasing reliability.

.github/workflows/main.yml

@@ -1,54 +1,19 @@
-name: Tests
+name: CI
 
 on:
   push:
-    branches: [ main ]
+    branches: [main]
   pull_request:
-    branches: [ main ]
-
+    branches: [main]
 
 jobs:
-  Formatting:
+  # pixi test
+  Test:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
-      - name: Formatting
-        uses: github/super-linter@v4
-        env:
-          VALIDATE_ALL_CODEBASE: false
-          DEFAULT_BRANCH: main
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          VALIDATE_SNAKEMAKE_SNAKEFMT: true
-
-  Linting:
-    runs-on: ubuntu-latest
-    steps:
-    - uses: actions/checkout@v2
-    - name: Lint workflow
-      uses: snakemake/[email protected]
-      with:
-        directory: .
-        snakefile: workflow/Snakefile
-        args: "--lint"
-
-  Testing:
-    runs-on: ubuntu-latest
-    needs: 
-      - Linting
-      - Formatting
-    steps:
-    - uses: actions/checkout@v2
-
-    - name: Test workflow
-      uses: snakemake/[email protected]
-      with:
-        directory: .test
-        snakefile: workflow/Snakefile
-        args: "--use-conda --show-failed-logs --cores 3 --conda-cleanup-pkgs cache --all-temp --configfile config.yml"
-
-#    - name: Test report
-#      uses: snakemake/[email protected]
-#      with:
-#        directory: .test
-#        snakefile: workflow/Snakefile
-#        args: "--report report.zip"
+      - uses: actions/checkout@v4
+      - uses: prefix-dev/[email protected]
+        with:
+          pixi-version: v0.37.0
+          cache: true
+      - run: pixi run test

.gitignore

@@ -142,7 +142,7 @@ figures/
 old/
 phasing/
 run_scripts/
-trackHub/
+trackHub-{v}/
 trackHub_altius/
 XCI/
 altius/
@@ -158,7 +158,6 @@ config/**
 .vscode/*
 .DS_Store
 
-
 # stats
 Rdata/
 Rplots.pdf
@@ -177,4 +176,6 @@ misc/
 new-agg-test/
 scATAC/
 test/
-
+test-data/
+bigtools-test/
+fire-test-data/

CHANGELOG.md

@@ -0,0 +1,3 @@
+# v0.1.0
+
+First major release of the FIRE pipeline. This release includes a refactor to reduce the computation by increased use of ft, changes to the output file names to include the fire version among other things, and finally a new launching method for the pipeline that uses pixi. Results are very similar to v0.0.7 of the pipeline; however, there are minor differences in the peak calls and the output names.

INSTALL.md

@@ -1,18 +1,17 @@
 # Install
 
-You will need **snakemake** which you can install using conda/mamba, e.g.:
-```
-mamba create -c conda-forge -c bioconda -n snakemake 'snakemake>=8.4'
-```
+If you wish to distribute jobs across a cluster you will need to install the appropriate [snakemake executor plugin](https://snakemake.github.io/snakemake-plugin-catalog/). For example, to use SLURM you can install the `snakemake-executor-slurm` plugin using pip:
 
-Finally, if you wish to distribute jobs across a cluster you will need to install the appropriate [snakemake executor plugin](https://snakemake.github.io/snakemake-plugin-catalog/). For example, to use SLURM you can install the `snakemake-executor-slurm` plugin using pip:
-```  
+```
+pixi shell
 pip install snakemake-executor-plugin-slurm
 ```
 
 We recommend adding a snakemake conda prefix to your `bashrc`, e.g. in the Stergachis lab add:
+
 ```bash
 export SNAKEMAKE_CONDA_PREFIX=/mmfs1/gscratch/stergachislab/snakemake-conda-envs
 export APPTAINER_CACHEDIR=/mmfs1/gscratch/stergachislab/snakemake-conda-envs/apptainer-cache
 ```
+
 Then snakemake installs all the additional requirements as conda envs in that directory.

README.md

@@ -1,47 +1,10 @@
 # 🔥 **FIRE**: <ins>F</ins>iber-seq <ins>I</ins>nferred <ins>R</ins>egulatory <ins>E</ins>lements
 
 [![DOI](https://zenodo.org/badge/561430995.svg)](https://zenodo.org/doi/10.5281/zenodo.10023811)
+[![Actions Status](https://github.com/fiberseq/FIRE/workflows/CI/badge.svg)](https://github.com/mrvollger/FIRE/actions)
 
-A Snakemake workflow for calling Fiber-seq Inferred Regulatory Elements (FIREs) on single molecules. For a more detailed description and methods see the [docs](/docs/README.md), or [watch](https://youtu.be/RiZrMltAiWM?si=sSo64goaNQxgyfcc) my lab meeting on FIRE.
+A Snakemake workflow for calling Fiber-seq Inferred Regulatory Elements (FIREs) on single molecules.
 
-## Install
+## Installation and Usage
 
-Please install `snakemake` and all the UCSC Kent utilities. For detailed instructions see the [installation README](/INSTALL.md).
-
-## Configuring
-
-See the [configuration README](/config/README.md), the example [configuration file](/config/config.yaml), and the example [manifest file](/config/config.tbl) for configuration options.
-
-
-## Run
-
-We have a run script that executes the FIRE snakemake called `fire`, and any extra parameters are passed directly to snakemake. For example:
-
-```bash
-./fire --configfile config/config.yaml
-```
-
-If you want to do a dry run:
-
-```bash
-./fire --configfile config/config.yaml -n
-```
-
-If you want to execute across a cluster (modify `profiles/slurm-executor` as needed for distributed execution):
-
-```bash
-./fire --configfile config/config.yaml --profile profiles/slurm-executor
-```
-
-You can also run snakemake directly, e.g.:
-
-```bash
-snakemake \
-  --configfile config/config.yaml \
-  --profile profiles/slurm-executor \
-  --local-cores 8 -k
-```
-
-## Test data
-
-You can find input data to test against at [this url](https://s3-us-west-2.amazonaws.com/stergachis-public1/index.html?prefix=FIRE/test-data/).
+See the [docs](https://fiberseq.github.io/fire/run.html) for detailed installation and usage instructions.

config/README.md

@@ -17,6 +17,11 @@ manifest: config/config.tbl
 
 
 ## Optional input options
+Specify that the input BAM file is an ONT Fiber-seq file. Default is `False`.
+```
+ont: True
+```
+
 Max number of threads to use in very distributed steps:
 ```
 max_t: 4