Merge branch 'main' into data-trigger

README.md

@@ -18,12 +18,13 @@ This repository contains the code to handle various aspects of this portal, each
 The analytics pipeline was built with a module architecture stored in series of docker images to enable the pipeline to be processed in individual stages. Currently there are five modules:
 1. dbSchema: This module validates files when they are input into system and when they are pushed to the database to ensure they align to expected schemas.
 2. zfBmd: This module processes the zebrafsih benchmark dose data to assess the endpoints described below.
-3. bmd2Samps: This module combines existing zebrafish data with new data from the `zfBmd` module and also integratees environmental sample information
-4. exposome: This module pulls exposome data and creates an outputed file
+3. zfExp: This module formats gene expression data
+4. bmd2Samps: This module combines existing zebrafish data with new data from the `zfBmd` module and also integratees environmental sample information
+5. exposome: This module pulls exposome data and creates an outputed file
 
 A depiction of the framework is shown below:
 
-<img src="schemaFig.png" width="500">
+<img src="schemaFig.jpg" width="500">
 
 ### To run pipeline
 

data/srp_build_files.csv

@@ -32,9 +32,10 @@ fit7,fit,extract,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/mai
 fit8,fit,extract,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/extracts/fit_vals_all_qc.csv,1
 gene1,expression,zebrafish,,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/zfExp/ZF_gex.xlsx,3
 geneInfo,mapping,zebrafish,,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/zfExp/allianceGenomeInfo.csv,3
-chemMap,mapping,chemical,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/chemicalIdMapping.csv,1
-sampMap,mapping,sample,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/sampleIdMapping.csv,1
+chemId,mapping,chemical,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/chemicalIdMapping.csv,1
+sampId,mapping,sample,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/sampleIdMapping.csv,1
 class1,classification,chemical,https://raw.githubusercontent.com/PNNL-CompBio/srpAnalytics/main/data/MASV%20Classifications%202021.xlsx,1
-compTox,mapping,chemical,https://github.com/PNNL-CompBio/srpAnalytics/blob/main/data/CCDBatch_2022-01-26_AND_2021_11_05.xlsx,1
-endpointMap,mapping,endpoint,https://github.com/PNNL-CompBio/srpAnalytics/blob/main/data/SuperEndpoint%20Mapping%202021NOV04.xlsx,1
-chemdesc,classification,chemical,https://github.com/PNNL-CompBio/srpAnalytics/blob/main/data/ChemicalDescriptions.xlsx,1
+compTox,mapping,chemical,https://github.com/PNNL-CompBio/srpAnalytics/raw/main/data/CCDBatch_2022-01-26_AND_2021_11_05.xlsx,1
+endpointMap,mapping,endpoint,https://github.com/PNNL-CompBio/srpAnalytics/raw/main/data/SuperEndpoint%20Mapping%202021NOV04.xlsx,1
+chemdesc,classification,chemical,https://github.com/PNNL-CompBio/srpAnalytics/raw/main/data/ChemicalDescriptions.xlsx,1
+sampMap,mapping,sample,https://github.com/PNNL-CompBio/srpAnalytics/raw/main/data/envSampCleanMapping.xlsx,1