When running for the first time
python -m virtualenv myvenv
Once a virtual environment is created, some libraries may need to be downloaded before running.
First, to setup spark and the virtual environment:
source setup.sh
In order for analysis to be completed on a specific run, dcs data must be offloaded. The arguments required for this are a current run and the previous run. The dcs data from the current run will be offloaded, but some data is needed from the previous run.
To offload dcs data for a single run:
python ./analysis/src/spark/runOffload.py -r 451295 -p 451140
To offload a group of runs, specify a year and a starting run:
python ./analysis/src/spark/runOffload_all.py -y 2022 -r 430702
To generate the plots and figures needed for the webpage, the dcs offloaded data, a current run, and the previous run are needed. To generate the graphics from a specific run:
python ./analysis/src/python_analysis/runAnalysis.py -r 451295 -p 451140
To generate graphics a group of runs, specify a year and a starting run:
python ./analysis/src/python_analysis/runAnalysis_all.py -y 2022 -r 430702
To generate the web server with Flask:
python ./web/app.py