Skip to content
Theo edited this page May 21, 2020 · 1 revision

An overview of each DAG

CreateConnections.py
Will create each Airflow connection referenced in the resources/connections.json file

CreateUsersDag.py
Will create Airflow webserver users for each user referenced in the resources/defaultUsers.json file

ImportDags.py
Will git pull each project referenced in the resources/projects.csv file to /efs/ then creates a symbolic link for each project in /usr/local/airflow/dags/

InstallDependenciesProd.py
Will install any Python library referenced in the requirements.txt file at the root of each imported project. It will also install any non Python libraries referenced in the bash commands located in the DAG itself.

Clone this wiki locally