To ease and speed up the process of developing DAGs, you can use py:meth:~airflow.models.dag.DAG.test, which will run a dag in a single process.
To set up the IDE:
- Add
main
block at the end of your DAG file to make it runnable.
if __name__ == "__main__":
dag.test()
- Run and debug the DAG file.
You can also run the dag in the same manner with the Airflow CLI command airflow dags test
:
# airflow dags test [dag_id] [execution_date]
airflow dags test example_branch_operator 2018-01-01
By default /files/dags
folder is mounted from your local <AIRFLOW_SOURCES>/files/dags
and this is
the directory used by airflow scheduler and webserver to scan dags for. You can place your dags there
to test them.
The DAGs can be run in the main version of Airflow but they also work with older versions.
For other kinds of tests look at Testing document