Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question with C8306 For consistency match the DAG filename with the dag_id #2

Open
louisguitton opened this issue May 25, 2019 · 1 comment

Comments

@louisguitton
Copy link

I personally version my DAGs (when we add tasks or change start_date) using only the dag_id, but leaving the filename.

For example

# dags/mailed_report.py

mailed_report = DAG(
    dag_id='mailed_report_v2',
    default_args=default_args,
    catchup=True,
    template_searchpath=['/usr/local/airflow/sql'],
    schedule_interval='@daily'
)

This triggers C8306.
I'm curious what your thoughts are: should the filename be versioned as well? or could the match in https://github.com/BasPH/pylint-airflow/blob/master/src/pylint_airflow/checkers/dag.py#L121 be more flexible?

Here is a quick and dirty way to show you what I mean

dagid_without_version =  f"{dagid}".split("_v")[0]
expected_filename = f"{dagid_without_version}.py" 
@BasPH
Copy link
Owner

BasPH commented May 26, 2019

Thanks for pointing this out. I have seen other people version their DAGs too, it would indeed make sense to allow the DAG to start with the filename and have a postfix. Perhaps by checking if the dag_id is contained in the filename or vice versa, to allow for versions/other postfixes in either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants