Airflow - TriggerDagRunOperator - 使用不同的 Conf 调用相同的 DAG

Question

I've one dynamic DAG (dag_1) that is orchestrated by another DAG (dag_0) using TriggerDagRunOperator.

The dag_1 is a very simple script: `from datetime import datetime from airflow.models import DAG from airflow.operators.python_operator import PythonOperator

default_args = { 'provide_context': True, }

def get_list(**context): p_list = ['a','b','c'] res = context["dag_run"].conf['message'] p_list.append(res) return p_list

with DAG( dag_id='dag_1', schedule_interval='@once', start_date=datetime(2022, 3, 1) , default_args=default_args ) as dag: data_list = PythonOperator( task_id='get_lists', python_callable=get_list, dag=dag) for i in get_list(): bash_task = BashOperator(task_id='bash_task_{}'.format(i), bash_command="echo 'command executed {}'.format(i)")

And my dag_0 that will run orchestrate the dag_1 based on the parameters passed: `from airflow.operators.trigger_dagrun import TriggerDagRunOperator from datetime import datetime from airflow import DAG

dag = DAG( 'dag_0', description='create the table structure', schedule_interval='@once', catchup=False, max_active_runs=1, start_date=datetime(2022, 3, 1) )

trigger_step = TriggerDagRunOperator( task_id="trigger_step", trigger_dag_id="dag_1", conf={"message": "Hello Word 1"}, dag=dag )

trigger_step2 = TriggerDagRunOperator( task_id="trigger_step2", trigger_dag_id="dag_1", conf={"message": "Hello Word 2"}, dag=dag )

trigger_step >> trigger_step2`

If I run this without the need to pass the PythonOperator to a list, it works fine by when I put that part "for i in get_list()" on first script I got: res = context["dag_run"].conf['message'] KeyError: 'dag_run'

What I am doing wrong?

Answer 1

Try the below. I think because that dag_run object is created at run time.

def get_list(dag_run=None,**context): 
    p_list = ['a','b','c'] 
    res = dag_run.conf.get('message') 
    p_list.append(res) 
    return p_list