[英]Problem with BranchPythonOperator to TaskGroup
I got the following DAG我得到了以下 DAG
import logging
from airflow import DAG
from airflow.operators.dummy import DummyOperator
from airflow.operators.python import BranchPythonOperator
from airflow.utils.dates import days_ago
from airflow.utils.task_group import TaskGroup
def select_next_branch():
if some_condition:
next_task_ = 'tasks.inner'
else:
next_task_ = 'end'
logging.info(f'next_task: {next_task_}')
return next_task_
with DAG(dag_id='poc_branch_tasks',
description='Branching with task group POC',
schedule_interval=None,
start_date=days_ago(1),
tags=['poc', 'branch', 'task_group']) as dag:
start = DummyOperator(task_id='start')
branch = BranchPythonOperator(task_id='branch',
python_callable=select_next_branch)
with TaskGroup(group_id='tasks') as task_group:
inner_one = DummyOperator(task_id='inner')
end = DummyOperator(task_id='end')
start >> branch
branch >> end
branch >> task_group >> end
When some_condition
is satisfied the flow go correctly from branch
to task_group.inner
, otherwise it should be from branch
to end
, but instead of execute end
this is skipped.当some_condition
被满足时,流 go 正确地从branch
到task_group.inner
,否则它应该从branch
到end
,但不是执行end
,而是跳过。 What I'm doing wrong?我做错了什么?
Thanks in advance.提前致谢。
Try adding trigger_rule='one_success'
for end
task.尝试为end
任务添加trigger_rule='one_success'
。 The default trigger_rule
is all_success.默认的trigger_rule
是 all_success。
all_success (default): All upstream tasks have succeeded all_success(默认):所有上游任务都已成功
However, your end
task is dependent for both Branch operator and inner
task.但是,您的end
任务取决于 Branch 运算符和inner
任务。 When inner
task is skipped, end
cannot triggered because one of the upstream task is not "success".跳过inner
任务时,无法触发end
,因为上游任务之一不是“成功”。
The trigger rule one_success will try to execute this end
task if either of Branch operator or inner
task is succeeded.如果分支运算符或inner
任务成功,触发规则 one_success 将尝试执行此end
任务。
end = DummyOperator(task_id='end', trigger_rule='one_success')
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.