[英]How to trigger task in airflow when pre-task failed and depends_on_past = true?
I have a task needs to be triggered every 5 minutes. 我有一个任务需要每5分钟触发一次。 And the next task needs to wait for the previous one to be done (no matter success or failure on the previous task).
下一个任务需要等待上一个任务完成(无论上一个任务是成功还是失败)。
Here's how I configure the parameters. 这是我配置参数的方式。
default_args = {
'owner': 'airflow',
'start_date': datetime(2019, 7, 1, 8, 30),
'max_active_runs': 1,
'depends_on_past': True,
'execution_timeout': timedelta(seconds=300)
}
dag = DAG(
dag_id='dag1', default_args=default_args,
schedule_interval='*/5 8-16 * * *',
dagrun_timeout=timedelta(minutes=600))
def task1(ds, **kwargs):
#do something
task1 = PythonOperator(
task_id='task1',
provide_context=True,
python_callable=task1,
trigger_rule= 'all_done',
dag=dag)
Under my configuration, task1 will be triggered while the previous state is a success, and be blocked while the previous state is failed. 在我的配置下,task1将在先前状态成功时被触发,而在先前状态失败时被阻塞。 I found the description in airflow's doc, "trigger_rule can be used in conjunction with depends_on_past (boolean) that, when set to True, keeps a task from getting triggered if the previous schedule for the task hasn't succeeded".
我在airflow的文档中找到了描述,“ trigger_rule可以与depends_on_past(布尔值)结合使用,当设置为True时,如果先前的任务计划未成功,则不会触发任务”。
So how can I achieve the purpose of triggering task1 while the previous task1 is done and no matter what the pre-state is? 那么,无论前状态如何,在完成前一个task1时如何达到触发task1的目的?
max_active_runs: 1 will only allow one execution of the dag at once, covering your waiting for previous requirement. max_active_runs:1将一次只允许执行一次dag,涵盖了您等待之前的要求。
You have 2 depends_on_past settings. 您有2个depends_on_past设置。 Remove one and have it set to false.
删除一个并将其设置为false。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.