[英]How to schedule multiple timings in single Airflow DAG
I understand that Airflow DAG runs has presets like @hourly, @daily, etc. However, I have interesting use case that I would like to perform multiple tasks in a single DAG with different schedules .我知道 Airflow DAG 运行具有@hourly、@daily 等预设。但是,我有一个有趣的用例,我想在具有不同计划的单个 DAG 中执行多个任务。
For example,例如,
.. And so on. .. 等等。 Is this possible?
这可能吗? These are very simple tasks (such as an API call) with no need to process results so I would like to avoid creating multiple DAGs for each of these tasks.
这些是非常简单的任务(例如 API 调用),不需要处理结果,因此我想避免为每个任务创建多个 DAG。
You could solve this by using a PythonOperator and the croniter
package.您可以通过使用 PythonOperator 和
croniter
包来解决这个问题。
The DAG could run ie @daily and then the PythonOperator calls a functions that checks first if a certain task-specific cron expression matches the execution_date and then executes the code. DAG 可以运行 ie @daily,然后 PythonOperator 调用一个函数,该函数首先检查某个特定任务的 cron 表达式是否与 execution_date 匹配,然后执行代码。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.