简体   繁体   English

如何在单个 Airflow DAG 中安排多个时间

[英]How to schedule multiple timings in single Airflow DAG

I understand that Airflow DAG runs has presets like @hourly, @daily, etc. However, I have interesting use case that I would like to perform multiple tasks in a single DAG with different schedules .我知道 Airflow DAG 运行具有@hourly、@daily 等预设。但是,我有一个有趣的用例,我想在具有不同计划的单个 DAG 中执行多个任务

For example,例如,

  • Task 1 - Every first of the month任务 1 - 每个月的第一天
  • Task 2 - Every second of the month at 12pm任务 2 - 每月 12 点的每一秒

.. And so on. .. 等等。 Is this possible?这可能吗? These are very simple tasks (such as an API call) with no need to process results so I would like to avoid creating multiple DAGs for each of these tasks.这些是非常简单的任务(例如 API 调用),不需要处理结果,因此我想避免为每个任务创建多个 DAG。

You could solve this by using a PythonOperator and the croniter package.您可以通过使用 PythonOperator 和croniter包来解决这个问题。

The DAG could run ie @daily and then the PythonOperator calls a functions that checks first if a certain task-specific cron expression matches the execution_date and then executes the code. DAG 可以运行 ie @daily,然后 PythonOperator 调用一个函数,该函数首先检查某个特定任务的 cron 表达式是否与 execution_date 匹配,然后执行代码。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM