简体   繁体   English

Airflow 调度器启动任务失败

[英]Airflow scheduler fails to start tasks

My problem:我的问题:

Airflow scheduler is not assigning tasks. Airflow 调度程序未分配任务。

Background:背景:

I have Airflow running successfully on my local machine with sqlitedb.我的 Airflow 使用 sqlitedb 在我的本地机器上成功运行。 The sample dags as well as my custom DAGs ran without any issues.示例 dag 以及我的自定义 DAG 运行没有任何问题。 When I try to migrate from sqlite database to Postgres (using this guide), the scheduler no longer seems to be assigning tasks.当我尝试从 sqlite 数据库迁移到 Postgres(使用指南)时,调度程序似乎不再分配任务。 The DAG get stuck on "running" state but no task in any DAGs ever gets assigned a state. DAG 卡在“正在运行”的 state 上,但任何 DAG 中的任务都没有被分配 state。

Troubleshooting steps I've taken我已采取的故障排除步骤

  • The web server and the scheduler are running web 服务器和调度程序正在运行
  • The DAG is set to "ON". DAG 设置为“开”。
  • After running airflow initdb, the public schema is populated with all of the airflow tables.运行 airflow initdb 后,公共模式将填充所有 airflow 表。
  • The user in my connection string owns the database as well as every table in the public schema.我的连接字符串中的用户拥有数据库以及公共模式中的每个表。

Scheduler Log调度程序日志

The scheduler log keeps posting out this WARNING but I have not been able to use it to find any useful information aside form this other post with no responses.调度程序日志不断发布此警告,但我无法使用它来找到除此其他帖子之外的任何有用信息而没有回复。

[2020-04-08 09:39:17,907] {dag_processing.py:556} INFO - Launched DagFileProcessorManager with pid: 44144
[2020-04-08 09:39:17,916] {settings.py:54} INFO - Configured default timezone <Timezone [UTC]>
[2020-04-08 09:39:17,927] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=44144
[2020-04-08 09:39:19,914] {dag_processing.py:663} WARNING - DagFileProcessorManager (PID=44144) exited with exit code -11 - re-launching

Environment环境

  • PostgreSQL version 12.1 PostgreSQL 12.1版
  • Airflow v1.10.9 Airflow v1.10.9
  • This is all running on a MacBook Pro (Catalina) in a conda virtual environment.这一切都在 conda 虚拟环境中的 MacBook Pro (Catalina) 上运行。

Postgres was installed using postgresapp . Postgres 是使用postgresapp安装的。 Updated postgresapp to version 2.3.3e.将 postgresapp 更新到 2.3.3e 版本。 PostgresSQL is still version 12.1 but by updating the postgresapp, the issue was solved. PostgresSQL 仍然是 12.1 版,但通过更新 postgresapp,问题得到了解决。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM