Airflow DAGs are queued up

Question

I am working on a project where I can see all of the dags are queued up and not moving (appx over 24H or more) Looks like its scheduler is broken but I need to confirm that.

So here are my questions

1. How to see if scheduler is broken
2. How to reset my airflow (web server) scheduler?

Expecting some help regarding how to reset airflow schedulers

Answer 1

The answer will depend a lot on how you are running Airflow (standalone, in Docker, Astro CLI, managed solution...?).

• If your scheduler is broken the Airflow UI will usually tell you the time since the last heartbeat like this:

• There is also an API endpoint for a scheduler health check at http://localhost:8080/health (if Airflow is running locally).

• Check the scheduler logs. By default they are in a file at $AIRFLOW_HOME/logs/scheduler .

You might also want to look at how to do health checks in Airflow in general.

In terms of resetting it is usually best to restart the scheduler and again this will depend on how you started it in the first place. If you are using a standalone instance and have the processes in the foreground simply do ctr+c or close the terminal to stop it. If you are running airflow in docker restart the container, for the Astro CLI there is astro dev restart .