airflow sqlite3.operationalerror database is locked then scheduler crashes

Question

I'm running airflow on a linux server and it runs fine for a couple hours than the airflow schedular fails.

This is the output of the log:

OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT connection.conn_id AS connection_conn_id \nFROM connection GROUP BY connection.conn_id'] (Background on this error at: http://sqlalche.me/e/e3q8)
[2019-08-14 11:20:20 +0000] [14173] [INFO] Worker exiting (pid: 14173)
[2019-08-14 11:20:20 +0000] [14011] [INFO] Worker exiting (pid: 14011)
[2019-08-14 11:20:20 +0000] [14068] [INFO] Worker exiting (pid: 14068)
[2019-08-14 11:20:20 +0000] [14150] [INFO] Worker exiting (pid: 14150)
[2019-08-14 11:20:20 +0000] [13955] [INFO] Worker exiting (pid: 13955)
[2019-08-14 11:20:20 +0000] [32013] [INFO] Shutting down: Master
[2019-08-14 11:20:20 +0000] [32013] [INFO] Reason: Worker failed to boot.
[2019-08-14 11:20:21,813] {cli.py:808} ERROR - [0 / 0] some workers seem to have died and gunicorndid not restart them as expected
[2019-08-14 11:22:31,885] {cli.py:815} ERROR - No response from gunicorn master within 120 seconds
[2019-08-14 11:22:31,886] {cli.py:816} ERROR - Shutting down webserver

How can I stop this from happening?

Answer 1

That error can happen when you are making requests to sqlite while it is still processing another. It happened to me and i just waited a little to make another request.