SQLITE3 database getting locked Django and Celery

Question

We have aa legacy django app (django 1.4.2) which is being run to publish some reports. It uses celery and sqlite3. Because of multiple concurrent connections, sqlite database is getting locked.

The applications was written almost four years back, and is working very well except for occasional database locking which is becoming a pain. Upgrading to a newer version of Django assuming that setting CONN_MAX_AGE (introduced in django 1.6) will work fine is being overruled because the application is fairly large and may have some dependency issues.

Note this is not web front ending application has less than 30 internal users. So security is not a big concern.

Question is how do we ensure that database is not getting locked. Is there a way to implement CONN_MAX_AGE for an older version of django.

Answer 1

You should change to use a database that has better support for multiple connections such as postgres or mysql, sqlite really isn't built up to do it and is more used for development where its very rare you'd have multiple concurrent connections

You can find more about the locked error on the django documentation - “Database is locked” errors

If you're getting this error, you can solve it by:

• Switching to another database backend. At a certain point SQLite becomes too “lite” for real-world applications, and these sorts of concurrency errors indicate you've reached that point.

• Rewriting your code to reduce concurrency and ensure that database transactions are short-lived.

• Increase the default timeout value by setting the timeout database option option:

---

_{Just noticed that this only applies to >= django 1.8}

In addition to this, if you went with postgres, you'd find yourself able to use more django queryset methods, model fields , and more