How to run tests in parallel in Django?

Question

In my Django projects I use sqlite database to run tests. Since it uses only memory, it's much faster than MySQL, but it's still not fast enough. During tests, only one of 4 processors is used, and not much memory is consumed. So, I'd like to have 4 sqlite databases in memory to run 4 tests in parallel.

Has anyone tried this?

Answer 1

Since Django 1.9 it's possible to run the tests in parallel by Django with its built-in unit-test features.

Django Docs: https://docs.djangoproject.com/en/3.0/ref/django-admin/#cmdoption-test-parallel

Answer 2

You can use a parallel test runner for Django and Twisted described here: http://www.tomaz.me/2011/04/03/making-django-and-twisted-tests-faster.html (the source lives herehttps://github.com/Kami/parallel-django-and-twisted-test-runner - link at the end of the post). You can use it as described in Django docs on testing.

There is also a nose parallel test runner.

Answer 3

According to the Django 3.0 documentation there is a --parallel option that you can use.

--parallel [N] Runs tests in separate parallel processes. Since modern processors have multiple cores, this allows running tests significantly faster.

So you can use the following command to execute the tests in parallel.

python manage.py test --parallel

You can adjust the number of processes either by providing it as the option's value, eg --parallel=4, or by setting the DJANGO_TEST_PROCESSES environment variable.

This can help considerably reduce your test execution time if you have a large Django project with quite a few test unit cases.

Answer 4

You can easily split the testing for apps on parrallalel on linux by:

$ python manage.py test cms & \
python manage.py test blog & \
python manage.py test store & \
python manage.py test registration
$ 

Could be helpfull for big projects with a lot of apps, the best would be a bash scripts that runs tests every four apps.