Celeryd launching too many processes

Question

How do you ensure celeryd only runs as a single process? When I run manage.py celeryd --concurrency=1 and then ps aux | grep celery ps aux | grep celery I see 3 instances running:

www-data  8609  0.0  0.0  20744  1572 ?        S    13:42   0:00 python manage.py celeryd --concurrency=1
www-data  8625  0.0  1.7 325916 71372 ?        S    13:42   0:01 python manage.py celeryd --concurrency=1
www-data  8768  0.0  1.5 401460 64024 ?        S    13:42   0:00 python manage.py celeryd --concurrency=1

I've noticed a similar problem with celerybeat, which always runs as 2 processes.

Answer 1

As per this link .. The number of processes would be 4: one main process, two child processes and one celerybeat process, also if you're using FORCE_EXECV there's another process started to cleanup semaphores.

If you use celery+django-celery development, and using RabbitMQ or Redis as a broker, then it shouldn't use more than one extra thread (none if CELERY_DISABLE_RATE_LIMITS is set)