限制celery中同时特定任务的数量

Question

I have 2 tasks taskA and taskB which run using the GPU. They can both run on the GPU at the same time. But if celery decides to launch an additional instance, then I will have a crash (out of memory on the GPU).

I therefore would like to be able to have control over this (ie only allow taskA + taskB , or taskB + taskB , or taskA + taskA at the same time, but prevent for instance taskA + taskA + taskB ).

How can I control this?

Answer 1

You could limit the number of simultaneous tasks by using the --concurrency argument. Set this to 2 if you're using the same worker for both tasks or to 1 if you have two separate workers.