Celery - execute task on all nodes

Question

I have a special use case where I need to run a task on all workers to check if a specific process is running on the celery worker. The problem is that I need to run this on all my workers as each worker represents a replica of this specific process.

In the end I want to display 8/20 workers are ready to process further tasks. But currently I'm only able to process a task on either a random selected worker or just on one specific worker which does not solve my problem at all...

Thanks in advance

Answer 1

I can't think of a good way to do this on Celery. However, a nice workaround perhaps could be to implement your own command, and then you can broadcast that command to every worker (just like you can broadcast shutdown or status commands for an example). When I think of it, this does indeed sound like some sort of monitoring/maintenance operation, right?