芹菜链任务用例？

Question

Suppose functions a, b, ..., that must be run sequentially. We can make a task that does that:

def a(...):
    ...
def b(...):
    ...
...
@app.task
def abc(...):
    a(...)
    b(...)
    ...

Alternatively, we can make them separate tasks:

@app.task
def a(...):
    ...
@app.task
def b(...):
    ...
...

and chain them:

@app.task
def abc(...):
    chain(a.s(...), b.s(...), ...)
    ...

The first version is faster that the second (maybe not by much, depends on the task overhead vs. the average time spent in a, b, ...). I suppose there must be a reason for chain but I haven't figured it out.

Answer 1

There are several use cases:

1. You just want your tasks to be composable by the celery client rather than predefining workflows in the celery code base (ie, you can use chain on the client to submit tasks!)
2. You want to monitor each subtask independently. Because chain launches each subtask as a separate task, each one can be monitored independently of the other. If you have one function that calls the other two as python functions, you can't hook into celery monitoring.
3. You want to specialize workers/servers so that task A of the chain runs on one type of server and task B of the chain runs on another (eg for security).

The list can go on, but I've definitely used all three in production environments. Good luck!