Python multiprocessing get the results

Question

I'm trying to understand the Python multiprocessing module. The below code creates 4 process and tries run the function f(x) .

Questions

1. I'm creating 4 processes but submitting multiple tasks (f,[10]). Will the module will automatically run every 4 process at any time?
2. I capture the result in a list. What will the pool.apply_async will return, Will it be print statements in the job or whatever returned by the method?

When I execute this code, it just displays 100 but not other values.

from multiprocessing import Pool

def f(x):

    print x*x
    return x*x


if __name__ == '__main__':

    result = []
    pool = Pool(processes=4)            
    result.append(pool.apply_async(f, [10]))
    out = map(lambda x: x.get(), result)   

Answer 1

The pool you create has 4 processes available to it. You then give it a single job to do and it farms it out to one of its 4 workers. If you give it more jobs to do it will use more workers.

Answer 2

You're actually only submitting one task here: The function f with the argument tuple (10,) , meaning you're just executing f(10) in a single worker. If you had actually submitted more than 4 processes, the Pool would always execute at most 4 tasks simultaneously. As soon as one task finishes, a pending task will be started to replace it, until there are no remaining tasks. Also note that the same four processes will be used the entire time; they are not restarted after they complete a task, unless you supply the maxtasksperchild keyword argument when you create the Pool .

apply_async will return an AsyncResult object, which can be used to fetch the result of the task, once it's available. You do that by calling get() on the AsyncResult object, so the out list will contain the return value of your one call to f . The stuff you print in the worker process will appear in stdout like it would if you ran it in the main process, but it doesn't get captured anywhere.