澄清Python的`multiprocessing.Pool`的`processes`参数

Question

My question is if I execute [pool.apply_async(myfunc, args=(y,)) for i in range(8)] like shown below, and I initialized the Pool with multiple processes, eg, here 4 ,
does it mean that every function call is running parallel on 4 processes, and I am running 8 function calls parallel, too, so 4x8 = 32 processes, or does it run 4-times 1 function call, waits until they finish and then runs another 4 function calls?

import multiprocessing
pool = multiprocessing.Pool(processes=4)
results = [pool.apply_async(myfunc, args=(i,)) for i in range(8)]
results = [res.get() for res in results]

Answer 1

A multiprocessing.Pool will never run more processes in parallel than the number you specified at creation time. Instead, it immediately spawns as many processes as you specified, and leaves them running until the pool is closed/joined. So in your case, the Pool will always be running exactly four processes, even if none of them are doing any work. If you give the pool eight work items, the first four will immediately begin executing in parallel, while the next four are queued. As soon as one of the worker processes finishes running myfunc , the first queued item will start being processed by the now idle worker process.

You can see this for yourself if you run this example:

def myfunc(num):
    print("in here %s" % num)
    time.sleep(2)
    print("done with %s" % num)
    return num+2

if __name__ == "__main__":
    pool = multiprocessing.Pool(4)
    results = [pool.apply_async(myfunc, args=(i,)) for i in range(8)]
    results = [res.get() for res in results]
    print results

Output:

in here 0
in here 1
in here 2
in here 3
<2 second pause>
done with 0
done with 3
done with 1
in here 4
in here 5
in here 6
done with 2
in here 7
<2 second pause>
done with 6
done with 7
done with 4
done with 5
[2, 3, 4, 5, 6, 7, 8, 9]