Using apply_async with callback function for a pool of processes

Question

I am trying to understand how multiprocess pools work. In the following programing I created a pool of 4 processes.

And I call apply_async with a callback function that should update a list called result_list

import Queue
from multiprocessing import Process
from multiprocessing import Pool

result_list = []

def foo_pool(q): #Function for each process
    print "foo_pool" 
    if(q.qsize() > 0):
        number = q.get()
    return number * 2

def log_result(result):
    # This is called whenever foo_pool(i) returns a result.
    # result_list is modified only by the main process, not the pool workers.
    result_list.append(result)

if __name__ == "__main__": 
    q = Queue.Queue()
    for i in range(4):
        q.put(i + 1) #Put 1..4 in the queue

    p = Pool(4)
    p.apply_async(foo_pool, args = (q, ), callback = log_result)

I realize I don't need to use a queue here. But I am testing this for another program which requires me to use a queue. When I run the program, the function foo_pool is not being called. The print statement print "foo_pool" does not execute. Why is this?

Answer 1

Roughly speaking, apply_async only schedule async task, but not run it. You need to call p.close() and p.join() to trigger execution or r = p.apply_async() and r.get() .