如何停止 multiprocessing.Pool.map 异常

Question

When I raise an exception inside my thread_function , it doesn't stop the rest of the map processing. I'd like to stop it.

def thread_function(n):
    if n == 10:
        raise Exception('Stop everything!')

pool = Pool(processes = 4)
pool.map(thread_function, range(1, 1000), chunksize = 1)

I'd expect no more processing after one thread reached n == 10 .

Answer 1

I don't know of a way to do this directly with map but you can monitor an async_map like this...

from multiprocessing import Pool
import time

def thread_function(n):
    if n == 10:
        print('Raising Exception')
        raise Exception('Stop everything!')
    print(n)
    time.sleep(0.1)

pool = Pool(processes = 4)
result = pool.map_async(thread_function, range(1, 1000), chunksize = 1)
while not result.ready():
    if not result._success:
        print('Exiting for failure')
        pool.terminate()
        pool.join()
        break