如何停止 multiprocessing.Pool on Exception

Question

在下面的代碼中，我在第一次調用時引發了異常，但似乎異常被吸收了，我仍然執行了所有其他進程，這是什么問題？ 我想要的是，每當發生第一個異常時，打印它，然后直接停止多處理池。

def func(i):

    if i==0:
        raise Exception()
    else:
        time.sleep(1)
        print(i)

num_workers = 4

pool = multiprocessing.Pool(num_workers)

try:
    for i in range(4):
        pool.apply_async(func,args=(i,))
except:
    print("err")

pool.close()
pool.join()

以下根據 HTF 編輯的代碼

import multiprocessing
import time

if __name__ == '__main__':
    def func(i):

        if i == 0:
            raise Exception()
        else:
            time.sleep(1)
            print(i)


    num_workers = 4

    pool = multiprocessing.Pool(num_workers)

    results = [pool.apply_async(func, args=(i,)) for i in range(4)]

    try:
        for result in results:
            result.get()
    except:
        print("err")

    pool.close()
    pool.join()

給出 output

err
1
2
3

我期望只有err的地方

Answer 1

您剛剛安排了任務，但您需要等待結果：

results = [pool.apply_async(func,args=(i,)) for i in range(4)]

try:
    for result in results:
        result.get()
except:
    print("err")

2021 年 4 月 7 日星期三 20:42:59 UTC 更新：

你可以嘗試這樣的事情：

import time
  
from functools import partial
from multiprocessing import Pool


def func(i):
    if i == 0:
        raise Exception("something bad happened")
    else:
        time.sleep(1)
        print(i)


def quit(pool, err):
    print(f"ERROR: {err}")
    pool.terminate()


def main():
    pool = Pool()
    partial_quit = partial(quit, pool)

    for i in range(4):
        pool.apply_async(func, args=(i,), error_callback=partial_quit)

    pool.close()
    pool.join()


if __name__ == "__main__":
    main()

測試：

$ python test1.py
ERROR: something bad happened

如果您需要返回值，則使用裸進程和隊列實際上可能更容易：

import time
  
from multiprocessing import Process, Queue

PROCS = 4


def worker(q, i):
    if i == 10:
        print("error")
        q.put_nowait("ERROR")
    else:
        time.sleep(1)
        print(i)
        q.put_nowait(i)


def main():
    q = Queue()
    procs = []

    for i in range(PROCS):
        p = Process(target=worker, args=(q, i))
        p.start()
        procs.append(p)

    count = len(procs)

    while count:
        result = q.get()

        if result == "ERROR":
            for p in procs:
                p.terminate()
            break

        print(f"Result for: {result}")
        count -= 1


if __name__ == "__main__":
    main()

測試：

$ python test2.py
0
2
1
3
Result for: 0
Result for: 2
Result for: 1
Result for: 3