Python 多處理：如果返回時間太長，則終止進程

Question

下面是一個簡單的例子，它凍結了，因為子進程退出時沒有返回任何東西，而父進程一直在等待。 如果花費太長時間，有沒有辦法讓進程超時並讓其余進程繼續？ 我是 python 中多處理的初學者，我發現文檔不是很有啟發性。

import multiprocessing as mp
import time

def foo(x):
    if x == 3:
        sys.exit()
    #some heavy computation here
    return result

if __name__ == '__main__':  
    pool = mp.Pool(mp.cpu_count)
    results = pool.map(foo, [1, 2, 3])

Answer 1

我遇到了同樣的問題，這就是我解決它的方法。 也許有更好的解決方案，但是，它也解決了未提及的問題。 例如，如果進程占用了許多資源，則可能會發生正常終止需要一段時間才能通過進程 - 因此我使用強制終止（ kill -9 ）。 這部分可能僅適用於 Linux，因此如果您使用其他操作系統，則可能需要調整終止。

它是我自己代碼的一部分，因此可能無法復制粘貼。

from multiprocessing import Process, Queue
import os 
import time 

timeout_s = 5000 # seconds after which you want to kill the process

queue = Queue()  # results can be written in here, if you have return objects

p = Process(target=INTENSIVE_FUNCTION, args=(ARGS_TO_INTENSIVE_FUNCTION, queue))
p.start()

start_time = time.time()
check_interval_s = 5  # regularly check what the process is doing

kill_process = False
finished_work = False

while not kill_process and not finished_work:
    time.sleep(check_interval_s)  
    now = time.time()
    runtime = now - start_time

    if not p.is_alive():
        print("finished work")
        finished_work = True

    if runtime > timeout_s and not finished_work:
        print("prepare killing process")
        kill_process = True

if kill_process:
    while p.is_alive():
        # forcefully kill the process, because often (during heavvy computations) a graceful termination
        # can be ignored by a process.
        print(f"send SIGKILL signal to process because exceeding {timeout_s} seconds.")
        os.system(f"kill -9 {p.pid}")

        if p.is_alive():
            time.sleep(check_interval_s)
else:
    try:
        p.join(60)  # wait 60 seconds to join the process
        RETURN_VALS = queue.get(timeout=60)
    except Exception:
        # This can happen if a process was killed for other reasons (such as out of memory)
        print("Joining the process and receiving results failed, results are set as invalid.")