python：如何为多进程应用回调 function？

Question

在我的 GUI 应用程序中，我想使用multiprocessing来加速计算。 现在，我可以使用multiprocessing ，并收集计算结果。 现在，我希望子进程可以通知主进程计算完成，但我找不到任何解决方案。

我的multiprocessing看起来像：

import multiprocessing
from multiprocessing import Process
import numpy as np

class MyProcess(Process):
    def __init__(self,name, array):
        super(MyProcess,self).__init__()
        self.name = name
        self.array = array
        recv_end, send_end = multiprocessing.Pipe(False)
        self.recv = recv_end
        self.send = send_end

    def run(self):

        s = 0
        for a in self.array:
            s += a
        self.send.send(s)

    def getResult(self):
        return self.recv.recv()


if __name__ == '__main__':
    process_list = []
    for i in range(5):
        a = np.random.random(10)
        print(i, ' correct result: ', a.sum())
        p = MyProcess(str(i), a)
        p.start()
        process_list.append(p)

    for p in process_list:
        p.join()

    for p in process_list:
        print(p.name, ' subprocess result: ', p.getResult())

我希望子进程可以通知主进程计算已完成，以便我可以在我的 GUI 中显示结果。

任何建议表示赞赏~~~

Answer 1

假设您想在生成结果后立即对结果（在您的情况下为numpy数组的总和）做某事，那么我将使用带有方法multiprocessing.pool.Pool和方法imap_unordered的多处理池，这将按生成的顺序返回结果。 在这种情况下，您需要将 arrays 列表中的数组索引与数组本身一起处理并让它返回这个索引以及数组的总和，因为这是主要的唯一方法了解为哪个数组生成总和的过程：

from multiprocessing import Pool, cpu_count
import numpy as np

def compute_sum(tpl):
    # unpack tuple:
    i, array = tpl
    s = 0
    for a in array:
        s += a
    return i, s

if __name__ == '__main__':
    array_list = [np.random.random(10) for _ in range(5)]
    n = len(array_list)
    pool_size = min(cpu_count(), n)
    pool = Pool(pool_size)
    # get result as soon as it has been returned:
    for i, s in pool.imap_unordered(compute_sum, zip(range(n), array_list)):
        print(f'correct result {i}: {array_list[i].sum()}, actual result: {s}')
    pool.close()
    pool.join()

印刷：

correct result 0: 4.760033809335711, actual result: 4.76003380933571
correct result 1: 5.486818812843256, actual result: 5.486818812843257
correct result 2: 5.400374562564179, actual result: 5.400374562564179
correct result 3: 4.079376706247242, actual result: 4.079376706247242
correct result 4: 4.20860716467263, actual result: 4.20860716467263

在上述运行中，生成的实际结果恰好与提交任务的顺序相同。 为了证明通常可以根据工作人员 function 计算其结果所需的时间以任意顺序生成结果，我们在处理时间中引入了一些随机性：

from multiprocessing import Pool, cpu_count
import numpy as np

def compute_sum(tpl):
    import time

    # unpack tuple:
    i, array = tpl
    # results will be generated in random order:
    time.sleep(np.random.sample())
    s = 0
    for a in array:
        s += a
    return i, s

if __name__ == '__main__':
    array_list = [np.random.random(10) for _ in range(5)]
    n = len(array_list)
    pool_size = min(cpu_count(), n)
    pool = Pool(pool_size)
    # get result as soon as it has been returned:
    for i, s in pool.imap_unordered(compute_sum, zip(range(n), array_list)):
        print(f'correct result {i}: {array_list[i].sum()}, actual result: {s}')
    pool.close()
    pool.join()

印刷：

correct result 4: 6.662288433360379, actual result: 6.66228843336038
correct result 0: 3.352901187256162, actual result: 3.3529011872561614
correct result 3: 5.836344458981557, actual result: 5.836344458981557
correct result 2: 2.9950208717729656, actual result: 2.9950208717729656
correct result 1: 5.144743159869513, actual result: 5.144743159869513

如果您对在任务提交而不是任务完成顺序中返回结果感到满意，那么使用方法imap并且不需要来回传递数组索引：

from multiprocessing import Pool, cpu_count
import numpy as np

def compute_sum(array):
    s = 0
    for a in array:
        s += a
    return s

if __name__ == '__main__':
    array_list = [np.random.random(10) for _ in range(5)]
    n = len(array_list)
    pool_size = min(cpu_count(), n)
    pool = Pool(pool_size)
    for i, s in enumerate(pool.imap(compute_sum, array_list)):
        print(f'correct result {i}: {array_list[i].sum()}, actual result: {s}')
    pool.close()
    pool.join()

印刷：

correct result 0: 4.841913985702773, actual result: 4.841913985702773
correct result 1: 4.836923014762733, actual result: 4.836923014762733
correct result 2: 4.91242274200897, actual result: 4.91242274200897
correct result 3: 4.701913574838348, actual result: 4.701913574838349
correct result 4: 5.813666896917504, actual result: 5.813666896917503