具有异步工作者的多处理池

Question

考虑以下代码：

class Sandbox:
    def __init__(self):
        self.pool = Pool(4)

    def worker(self, x):
        print(x) # Should be printing "testing123"

    def run(self):
        res = self.pool.apply_async(self.worker, ("testing123",))
        print(res.get()) # NotImplementedError
        self.pool.close()
        self.pool.join()


sandbox = Sandbox()
sandbox.run()

当我运行这个时，我得到

NotImplementedError：池对象不能在进程之间传递或腌制

res.get() 揭示了异常——如果我删除它，什么也不会发生。

Answer 1

async_apply获取 state 并设置 state，因此您需要修改这些方法，例如：

from multiprocessing import Pool
class Sandbox:
    def __init__(self):
        self.pool = Pool(4)

    def worker(self, x):
        print(x) # Should be printing "testing123"

    def run(self):
        res = self.pool.apply_async(self.worker, ("testing123",))
        print(res.get()) # NotImplementedError
        self.pool.close()
        self.pool.join()

    def __getstate__(self):
        self_dict = self.__dict__.copy()
        del self_dict['pool']
        return self_dict

    def __setstate__(self, state):
        self.__dict__.update(state)


sandbox = Sandbox()
sandbox.run()