[英]Multiprocessing Pool getting stuck
I don`t have much experience with multiprocessing but thus far it seems to hate me.我对多处理没有太多经验,但到目前为止它似乎讨厌我。 I am trying to run the most simplest of examples, and yet it gets stuck and I have to restart my kernel.
我正在尝试运行最简单的示例,但它卡住了,我必须重新启动我的 kernel。
My example:我的例子:
import multiprocessing as mp
aa = [x for x in range(3,16)]
def f(x):
return x**2
if __name__ == '__main__':
with mp.Pool(processes = 4) as p:
res = p.map(f, aa)
print(res)
It worked just fine up until I experimented with using class methods and atributes inside of the pool.map()
.在我尝试使用 class 方法和
pool.map()
内部的属性之前,它工作得很好。 I think that it broke after I tried to execute this:我认为在我尝试执行此操作后它坏了:
class clasy():
def __init__(self, a, b):
self.a = a
self.b = b
def gen(self):
for i in range(self.a, self.b):
yield i
def dodo(self):
gg = gen()
with mp.Pool() as pool:
self.res = pool.map(f, gg)
return self.res
paralel = clasy(3,16)
print(paralel.dodo())
What went wrong and can it be fixed?出了什么问题,可以修复吗?
And further more, is it possible to use class method for generator inside of the pool.map(), can it be used inside a class.此外,是否可以在 pool.map() 内部使用 class 方法生成生成器,是否可以在 class 内部使用。 Or should the pool, function and iterator be top level objects?
或者池 function 和迭代器应该是顶级对象吗?
You need to either pass the function f
to the class clasy
or make it as an instance of the class.您需要将 function
f
传递给clasy
或将其作为 class 的实例。 For example, the following should work:例如,以下应该有效:
class clasy():
def __init__(self, a, b):
self.a = a
self.b = b
def gen(self):
for i in range(self.a, self.b):
yield i
def f(self, x):
return x**2
def dodo(self):
gg = self.gen()
with mp.Pool(processes = 4) as pool:
self.res = pool.map(self.f, gg)
return self.res
paralel = clasy(3,16)
print(paralel.dodo())
Out[1]:
[9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.