[英]Parallellising a for loop for a function with multiple inputs and outputs
I've read several posts on multiprocessing in Python, but it's not clear to me how can I use them for my problem (where I have multiple inputs and outputs). 我已经阅读了几篇有关使用Python进行多处理的文章,但是我不清楚如何使用它们解决问题(我有多个输入和输出)。 Most of the available examples consider a single output function with a rather simple structure. 大多数可用示例都考虑了具有相当简单结构的单个输出函数。
Here is the code in Python: 这是Python中的代码:
import numpy as np
n = 1000
i1 = np.random.random(n)
i2 = np.random.random(n)
i3 = np.random.random(n)
i4 = np.random.random(n)
o1 = np.zeros(n)
o2 = np.zeros(n)
o3 = np.zeros(n)
def fun(i1,i2,i3,i4):
o1 = i1 + i2 + i3 + i4
o2 = i2*i3 - i1 + i4
o3 = i1 - i2 + i3 + i4
if o1 < o2:
o1 = o2
else:
o2 = o1
while o1 + o2 > o3:
o3 = o3 + np.random.random()
return o1,o2,o3
for i in range(n): # I want to parallellise this loop
o1[i],o2[i],o3[i] = fun(i1[i],i2[i],i3[i],i4[i])
I am only looking for a way to parallelize the for
loop. 我只在寻找一种并行化for
循环的方法。 How can I accomplish this? 我该怎么做?
I will use a generator to combine your input lists i1 to i4. 我将使用一个生成器来组合您的输入列表i1至i4。 Your math-function fun
will return a list-object. 您的数学函数fun
将返回一个列表对象。 Now I have a single argument as input (the generator) and get a single object as output (the list). 现在,我有一个参数作为输入(生成器),并有一个对象作为输出(列表)。 I have tried the code below and it works. 我已经尝试过下面的代码,它可以工作。
You can add a sleep-command in your fun
-function to see the speed-gain when using multiple processes. 您可以在fun
函数中添加一个sleep命令,以查看使用多个进程时的速度增益。 Otherwise your fun
-function is too simple to really benefit from multi-processing. 否则,您的fun
功能太简单了,无法真正从多处理中受益。
import numpy as np
from multiprocessing import Pool
n = 1000
i1 = np.random.random(n)
i2 = np.random.random(n)
i3 = np.random.random(n)
i4 = np.random.random(n)
def fun(a):
o1 = a[0] + a[1] + a[2] + a[3]
o2 = a[1]*a[2] - a[0] + a[3]
o3 = a[0] - a[1] + a[2] + a[3]
if o1 < o2:
o1 = o2
else:
o2 = o1
while o1 + o2 > o3:
o3 = o3 + np.random.random()
return [o1,o2,o3]
# a generator that fills the Pool input
def conc(lim):
n = 0
while n < lim:
yield [i1[n], i2[n], i3[n], i4[n]]
n += 1
if __name__ == '__main__':
numbers = conc(n)
with Pool(5) as p:
print(p.map(fun, numbers))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.