将变量传递给并行化函数

Question

I'm parallelizing the generation of a matrix where each element in the matrix is computed by a function fun. I can get it to work if the only thing I pass into this function are the indices i and j. However, I want to pass another variable into this function say x, how do I do this?

I'm using Python 2.7

import numpy as np                                                                  
import multiprocess as mp                                                           
import itertools                                                                    
p = mp.Pool()                                                                       

def fun((i,j)):                                                                     

   print i,j                                                                         
   prod =  i * j  
   # what if I want to have a variable x in this function
   # prod = i * j * x

   return prod                                                                       

combs = ((i,j) for i,j in itertools.product(xrange(5), repeat=2) if i <= 5)         
result = p.map(fun, combs)                                                          
p.close()                                                                           
p.join() 

newresult = np.array(result).reshape(5,5)                                           
print newresult

Answer 1

def fun((i,j,x)):                                                                     
   print i,j,x                                                                         
   prod =  i * j * x 
   return prod    

Why this works: You are actually just passing one object into the function, which turns out to be a tuple. def fun((i,j)) is just simply breaking the tuple apart again from the object. So to answer your question, you can just add another element to the tuple and it works fine.

A more visibly clear representation of what you are doing:

def fun(data):
    i,j,x = data
    print i,j,x
    prod =  i * j * x
    return prod

data = (2,4,10)
print(fun(data))

Or you can do this:

def fun((i,j), x):
    print i,j, x
    prod =  i * j * x
    # what if I want to have a variable x in this function
    # prod = i * j * x

    return prod

print(fun((2,4), 10))