I've coded a 1 vs rest classifier in Python that trains 11 different classifiers, one for each class. The code is shown below:
def onevsrest(X_train,y_train,lamb):
beta=[]
beta_init=np.zeros(X_train.shape[1])
for i in range(1,12):
print(i)
y=np.copy(y_train)
y[y != i] = -1
y[y == i] = 1
beta_temp,objs = svm(lamb, 0.1, 200, X_train, y)
beta.append(beta_temp[-1])
return beta
How would I parallelize the above program using the Python multiprocessing module? From my understanding, multiprocessing can only be used for code with a single argument. How would I scale that towards this function that takes multiple arguments?
You can "partial" your function. Example:
# multiple arguments function
def calc(a, b, c):
return a + b + c
# prepare a single argument partial function, freezing `b` and `c`
from functools import partial
calc2 = partial(calc, b=3, c=7)
from multiprocessing import Pool
p = Pool(5)
print(p.map(calc2, [1, 2, 3, 4, 5, 6]))
You could also use tuple , like:
from multiprocessing import Pool
def f(data):
x, y = data
return x*y
if __name__ == '__main__':
with Pool(5) as p:
X = (1,2,3)
Y = (1,4,9)
print(p.map( f, list(zip(X,Y)) )) # returns [1, 8, 27]
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.