[英]How to parallelise .predict() method of a scikit-learn SVM (SVC) Classifier?
我最近遇到一个要求,我有一个.fit()
训练有素的scikit-learn
SVC
分类器实例,并且需要.predict()
很多实例。
有没有办法通过任何scikit-learn
内置工具仅并行化这个.predict()
方法?
from sklearn import svm
data_train = [[0,2,3],[1,2,3],[4,2,3]]
targets_train = [0,1,0]
clf = svm.SVC(kernel='rbf', degree=3, C=10, gamma=0.3, probability=True)
clf.fit(data_train, targets_train)
# this can be very large (~ a million records)
to_be_predicted = [[1,3,4]]
clf.predict(to_be_predicted)
如果有人确实知道解决方案,如果您能分享它,我会非常高兴。
这可能有问题,但像这样的事情应该可以解决问题。 基本上,将您的数据分解成块并在joblib.Parallel
循环中分别在每个块上运行您的模型。
from sklearn.externals.joblib import Parallel, delayed
n_cores = 2
n_samples = to_be_predicted.shape[0]
slices = [
(n_samples*i/n_cores, n_samples*(i+1)/n_cores))
for i in range(n_cores)
]
results = np.vstack( Parallel( n_jobs = n_cores )(
delayed(clf.predict)( to_be_predicted[slices[i_core][0]:slices[i_core][1]
for i_core in range(n_cores)
))
上面的工作示例...
from joblib import Parallel, delayed
from sklearn import svm
data_train = [[0,2,3],[1,2,3],[4,2,3]]
targets_train = [0,1,0]
clf = svm.SVC(kernel='rbf', degree=3, C=10, gamma=0.3, probability=True)
clf.fit(data_train, targets_train)
to_be_predicted = np.array([[1,3,4], [1,3,4], [1,3,5]])
clf.predict(to_be_predicted)
n_cores = 3
parallel = Parallel(n_jobs=n_cores)
results = parallel(delayed(clf.predict)(to_be_predicted[i].reshape(-1,3))
for i in range(n_cores))
np.vstack(results).flatten()
array([1, 1, 0])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.