回归与多维目标

Question

I am using scikit-learn to do regression and my problem is the following. I need to do regression on several parameters (vectors). This works fine with some regression approaches such as ensemble.ExtraTreesRegressor and ensemble.RandomForestRegressor . Indeed, one can give a vector of vectors as targets to fit the model ( fit(X,y) method) for the two aforementionned regression methods.

However when I try with ensemble.GradientBoostingRegressor , ensemble.AdaBoostRegressor and linear_model.SGDRegressor , the classifier fails to fit the model because it expects 1-dimensional values as targets (y argument of the fit(X,y) method). This means, with those Regression methods I can estimate only one parameter at a time. This is not suitable for my problem because it might take some time while I need to estimate about 20 parameters. On the other hand, I really would like to test those approaches.

So, my question is: Does anyone know if there is a solution to fit the model once and estimate several parameters for ensemble.GradientBoostingRegressor , ensemble.AdaBoostRegressor and linear_model.SGDRegressor ?

I hope I've been clear enough ...

Answer 1

I interpret that what you have is a problem of multiple multivariate regression .

Not every regression method in scikit-learn can handle this sort of problem and you should consult the documentation of each one to find it out. In particular, neither SGDRegressor , GradientBoostingRegressor nor AdaBoostRegressor support this at the moment: fit(X, y) specifies X : array-like, shape = [n_samples, n_features] and y: array-like, shape = [n_samples].

However, you can use other methods in scikit-learn. For example, linear models:

from sklearn import linear_model
# multivariate input
X = [[0., 0.], [1., 1.], [2., 2.], [3., 3.]]
# univariate output
Y = [0., 1., 2., 3.]
# multivariate output
Z = [[0., 1.], [1., 2.], [2., 3.], [3., 4.]]

# ordinary least squares
clf = linear_model.LinearRegression()
# univariate
clf.fit(X, Y)
clf.predict ([[1, 0.]])
# multivariate
clf.fit(X, Z)
clf.predict ([[1, 0.]])

# Ridge
clf = linear_model.BayesianRidge()
# univariate
clf.fit(X, Y)
clf.predict ([[1, 0.]])
# multivariate
clf.fit(X, Z)
clf.predict ([[1, 0.]])

# Lasso
clf = linear_model.Lasso()
# univariate
clf.fit(X, Y)
clf.predict ([[1, 0.]])
# multivariate
clf.fit(X, Z)
clf.predict ([[1, 0.]])

Answer 2

As already mentioned, only some models support multivariate output. If you want to use one of the others, you can use a new class for parallelization of regressors for multivariate output: MultiOutputRegressor .

You can use it like this:

from sklearn.datasets import load_linnerud
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor

linnerud = load_linnerud()

X = linnerud.data
Y = linnerud.target

# to set number of jobs to the number of cores, use n_jobs=-1
MultiOutputRegressor(GradientBoostingRegressor(), n_jobs=-1).fit(X, Y)