使用sklearn训练不同的回归器

Question

I have a list of Xs and their output value Ys . And using the following code, I am able to train the following regressors:

• Linear Regressor
• Isotonic Regressor
• Baysian Ridge Regressor
• Gradient Boosting Regressor

The code:

import numpy as np

from sklearn.linear_model import LinearRegression, BayesianRidge
from sklearn.isotonic import IsotonicRegression
from sklearn import ensemble
from sklearn.svm import SVR
from sklearn.gaussian_process import GaussianProcess


import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection


def get_meteor_scores(infile):
    with io.open(infile, 'r') as fin:
        meteor_scores = [float(i.strip().split()[-1]) for 
                               i in re.findall(r'Segment [0-9].* score\:.*\n', 
                                               fin.read())]
        return meteor_scores

def get_sts_scores(infile):
    with io.open(infile, 'r') as fin:
        sts_scores = [float(i) for i in fin]
        return sts_scores

Xs = 'meteor.output.train'
Ys = 'score.train'
# Gets scores from https://raw.githubusercontent.com/alvations/USAAR-SemEval-2015/master/task02-USAAR-SHEFFIELD/x.meteor.train
meteor_scores = np.array(get_meteor_scores(Xs))
# Gets scores from https://raw.githubusercontent.com/alvations/USAAR-SemEval-2015/master/task02-USAAR-SHEFFIELD/score.train
sts_scores = np.array(get_sts_scores(Ys))

x = meteor_scores
y = sts_scores
n = len(sts_scores)

# Linear Regression
lr = LinearRegression()
lr.fit(x[:, np.newaxis], y)

# Baysian Ridge Regression
br = BayesianRidge(compute_score=True)
br.fit(x[:, np.newaxis], y)

# Isotonic Regression
ir = IsotonicRegression()
y_ = ir.fit_transform(x, y)

# Gradient Boosting Regression
params = {'n_estimators': 500, 'max_depth': 4, 'min_samples_split': 1,
          'learning_rate': 0.01, 'loss': 'ls'}
gbr = ensemble.GradientBoostingRegressor(**params)
gbr.fit(x[:, np.newaxis], y)

But how do I train regressors for Support Vector Regression , Gaussian Process and Decision Tree Regressor ?

---

When i tried the following to train Support Vector Regressors , I get an error:

from sklearn.svm import SVR
# Support Vector Regressions
svr_rbf = SVR(kernel='rbf', C=1e3, gamma=0.1)
svr_lin = SVR(kernel='linear', C=1e3)
svr_poly = SVR(kernel='poly', C=1e3, degree=2)
y_rbf = svr_rbf.fit(x, y)
y_lin = svr_lin.fit(x, y)
y_poly = svr_poly.fit(x, y)

[out]:

Traceback (most recent call last):
  File "/home/alvas/git/USAAR-SemEval-2015/task02-somethingLiddat/carolling.py", line 47, in <module>
    y_rbf = svr_rbf.fit(x, y)
  File "/home/alvas/.local/lib/python2.7/site-packages/sklearn/svm/base.py", line 149, in fit
    (X.shape[0], y.shape[0]))
ValueError: X and y have incompatible shapes.
X has 1 samples, but y has 10597.

The same happens when I tried Gaussian Process :

from sklearn.gaussian_process import GaussianProcess
# Gaussian Process
gp = GaussianProcess(corr='squared_exponential', theta0=1e-1,
                     thetaL=1e-3, thetaU=1,
                     random_start=100)
gp.fit(x, y)

[out]:

Traceback (most recent call last):
  File "/home/alvas/git/USAAR-SemEval-2015/task02-somethingLiddat/carolling.py", line 57, in <module>
    gp.fit(x, y)
  File "/home/alvas/.local/lib/python2.7/site-packages/sklearn/gaussian_process/gaussian_process.py", line 271, in fit
    X, y = check_arrays(X, y)
  File "/home/alvas/.local/lib/python2.7/site-packages/sklearn/utils/validation.py", line 254, in check_arrays
    % (size, n_samples))
ValueError: Found array with dim 10597. Expected 1

When running the gp.fit(x[:,np.newaxis], y) I get this error:

Traceback (most recent call last):
  File "/home/alvas/git/USAAR-SemEval-2015/task02-somethingLiddat/carolling.py", line 95, in <module>
    gp.fit(x[:,np.newaxis], y) 
  File "/home/alvas/.local/lib/python2.7/site-packages/sklearn/gaussian_process/gaussian_process.py", line 301, in fit
    raise Exception("Multiple input features cannot have the same"
Exception: Multiple input features cannot have the same target value.

When I tried Decision Tree Regressor :

from sklearn.tree import DecisionTreeRegressor
# Decision Tree Regression
dtr2 = DecisionTreeRegressor(max_depth=2)
dtr5 = DecisionTreeRegressor(max_depth=2)
dtr2.fit(x,y)
dtr5.fit(x,y)

[out]:

Traceback (most recent call last):
  File "/home/alvas/git/USAAR-SemEval-2015/task02-somethingLiddat/carolling.py", line 47, in <module>
    dtr2.fit(x,y)
  File "/home/alvas/.local/lib/python2.7/site-packages/sklearn/tree/tree.py", line 140, in fit
    n_samples, self.n_features_ = X.shape
ValueError: need more than 1 value to unpack

Answer 1

All these regressors require multidimensional x-array but your x-array is a 1D array. So only requirement is to convert x-array into 2D array for these regressors to work. This can be achieved using x[:, np.newaxis]

Demo:

>>> from sklearn.svm import SVR
>>> # Support Vector Regressions
... svr_rbf = SVR(kernel='rbf', C=1e3, gamma=0.1)
>>> svr_lin = SVR(kernel='linear', C=1e3)
>>> svr_poly = SVR(kernel='poly', C=1e3, degree=2)
>>> x=np.arange(10)
>>> y=np.arange(10)
>>> y_rbf = svr_rbf.fit(x[:,np.newaxis], y)  
>>> y_lin = svr_lin.fit(x[:,np.newaxis], y)
>>> svr_poly = svr_poly.fit(x[:,np.newaxis], y)
>>> from sklearn.gaussian_process import GaussianProcess
>>> # Gaussian Process
... gp = GaussianProcess(corr='squared_exponential', theta0=1e-1,
...                      thetaL=1e-3, thetaU=1,
...                      random_start=100)
>>> gp.fit(x[:, np.newaxis], y)
GaussianProcess(beta0=None,
        corr=<function squared_exponential at 0x7f46f3ebcf50>,
        normalize=True, nugget=array(2.220446049250313e-15),
        optimizer='fmin_cobyla', random_start=100,
        random_state=<mtrand.RandomState object at 0x7f4702d97150>,
        regr=<function constant at 0x7f46f3ebc8c0>, storage_mode='full',
        theta0=array([[ 0.1]]), thetaL=array([[ 0.001]]),
        thetaU=array([[1]]), verbose=False)
>>> from sklearn.tree import DecisionTreeRegressor
>>> # Decision Tree Regression
... dtr2 = DecisionTreeRegressor(max_depth=2)
>>> dtr5 = DecisionTreeRegressor(max_depth=2)
>>> dtr2.fit(x[:,np.newaxis],y)
DecisionTreeRegressor(compute_importances=None, criterion='mse', max_depth=2,
           max_features=None, min_density=None, min_samples_leaf=1,
           min_samples_split=2, random_state=None, splitter='best')
>>> dtr5.fit(x[:,np.newaxis],y)
DecisionTreeRegressor(compute_importances=None, criterion='mse', max_depth=2,
           max_features=None, min_density=None, min_samples_leaf=1,
           min_samples_split=2, random_state=None, splitter='best')

Preprocessing for GaussianProcess :

xu = np.unique(x)  # get unique x values
idx = [np.where(x==x1)[0][0] for x1 in xu]  # get corresponding indices for unique x values
gp.fit(xu[:,np.newaxis], y[idx])  # y[idx] selects y values corresponding to unique x values

Answer 2

Multiple input features cannot have the same target value.

This means that one data point is repeating in your input data, and the Gaussian process does not allow for one data point to be listed twice. Unfortunately, your dataset is no longer available, so I cannot check this, but that is what I think should be the case.