多类分类中的输入形状不好（）

Question

I'am performing a multi-class classification task using sci-kit learn. In the setup i created, i want to compare different classification algorithms.

I use a pipeline, where text is inserted as X and Y is the class (multi-class, N = 5). Textual features are extracted in the pipeline using TfidfVectorizer().

KNN does the job, but other classifiers give this: ValueError: bad input shape (670, 5)

Full traceback:

"/Users/Robbert/pipeline.py", line 62, in <module>
train_pipeline.fit(X_train, Y_train)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/pipeline.py", line 130, in fit
self.steps[-1][-1].fit(Xt, y, **fit_params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/svm/base.py", line 138, in fit
y = self._validate_targets(y)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/svm/base.py", line 441, in _validate_targets
y_ = column_or_1d(y, warn=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/utils/validation.py", line 319, in column_or_1d
raise ValueError("bad input shape {0}".format(shape))
ValueError: bad input shape (670, 5)

The code i use:

def read_data(f):
data = []
for row in csv.reader(open(f), delimiter=';'):
    if row:
        plottext = row[8]
        target = { 'Age': row[4] }
        data.append((plottext, target))
(X, Ycat) = zip(*data) 
Y = DictVectorizer().fit_transform(Ycat)
Y = preprocessing.LabelBinarizer().fit_transform(Y)
return (X, Y)

X, Y = read_data('development2.csv')

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.33, random_state=42)

###KNN Pipeline
#train_pipeline = Pipeline([
#    ('vect', TfidfVectorizer(ngram_range=(1, 3), min_df=1)),
#    ('clf', KNeighborsClassifier(n_neighbors=350, weights='uniform'))])

###Logistic regression Pipeline
#train_pipeline = Pipeline([
#    ('vect', TfidfVectorizer(ngram_range=(1, 3), min_df=1)),
#    ('clf', LogisticRegression())])

##SVC
train_pipeline = Pipeline([
('vect', TfidfVectorizer(ngram_range=(1, 3), min_df=1)),
('clf', SVC(C=1, kernel='rbf', gamma=0.001, probability=True))])

##Decision tree
#train_pipeline = Pipeline([
#    ('vect', TfidfVectorizer(ngram_range=(1, 3), min_df=1)),   
#    ('clf', DecisionTreeClassifier(random_state=0))])

train_pipeline.fit(X_train, Y_train)

predicted = train_pipeline.predict(X_test)

print accuracy_score(Y_test, predicted)

How is it possible that KNN accepts the shape of the array and other classifiers don't? And how to change this shape?

Answer 1

If you compare documentation for fit(X, y) function in KNeighborsClassifier and SVC , you will see that only the former one accepts the y in the form [n_samples, n_outputs].

Possible solution: why do you need LabelBinarizer at all? Just do not use it.

Answer 2

If your Y vector is of size (n_samples, n_classes) and contains at least a single row which has more than one non-zero element, then you are solving a multi-label classification problem. If that is the case, The multiclass and multilabel algorithms page in scikit-learn docs lists KNN as one of the classifiers that supports multi-label classification. You might want to try out other classifiers from that list

* sklearn.tree.DecisionTreeClassifier
* sklearn.tree.ExtraTreeClassifier
* sklearn.ensemble.ExtraTreesClassifier
* sklearn.neural_network.MLPClassifier
* sklearn.neighbors.RadiusNeighborsClassifier
* sklearn.ensemble.RandomForestClassifier
* sklearn.linear_model.RidgeClassifierCV