用sci-kit分类学习多值输出

Question

Let us say I have chosen a single training document from a training set. I have put it into feature vector X for my chosen features.

I am trying to do:

self.clf = LogisticRegression()
self.clf.fit(X, Y)

My Y would be something like: [0 0 0 1 1 0 1 0 0 1 0]

I would like to train my one single model so that it best fits each of the 11 output values simultaneously. This doesn't seem to work for fit as I get a unhashable type 'list' error because it is expecting a single value which is ether binary or multi-class but does not allow for more than one value.

Is there anyway to do this with sci-kit learn?

Answer 1

Multi-label classification has a somewhat different API than ordinary classification. Your Y should be a sequence of sequences, eg a list of lists, like

Y = [["foo", "bar"],          # the first sample is a foo and a bar
     ["foo"],                 # the second is only a foo
     ["bar", "baz"]]          # the third is a bar and a baz

Such a Y can then be fed to an estimator that handles multiple classifications. You can construct such an estimator using the OneVsRestClassifier wrapper:

from sklearn.multiclass import OneVsRestClassifier
clf = OneVsRestClassifier(LogisticRegression())

then train with clf.fit(X, Y) . clf.predict will now produce sequences of sequences as well.

UPDATE as of scikit-learn 0.15, this API is deprecated because its input is ambiguous. You should convert the Y I gave above to a matrix with a MultiLabelBinarizer :

>>> from sklearn.preprocessing import MultiLabelBinarizer
>>> mlb = MultiLabelBinarizer()
>>> mlb.fit_transform(Y)
array([[1, 0, 1],
       [0, 0, 1],
       [1, 1, 0]])

Then feed this to an estimator's fit method. Converting back is done with inverse_transform on the same binarizer:

>>> mlb.inverse_transform(mlb.transform(Y))
[('bar', 'foo'), ('foo',), ('bar', 'baz')]

Answer 2

Could you please be more specific what your task is? Is a label a fixed length vector of binary variables? Then this would be called multi label classification (ie multiple labels are either on or off). If each label can have more than two values it is called "multi output" in scikit-learn and can only be done by trees and ensembles.

PS: if you use a linear classifier such as logistic regression, the output variables will be treated independently any way.