如何计算Scikit中多类分类的混淆矩阵？

Question

I have a multiclass classification task. When I run my script based on the scikit example as the follows:

classifier = OneVsRestClassifier(GradientBoostingClassifier(n_estimators=70, max_depth=3, learning_rate=.02))

y_pred = classifier.fit(X_train, y_train).predict(X_test)
cnf_matrix = confusion_matrix(y_test, y_pred)

I get this error:

File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 242, in confusion_matrix
    raise ValueError("%s is not supported" % y_type)
ValueError: multilabel-indicator is not supported

I tried to pass the labels=classifier.classes_ to confusion_matrix() , but it doesn't help.

y_test and y_pred are as the follow:

y_test =
array([[0, 0, 0, 1, 0, 0],
   [0, 0, 0, 0, 1, 0],
   [0, 1, 0, 0, 0, 0],
   ..., 
   [0, 0, 0, 0, 0, 1],
   [0, 0, 0, 1, 0, 0],
   [0, 0, 0, 0, 1, 0]])


y_pred = 
array([[0, 0, 0, 0, 0, 0],
   [0, 0, 0, 0, 0, 0],
   [0, 0, 0, 0, 0, 0],
   ..., 
   [0, 0, 0, 0, 0, 1],
   [0, 0, 0, 0, 0, 1],
   [0, 0, 0, 0, 0, 0]])

Answer 1

First you need to create the label output array. Lets say you have 3 classes: 'cat', 'dog', 'house' indexed: 0,1,2 . And the prediction for 2 samples is: 'dog', 'house'. Your output will be:

y_pred = [[0, 1, 0],[0, 0, 1]]

run y_pred.argmax(1) to get: [1,2] This array stands for the original label indexes, meaning: ['dog', 'house']

num_classes = 3

# from lable to categorial
y_prediction = np.array([1,2]) 
y_categorial = np_utils.to_categorical(y_prediction, num_classes)

# from categorial to lable indexing
y_pred = y_categorial.argmax(1)

Answer 2

This worked for me:

y_test_non_category = [ np.argmax(t) for t in y_test ]
y_predict_non_category = [ np.argmax(t) for t in y_predict ]

from sklearn.metrics import confusion_matrix
conf_mat = confusion_matrix(y_test_non_category, y_predict_non_category)

where y_test and y_predict are categorical variables like one-hot vectors.

Answer 3

I just subtracted the outputs y_test matrix from the prediction y_pred matrix while keeping the categorical format. In case of -1 , I assumed a false negative while in case of 1 , a false positive.

Next:

if output_matrix[i,j] == 1 and predictions_matrix[i,j] == 1:  
    produced_matrix[i,j] = 2 

Ending up with the following notation:

• -1: false negative
• 1: false positive
• 0: true negative
• 2: true positive

Finally, be performing some naive counting you can produce any confusion metric.