ValueError：分类指标无法在 ROC 曲线计算中处理多类和多标签指标目标的混合

Question

I'm trying to draw a roc curve for multiclass classification.

At first I calculate y_pred and y_proba using the following code

X_train, X_test, y_train, y_test = train_test_split(X, Y, random_state = 0)
  
# training a DescisionTreeClassifier
from sklearn.tree import DecisionTreeClassifier

dtree_model = DecisionTreeClassifier(max_depth = 2).fit(X_train, y_train)

y_pred = dtree_model.predict(X_test)
y_proba= dtree_model.predict_proba(X_test)

After that I use the following function to calculate tpr and fpr

from sklearn.metrics import confusion_matrix

def calculate_tpr_fpr(y_test, y_pred):
    '''
    Calculates the True Positive Rate (tpr) and the True Negative Rate (fpr) based on real and predicted observations
    
    Args:
     y_real: The list or series with the real classes
     y_pred: The list or series with the predicted classes
    
    Returns:
     tpr: The True Positive Rate of the classifier
     fpr: The False Positive Rate of the classifier
    '''
    
    # Calculates the confusion matrix and recover each element
    cm = confusion_matrix(y_test, y_pred)
    TN = cm[0, 0]
    FP = cm[0, 1]
    FN = cm[1, 0]
    TP = cm[1, 1]
    
    # Calculates tpr and fpr
    tpr = TP / (TP + FN) # sensitivity - true positive rate
    fpr = 1 - TN / (TN + FP) # 1-specificity - false positive rate
    
    return tpr, fpr

Then, I try using this function to calculate a list of fpr and tpr to draw the curve

def get_all_roc_coordinates(y_test, y_proba):
    '''
    Calculates all the ROC Curve coordinates (tpr and fpr) by considering each point as a treshold for the predicion of the class.
    
    Args:
     y_test: The list or series with the real classes.
     y_proba: The array with the probabilities for each class, obtained by using the `.predict_proba()` method.
     
    Returns:
     tpr_list: The list of TPRs representing each threshold.
     fpr_list: The list of FPRs representing each threshold.
    '''
    
    tpr_list = [0]
    fpr_list = [0]
    
    for i in range(len(y_proba)):
        threshold = y_proba[i]
        y_pred = y_proba = threshold
        tpr, fpr = calculate_tpr_fpr(y_test, y_pred)
        tpr_list.append(tpr)
        fpr_list.append(fpr)
        
    return tpr_list, fpr_list

but it gives me the following error

ValueError: Classification metrics can't handle a mix of multiclass and multilabel-indicator targets

Note that the Y column is multiclass {0,1,2}. I also tried to ensure that y is string not integer, but it gives me the same error.

Answer 1

You've got 3 classes but you only use 2 classes in your calculate_tpr_fpr() . Also, you probably meant y_pred = y_proba > threshold . Either way, it won't be that easy since you've got 3 columns of class scores. The easiest way seems to be drawing one vs rest curves, treating each column individually:

from sklearn.metrics import roc_curve
from sklearn.preprocessing import label_binarize
import matplotlib.pyplot as plt

classes = range(y_proba.shape[1])

for i in classes:
    fpr, tpr, _ = roc_curve(label_binarize(y_test, classes=classes)[:,i], y_proba[:,i])
    plt.plot(fpr, tpr, alpha=0.7)
    plt.legend(classes)