使用Sklearn在ROC曲线下的面积？

Question

I can't figure out why the Sklearn function roc_auc_score returns 1 in the following case:

y_true = [0, 0, 1, 0, 0, 0, 0, 1]

y_scores = [0.18101096153259277, 0.15506085753440857, 
            0.9940806031227112, 0.05024950951337814, 
            0.7381414771080017, 0.8922111988067627, 
            0.8253260850906372, 0.9967281818389893]

roc_auc_score(y_true,y_scores)

The three scores 0.7381414771080017, 0.8922111988067627, 0.8253260850906372 at the end don't match the labels 0, 0, 0 . So, how can the AUC be 1? What am I getting wrong here?

Answer 1

The auc of ROC curve just measures the ability of your model to rank order the datapoints, with respect to your positive class.

In your example, the score of the positive class is always greater than the negative class datapoints. Hence, the auc_roc_score of 1 is correct.

pd.DataFrame({'y_true':y_true,'y_scores':y_scores}).sort_values('y_scores',ascending=False)

    y_scores    y_true
7   0.996728    1
2   0.994081    1
5   0.892211    0
6   0.825326    0
4   0.738141    0
0   0.181011    0
1   0.155061    0
3   0.050250    0

Answer 2

If you look at the ROC itself, it's easier to see why:

> roc_curve(y_true, y_scores)

(array([0., 0., 0., 1.]),
 array([0. , 0.5, 1. , 1. ]),
 array([1.99672818, 0.99672818, 0.9940806 , 0.05024951]))

The first value in the returned tuple is the FPR, the second is the TPR, and the third is the threshold points where the value changes.

For a threshold of 0.99672818, the FPR is indeed 0.5, and not 0, which would cause you to think that the AUC of the ROC is not 0. However, the FPR/TPR points are just the lines 0, 0 -> 0, 1 -> 1, 1, and the area beneath that is indeed 1.