如何使用 CNN (Keras) 计算多类的一对一 ROC

Question

I am trying to get ROC for 10 classes. I have used CNN model (keras). I am able to get one vs rest curve but I want to get one vs one. The following is the snippet of my code.

model.compile(optimizer=keras.optimizers.Adam(0.001),
          loss='categorical_crossentropy',
          metrics=['acc'])


   from keras.callbacks import History
history = History()

model.fit_generator(generator=train_generator, callbacks= [history],
                validation_data=valid_generator,epochs=10)

score = model.evaluate_generator(test_generator)

x, y = test_generator.next()
prediction = model.predict(x)

predict_label1 = np.argmax(prediction, axis=-1)
true_label1 = np.argmax(y, axis=-1)

y = np.array(true_label1)

scores = np.array(predict_label1)
fpr, tpr, thresholds = metrics.roc_curve(y, scores, pos_label=9)
roc_auc = metrics.auc(fpr, tpr)


plt.figure()
lw = 2
plt.plot(fpr, tpr, color='darkorange',
     lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic (ROC)')
plt.legend(loc="lower right")
plt.show()

This code generates one vs rest curve however I want to get the curve like this picture Is it possible to do so? Help will be appreciated.

Answer 1

The figure you posted illustrates a plot of multiple one vs rest ROC curves. The blue plot is the 0class vs rest, the red plot is the 1 class vs rest, and the green plot denotes the 2 class vs rest.

The ROC curve plots precision/recall based on various thresholds, and help find a trade-off between them. Because a threshold can't be defined in 3+ classes, we imploy a one vs rest method to enable the usage of ROC curves.

I believe you could look up on some blogs on the notion of ROC curves, precision/recall, and f1 score.