How can I plot a ROC curve with AUC?

Question

I am trying to classify two different cases. Therefore I built decision trees, the confusion matrix and calculated the accuracy, sensitivity and the specificitvy. I ran my program 100 times, so I have 100 accuracy, sensitivity and specificitvy values.

What I want to do now is to plot the ROC curve with the AUC. I did some researches and all these examples were discussing probabilities and I don't know what exactly these are.

So can anyone help me to plot this? I don't know which part of my code I should present, so please feel free to ask for it.

Answer 1

You could use a package like pROC, which may be easier than creating it yourself. You use the roc() function to create a roc object and then plot() to plot the object and create the ROC curve.

Also, it doesn't sound like you are going about this in the right way. The ROC curve is calculated from sensitivity and specificity values taken at different cut points across the range of possible probabilities from you model ie at cut points between 0 and 1. You don't need to create 100 sets of model predictions, just one model will suffice.

Try something like this, where y is your response variable and p is the vector of probability values output by your model:

plot(roc(y, p)), print.auc = TRUE)