Change ROC cutoff values using AUC package in R

Question

I am constructing ROC plots in R using the AUC package .

These are the first 5 points of a 300 point dataset comparing probability of survival to observed survival.

predict1 <- c(0.755, 0.755, 0.937, 0.978, 0.755)
y <- c(1,1,1,0,1)

ROC_null_a <- auc(roc(predict1, as.factor(y)))
plot(roc(predict1, as.factor(y)))

I would like to change the cutoff values for the ROC plot. The documentation describes these values but does not specify how to actually use them in the roc or auc functions:

"Value

A list containing the following elements:

cutoffs  A numeric vector of threshold values

fpr  A numeric vector of false positive rates corresponding to the threshold values

tpr  A numeric vector of true positive rates corresponding to the threshold values"

The examples only include the basic function and do not demonstrate using cutoffs, tpr, or fpr.

I'm stumped on how to incorporate the cutoff values into the roc function. Has anyone used cutoff values in the AUC package before? I know it can be done with other packages, but would like to stick within this package if possible since my data and code is already set up for it.

Answer 1

It sounds like you want to compute the true positive rate (TPR) and false positive rate (FPR) corresponding to a specified probability cutoff. Consider computing the roc object for your data:

library(AUC)
predict1 <- c(0.755, 0.755, 0.937, 0.978, 0.755)
y <- c(1,1,1,0,1)
r <- roc(predict1, as.factor(y))

Given a cutoff p (I set it to 0.85 below), you can use r$cutoffs to compute the location within the outputted ROC object corresponding to your selected cutoff:

p <- 0.85
index <- max(which(r$cutoffs >= p))

Finally, you can look up the TPR and FPR at the computed location:

r$tpr[index]
# [1] 0.25
r$fpr[index]
# [1] 1

In this case we can manually confirm this result is correct: 1/4 (25%) of positive observations have predicted value of 0.85 or more, confirming the TPR of 0.25, and 1/1 (100%) of negative observations have predicted value of 0.85 or more, confirming the FPR of 1.