在 R 中计算随机森林 Model 的 AUC

Question

I'm trying to calculate AUC on my two models Random Forest and Naive Bayes but getting the same error ""$ operator is invalid for atomic vectors". Would you have some ideas please?

Background: Target variable "Diagnosis" is non-numerical with values B and M

Here is sample code for RF model

fitControl <- trainControl(method="cv",number = 5,preProcOptions = list(thresh = 0.4),classProbs = TRUE,summaryFunction = twoClassSummary)

wdbc_model_rf <- train(Diagnosis~.,train_wdbc,method="ranger",metric="ROC",preProcess = c('center', 'scale'),trControl=fitControl)

Answer 1

Below is an example of R code that works. Please note: your interest in ROC implies there are only two classes.

Predict <- function(class_obj, newdata, Param) {

if(Param$method == 'RF') {
    Predicted_Probs         <- predict(class_obj, newdata = newdata, type = 'prob')
} else if(Param$method == 'GBM') {
    Predicted_Probs         <- predict(class_obj, newdata = newdata, type = 'response', n.trees = Param$n.trees)[,,1]
} else if(Param$method == 'SVM') {
    Predicted_Probs         <- predict(class_obj, newdata = newdata, type = 'probabilities')
} else if(Param$method == 'logit') {
    Predicted_Probs         <- predict(class_obj, newdata = newdata, type = 'response')
    Predicted_Probs         <- cbind(1 - Predicted_Probs, Predicted_Probs)
} else { 
    cat('\nPredict(): unknown classification method.')
}

Predicted_Probs[,2]

}

@@@

AUC <- function(Truth, Predicted_Probs) {

###########################################################################################################
# SETTINGS

d_Prob              <- 0.01

###########################################################################################################
# CALCULATIONS

Prob_Grid               <- seq(1, 0, -d_Prob)
NP                  <- length(Prob_Grid)
True_Positive_Rate      <- c()
False_Positive_Rate     <- c()

for(Prob_Threshold in Prob_Grid) {
    Forecast                <- as.factor( c(0, 1, 1 * (Predicted_Probs >= Prob_Threshold)) )
    levels(Forecast)            <- c('0', '1')
    Forecast                <- Forecast[-c(1,2)]
    Table               <- xtabs(~Truth + Forecast)
    False_Positive_Rate     <- c(False_Positive_Rate, Table[1,2] / (Table[1,1] + Table[1,2]))
    True_Positive_Rate      <- c(True_Positive_Rate, Table[2,2] / (Table[2,1] + Table[2,2]))
}

AUC                 <- 0

for(i in 2:NP) {
    AUC                 <- AUC + True_Positive_Rate[i] * (False_Positive_Rate[i] - False_Positive_Rate[i-1])
}

AUC

}

Please note: the code is quite generic and can be applied to many methods, like support vector machines , gradient boosting , random forests , etc. Hopefully, it is straightforward to modify the code to your needs.