How to find Out of bag error in train() method=“treebag”

Question

How can I extract out of bag error from train() function with method="treebag".

control=trainControl(method="cv",number=10) bag=train(X_train,as.factor(y_train),method="treebag",trControl=control,verbose=F)

Came across an option 'coob' in 'ipred' package to get out of bag error. Please assist.

Is below step is the right method to find OOB? oob=table(y_train,predict(btree$finalModel,X_train,OOB=T)) sum(diag(as.matrix(oob)))/nrow(X_train)

Answer 1

There is some built-in code for a few different models (so that you can tune with trainControl(method = "oob") ).

Note that you'll need the keepX option set:

> library(caret)
> 
> set.seed(422)
> dat <- twoClassSim(100)
> 
> mod <- train(Class ~ ., data = dat, method = "treebag",
+              trControl = trainControl(method = "none"),
+              # you'll need this to bass to the bagging function
+              keepX = TRUE)
> 
> tb_code <- getModelInfo("treebag")[[1]]
> tb_code$oob(mod$finalModel)
  Accuracy      Kappa AccuracySD    KappaSD 
0.72787041 0.45005686 0.08011663 0.16212862