R using my own model in RFE(recursive feature elimination) to pick important feature

Question

Using RFE, you can get a importance rank of the features, but right now I can only use the model and parameter inner the package like: lmFuncs(linear model),rfFuncs(random forest) it seems that

caretFuncs

can do some custom settings for your own model and parameter,but I don't know the details and the formal document didn't give detail, I want to apply svm and gbm to this RFE process,because this is the current model I used to train, anyone has any idea?

Answer 1

I tried to recreate working example based on the documentation. You correctly identified use of caretFuncs , you can then set your model parameters in rfe call (you can also define trainControl object etc).

# load caret
library(caret)

# load data, get target and feature column labels
data(iris)
col_names = names(iris);target = "Species"
feature_names = col_names[col_names!=target]

# construct rfeControl object
rfe_control = rfeControl(functions = caretFuncs, #caretFuncs here
                     method="cv",
                     number=5)

# construct trainControl object for your train method 
fit_control = trainControl(classProbs=T,
                        search="random")

# get results
rfe_fit = rfe(iris[,feature_names], iris[,target],
             sizes = 1:4,
             rfeControl = rfe_control,
             method="svmLinear",
             # additional arguments to train method here
             trControl=fit_control)

If you want to dive deeper into the matter you might want to visit links below.

rfe documentation with basic code snippets:
https://www.rdocumentation.org/packages/caret/versions/6.0-80/topics/rfe

caret documentation on rfe :
https://topepo.github.io/caret/recursive-feature-elimination.html

Hope this helps!