随机森林预测模型

Question

I am trying to build a random forest model for prices prediction problem. I have went through the following steps:

1) split the data into 3 sets train, test and valid (it is required to split into 3 sets not only train and test)

set.seed(1234)
assignment <- sample(1:3, size = nrow(train), prob = c(0.7, 0.15, 0.15), replace = TRUE) 
#Create a train, validation and tests from the train data
train_train <- train[assignment == 1, ]  
train_valid <- train[assignment == 2, ]  
train_test <- train[assignment == 3, ] 

2) I have built the model with x and y being from the train set

fit_rf_train <- train(x = train_train[, -which(names(train_train) %in% 
c("Item_Identifier", "Item_Outlet_Sales"))], 
                y = train_train$Item_Outlet_Sales,
                method = "ranger",
                metric = "RMSE",
                tuneGrid = expand.grid(
                  .mtry = 6,
                  .splitrule = "variance",
                  .min.node.size = c(10,15,20)),
                trControl = trControl,
                importance = "permutation",
                num.trees = 350)

I have the following screenshot for model output on the same train data:

3) Using predict function I used the model with the two other data sets, valid and test using the following line of code:

prediction_test <- predict(fit_rf_train, train_test)
prediction_valid <- predict(fit_rf_train, train_valid)

My question is how can I measure the performance of the model on the un seen data (test and valid)?

Answer 1

If you want to stick with caret , then you can do the following:

library(caret)
trainda<-createDataPartition(iris$Sepal.Length,p=0.8,list=F)
valid_da<-iris[-trainda,]
trainda<-iris[trainda,]
ctrl<-trainControl(method="cv",number=5)
set.seed(233)
m<-train(Sepal.Length~.,data=trainda,method="rf",metric="RMSE",trControl = ctrl,verbose=F)
m1<-predict(m,valid_da)
RMSE(m1,valid_da$Sepal.Length)

Result:

[1] 0.3499783