Extracting values and plot Box-plot from forecast objects

Question

I made some forecast with forecast package with several models.Example of this models you can see below:

    # CODE

library(fpp2) # required for the data
library(dplyr)
library(forecast)
#HOLT WINTER
fc <- hw(subset(hyndsight,end=length(hyndsight)-35),
         damped = TRUE, seasonal="multiplicative", h=35)
autoplot(hyndsight) +
  autolayer(fc, series="HW multi damped", PI=FALSE)+
  guides(colour=guide_legend(title="Daily forecasts"))
#ETS
ets_f <- forecast(subset(hyndsight,end=length(hyndsight)-35),
        , h=35)
autoplot(hyndsight) +
  autolayer(ets_f, series="ETS", PI=FALSE)+
  guides(colour=guide_legend(title="Daily forecasts"))

So next steep is comparation between models with RMSE. Namely this package can automatically plot all this results RMSE and also (Rsquared and MAE).You can see that with code below:

#CARET
library(caret)
library(caretEnsemble)

MY_DATA111<-data.frame(uschange[,1:2])

trainControl1 <- trainControl(method="repeatedcv", number=10, repeats=3,
                             savePredictions=TRUE, classProbs=TRUE)
algorithmList1 <- c('lm', 'rpart')
set.seed(7)
models1 <- caretList(Consumption ~., data=MY_DATA111, trControl=trainControl1, methodList=algorithmList1)
results1 <- resamples(models1)
summary(results1)
dotplot(results1)

Output of this model for RMSE with 30-resamples is as fallow:

Call:
summary.resamples(object = results1)

Models: lm, rpart 
Number of resamples: 30 

RMSE 
           Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
lm    0.3624163 0.4959485 0.5582943 0.5940259 0.6851297 0.9484492    0
rpart 0.4514924 0.5046020 0.6208663 0.6332712 0.7370780 0.9759921    0

So can anybody help me how to resolve this problem and extract values from first two model from forecast package in order to plot box-plot like example of Caret package for RMSE ?

Answer 1

You can get forecast mean with ets_f$mean and calculate rmse by hand.

# Extract forecasted values
forcasted_values <- ets_f$mean
actual_values <- subset(hyndsight,start=length(hyndsight)-36)

# Example ar model
fc2 <- ar(subset(hyndsight,end=length(hyndsight)-35))
for_values2 <- forecast(fc2, h=35)$mean

# Prepare output data.frame
result <- data.frame(
  model = c("hw",  
            "ar"),
  rmse = c(sqrt(mean((forcasted_values - actual_values)^2)), 
           sqrt(mean((for_values2 - actual_values)^2)))
)

#Box plot
boxplot(result$rmse ~ result$model)