R中auto.arima的训练数据集

Question

I have around 10000 time series.

I wanted to use auto.arima function http://www.inside-r.org/packages/cran/forecast/docs/auto.arima

I wanted to test the accuracy of my auto.arima model for the 10000 time series. I am holding off 20% of data points (if you see sample out of 40 I will hold off 8) and then let auto.arima predict. Then I can compare generated 8 values with actual 8 values.
But is there a formal way to test accuracy in ARIMA model? Is my approach correcT?

y=auto.arima(x)
plot(forecast(y,h=8))

Sample time series 1

0.0003748,0.0003929,0.0003653,0.0003557,0.0004463,0.000349,0.0003099,0.0003395,0.0003157,0.0002871,0.0002604,0.0002422,0.0001917,0.0002117,0.0002689

time series 2

0.0003977,0.0003481,0.0002413,0.0002069,0.0002127,0.0002108,0.0002003,0.0002174,0.0002098,0.0002069,0.0001955,0.0001926,0.0002108,0.0002146,0.0002079

Answer 1

It sounds to me like your Q is about the different metrics for comparing forecast accuracy, more than the specific use of auto.arima() and forecast() . If so, then there are a number of metrics that can be used. For an overview, see

https://en.wikipedia.org/wiki/Forecasting#Forecasting_accuracy

Each of them has its proponents and detractors; for example, see this paper:

http://robjhyndman.com/papers/mase.pdf

Independent of what accuracy metric you use, you still need to be able to justify why you are holding back 20% of the data for forecasting.

If, however, you are interested in the different model forms, then you also have some options. For example, as suggested in the comments,

1. fit the same univariate model (specified a priori ) to each time series using arima() (or some equivalent);
2. fit a (potentially) different univariate model to each time series using auto.arima() ; or
3. fit a multivariate model to all time series.

If it's #3 you're interested in, I'd suggest the MARSS pkg here:

https://cran.r-project.org/web/packages/MARSS/index.html

and user's guide here:

https://cran.r-project.org/web/packages/MARSS/vignettes/UserGuide.pdf