Straight line through time series with atocorrelated residuals (auto.arima in forecast package)

Question

I have a time series with autocorrelated residuals. Due to this, my thought was to use the auto.arima() function in the forecast package in R in order to find the intercept and slope.

Unfortunately, the output from the auto.arima() function results in a fitted line that is clearly incorrect.The slope could very well be correct, but the intercept is far too low.

The r code that I am using:

require(forecast)
y <- as.numeric(readClipboard())
x <- 1:length(y)
arimareg <- auto.arima(y, xreg=x, max.p=5, max.q=10, max.order = 12, stepwise=FALSE,  approximation=FALSE, seasonal=FALSE)
intercept <- unname(arimareg$coef[length(arimareg$coef)-1])
slope <- unname(arimareg$coef[length(arimareg$coef)])

Results: Intercept: 0.474869856067 Slope: 0.00539552660742

The time series that I an copying to the clipboard can be seen below:

 1.0000 0.9999 0.9993 1.0299 1.0267 1.0305 1.1427 1.1339 1.1323 1.2051 1.2091 1.2087 1.3343 1.3381 1.3361 1.3848 1.3853 1.3865 1.4207 1.4378 1.4394 1.4546 1.4506 1.4518 1.4495 1.4458 1.4461 1.4468 1.4461 1.4470 1.4645 1.4621 1.4592 1.4833 1.4773 1.4738 1.5398 1.5284 1.5302 1.5562 1.5541 1.5501 1.5740 1.5703 1.5715 1.5893 1.5874 1.5867 1.6213 1.6211 1.6175 1.6206 1.6131 1.6038 1.6034 1.6041 1.6015 1.6009 1.5940 1.5865 1.5877 1.5890 1.5885 1.6111 1.6100 1.6065 1.6241 1.6263 1.6250 1.6490 1.6565 1.6560 1.6911 1.6864 1.6837 1.7143 1.7103 1.7102 1.7298 1.7253 1.7341 1.7567 1.7512 1.7455 1.7338 1.7329 1.7276 1.7022 1.6975 1.6928 1.6454 1.6361 1.6293 1.5601 1.5591 1.5508 1.4704 1.4650 1.4655 1.4439 1.4464 1.4486 1.4669 1.4810 1.4872 1.5151 1.5191 1.5223 1.5384 1.5417 1.5449 1.5650 1.5714 1.5741 

I am sure I am doing something stupid, but unfortunately I cannot figure out what.

Any help would be greatly appreciated!

Edit: I understand what is going wrong now. If I have less than ~160 observations then there will be no intercept in the output. This means that the value I store as intercept is actually the last MA() lag coefficient.

The code above should be adjusted from:

intercept <- unname(arimareg$coef[length(arimareg$coef)-1])

To:

intercept <- unname(arimareg$coef['intercept'])

The question still remains; Why is there no intercept in the output when the number of observations is small.

Answer 1

According the the forecast developers , the auto.arima() was designed to automate inclusion on the constant:

By default, for d=0 or d=1, a constant will be included if it improves the AIC value; for d>1 the constant is always omitted.

Thus, it seems that your models may not have an intercept term and it's perfectly normal.

If you want to check that the model works properly for the data supplied you may use fitted() functon:

plot(x = x, y = fitted(arimareg), col = "blue", type = "l", lwd = 3, 
    panel.first = grid(col = "black"))
points(x = x, y = y, bg = "red", col = "darkred", pch = 21)

Which gives

Does it answer your question?