How to find accuracy of ARIMA model?

Question

Problem description : Prediction on CPU utilization.
Approach: Used time series algorithm.

Step 1 : From Elasticsearch I collected 1000 observations and exported on Python.

Step 2 : Plotted the data and checked whether data is stationary or not.

Step 3 : Used log to convert the data into stationary form.

Step 4 : Done DF test, ACF and PACF.

Step 5 : Build ARIMA(3,0,2) model.

Step 6 : Forecast.

I built an ARIMA (3,0,2) time-series model but was unable to find the accuracy of model. Is there any command through which we can check the accuracy of model in Python?

Could you please advice if my approach was correct or not and how to find accuracy of model in Python?

Answer 1

Approach is correct or not-

I hope you would have found out best P,Q values from ACF and PACF. There are github codes in python that will do sth like Auto Arima (automatically find best parameter), so you dont have to worry about P,q values. Basically one takes P,Q values where BIC of model is least.

Pyhton code-

There are three primary metrics used to evaluate linear models. These are: Mean absolute error (MAE), Mean squared error (MSE), or Root mean squared error (RMSE).

MAE : The easiest to understand. Represents average error

MSE : Similar to MAE but noise is exaggerated and larger errors are “punished”. It is harder to interpret than MAE as it's not in base units, however, it is generally more popular.

RMSE : Most popular metric, similar to MSE, however, the result is square rooted to make it more interpretable as it's in base units. It is recommended that RMSE be used as the primary metric to interpret your model.

Below, you can see how to calculate each metric. All of them require two lists as parameters, with one being your predicted values and the other being the true values-

Answer 2

I have been doing some research on this, unfortunately,I could not find a score function with regard to statsmodels in python. I would recommend to visit this site as recommended as an answer from an earlier post .

Also, as noted in the answer "statsmodels does have performance measures for continuous dependent variables."

Hopefully some geek would find and answer and if I find anything with regard to this, I will definitely post it to the community.