这个 R 神经网络是不是过拟合了？

Question

I have built an neural network with the neuralnet package in R to predict stock prices. My model and code works well, but I am getting an accuracy around 97-99%, which makes me a bit suspicious:

Is my model overfitting?

This is the Dataset I am using (which is already scaled), and this is my original Dataset (not scaled), which I need to calc the accuracy for. This is the code to build and test the model:

normalize <- function(x) {
return ((x - min(x)) / (max(x) - min(x)))
}

nn_df <- as.data.frame(lapply(nn_df, normalize))    

nn_df_train = as.data.frame(nn_df[1:1965,]) #1965
nn_df_test = as.data.frame(nn_df[1966:2808,]) #843

# NN for Sentiment GI
nn_model <- neuralnet(GSPC.Close ~ GSPC.Open +GSPC.Low + GSPC.High + SentimentGI, data = nn_df_train, hidden=5, linear.output=TRUE, threshold=0.01)

plot(nn_model)

nn_model$result.matrix

nn_pred <- compute(nn_model, nn_df_test)
nn_pred$net.result

results <- data.frame(actual = nn_df_test$GSPC.Close, prediction = nn_pred$net.result)
results

#calc accuracy
predicted = results$prediction * abs(diff(range(nn_org$GSPC.Close))) + min(nn_org$GSPC.Close)
actual = results$actual * abs(diff(range(nn_org$GSPC.Close))) + min(nn_org$GSPC.Close)
comparison = data.frame(predicted,actual)
#deviation=((actual-predicted)/actual)
deviation= abs((actual-predicted)/actual)
comparison=data.frame(predicted,actual,deviation)
accuracy=1-abs(mean(deviation))
accuracy

Answer 1

I would say that there's only a risk of overfitting if:

• the model has been trained on several iterations, for which every iteration shuffles the content of the training and test groups
• the model had been optimized after the parameters have been tweaked iteratively in order to get good accuracy in your test data.

In both situations, a validation set would be necessary.

If none of the above is the case, then that accuracy is clearly more reliable. Even so, you could test the same model with an extra set of data just to confirm your results.

Edit : let me add that, although surprisingly, I got a similar accuracy in a 30-day in the future stock prediction model by only using linear regression; so I do not know whether this level of accuracy is good enough in the stock market prediction area.