4000 个 Epoch 后损失增加

Question

I am trying to train a RNN Network for stock price prediction for my Master Thesis. I have additional input values (6), not just the stock prices by itself.

Using an LSTM Network with the "optimal" structure based on Hyperparameter Tuning with Keras Tuner, i observed a significant increase in the losses for training and validation in my case after 4000 Epochs.

My dataset consists of about 12 000 datapoints and i use the Adam optimizer with mean_absolute_error loss function

The Network is quite deep with the following layers:

LSTM (24 unit)
Dropout
LSTM (366 unit)
Dropout
LSTM (150 unit)
Dropout
Dense (1 unit)

I attached a graph of the loss (sorry for the german)

I would really like to understand what leads to this (for me) unexpected behaviour.

Answer 1

Your learning rate (often called alpha) is probably too large. Try to factor the learning rate down and try again.

Answer 2

it may be because of overfitting to the training data. if your setup for training uses a validation set for computing the accuracy, after a number of epochs the model becomes overfit on the training data which means it does very well on the training data but it won't generalize well on verification and test data. to solve this issue you could use regularization techniques or the easier approach is to just stop training when the accuracy starts to drop.