LSTM模型的随机性

Question

I have one LSTM model like below:

model = Sequential()
model.add(Conv1D(3, 32, input_shape=(60, 12)))
model.add(LSTM(units=256, return_sequences=False, dropout=0.25))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.summary() 

Each time I use the same dataset to train it, I will get a different model. Most of the time, the performance of the trained model is acceptable, but sometime is really bad. I think that there are some randomness during the training or initialization. So how can I fix everything to get same model for each training?

Answer 1

I've experienced this problem with Keras as well, it has to do with the random seed, you can fix your random seed like this before importing the Keras, so that you could get the consistent result.

import numpy as np
np.random.seed(1000)

import os
import random

os.environ['PYTHONHASHSEED'] = '0'
random.seed(12345)

# Also set the tf randomness to some fixed values like this if you need:
tf.set_random_seed(1234)

This worked for me.

Answer 2

Weights are initialized randomly in neural networks, so it is possibly to get different results by design. If you think about how backpropagation works and how the cost function is minimized, you will notice that you don´t have any guarantee that your network will find the "global minima". Fixing the seed is one idea to get reproducible results, but on the other hand you limit your network to a fixed starting position, where it probably will never reach the global minima.

A lot of complex models, especially LSTMs are unstable. You could look at convolutional approaches. I noticed, they are performing almost equally and are much more stable. https://arxiv.org/pdf/1803.01271.pdf

Answer 3

You can save it

from keras.models import load_model
model.save("lstm_model.h5")

And load it later on

model = model.load("lstm_model.h5")