how to use set_random_seed to get same results

Question

I am trying to learn about LSTM cells. I have read a few articles and am now trying to write my own code using tensorflow.

I came across this code from this tutorial . So I copied the lstm.py file and the data file sp500.csv. The code below is what I written.

Everything works fine. Each time I run the model I get very different predictions. I understand that because this model is very basic (trying to predict stock prices using just the closing price) this is to be expected. Also I believe in the background tensorflow uses some randomisation process to initiate the variables.

I wanted though to run the model and get the same results each time. So I read that to do this I need to use set_random_seed.

I have added this to my code below but when I re-run my code I get very different predictions. How am I supposed to use set_random_seed and why is the parameter 1234?

from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense, LSTM, Dropout, Activation
import lstm, time
import tensorflow as tf

def RunML():

    tf.set_random_seed(1234)

    X_train, y_train, X_test, y_test = lstm.load_data('sp500.csv', 50, True)

    model = Sequential()

    model.add(LSTM(50, 
               input_shape=(50, 1),
               return_sequences=True))
    model.add(Dropout(0.2))

    model.add(LSTM(
        100,
        return_sequences=False))
    model.add(Dropout(0.2))

    model.add(Dense(1))
    model.add(Activation('linear'))

    start = time.time()
    model.compile(loss='mse', optimizer='rmsprop')
    print ('compilation time : ', time.time() - start)

    #Step 3 Train the model
    model.fit(
        X_train,
        y_train,
        batch_size=512,
        nb_epoch=1,
        validation_split=0.05)

    predictions = lstm.predict_sequences_multiple(model, X_test, 50, 50)
    lstm.plot_results_multiple(predictions, y_test, 50)

Answer 1

If you set the random seed before building the graph, you will get reproducible results. For example, given a random input:

random_input = np.random.rand(2, 50, 1)

We can define one tf.keras.model :

tf.keras.backend.clear_session()
tf.set_random_seed(42)

model_1 = Sequential()

model_1.add(LSTM(50,  input_shape=(50, 1), return_sequences=True))
model_1.add(Dropout(0.2))

model_1.add(LSTM(100, return_sequences=False))
model_1.add(Dropout(0.2))

model_1.add(Dense(1))
model_1.add(Activation('linear'))

model_1.compile(loss='mse', optimizer='rmsprop')

random_predict_1 = model_1.predict(random_input)

Then another:

tf.keras.backend.clear_session()
tf.set_random_seed(42)

model_2 = Sequential()

model_2.add(LSTM(50,  input_shape=(50, 1), return_sequences=True))
model_2.add(Dropout(0.2))

model_2.add(LSTM(100, return_sequences=False))
model_2.add(Dropout(0.2))

model_2.add(Dense(1))
model_2.add(Activation('linear'))

model_2.compile(loss='mse', optimizer='rmsprop')

random_predict_2 = model_2.predict(random_input)

Now, if we perform inference on the random_input tensor, we will get the same results with both models because they had the same random seed to start.

np.testing.assert_array_equal(random_predict_1, random_predict_2)

To sum it up, no matter which random seed you use, if you set the random seed to the same number before building the graph with tf.set_random_seed(seed) , you will get the same results.