无法使用 Keras CNN Model 获得可重现的结果

Question

I have read several other posts about how to get reproducible results using tensorflow/keras. However, I am still getting varying results.

The additional random components of my script include a sklearn.train_test_split() which I set a random_state = 123456 and also a ImageDataGenerator from Keras for data augmentation. I am setting the seed there as well.

Below is the code and where I set the seed. I am running my model for 1 epoch to compare the accuracies and they are different every time. I am setting the random seeds at the top of my code and then within the functions I use the random state. If I don't utilize the random_state in sklearn.model_selection.train_test_split or seed in keras.preprocessing.image.ImageDataGenerator I also do not get the same results.

# seeding necessary for neural network reproducibility at top of the code
SEED = 123456
import os
import random as rn
import numpy as np

os.environ['PYTHONHASHSEED']=str(SEED)
np.random.seed(SEED)
tf.random.set_seed(SEED)
rn.seed(SEED)

# within another function
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size = validation_size, random_state = 123456)

# in the model building function
data_generator = ImageDataGenerator(
    rotation_range = 15,
    width_shift_range = 0.1,
    height_shift_range = 0.1,
    horizontal_flip = True
    #zoom_range = [0.5, 1.0] # half zoom to double zoom possibilities
)
data_generator.fit(xtrain, seed = 123456)

Answer 1

After some trial and error. I found there is a library for NVIDIA GPU (which I am using) that will give me deterministic results.

First I install the library with pip install tensorflow-determinism Then set os environment variable:

os.environ['TF_DETERMINISTIC_OPS'] = '1' . I am now getting reproducible results:

Test error: 1.6772334575653076, test accuracy: 0.5085999965667725

Test error: 1.6772334575653076, test accuracy: 0.5085999965667725

Answer 2

I think your problem are how the keras model parameters are initialized. You can specify your own initializers to get reproducible behaviour as documented here