Python: Keras model returns different results for the same data and same model

Question

Last few hours, I have been trying to make my first model for image classification. For this purpose, I have used Image classification from scratch tutorial. As I followed the steps I managed to reach the end of the tutorial.

The only differences I made compared to the code in tutorial are:

• in order to make training process faster, I changed number of epochs from 50 to 10,
• I deleted Image augmentation block from make_model function (one row of code).

Now, I am getting to what my problem is. At the end, when I try to get the prediction results for the same data and the same model again, the results are different. Look at this simple code:

>>> for i in range(5):
...     predictions = model.predict(val_ds)
...     predictions_list = [round(pred[0], 3) for pred in predictions]
...     print(predictions_list[:10])

and the result:

[0.937, 0.905, 1.0, 0.094, 0.021, 0.095, 0.07, 0.006, 1.0, 1.0]
[0.905, 1.0, 1.0, 1.0, 1.0, 1.0, 0.122, 1.0, 1.0, 0.0]
[0.996, 0.003, 1.0, 0.887, 1.0, 1.0, 0.798, 1.0, 1.0, 1.0]
[1.0, 1.0, 0.819, 0.999, 1.0, 0.887, 0.087, 1.0, 0.914, 1.0]
[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.916, 0.102]

I assume, that results can be different only in case I retrain the model. But that is not my case! I only rerun .predict() method. So, my question is - can you help me, what I am doing wrong, please? Am I missing something?

Answer 1

Can you try with the code below, please?

>>> val_item = val_ds.take(1)
>>> for i in range(5):
...     predictions = model.predict(val_item)
...     predictions_list = [round(pred[0], 3) for pred in predictions]
...     print(predictions_list[:10])

In your code, you are using different items not the same one. You can check it by manually printing the val_ds value.

Answer 2

The problem was with reading data function tf.keras.preprocessing.image_dataset_from_directory , which has got its shuffle argument set to True .

When I reload the data again and set shuffle=False like this:

>>> val_ds = tf.keras.preprocessing.image_dataset_from_directory(
...     'PetImages',
...     shuffle=False,
...     validation_split=0.2,
...     subset="validation",
...     seed=1337,
...     image_size=image_size,
...     batch_size=batch_size,
... )
>>> for i in range(5):
...     predictions = model.predict(val_ds)
...     predictions_list = [round(pred[0], 3) for pred in predictions]
...     print(predictions_list[:10])

then the result looks as I expected:

[0.998, 0.994, 1.0, 1.0, 0.885, 1.0, 0.998, 1.0, 0.979, 1.0]
[0.998, 0.994, 1.0, 1.0, 0.885, 1.0, 0.998, 1.0, 0.979, 1.0]
[0.998, 0.994, 1.0, 1.0, 0.885, 1.0, 0.998, 1.0, 0.979, 1.0]
[0.998, 0.994, 1.0, 1.0, 0.885, 1.0, 0.998, 1.0, 0.979, 1.0]
[0.998, 0.994, 1.0, 1.0, 0.885, 1.0, 0.998, 1.0, 0.979, 1.0]