Different predictions when running Keras model in TensorFlow Lite

Question

Trying out TensorFlow Lite with a pretrained Keras image classifier, I'm getting worse predictions after converting the H5 to the tflite format. Is this intended behaviour (eg weight quantization), a bug or am I forgetting something when using the interpreter?

Example

from imagesoup import ImageSoup
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input, decode_predictions
from tensorflow.keras.preprocessing.image import load_img, img_to_array

# Load an example image.
ImageSoup().search('terrier', n_images=1)[0].to_file('image.jpg')
i = load_img('image.jpg', target_size=(224, 224))
x = img_to_array(i)
x = x[None, ...]
x = preprocess_input(x)

# Classify image with Keras.
model = ResNet50()
y = model.predict(x)
print("Keras:", decode_predictions(y))

# Convert Keras model to TensorFlow Lite.
model.save(f'{model.name}.h5')
converter = tf.contrib.lite.TocoConverter.from_keras_model_file
tflite_model = converter(f'{model.name}.h5').convert()
with open(f'{model.name}.tflite', 'wb') as f:
    f.write(tflite_model)

# Classify image with TensorFlow Lite.
f = tf.contrib.lite.Interpreter(f'{model.name}.tflite')
f.allocate_tensors()
i = f.get_input_details()[0]
o = f.get_output_details()[0]
f.set_tensor(i['index'], x)
f.invoke()
y = f.get_tensor(o['index'])
print("TensorFlow Lite:", decode_predictions(y))

Keras: [[('n02098105', 'soft-coated_wheaten_terrier', 0.70274395), ('n02091635', 'otterhound', 0.0885325), ('n02090721', 'Irish_wolfhound', 0.06422518), ('n02093991', 'Irish_terrier', 0.040120784), ('n02111500', 'Great_Pyrenees', 0.03408164)]]

TensorFlow Lite: [[('n07753275', 'pineapple', 0.94529104), ('n03379051', 'football_helmet', 0.033994876), ('n03891332', 'parking_meter', 0.011431991), ('n04522168', 'vase', 0.0029440755), ('n02094114', 'Norfolk_terrier', 0.0022089847)]]

Answer 1

Did you make sure to set the learning phase to 0 in order to avoid Dropout and other non-deterministic layers during the prediction phase?

https://www.tensorflow.org/api_docs/python/tf/keras/backend/learning_phase

Answer 2

There was a bug in from_keras_model_file in TensorFlow 1.10. It was fixed in the August 9th nightly release in this commit .

The nightly can be installed via pip install tf-nightly . Additionally, it will be fixed in TensorFlow 1.11.