Convert Keras model to quantized Tensorflow Lite model that can be used on Edge TPU

Question

I have a Keras model that I want to run on the Coral Edge TPU device. To do this, it needs to be a Tensorflow Lite model with full integer quantization. I was able to convert the model to a TFLite model:

model.save('keras_model.h5')

converter = tf.lite.TFLiteConverter.from_keras_model_file("keras_model.h5")
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

But when I run edgetpu_compiler converted_model.tflite , I get this error:

Edge TPU Compiler version 2.0.267685300
Invalid model: converted_model.tflite
Model not quantized

This is because I need to quantize the model, but I'm not sure how to do that. I found this page which tells me how to do this, but it wants me to make an input data generator. This is the example it provides:

def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [input]

How can I adapt this code to work with my input data? Where does num_calibration_steps come from? Is there a better way to do this? (I saw references to tf.contrib.tpu.keras_to_tpu_model but it has been deprecated)

Answer 1

I believe num_calibration_steps is just the number of times the converter uses your rep set to determine the quantization levels. Just a guess, but maybe it subsamples from your rep set multiple times (bootstrapping or jackknifing). I'm still investigating the whole process myself, but it seems to work for me if I just pass it a single image for each yield, and use num_calibration_steps at about 100 (eg, 100 representative images). You can see my demo script on github .

The key part is:

image_shape = (56, 56, 32)

def representative_dataset_gen():
    num_calibration_images = 10
    for i in range(num_calibration_images):
        image = tf.random.normal([1] + list(image_shape))
        yield [image]

Also see my similar response to this question: Post-training full integer quantization of Keras model