将 keras model 转换为量化的 tflite 丢失精度

Question

I'm trying to convert my keras model into tflite quantized model so that I can run my model on coral TPU, but the output of my keras model and tflite model are significantly different.

The red points are quantized tflite model output, and blue points are original keras model output.

这是我将 keras model 转换为量化 tflite model 的代码：

quant = True
gc.collect()
import tensorflow as tf
import numpy as np
import pathlib
print(tf.__version__)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
if quant:
    print("Converting quant....")
    sample_size = 200
    rdm_idx = np.random.choice(range(len(X_test)),sample_size)
    rep_data = tf.cast(X_train[rdm_idx], tf.float32) / 255.0
    dataset = tf.data.Dataset.from_tensor_slices(rep_data).batch(1)

    def representative_data_gen():
        for input_value in dataset.take(sample_size):
            yield [input_value]

    converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
    converter.representative_dataset = representative_data_gen
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
    converter.inference_input_type = tf.uint8
    converter.inference_output_type = tf.uint8
    tflite_model_quant = converter.convert()
    open("MaskedLandMarkDetction_MobileNetV2_quant_fromKeras_v5.tflite", "wb").write(tflite_model_quant)

    print("Write quantization tflite done.")
else:
    print("Converting normal....")
    tflite_model = converter.convert()
    open("MaskedLandMarkDetction_MobileNetV2_fromKeras.tflite", "wb").write(tflite_model)
    print("Write tflite done.") 

X_train是我的训练数据，我会将输入图像值从 0 缩放到 1 除以255. ，所以我在representative_data_gen数据生成函数中做同样的事情。

您能提供的任何帮助将不胜感激。

我用的tensorflow版本是gpu 2.2.0

Answer 1

看起来 api 的用法是正确的。 并非所有模型都保证通过训练后量化获得良好的准确性。 例如，需要更高精度或小型模型的任务可能会遭受更多损失。

对于这些更困难的任务，我们建议使用量化感知训练，它适用于 keras 模型： https://www.tensorflow.org/model_optimization/guide/quantization 。