將 TF model 轉換為 TFLite，然后再轉換為 EdgeTPU

Question

我正在嘗試使用一個簡單的 keras model 進行添加操作，然后轉換為 TFLite，然后轉換為 EdgeTPU。 需要對 int8 進行量化，但取決於提供的轉換參數，它會導致 FlexAddV2 操作不受支持，或數據類型 int32 不受支持，或 AddV2 錯誤代碼：ERROR_NEEDS_FLEX_OPS。

model 和轉換相對簡單明了：

from tensorflow import keras
import numpy as np
import random

def representative_dataset():
    for _ in range(100):
        #data = random.randint(0, 1)
        #yield [data]
        data = np.random.rand(32)*2
        yield [data.astype(np.int8)]

input = keras.Input(shape=(32,), name="dummy_input", dtype=tf.int8)
output = tf.add(input, 1)
model = keras.Model(inputs=input, outputs=output)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS_INT8, # enable TensorFlow Lite ops.
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.int8 # or tf.uint8
converter.inference_output_type = tf.int8 # or tf.uint8
converter.experimental_new_quantizer = True # It will enable conversion and quantization of MLIR ops
converter.experimental_new_converter = False
tflite_quant_model = converter.convert()

Output 從運行轉換：

Traceback (most recent call last):
File "/home/gsosnow/doc/gt2tf.py", line 27, in
tflite_quant_model = converter.convert()
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 929, in wrapper
return self._convert_and_export_metrics(convert_func, *args, **kwargs)
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 908, in _convert_and_export_metrics
result = convert_func(self, *args, **kwargs)
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1338, in convert
saved_model_convert_result = self._convert_as_saved_model()
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1320, in _convert_as_saved_model
return super(TFLiteKerasModelConverterV2,
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/lite.py", line 1131, in convert
result = _convert_graphdef(
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert_phase.py", line 212, in wrapper
raise converter_error from None # Re-throws the exception.
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper
return func(*args, **kwargs)
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert.py", line 794, in convert_graphdef
data = convert(
File "/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/lite/python/convert.py", line 311, in convert
raise converter_error
tensorflow.lite.python.convert_phase.ConverterError: /home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/save.py:1325:0: error: 'tf.AddV2' op is neither a custom op nor a flex op
:0: note: loc(fused["PartitionedCall:", "PartitionedCall"]): called from
/home/gsosnow/anaconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/save.py:1325:0: note: Error code: ERROR_NEEDS_FLEX_OPS
:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: AddV2
Details:
tf.AddV2(tensor<?x32xi8>, tensor) -> (tensor<?x32xi8>) : {device = ""}

Answer 1

這在這里解決了： https://github.com/google-coral/edgetpu/issues/655

這是完成此操作的 python 轉換代碼：

import tensorflow as tf
from tensorflow import keras
import numpy as np
import random

def representative_dataset():
  for _ in range(100):
      #data = random.randint(0, 1)
      #yield [data]
      data = np.random.rand(32)*2
      yield [data.astype(np.float32)]

input = keras.Input(shape=(32,), name="dummy_input", dtype=tf.float32)
output = tf.add(input, 1)
# output = tf.keras.layers.Add()([input, input])
model = keras.Model(inputs=input, outputs=output)
print(model.summary())
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.int8 # or tf.uint8 
converter.inference_output_type = tf.int8 # or tf.uint8 

tflite_quant_model = converter.convert()