[英]Coral Edge TPU Compiler cannot convert tflite model: Model not quantized
I am trying to deploy a simple test application with TensorFlow lite.我正在尝试使用 TensorFlow lite 部署一个简单的测试应用程序。 I want to use the Coral Edge TPU Stick on my device, so I have to perform Quantization Aware Training.我想在我的设备上使用 Coral Edge TPU Stick,所以我必须执行量化感知训练。 I want to fit a function f(x) = 2 x - 1
.我想拟合一个函数f(x) = 2 x - 1
。 My training code looks like this:我的训练代码如下所示:
import tensorflow as tf
import numpy as np
from tensorflow import keras
from tensorflow.contrib import lite
# Create model
model = keras.models.Sequential([keras.layers.Dense(units=1, input_shape=[1])])
# Quantization aware training
sess = keras.backend.get_session()
tf.contrib.quantize.create_training_graph(sess.graph)
sess.run(tf.global_variables_initializer())
tf.summary.FileWriter('logs/', graph=sess.graph)
model.compile(optimizer='sgd', loss='mean_squared_error')
# Training data
xs = np.array([ -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = np.array([ -3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)
model.fit(xs, ys, epochs=500, batch_size=2)
# Test the model for plausbility
print(model.predict([10.0]))
# Display the quantization-relevant variables
for node in sess.graph.as_graph_def().node:
if 'weights_quant/AssignMaxLast' in node.name \
or 'weights_quant/AssignMinLast' in node.name:
tensor = sess.graph.get_tensor_by_name(node.name + ':0')
print('{} = {}'.format(node.name, sess.run(tensor)))
# Save the keras model
keras_file = 'quant_linear.h5'
keras.models.save_model(model, keras_file)
# Convert the keras model into a tflite model
converter = lite.TocoConverter.from_keras_model_file(keras_file)
converter.post_training_quantize = True
tflite_model = converter.convert()
open('quant_linear.tflite', 'wb').write(tflite_model)
As output, I get (keras and CUDA specific output is omitted):作为输出,我得到(省略了 keras 和 CUDA 特定输出):
[[18.86733]]
dense/weights_quant/AssignMinLast = 0.0
dense/weights_quant/AssignMaxLast = 1.984399676322937
Two things to note here: the model is plausible, it should output a value close to 19. Obviously, it also uses quantized weights.这里需要注意两点:模型是合理的,它应该输出接近 19 的值。显然,它也使用了量化权重。 If I do not enable quantization aware training, the two variables won't show up.如果我不启用量化感知训练,这两个变量将不会出现。
Additionally, this model can be loaded and executed by a tf-lite interpreter instance.此外,该模型可以由 tf-lite 解释器实例加载和执行。 To be able to use it with TPU support, however, I have to convert it with the tpuedge_compiler
.但是,为了能够在 TPU 支持下使用它,我必须使用tpuedge_compiler
进行转换。 After installing it, I execute安装后,我执行
edgetpu_compiler quant_linear.tflite
Unfortunately, it seems to be unable to recognize that the model is quantized.不幸的是,它似乎无法识别模型是量化的。 It outputs它输出
user@ubuntu:~/TensorFlow$ edgetpu_compiler quant_linear.tflite
Edge TPU Compiler version 1.0.249710469
INFO: Initialized TensorFlow Lite runtime.
Invalid model: quant_linear.tflite
Model not quantized
I have tried to compile it online, which also fails.我试过在线编译它,也失败了。 Is this a bug or did I mess something up during training/converting?这是一个错误还是我在训练/转换过程中搞砸了什么? Also, maybe there is tool to verify that I really use a quantized model?另外,也许有工具可以验证我是否真的使用了量化模型?
Thanks!谢谢!
You have to use explicit quantization during the TFLite conversion AFAIK.您必须在 TFLite 转换 AFAIK 期间使用显式量化。 Code example which quantizes a Keras model:量化 Keras 模型的代码示例:
dataset = tf.data.Dataset(...)
def generator():
for item in dataset:
image = # get image from dataset item
yield [np.array([image.astype(np.float32)])]
converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = tf.lite.RepresentativeDataset(generator)
model = converter.convert()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.