Error converting Facenet model .pb file to TFLITE format

Question

i'm trying to convert a pre-trained frozen .pb based on Inception ResNet i got from David Sandbergs Github with the Tensorflow Lite Converter on Ubuntu using the following command:

/home/nils/.local/bin/tflite_convert
--output_file=/home/nils/Documents/frozen.tflite
--graph_def_file=/home/nils/Documents/20180402-114759/20180402-114759.pb 
--input_arrays=input 
--output_arrays=embeddings 
--input_shapes=1,160,160,3

However, i get the following error:

2018-12-03 15:03:16.807431: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "/home/nils/.local/bin/tflite_convert", line 11, in <module>
sys.exit(main())
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 412, in main
app.run(main=run_main, argv=sys.argv[:1])
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 408, in run_main
_convert_model(tflite_flags)
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/contrib/lite/python/tflite_convert.py", line 162, in _convert_model
output_data = converter.convert()
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/contrib/lite/python/lite.py", line 453, in convert
**converter_kwargs)
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/contrib/lite/python/convert.py", line 342, in toco_convert_impl
input_data.SerializeToString())
File "/home/nils/.local/lib/python3.6/site-packages/tensorflow/contrib/lite/python/convert.py", line 135, in toco_convert_protos
(stdout, stderr))
RuntimeError: TOCO failed see console for info.
b'2018-12-03 15:03:26.006252: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: FIFOQueueV2\n2018-12-03 15:03:26.006322: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: batch_join/fifo_queue\n2018-12-03 15:03:26.006339: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: QueueDequeueUpToV2\n2018-12-03 15:03:26.006352: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: batch_join\n2018-12-03 15:03:27.496676: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 5601 operators, 9399 arrays (0 quantized)\n2018-12-03 15:03:28.603936: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After Removing unused ops pass 1: 3578 operators, 6254 arrays (0 quantized)\n2018-12-03 15:03:29.418074: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 3578 operators, 6254 arrays (0 quantized)\n2018-12-03 15:03:29.420354: F tensorflow/contrib/lite/toco/graph_transformations/resolve_batch_normalization.cc:42] 
Check failed: IsConstantParameterArray(*model, bn_op->inputs[1]) && IsConstantParameterArray(*model, bn_op->inputs[2]) && IsConstantParameterArray(*model, bn_op->inputs[3]) Batch normalization resolution requires that mean, multiplier and offset arrays be constant.\nAborted (core dumped)\n'
None

If i get this right, this might be because of two unsupported Ops, QueueDequeueUpToV2 and FIFOQueueV2, but i don't know for sure. Do you have any ideas what might be the problem or how i can solve this error? What does that error even mean? I want this model to run on a mobile android device, are there any alternatives? Versions: Tensorflow V1.12 Python 3.6.7 Ubuntu 18.04.1 LTS on a VirtualBox Thanks in advance!

Answer 1

I have solved this problem here , adding the snippet here too:

I could able to convert FaceNet .pb to .tflite model, and following are the instructions to do so:

We will quantise pre-trained Facenet model with 512 embedding size. This model is about 95MB in size before quantization.

$ ls -l model_pc
total 461248
-rw-rw-r--@ 1 milinddeore  staff   95745767 Apr  9  2018 20180402-114759.pb

create a file inference_graph.py with following code:

import tensorflow as tf
from src.models import inception_resnet_v1
import sys
import click
from pathlib import Path

@click.command()
@click.argument('training_checkpoint_dir', type=click.Path(exists=True, file_okay=False, resolve_path=True))
@click.argument('eval_checkpoint_dir', type=click.Path(exists=True, file_okay=False, resolve_path=True))

def main(training_checkpoint_dir, eval_checkpoint_dir):
    traning_checkpoint = Path(training_checkpoint_dir) / "model-20180402-114759.ckpt-275"
    eval_checkpoint = Path(eval_checkpoint_dir) / "imagenet_facenet.ckpt"
    data_input = tf.placeholder(name='input', dtype=tf.float32, shape=[None, 160, 160, 3])
    output, _ = inception_resnet_v1.inference(data_input, keep_probability=0.8, phase_train=False, bottleneck_layer_size=512)
    label_batch= tf.identity(output, name='label_batch')
    embeddings = tf.identity(output, name='embeddings')
    init = tf.global_variables_initializer()
    with tf.Session() as sess:
        sess.run(init)
        saver = tf.train.Saver()
        saver.restore(sess, traning_checkpoint.as_posix())
        save_path = saver.save(sess, eval_checkpoint.as_posix())
        print("Model saved in file: %s" % save_path)

if __name__ == "__main__":
     main()

Run this file on pre-trained model, would generate model for inference. Download pre-trained model and unzip it to model_pre_trained/ directory. Make sure you have python ≥ 3.4 version.

python3 eval_graph.py model_pre_trained/ model_inference/

FaceNet provides freeze_graph.py file, which we will use to freeze the inference model.

python3  src/freeze_graph.py model_inference/  my_facenet.pb

Once the frozen model is generated, time to convert it to .tflite

$ tflite_convert --output_file model_mobile/my_facenet.tflite --graph_def_file my_facenet.pb  --input_arrays "input" --input_shapes "1,160,160,3" --output_arrays embeddings --output_format TFLITE --mean_values 128 --std_dev_values 128 --default_ranges_min 0  --default_ranges_max 6 --inference_type QUANTIZED_UINT8 --inference_input_type QUANTIZED_UINT8

Let us check the quantized model size:

$ ls -l model_mobile/
total 47232
-rw-r--r--@ 1 milinddeore  staff  23667888 Feb 25 13:39 my_facenet.tflite

Interpeter code:

 import numpy as np
 import tensorflow as tf


 # Load TFLite model and allocate tensors.
 interpreter = tf.lite.Interpreter(model_path="/Users/milinddeore/facenet/model_mobile/my_facenet.tflite")
 interpreter.allocate_tensors()

 # Get input and output tensors.
 input_details = interpreter.get_input_details()
 output_details = interpreter.get_output_details()

 # Test model on random input data.
 input_shape = input_details[0]['shape']
 input_data = np.array(np.random.random_sample(input_shape), dtype=np.uint8)
 interpreter.set_tensor(input_details[0]['index'], input_data)

 interpreter.invoke()
 output_data = interpreter.get_tensor(output_details[0]['index'])

 print('INPUTS: ')
 print(input_details)
 print('OUTPUTS: ')
 print(output_details)

Interpeter output:

$ python inout.py
INPUTS:
[{'index': 451, 'shape': array([  1, 160, 160,   3], dtype=int32), 'quantization': (0.0078125, 128L), 'name': 'input', 'dtype': <type 'numpy.uint8'>}]
OUTPUTS:
[{'index': 450, 'shape': array([  1, 512], dtype=int32), 'quantization': (0.0235294122248888, 0L), 'name': 'embeddings', 'dtype': <type 'numpy.uint8'>}]

Hope this helps!

Answer 2

I had no luck with @milind-deore's suggestions. The model does reduce to 23 MB but the embeedings seems to be broken.

I found an alternative way : TF -> Keras -> TF Lite

David Sandberg's FaceNet implementation can be converted to TensorFlow Lite, first converting from TensorFlow to Keras, and then from Keras to TensorFlow Lite .

I created this Google Colab that does the conversion. Most part of the code was taken from here .

What it does is as follows:

1. Download Hiroki Taniai's Keras FaceNet implementation
2. Override the inception_resnet_v1.py file with my patched version (which does adds an extra layer to the model to have normalized embeedings as output)
3. Download Sandberg's pre-trained model (20180402-114759) from here , and unzips it
4. Extract the tensors from the checkpoint file and writes the weights to numpy arrays on disk, mapping the name of each corresponding layer.
5. Create a new Keras model with random weights (Important: using 512 classes).
6. Write the weights for each corresponding layer reading from the numpy arrays.
7. Store the model with the Keras format .h5

8. Convert Keras to TensorFlow Lite using the command "tflite_convert".

   tflite_convert --post_training_quantize --output_file facenet.tflite --keras_model_file /content/keras-facenet/model/keras/model/facenet_keras.h5

Also in my Colab I provide some code to show that the conversion is good, and the TFLite model does work.

distance bill vs bill 0.7266881 distance bill vs larry 1.2134411

So even though I'm not aligning the faces, a threshold of about 1.2 would be good to the recognition.

Hope it helps!