TensorFlow Lite: toco_convert for arbitrary sized input tensor

Question

Looking at converting my TensorFlow model to the Flatbuf format ( .tflite ).

However, my model allows input of arbitrary size, ie you can classify one item, or N items at once. When I try to convert, it throws an error since one of my input/output devices is of type NoneType .

Think of something like theTensorFlow MNIST tutorial , where in the computation graph, our input x is of shape [None, 784] .

From the tflite dev guide , you can convert your model to FlatBuf like so:

import tensorflow as tf

img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3))
val = img + tf.constant([1., 2., 3.]) + tf.constant([1., 4., 4.])
out = tf.identity(val, name="out")

with tf.Session() as sess:
  tflite_model = tf.contrib.lite.toco_convert(sess.graph_def, [img], [out])
  open("converteds_model.tflite", "wb").write(tflite_model)

However, this does not work for me. A MWE could be:

import tensorflow as tf

img = tf.placeholder(name="inputs", dtype=tf.float32, shape=(None, 784))
out = tf.identity(inputs, name="out")


with tf.Session() as sess:
  tflite_model = tf.contrib.lite.toco_convert(sess.graph_def, [img], [out])
  open("converteds_model.tflite", "wb").write(tflite_model)

Error: TypeError: __int__ returned non-int (type NoneType)

Looking at the tf.contrib.lite.toco_convert docs, we have "input_tensors: List of input tensors. Type and shape are computed using foo.get_shape() and foo.dtype.". So that's where our failure likely is. But I'm not sure if there's an argument I should be using or something that would allow me to export a model like this

Answer 1

This problem is already resolved in the newest converter code. You can pass an input tensor where the 1st dimension is None (the 1st dimension is usually batch), and the converter will handle it correctly.

BTW, before invoking the interpreter, you can call interpreter.resize_tensor_input to change the batch size.