Tensorflow Lite GPU Support on object detector

Question

I have read the page https://www.tensorflow.org/lite/performance/gpu And I was wondering if anyone can help me figure out how to shape my inputs to the "MobileNet SSD object detection". You only get a file: "mobile_ssd_v2_float_coco.tflite"

I've tried 320x320*3*4, because if I tried the "old" image resolution of 300x300 i got an error saying the arrays sizes didn't match, and the new size matches 320x320.

But now when I call run(input, output) the method never returns... no errors or anything :(

Does anyone have any advise, because I'm really grasping straws at the moment.

Thank you in advance.

Answer 1

You can check spec of some models by executing this commands in python How to convert output from interpreter.run in java Printing input_details results in:

[{'name': 'normalized_input_image_tensor', 'index': 306, 'shape': array([  1, 320, 320,   3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]

array shape [1, 320, 320, 3] so input must be image 320x320. Output details:

[{'name': 'raw_outputs/box_encodings', 'index': 307, 'shape': array([   1, 2034,    4], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}, {'name': 'raw_outputs/class_predictions', 'index': 308, 'shape': array([   1, 2034,   91], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0)}]

We have 2 outputs with following shape: [1, 2034, 4] and [1, 2034, 91] So prepare 2 arrays with exact size, for example in java android

private float[][][] out1;
private float[][][] out2;
out1 = new float[1][2034][4];
out2 = new float[1][2034][91];

then add them to map example java:

private Map<Integer, Object> output_map = new TreeMap<>();
output_map.put(0, out1);
output_map.put(1, out2);

and run with:

tflite.runForMultipleInputsOutputs(input_data, output_map);

tflite - interpreter, input_data - converted image

You will have data in arrays. I assume that in out1 will be bounding boxes coordinates and in out2 classes/scores? You need to do some more testing.

Example code https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/java/demo/app/src/main/java/com/example/android/tflitecamerademo Of course you need to make some changes, so it will work with object detection model.