Different model on speech recognition

Question

I have tried so much time to run this example project https://github.com/tensorflow/examples/tree/master/lite/examples/speech_commands/ml and finally produced this tflite model https://imgur.com/bVpesdd using convert_keras_lite.py inside export direcory.However, i checked the tflite model inside assets directory in this android project https://github.com/tensorflow/examples/tree/master/lite/examples/speech_commands/android . I found that the tflite model is different with the first one. https://imgur.com/7Cn69qx .

I tried to replace the tflite model inside assets android directory to the first tflite model, but the app suddenly crashed with this error code in Android Studio logcat:

2019-12-22 16:45:22.262 795-795/org.tensorflow.lite.examples.speech E/AndroidRuntime: FATAL EXCEPTION: main
    Process: org.tensorflow.lite.examples.speech, PID: 795
    java.lang.RuntimeException: Unable to start activity ComponentInfo{org.tensorflow.lite.examples.speech/org.tensorflow.lite.examples.speech.SpeechActivity}: java.lang.IllegalArgumentException: Input error: Can not resize 1-th input for a model having 1 inputs.
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3139)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3282)
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1970)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7156)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:494)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:975)
     Caused by: java.lang.IllegalArgumentException: Input error: Can not resize 1-th input for a model having 1 inputs.
        at org.tensorflow.lite.NativeInterpreterWrapper.resizeInput(Native Method)
        at org.tensorflow.lite.NativeInterpreterWrapper.resizeInput(NativeInterpreterWrapper.java:194)
        at org.tensorflow.lite.Interpreter.resizeInput(Interpreter.java:285)
        at org.tensorflow.lite.examples.speech.SpeechActivity.onCreate(SpeechActivity.java:190)
        at android.app.Activity.performCreate(Activity.java:7335)
        at android.app.Activity.performCreate(Activity.java:7326)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1275)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3119)**

Any help is appreciated.

Answer 1

Okay I did not solve the problem yet but I think I discovered some things that may help:

Firstly, there is other code for a speech command example that is in the TensorFlow repo itself (not the dedicated example repo): https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/speech_commands

It does not use keras like the one you are using and has no information about tflite conversion, so I guess that example was created before tflite was around. After looking at the architecture of the (conv) model that was used there, I found out that it is the same architecture as the prepackaged tflite model in the android project and it matches the second imgur link you posted.

So it pretty much looks like the tflite model in the android project is a conversion of the model in the old example. And the model that results from https://github.com/tensorflow/examples/tree/master/lite/examples/speech_commands/ml is not compatible with the android code at all, since the inputs are different.

And that is not surprising since the android code is pretty much copy paste from the old examples android code with a few tweaks for tflite usage.

So I guess the best take is to work on the old ml code and convert the resulting frozen pb graph to tflite.

Update

I did it!!! Here's how it works:

• Download the frozen graph fromhere .
• Run the following in python

import tensorflow as tf

converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph("./conv_actions_frozen.pb", input_arrays=['decoded_sample_data', 'decoded_sample_data:1'], output_arrays=['labels_softmax'])
converter.allow_custom_ops=True
tflite_model = converter.convert()
open("model.tflite", "wb").write(model)

If you view the model with netron you'll see it's identical to the model that comes with the android example.