如何使用 Java 在 android 中将图像提供给 TFLite 预训练的 model？

Question

I am new to android development, I need to get a pre-trained image classification model and run it with the image I choose, I managed to choose images from the gallery and retrieve the image from URI beside loading the TFLite model

I also saw tutorials about TFLite model conversions and managed to run a simple model converting C degrees to F degrees

I can't do this with an image, any help?

Android code for image selection and model loading:

import android.annotation.SuppressLint;
import android.app.Activity;
import android.content.Intent;
import android.content.res.AssetFileDescriptor;
import android.net.Uri;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;

import androidx.appcompat.app.AppCompatActivity;

import org.tensorflow.lite.Interpreter;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MainActivity extends AppCompatActivity {

    Button btnChoose;
    ImageView imageView;
    Interpreter tflite;
    TextView textView;

    String[] classes;

    private static int GALLERY_REQUEST_CODE = 35;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        imageView = findViewById(R.id.imageView);
        textView = findViewById(R.id.textView);

        btnChoose = findViewById(R.id.btnChoose);

        btnChoose.setOnClickListener(new View.OnClickListener() {
            @SuppressLint("IntentReset")
            @Override
            public void onClick(View v) {
                Intent intent=new Intent(Intent.ACTION_PICK);
                intent.setType("image/*");
                String[] mimeTypes = {"image/jpeg", "image/png"};
                intent.putExtra(Intent.EXTRA_MIME_TYPES,mimeTypes);
                startActivityForResult(intent,GALLERY_REQUEST_CODE);
            }
        });

        try{
            tflite = new Interpreter(loadModelFile());
        } catch (Exception ex){
            ex.printStackTrace();
        }

        try{
            InputStream is = getAssets().open("labels_mobilenet_quant_v1_224.txt");
            int size = is.available();
            byte[] buffer = new byte[size];
            is.read(buffer);
            is.close();
            classes = new String(buffer).split("\n");
            textView.setText(classes[1]);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }


    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);

        if (requestCode == GALLERY_REQUEST_CODE && resultCode == Activity.RESULT_OK && null != data) {
            Uri selectedImage = data.getData();
            imageView.setImageURI(selectedImage);
        }
    }

    // TODO:
    /*
    doInf Function
     */

    private MappedByteBuffer loadModelFile() throws IOException {
        AssetFileDescriptor fileDescriptor = this.getAssets().openFd("mobilenet_v1_1.0_224_quant.tflite");
        FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
        FileChannel fileChannel = inputStream.getChannel();
        long startOffset = fileDescriptor.getStartOffset();
        long declaredLength = fileDescriptor.getDeclaredLength();
        return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
    }

}

TFLite Simple model code i done before

Button btn;
TextView tvOutput;
EditText etInput;
Interpreter tflite;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    btn = findViewById(R.id.button);
    etInput = findViewById(R.id.etInput);
    tvOutput = findViewById(R.id.tvOutput);

    try{
        tflite = new Interpreter(loadModelFile());
    } catch (Exception ex){
        ex.printStackTrace();
    }

    btn.setOnClickListener(new View.OnClickListener() {
        @SuppressLint("SetTextI18n")
        @Override
        public void onClick(View v) {
            float pred = doInf(etInput.getText().toString());
            tvOutput.setText(Float.toString(pred));
        }
    });
}

public float doInf(String inputString){
    float[] inputVal = new float[1];
    inputVal[0] = Float.parseFloat(inputString);

    float[][] outputVal = new float[1][1];

    tflite.run(inputVal, outputVal);

    return outputVal[0][0];
}

private MappedByteBuffer loadModelFile() throws IOException{
    AssetFileDescriptor fileDescriptor = this.getAssets().openFd("linear.tflite");
    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
    FileChannel fileChannel = inputStream.getChannel();
    long startOffset = fileDescriptor.getStartOffset();
    long declaredLength = fileDescriptor.getDeclaredLength();
    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}

Answer 1

Seems like you are trying to integrate an image classification model of mobilenet. Try the TFLite Task library, which will take 5 lines of code to run and it encapsulate image processing and output processing automatically for you. See the instruction and the Android app example .