无法在 Kotlin 中获得正确的 output 的 TFLite model

Question

I used FER2013 dataset from kaggle and trained a CNN model. Saved the model as TFLite. Made a Kotlin app using it. Now I am not able to get proper output. Sample output of the model: [0. 0. 0. 1. 0. 0. 0.] [0. 0. 0. 1. 0. 0. 0.] for happy

Please check the code for MainActivity.kt. I am pure noobie. Thank you so much for bearing with me.

package com.example.mooddetector

import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.os.Bundle
import android.util.Log
import android.widget.Button
import android.widget.ImageView
import android.widget.TextView
import android.widget.Toast
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AppCompatActivity
import androidx.core.content.ContextCompat
import com.example.mooddetector.databinding.ActivityMainBinding
import com.example.mooddetector.ml.MoodDetector
import org.tensorflow.lite.DataType
import org.tensorflow.lite.support.image.ColorSpaceType
import org.tensorflow.lite.support.image.TensorImage
import org.tensorflow.lite.support.image.ops.TransformToGrayscaleOp
import org.tensorflow.lite.support.tensorbuffer.TensorBuffer


class MainActivity : AppCompatActivity() {
    private lateinit var binding: ActivityMainBinding
    private lateinit var imageView: ImageView
    private lateinit var button: Button
    private lateinit var tvOutput: TextView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        val view = binding.root
        setContentView(view)

        imageView = binding.imageView
        button = binding.btnCaptureImage
        tvOutput = binding.tvOutput
        val buttonLoad = binding.btnLoadImage
        button.setOnClickListener {
            if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)
                == PackageManager.PERMISSION_GRANTED
            ) {
                takePicturePreview.launch(null)
            } else {
                requestPermission.launch(android.Manifest.permission.CAMERA)
            }
        }

    }

        private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
            if (granted) {
                takePicturePreview.launch(null)
            } else {
                Toast.makeText(this, "Permission Denied.", Toast.LENGTH_SHORT).show()
            }
        }

        private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->
             if(bitmap!=null){
                 imageView.setImageBitmap(bitmap)
                 outputGenerator(bitmap)
                 }
            }


    private fun outputGenerator(bitmap: Bitmap){
        val model = MoodDetector.newInstance(this)
        
        val newBitmap = Bitmap.createScaledBitmap(bitmap, 48, 48, true)
        val tfimage = TensorImage(DataType.FLOAT32)
        tfimage.load(newBitmap)
        val tfimagegrayscale = TransformToGrayscaleOp().apply(tfimage)
        val tensorbuffr=tfimagegrayscale.tensorBuffer
        val tensorimg = TensorImage(DataType.FLOAT32)
        tensorimg.load(tensorbuffr,ColorSpaceType.GRAYSCALE)
        val byteBuffer = tensorimg.buffer

        // Creates inputs for reference.
        val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 48, 48, 1), DataType.FLOAT32)
        inputFeature0.loadBuffer(byteBuffer)
        
        // Runs model inference and gets result.
        val outputs = model.process(inputFeature0)
        val outputFeature0 = outputs.outputFeature0AsTensorBuffer
        tvOutput.text = outputFeature0.toString()
        Log.d("TAG", outputs.toString())
        Log.d("TAG", outputFeature0.toString())
        
//        val data1 = outputFeature0.floatArray
//        Log.d("TAG2", outputFeature0.dataType.toString())
//        Log.d("TAG2", data1[0].toString())

//        val probabilityBuffer = TensorBuffer.createFixedSize(intArrayOf(1, 1001), DataType.UINT8)

        // Releases model resources if no longer used.
        model.close()

    }
}

The output of the last 2 log files is: com.example.mooddetector.ml.MoodDetector$Outputs@a04fe1 org.tensorflow.lite.support.tensorbuffer.TensorBufferFloat@ca3b548

Answer 1

The docs for TensorBufferFloat list 2 methods that might be useful to you

float[] getFloatArray()

Returns a float array of the values stored in this buffer.

float getFloatValue(int absIndex)

Returns a float value at a given index.

The docs are for Java, which means that in Kotlin all getters (and setters) just become normal field/property access, ie .getFloatArray() becomes just .floatArray

So if you get the whole array with .floatArray you can just join the values together with some separator to get a string representation.

val output = outputFeature0.floatArray.joinToString(", ", "[", "]")

Log.d("TAG", output)

If you want to control the formatting use the DecimalFormat

val pattern = "#.0#" // rounds to 2 decimal places if needed
val locale = Locale.ENGLISH
val formatter = DecimalFormat(pattern, DecimalFormatSymbols(locale))
formatter.roundingMode = RoundingMode.HALF_EVEN // this is the default rounding mode anyway    

val output = outputFeature0.floatArray.joinToString(", ", "[", "]") { value ->
    formatter.format(value)
}

Log.d("TAG", output)

If you need to do this formatting in different places you can move the logic into an extension method

fun FloatArray.joinToFormattedString(): String {
    val pattern = "#.0#" // rounds to 2 decimal places if needed
    val locale = Locale.ENGLISH
    val formatter = DecimalFormat(pattern, DecimalFormatSymbols(locale))
    formatter.roundingMode = RoundingMode.HALF_EVEN // this is the default rounding mode anyway
    return this.joinToString(
        separator = ", ",
        prefix = "[",
        postfix = "]",
    ) { value ->
        formatter.format(value)
    }
}

Then you can simply call

Log.d("TAG", outputFeature0.floatArray.joinToFormattedString())