无法在 Kotlin 中获得正确的 output 的 TFLite model

Question

我使用了来自 kaggle 的 FER2013 数据集并训练了一个 CNN model。 将 model 保存为 TFLite。 使用它制作了一个 Kotlin 应用程序。 现在我无法获得正确的 output。 model 的样品 output： [0. 0. 0. 1. 0. 0. 0.] [0. 0. 0. 1. 0. 0. 0.]表示快乐

请检查 MainActivity.kt 的代码。 我是纯noobie。 非常感谢你对我的包容。

package com.example.mooddetector

import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.os.Bundle
import android.util.Log
import android.widget.Button
import android.widget.ImageView
import android.widget.TextView
import android.widget.Toast
import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AppCompatActivity
import androidx.core.content.ContextCompat
import com.example.mooddetector.databinding.ActivityMainBinding
import com.example.mooddetector.ml.MoodDetector
import org.tensorflow.lite.DataType
import org.tensorflow.lite.support.image.ColorSpaceType
import org.tensorflow.lite.support.image.TensorImage
import org.tensorflow.lite.support.image.ops.TransformToGrayscaleOp
import org.tensorflow.lite.support.tensorbuffer.TensorBuffer


class MainActivity : AppCompatActivity() {
    private lateinit var binding: ActivityMainBinding
    private lateinit var imageView: ImageView
    private lateinit var button: Button
    private lateinit var tvOutput: TextView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        val view = binding.root
        setContentView(view)

        imageView = binding.imageView
        button = binding.btnCaptureImage
        tvOutput = binding.tvOutput
        val buttonLoad = binding.btnLoadImage
        button.setOnClickListener {
            if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA)
                == PackageManager.PERMISSION_GRANTED
            ) {
                takePicturePreview.launch(null)
            } else {
                requestPermission.launch(android.Manifest.permission.CAMERA)
            }
        }

    }

        private val requestPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
            if (granted) {
                takePicturePreview.launch(null)
            } else {
                Toast.makeText(this, "Permission Denied.", Toast.LENGTH_SHORT).show()
            }
        }

        private val takePicturePreview = registerForActivityResult(ActivityResultContracts.TakePicturePreview()){bitmap->
             if(bitmap!=null){
                 imageView.setImageBitmap(bitmap)
                 outputGenerator(bitmap)
                 }
            }


    private fun outputGenerator(bitmap: Bitmap){
        val model = MoodDetector.newInstance(this)
        
        val newBitmap = Bitmap.createScaledBitmap(bitmap, 48, 48, true)
        val tfimage = TensorImage(DataType.FLOAT32)
        tfimage.load(newBitmap)
        val tfimagegrayscale = TransformToGrayscaleOp().apply(tfimage)
        val tensorbuffr=tfimagegrayscale.tensorBuffer
        val tensorimg = TensorImage(DataType.FLOAT32)
        tensorimg.load(tensorbuffr,ColorSpaceType.GRAYSCALE)
        val byteBuffer = tensorimg.buffer

        // Creates inputs for reference.
        val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 48, 48, 1), DataType.FLOAT32)
        inputFeature0.loadBuffer(byteBuffer)
        
        // Runs model inference and gets result.
        val outputs = model.process(inputFeature0)
        val outputFeature0 = outputs.outputFeature0AsTensorBuffer
        tvOutput.text = outputFeature0.toString()
        Log.d("TAG", outputs.toString())
        Log.d("TAG", outputFeature0.toString())
        
//        val data1 = outputFeature0.floatArray
//        Log.d("TAG2", outputFeature0.dataType.toString())
//        Log.d("TAG2", data1[0].toString())

//        val probabilityBuffer = TensorBuffer.createFixedSize(intArrayOf(1, 1001), DataType.UINT8)

        // Releases model resources if no longer used.
        model.close()

    }
}

The output of the last 2 log files is: com.example.mooddetector.ml.MoodDetector$Outputs@a04fe1 org.tensorflow.lite.support.tensorbuffer.TensorBufferFloat@ca3b548

Answer 1

TensorBufferFloat的文档列出了 2 个可能对您有用的方法

浮动[] getFloatArray()

返回存储在此缓冲区中的值的浮点数组。

浮动 getFloatValue(int absIndex)

返回给定索引处的浮点值。

这些文档适用于 Java，这意味着在 Kotlin 中，所有 getter（和 setter）都变成了正常的字段/属性访问，即.getFloatArray()变成了.floatArray

因此，如果您使用.floatArray获得整个数组，您只需将值与一些分隔符连接在一起即可获得字符串表示形式。

val output = outputFeature0.floatArray.joinToString(", ", "[", "]")

Log.d("TAG", output)

如果要控制格式，请使用DecimalFormat

val pattern = "#.0#" // rounds to 2 decimal places if needed
val locale = Locale.ENGLISH
val formatter = DecimalFormat(pattern, DecimalFormatSymbols(locale))
formatter.roundingMode = RoundingMode.HALF_EVEN // this is the default rounding mode anyway    

val output = outputFeature0.floatArray.joinToString(", ", "[", "]") { value ->
    formatter.format(value)
}

Log.d("TAG", output)

如果您需要在不同的地方进行这种格式化，您可以将逻辑移动到扩展方法中

fun FloatArray.joinToFormattedString(): String {
    val pattern = "#.0#" // rounds to 2 decimal places if needed
    val locale = Locale.ENGLISH
    val formatter = DecimalFormat(pattern, DecimalFormatSymbols(locale))
    formatter.roundingMode = RoundingMode.HALF_EVEN // this is the default rounding mode anyway
    return this.joinToString(
        separator = ", ",
        prefix = "[",
        postfix = "]",
    ) { value ->
        formatter.format(value)
    }
}

然后你可以简单地打电话

Log.d("TAG", outputFeature0.floatArray.joinToFormattedString())