在 Android Studio 中使用 OpenCV 和 Tensorflow 进行实时情绪检测

Question

I am trying to build an Android app with Android Studio(Java), that based on a tensorflow model(pretrained in python) is able to classify the emotion of the person standing in front of the device's camera. My app needs to take the frame, apply some opencv operations like facedetection, grayscale, resize, crop, and histogram equlisation, and after that, giving the result to the model, it will predict the emotion. I have an activity that is showing the camera, but i dont know how to do all this frame manipulation and prediction in the background, when on the phone's screen the preview of the camera and the tags of the predictions, are the only things showing.

I managed to do this on PC very easly, running on the max FPS of the webcam, so i think it should be possible on the phone too.

Is there any similar project that i can get inspiration from? or does anyone know a bit of documentation or a way to do something like this?

Answer 1

I would suggest checking out the Google ML Vision API's, they can easily be integrated into an Android App and are very accurate. There are many API's like the image labeller and facial labeller. Just check them out here: https://developers.google.com/ml-kit/vision/face-detection/android

Hope I could help!