Read a mp4 file frame by frame into inputimage of google ML Kit

Question

I'm trying to read a mp4 file and then use google ML kit to analyzes poses and faces. I can read in the file frame by frame using ffmpegframegrabber

FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(path);            frameGrabber.start();
IplImage frame;

try {
    frame = frameGrabber.grab();
} catch (Exception e) {
    e.printStackTrace();
}

Now the google ML kit uses inputImage for everything. So the question is how do I convert frame to inputimage? Or is there a better way to do this?

I can get BufferedImage using

BufferedImage  bi = frame.getBufferedImage();

It does'nt look like Android supports Buffered Image. Also I did'nt find a way to convert BufferedImage to InputImage

Answer 1

You can try using MediaExtractor to get the byte buffer of each frame and then build InputImage from the extracted buffer. https://developer.android.com/reference/android/media/MediaExtractor