Camera2 OpenGL 由 Matrix.rotateM 引起的视频随机故障

Question

I used the Grafika app and modified it to use the Camera2 API instead of the old API. I needed to use an OpenGL solution as I need to draw a watermark on top of the video, and Grafika was really useful. Unfortunately, my output videos are recording with random "flickers" of frames in the wrong orientation. I am looking to resolve the flickering issue, or at least understand why it is happening.

Originally, I managed to successfully record video, with sound, and drawing the watermark on top, but the video was in the wrong orientation as I required portrait videos. In order to achieve this, I used MediaMuxer.serOrientationHint() to configure the output file as portrait, and also applied a rotation to the transform matrix to ensure OpenGL frames were drawn in portrait, see below:

private void handleFrameAvailable(float[] transform, long timestampNanos) {

    mVideoEncoder.drainVideoEncoder(false);

    Matrix.rotateM(transform, 0, 270, 0, 0, 1); //Added these to rotate video frames
    Matrix.translateM(transform, 0, -1, 0, 0); //Added these to rotate video frames

    mFullScreen.drawFrame(mTextureId, transform);

    //...drawing of watermark happens here...//

    if (VERBOSE) { Log.e(TAG,"HandleVideo: "+timestampNanos); }
    mInputWindowSurface.setPresentationTime(timestampNanos);
    mInputWindowSurface.swapBuffers();
}

See below a regular frame and a glitched frame. In a 5 second video, around 20-30 non-consecutive frames may be like this.

Answer 1

If you aren't resetting transform to identity matrix then you are accumulating transforms on each frame. Try:

Matrix.setIndentityM(transform, m);

before apply translation and rotation.

Besides, could be orientation:

if (AppSetting.getValue(activity, Config.ORIENTATION, "").equalsIgnoreCase("Portrait")) {
    Matrix.rotateM(mTmpMatrix, 0, 270, 0, 0, 1);
    Matrix.translateM(mTmpMatrix, 0, -1, 0, 0);
}