如何在Android MediaCodec中设置演示文稿时间戳？

Question

I am rendering an h.264 video on Android using the MediaCodec (and SurfaceView). Here are some of my codes.

@Override
public void surfaceCreated(SurfaceHolder holder) {

}

@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
    if (videoPlayer == null) {
        videoPlayer = new PlayerThread(holder.getSurface());
        videoPlayer.start();
    }
}

@Override
public void surfaceDestroyed(SurfaceHolder holder) {
    if (videoPlayer != null) {
        videoPlayer.interrupt();
        videoPlayer.isEOS = true;
        videoPlayer = null;
    }
}

private class PlayerThread extends Thread {
    private MediaCodec decoder;
    private Surface surface;
    private final static String mimeType = "video/avc";
    public boolean isEOS = false;

    public PlayerThread(Surface surface) {
        this.surface = surface;
    }

    @Override
    public void run() {
        MediaFormat format = MediaFormat.createVideoFormat(mimeType, frame.width, frame.height);
        decoder = MediaCodec.createDecoderByType(mimeType);
        decoder.configure(format, surface, null, 0);
        decoder.start();

        ByteBuffer[] inputBuffers = decoder.getInputBuffers();
        ByteBuffer[] outputBuffers = decoder.getOutputBuffers();

        int startPTS = 0;
        while (!Thread.interrupted() && !isEOS) {
            frame = frameReader.nextFrame();
            if (startPTS == 0) {
                startPTS = frame.pts;
            }
            int relativePTS = frame.pts - startPTS;
            int inIndex = decoder.dequeueInputBuffer(-1);
            if (inIndex >= 0)
            {
                ByteBuffer inputBuffer = inputBuffers[inIndex];
                inputBuffer.clear();
                inputBuffer.put(frame.buf, 0, frame.size);

                decoder.queueInputBuffer(inIndex, 0, frame.size, relativePTS*1000, 0);
            }

            MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();

            int outIndex = decoder.dequeueOutputBuffer(info, -1);

            switch (outIndex) {
                case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
                    Log.d("DecodeActivity", "INFO_OUTPUT_BUFFERS_CHANGED");
                    outputBuffers = decoder.getOutputBuffers();
                    break;
                case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
                    Log.d("DecodeActivity", "New format " + decoder.getOutputFormat());
                    break;
                case MediaCodec.INFO_TRY_AGAIN_LATER:
                    Log.d("DecodeActivity", "dequeueOutputBuffer timed out!");
                    break;
                default:
                    ByteBuffer outputBuffer = outputBuffers[outIndex];
                    Log.v("DecodeActivity", "We can't use this buffer but render it due to the API limit, " + outputBuffer);

                    decoder.releaseOutputBuffer(outIndex, true);
            }
        }

        /* clean */
        decoder.stop();
        decoder.release();
        decoder = null;
    }
}

I set the PTS, but it's not working. The video playback very fast.

Anyone know how to set the correct presentation timestamp in this situation? Any help would be greatly appreciated.

Answer 1

When decoding video with MediaCodec, you are not the one setting the PTS, you are the one receiving the PTS. When you call releaseOutputBuffer() with the "render" flag set, you are telling the system to render the frame as soon as possible. It's your responsibility to pace the frames.

For an example of a MediaCodec-based video player that controls the playback rate, see Grafika , in particular the use of the SpeedControlCallback class.

Answer 2

In your example, you should consider employing info.mPresentationTimeUs which is received as part of decoder.dequeueOutputBuffer(info, -1); . Once the output frame is received, you should perform the AV Sync with a reference clock which could be audio clock or system clock.

Please refer to this link for a good reference as an example.