無法在vlc上播放android h264編碼視頻

Question

我正在使用MediaCodec將android攝像頭流編碼為h264格式，然后使用gstreamer通過gst-rtsp-server創建RTSP流。 我可以使用gstreamer客戶端管道播放流。 但是無法使用vlc播放。 任何人都可以幫我在vlc上播放流嗎？

這是MediaCodec代碼，

final int TIMEOUT_USEC = 10000;     // no timeout -- check for buffers, bail if none
    try {
        ByteBuffer[] encoderOutputBuffers = encoder.getOutputBuffers();
        bufferInfo = new MediaCodec.BufferInfo();

            while (true) {
                int encoderStatus = encoder.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
                if (encoderStatus == MediaCodec.INFO_TRY_AGAIN_LATER) {
                    // no output available yet
                    break;
                } else if (encoderStatus == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
                    // not expected for an encoder
                    encoderOutputBuffers = encoder.getOutputBuffers();
                } else if (encoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
                    // Should happen before receiving buffers, and should only happen once.
                    // The MediaFormat contains the csd-0 and csd-1 keys, which we'll need
                    // for MediaMuxer.  It's unclear what else MediaMuxer might want, so
                    // rather than extract the codec-specific data and reconstruct a new
                    // MediaFormat later, we just grab it here and keep it around.

                    // The PPS and PPS shoud be there

                    if (muxerStarted) {
                        throw new RuntimeException("format changed twice");
                    }

                    muxerStarted = true;
                } else if (encoderStatus < 0) {
                    Log.w(TAG, "unexpected result from encoder.dequeueOutputBuffer: " + encoderStatus);
                    // let's ignore it
                } else {
                    ByteBuffer encodedData = encoderOutputBuffers[encoderStatus];
                    if (encodedData == null) {
                        throw new RuntimeException("encoderOutputBuffer " + encoderStatus +
                                " was null");
                    }

                    if (bufferInfo.size != 0) {

                        encodedData.position(bufferInfo.offset);
                        encodedData.limit(bufferInfo.offset + bufferInfo.size);


                        byte[] outData = new byte[bufferInfo.size];
                        encodedData.get(outData);

                        if (spsPpsInfo == null) {
                            ByteBuffer spsPpsBuffer = ByteBuffer.wrap(outData);
                            if (spsPpsBuffer.getInt() == 0x00000001) {
                                spsPpsInfo = new byte[outData.length];
                                System.arraycopy(outData, 0, spsPpsInfo, 0, outData.length);
                            } else
                                return;
                        } else {
                            outputStream.write(outData);
                        }
                    }

                    encoder.releaseOutputBuffer(encoderStatus, false);

                    if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                        Log.w(TAG, "reached end of stream unexpectedly");
                        break;      // out of while
                    }
                }
            }

            byte[] ret = outputStream.toByteArray();


            if (ret.length > 5 && ret[4] == 0x65) {
                Log.d(TAG, "----> Setting SPS PPS info");
                byte[] concBuffer = new byte[spsPpsInfo.length + ret.length];
                System.arraycopy(spsPpsInfo, 0, concBuffer, 0, spsPpsInfo.length);
                System.arraycopy(ret, 0, concBuffer, spsPpsInfo.length, ret.length);

                outputStream.reset();
                outputStream.write(concBuffer);
                //outputStream.write(ret, spsPpsInfo.length, (spsPpsInfo.length + ret.length-1));
            }
        } catch (Throwable t) {
            t.printStackTrace();
        }

        byte ret[] = outputStream.toByteArray();
        outputStream.reset();

        // Log.d(TAG, "Got buffer with size " + ret.length + " and needData " + needData);

        if (needData == 1 && ret.length != 0) {
            if (streamMode == Native.STREAM_MODE_RTP) {
                // Log.d(TAG, "Sending buffer to RTP pipeline with size " + ret.length);

                Native.nativeRTPAddStream(ret, ret.length, bufferInfo.presentationTimeUs, native_custom);
            } else if (streamMode == Native.STREAM_MODE_RTSP) {
                // Log.d(TAG, "Sending buffer to RTSP pipeline with size " + ret.length);

                Native.nativeRTSPAddStream(ret, ret.length, bufferInfo.presentationTimeUs, native_custom);
            }
        }

這是gstreamer管道，

appsrc name=camsrc ! h264parse ! rtph264pay name=pay0 pt=96

在上面的管道的末尾，附加了gst-rtsp-server。

Answer 1

問題在於緩沖區時間戳（ bufferInfo.presentationTimeUs ）。 Android相機發送的時間戳錯誤。 在將每個緩沖區發送到GStreamer接收器之前，您需要對其進行重新定時。

一種方法是使用identity元素訪問每個緩沖區。 另一種方法是通過手動計算每個緩沖區的時間戳來糾正Android本身的時間戳。

GStreamer中的重新定時公式看起來像這樣

GstClockTime time = gst_util_uint64_scale_int (1, GST_SECOND, absCurFps);
GST_BUFFER_PTS (buffer) = GST_BUFFER_DTS (buffer) += time;
GST_BUFFER_DURATION (buffer) = 1 * GST_SECOND / absCurFps;

實際上它假設每個緩沖區是一幀並將表示時間戳設置為base_time + 1 / (current_fps)