[英]How to pack Android MediaCodec encoded H264 into RTP packets
如何將 H264 字節流正確打包到 RTP 數據包中，以便我可以使用 FFMPEG 接收幀？
當我啟動 FFMPEG 接收器時，它會產生很多這樣的錯誤：
Invalid UE golomb code
[h264 @ 0xd63060] pps_id 3199971767 out of range
[h264 @ 0xd63060] slice type 32 too large at -1
[h264 @ 0xd63060] decode_slice_header error
[h264 @ 0xd63060] non-existing PPS 0 referenced
[h264 @ 0xd63060] decode_slice_header error
[h264 @ 0xd63060] no frame!
[h264 @ 0xd63060] decode_slice_header error
[h264 @ 0xd63060] Unknown NAL code: 0 (0 bits)
[h264 @ 0xd63060] no frame!
[h264 @ 0xd63060] non-existing PPS 0 referenced
這是我使用的 SDP 文件：
c=IN IP4 192.168.2.30
t=0 0
m=video 51372 RTP/AVP 96
a=rtpmap:96 H264/90000
a=recv only
pps_id 錯誤很奇怪，它好像在尋找下一個 PPS，但找不到它，盡管我嘗試將 PPS 嵌入到每個 NALU 中。
我一直在閱讀RFC 6184並試圖理解它。 但是我感覺我還是不太明白H264和RTP是如何交互的。 目前，我正在嘗試對來自相機的像素進行編碼，並通過網絡上的 RTP 傳輸 1920x1080 H264 編碼的幀，然后由 FFMPEG 接收並解碼。 我正在用 Java 組裝 RTP 和 FU-A 標頭，並在它們對於 MTU 來說太大時將 NALU 分段。
我一直在 Wireshark 中密切關注流，這是我的第一個數據包的輸出：
Real-Time Transport Protocol
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...0 .... = Extension: False
.... 0000 = Contributing source identifiers count: 0
1... .... = Marker: True
Payload type: DynamicRTP-Type-96 (96)
Sequence number: 0
Timestamp: 2727179012
Synchronization Source identifier: 0x00000000 (0)
H.264
NAL unit header or first byte of the payload
0... .... = F bit: No bit errors or other syntax violations
.00. .... = Nal_ref_idc (NRI): 0
...0 0000 = Type: Undefined (0)
H264 NAL Unit Payload
我不明白為什么第一個有效負載的 NALU 類型為 0。不過，這是我的第二個數據包：
Real-Time Transport Protocol
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...0 .... = Extension: False
.... 0000 = Contributing source identifiers count: 0
0... .... = Marker: False
Payload type: DynamicRTP-Type-96 (96)
Sequence number: 1
Timestamp: 2727179019
Synchronization Source identifier: 0x00000000 (0)
H.264
FU identifier
0... .... = F bit: No bit errors or other syntax violations
.11. .... = Nal_ref_idc (NRI): 3
...1 1100 = Type: Fragmentation unit A (FU-A) (28)
FU Header
1... .... = Start bit: the first packet of FU-A picture
.0.. .... = End bit: Not the last packet of FU-A picture
..0. .... = Forbidden bit: 0
...0 0101 = Nal_unit_type: Coded slice of an IDR picture (5)
H264 NAL Unit Payload
0000 0000 0000 0000 0000 0000 0000 0001 0110 0101 1011 1000 0000 0100 0000 010. = first_mb_in_slice: 3000762881
.... ...1 = slice_type: P (P slice) (0)
0011 1... = pic_parameter_set_id: 6
所以我認為最后一個數據包是 I 幀？ 這是開始和結束片段之間的片段：
Real-Time Transport Protocol
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...0 .... = Extension: False
.... 0000 = Contributing source identifiers count: 0
0... .... = Marker: False
Payload type: DynamicRTP-Type-96 (96)
Sequence number: 1
Timestamp: 2727179019
Synchronization Source identifier: 0x00000000 (0)
H.264
FU identifier
0... .... = F bit: No bit errors or other syntax violations
.11. .... = Nal_ref_idc (NRI): 3
...1 1100 = Type: Fragmentation unit A (FU-A) (28)
FU Header
0... .... = Start bit: Not the first packet of FU-A picture
.0.. .... = End bit: Not the last packet of FU-A picture
..0. .... = Forbidden bit: 0
...0 0101 = Nal_unit_type: Coded slice of an IDR picture (5)
當然，這里是假設的 I-Frame 的最后一個數據包：
Real-Time Transport Protocol
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...0 .... = Extension: False
.... 0000 = Contributing source identifiers count: 0
1... .... = Marker: True
Payload type: DynamicRTP-Type-96 (96)
Sequence number: 1
Timestamp: 2727179019
Synchronization Source identifier: 0x00000000 (0)
H.264
FU identifier
0... .... = F bit: No bit errors or other syntax violations
.11. .... = Nal_ref_idc (NRI): 3
...1 1100 = Type: Fragmentation unit A (FU-A) (28)
FU Header
0... .... = Start bit: Not the first packet of FU-A picture
.1.. .... = End bit: the last packet of FU-A picture
..0. .... = Forbidden bit: 0
...0 0101 = Nal_unit_type: Coded slice of an IDR picture (5)
現在這是編碼器給我的下一個字節的數據包：
Real-Time Transport Protocol
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...0 .... = Extension: False
.... 0000 = Contributing source identifiers count: 0
0... .... = Marker: False
Payload type: DynamicRTP-Type-96 (96)
Sequence number: 2
Timestamp: 2727179089
Synchronization Source identifier: 0x00000000 (0)
H.264
FU identifier
0... .... = F bit: No bit errors or other syntax violations
.11. .... = Nal_ref_idc (NRI): 3
...1 1100 = Type: Fragmentation unit A (FU-A) (28)
FU Header
1... .... = Start bit: the first packet of FU-A picture
.0.. .... = End bit: Not the last packet of FU-A picture
..0. .... = Forbidden bit: 0
...0 0001 = Nal_unit_type: Coded slice of a non-IDR picture (1)
H264 NAL Unit Payload
0000 0000 0000 0000 0000 0000 0000 0001 0110 0001 1110 0000 0010 0000 0001 100. = first_mb_in_slice: 2968522763
.... ...0 0111 .... = slice_type: B (B slice) (6)
.... 0001 110. .... = pic_parameter_set_id: 13
這部分讓我感到困惑，當相機靜止時，編碼器給我提供了越來越小的未定義類型的 NALU，我不完全確定為什么，無論如何，下面的數據包作為一個完整的 NALU 被發送到 FFMPEG。
Real-Time Transport Protocol
10.. .... = Version: RFC 1889 Version (2)
..0. .... = Padding: False
...0 .... = Extension: False
.... 0000 = Contributing source identifiers count: 0
1... .... = Marker: True
Payload type: DynamicRTP-Type-96 (96)
Sequence number: 36
Timestamp: 2727180258
Synchronization Source identifier: 0x00000000 (0)
H.264
NAL unit header or first byte of the payload
0... .... = F bit: No bit errors or other syntax violations
.00. .... = Nal_ref_idc (NRI): 0
...0 0000 = Type: Undefined (0)
H264 NAL Unit Payload
我正在使用 Android MediaCodec 編碼器，這里是一些我配置編碼器的代碼：
mediaCodec = MediaCodec.createByCodecName("OMX.Nvidia.h264.encoder");
mediaFormat = MediaFormat.createVideoFormat("video/avc", 1920, 1080);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 125000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 0);
mediaFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 1920 * 1080);
編碼器是給我整個訪問單元還是只給我 NALU？
這是我的邏輯：
我覺得我很接近，但它顯然不適用於任何 RTP 接收器。 我感謝任何關於此事的想法或想法。
謝謝，
我終於設法解決了，我的數據包配置不正確。
我什至可以在流的中間啟動 FFmpeg 並且它可以工作！
Muxing Android MediaCodec將H264數據包編碼為RTMP
[英]Muxing Android MediaCodec encoded H264 packets into RTMP
使用 MediaCodec 在 Android 上使用 H264 數據解碼 RTP 流
[英]Decoding an RTP stream with H264 data on Android Using MediaCodec
如何使用Android中的MediaCodec Api在RTP上解碼H264
[英]How to decode H264 over RTP using MediaCodec Api in Android
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.