webrtc video encoder syncing through sdp

Question

I've webrtc application that uses h264/90000 media for video stream. My offer sdp is

`type: offer, sdp: v=0
o=- 6487162123143986811 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video data
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:6END
a=ice-pwd:qh8ZHjSd/8EiNkVnrfm0etc6
a=ice-options:trickle
a=fingerprint:sha-256 7D:A0:DD:31:74:EC:CC:29:71:EB:02:75:81:0A:98:48:50:31:A3:FF:2E:1E:16:04:12:AF:61:E8:1E:38:10:F2
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=fmtp:111 stereo=1;sprop-stereo=1;cbr=0
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 124 127 123 125 107 108
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:6END
a=ice-pwd:qh8ZHjSd/8EiNkVnrfm0etc6
a=ice-options:trickle
a=fingerprint:sha-256 7D:A0:DD:31:74:EC:CC:29:71:EB:02:75:81:0A:98:48:50:31:A3:FF:2E:1E:16:04:12:AF:61:E8:1E:38:10:F2
a=setup:actpass
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 H264/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:124 rtx/90000
a=fmtp:124 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:123 rtx/90000
a=fmtp:123 apt=127
a=rtpmap:125 red/90000
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 ulpfec/90000
m=application 9 DTLS/SCTP 5000
c=IN IP4 0.0.0.0
a=ice-ufrag:6END
a=ice-pwd:qh8ZHjSd/8EiNkVnrfm0etc6
a=ice-options:trickle
a=fingerprint:sha-256 7D:A0:DD:31:74:EC:CC:29:71:EB:02:75:81:0A:98:48:50:31:A3:FF:2E:1E:16:04:12:AF:61:E8:1E:38:10:F2
a=setup:actpass
a=mid:data
a=sctpmap:5000 webrtc-datachannel 1024`

H264/90000 has 3 enteries. a=rtpmap:100 H264/90000, a=rtpmap:102 H264/90000 and a=rtpmap:127 H264/90000

On server side, before setting this sdp to remotedescription, I've updated

m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 124 127 123 125 107 108 to

m=video 9 UDP/TLS/RTP/SAVPF 100 96 97 98 99 101 102 124 127 123 125 107 108

to ensure that it uses a=rtpmap:100 H264/90000 entry for video media. I always use the first entry of h264/90000 a=rtpmap:100 H264/90000 as video media.

This code was working fine. But now, I'm getting pink screen in my video element

After creating answer, what i got from localdescription of server side is m=video 9 UDP/TLS/RTP/SAVPF 102 96 97 124 125 107 108 It shows that 102(a=rtpmap:102 H264/90000) is used for video media. It shows entirely a pink screen in browser. Moreover, audio and data channel are still working fine.

Please help me to find out how should I know that which entry of h264/90000 is going to be used in server's local description.

Answer 1

It is working for me.

If an encoder (that you want to use) have different values in sdp, its all values must have more preference than others, ie there are three values for H264/90000, m=video must like

m=video 9 UDP/TLS/RTP/SAVPF 100 102 127 96 97 98 99 101 124 123 125 107 108

One question is not solved that is why there are three different values for H264/90000.

Answer 2

Multiple H264 lines are offered beacuse the encoding profiles are different. This is the line that looks like this:

a=fmtp:125 profile-level-id=42e01f

The 42e01f stands for Baseline constrained (for more information on this, check out this post on Stack Overflow )

So if multiple profiles for H264 are supported, you will see multiple a=rtpmap:XXX H264/90000 lines but with differing profile-level-id :

m=video 17456 RTP/AVP 100 102 127
a=rtpmap:100 H264/90000
a=fmtp:100 profile-level-id=42e00a
   :               :
a=rtpmap:102 H264/90000
a=fmtp:102 profile-level-id=42e01f
   :               :
a=rtpmap:127 H264/90000
a=fmtp:127 profile-level-id=640c1f