Recording RTP VP8 packets with FFMPEG with named pipe

Question

I'm developing a WebRTC video session recorder, in a gateway developed in C++, where I only have access to individual RTP packets.

When a session starts, I create two threads one that initializes a named pipe and an other that starts FFMPEG to fetch data from that pipe and store it in a matroska file, with the command:

ffmpeg -i \\.\pipe\screenRec -f matroska D:\djhfifj.mkv

Whenever I receive an RTP packet I send it through the pipe to FFMPEG . Although all communication is working fine, FFMPEG does not seem to be recognizing the RTP packet:

ffmpeg version N-73633-gdfc5858 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 28.100 / 54. 28.100
  libavcodec     56. 47.100 / 56. 47.100
  libavformat    56. 40.100 / 56. 40.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 21.100 /  5. 21.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
[aac @ 031b3fc0] Format aac detected only with low score of 1, misdetection possible!
[aac @ 031bd820] More than one AAC RDB per ADTS frame is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 031bd820] channel element 3.13 is not allocated
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (26) exceeds limit (9).
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (60) exceeds limit (44).
[aac @ 031bd820] Number of bands (6) exceeds limit (4).
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (30) exceeds limit (23).
[aac @ 031bd820] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 031bd820] Inconsistent channel configuration.
[aac @ 031bd820] get_buffer() failed
[aac @ 031bd820] Assuming an incorrectly encoded 7.1 channel layout instead of a spec-compliant 7.1(wide) layout, use -strict 1 to decode according to the specification instead.
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (16) exceeds limit (11).
[aac @ 031bd820] Dependent coupling is not supported together with LTP
    Last message repeated 9 times
[aac @ 031bd820] channel element 3.5 is not allocated
[aac @ 031bd820] channel element 3.13 is not allocated
[aac @ 031bd820] channel element 3.3 is not allocated
[aac @ 031bd820] Number of bands (16) exceeds limit (14).
[aac @ 031bd820] channel element 3.10 is not allocated
[aac @ 031bd820] channel element 3.2 is not allocated
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of scalefactor bands in group (61) exceeds limit (43).
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (29) exceeds limit (13).
[aac @ 031bd820] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 031bd820] Inconsistent channel configuration.
[aac @ 031bd820] get_buffer() failed
[aac @ 031bd820] channel element 0.7 is not allocated
[aac @ 031bd820] Number of bands (24) exceeds limit (15).
[aac @ 031bd820] channel element 1.1 is not allocated
[aac @ 031bd820] channel element 2.0 is not allocated
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of scalefactor bands in group (62) exceeds limit (41).
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (15) exceeds limit (13).
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (23) exceeds limit (2).
[aac @ 031bd820] channel element 1.4 is not allocated
[aac @ 031bd820] Assuming an incorrectly encoded 7.1 channel layout instead of a spec-compliant 7.1(wide) layout, use -strict 1 to decode according to the specification instead.
[aac @ 031bd820] channel element 1.2 is not allocated
[aac @ 031bd820] channel element 1.8 is not allocated
[aac @ 031bd820] channel element 3.7 is not allocated
[aac @ 031bd820] channel element 2.9 is not allocated
[aac @ 031bd820] channel element 3.8 is not allocated
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of scalefactor bands in group (48) exceeds limit (43).
[aac @ 031bd820] channel element 3.8 is not allocated
[aac @ 031bd820] channel element 2.13 is not allocated
[aac @ 031bd820] channel element 3.4 is not allocated
[aac @ 031bd820] Dependent coupling is not supported together with LTP
    Last message repeated 13 times
[aac @ 031bd820] channel element 2.14 is not allocated
[aac @ 031bd820] SBR was found before the first channel element.
[aac @ 031bd820] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 031bd820] Inconsistent channel configuration.
[aac @ 031bd820] get_buffer() failed
[aac @ 031bd820] Number of bands (6) exceeds limit (5).
[aac @ 031bd820] channel element 3.0 is not allocated
[aac @ 031bd820] channel element 1.5 is not allocated
[aac @ 031bd820] channel element 1.13 is not allocated
[aac @ 031bd820] channel element 1.7 is not allocated
[aac @ 031bd820] channel element 2.0 is not allocated
[aac @ 031bd820] Dependent coupling is not supported together with LTP
    Last message repeated 13 times
[aac @ 031bd820] channel element 3.0 is not allocated
[aac @ 031bd820] Assuming an incorrectly encoded 7.1 channel layout instead of a spec-compliant 7.1(wide) layout, use -strict 1 to decode according to the specification instead.
[aac @ 031bd820] SBR was found before the first channel element.
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (31) exceeds limit (30).
[aac @ 031bd820] channel element 1.12 is not allocated
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] Number of bands (29) exceeds limit (10).
[aac @ 031bd820] channel element 3.2 is not allocated
[aac @ 031bd820] channel element 3.15 is not allocated
[aac @ 031bd820] channel element 1.5 is not allocated
[aac @ 031bd820] channel element 2.7 is not allocated
[aac @ 031bd820] channel element 1.9 is not allocated
[aac @ 031bd820] Number of bands (54) exceeds limit (34).
[aac @ 031bd820] channel element 1.6 is not allocated
[aac @ 031bd820] channel element 1.2 is not allocated
[aac @ 031bd820] channel element 3.7 is not allocated
[aac @ 031bd820] Reserved bit set.
[aac @ 031bd820] ms_present = 3 is reserved.
[aac @ 031b3fc0] decoding for stream 0 failed
[aac @ 031b3fc0] Could not find codec parameters for stream 0 (Audio: aac (LTP), 4.0, fltp, 1506 kb/s): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
\\.\pipe\screenRec96: could not find codec parameters
Input #0, aac, from '\\.\pipe\screenRec96':
  Duration: N/A, bitrate: 1506 kb/s
    Stream #0:0: Audio: aac (LTP), 4.0, fltp, 1506 kb/s
[abuffer @ 0435cd00] Value inf for parameter 'time_base' out of range [0 - 2.14748e+009]
    Last message repeated 3 times
[abuffer @ 0435cd00] Error setting option time_base to value 1/0.
[graph 0 input from stream 0:0 @ 0319afe0] Error applying options to the filter.
Error opening filters!

Is it possible to make FFMPEG understand that the packet sent is RTP with VP8 ?

Answer 1

So, this is going to be tricky. Let me first explain why before we get to solutions. FFmpeg supports two types of RTP communication layers: UDP and TCP. Most people use TCP for things like radio, where the realtime element isn't that important and buffering a few seconds ahead is OK. UDP is used for p2p communication, like webrtc. So I'm going to assume you want to use UDP. (The TCP story is slightly different, but not by much.) UDP packets have packet boundaries, which get lost if you just "dump" them to a named pipe. Secondly, you're not telling ffmpeg what type of data is in your named pipe, so ffmpeg probably just thinks it's a raw aac file (with the rtp bits in between being random garbage). Garbage in, garbage out.

So. How do you fix it? Like I said, it's not easy. I think you absolutely want to use ffmpeg as a library, not as an executable. Then, I think you want to initialize ffmpeg to read your data stream as rtp, so open a localhost (127.0.0.1) socket in your application control code, open a rtp://127.0.0.1/ (localhost) connection through libavformat, and have your application write packets to it which will be received by ffmpeg. Now, it will understand that they're UDP packets and do the appropriate things. There's other ways to accomplish the same thing also - in fact, it's much easier if you just let ffmpeg do the UDP connection management for you, but if you don't want to do that, that's fine.