Merge audio and video RTP data into mp4 file

Question

I am receiving audio and video RTP data through socket connection. Now I want to merge the video and audio RTP data into MP4 file. How can I achieve this? Do I need to save the video RTP into h264 and audio RTP into PCMU separately and later merge these into MP4 file? Or is it possible to merge audio-video RTP into MP4 file directly?

Thanks in advance!

Answer 1

You don't need to create separate audio and video files to mux them to MP4 (or any container). But in your program you need to make H.264 and PCMU frames out of the RTP packets and transfer them to FFMPEG.

So in summary you program's pseudo code should look like this,

Main {
    //Setup RTP receiver
    //Configure MP4 Muxer of FFMPEG (set input and oputput format), your input is H.264 and PCMU and output is MP4
    //avformat_alloc_output_context2
    //avformat_new_stream

    //Create Thread1 to read audio RTP packets
    //Create Thread2 to read video RTP packets

    //Complete writing MP4 file
    //av_write_trailer
}

Thread1 ()
{
    //Receive audio RTP packets
    //Form an Audio frame

    //Send the frame to muxer av_interleaved_write_frame
}

Thread1 ()
{
    //Receive video RTP packets
    //Form a video frame

    //Send the frame to muxer av_interleaved_write_frame
}

Hope these helps. Muxer example available in ffmpeg source code will be helpful. https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/muxing.c