Render RTSP H.264 video stream using live555

Question

I would appreciate an example of using the Live555 library to render live streams to screen. Apparently it's simple, but it would really help to see it done rather than simply read that "it's simple"!. The gmane site states:

"To update (a copy of) the "testRTSPClient" code so that it renders video data is fairly straightforward: You simply have to change the "DummySink" class, so that it does the rendering (or calls a decoder library to do the rendering). In particular, you would change the implementation of the "afterGettingFrame()" function - at line 479. That's it!

(Actually, for H.264 video, there is one more thing that you'll probably need to do. H.264 streams have out-of-band configuration information (SPS and PPS NAL units) that you may need to feed to the decoder to initialize it. To get this information, call "MediaSubsession::fmtp_spropparametersets()" (on the video 'subsession' object). This will give you a (ASCII) character string. You can then pass this to "parseSPropParameterSets()", to generate binary NAL units for your decoder.) "

PS: I'm using visual studio and windows

Answer 1

I did something similar in a previous job. The trick was to use DirectShow for rendering. Basically, live555 does not decode or render anything by itself, you need some kind of h264 decoder as well as a video surface. Fortunately for you, DirectShow provides both, but need quite a lot of programming to get to work.

live555 only provides mechanism for streaming, meaning that it will take NAL packets (h264 packets if you will) from the network or file source and push it to the "DummySink", you still need to decode (transform the NAL units to bitmaps) and render (draw the bitmaps to the screen). This is not something that live555 will do for you, but other libraries can, such as ffmpeg, but I didn't managed to get it to work so we moved to the DirectShow solution. Namely, the "MS DTV-DVD Decoder" was very useful and could use some hardware acceleration provided by the chipset we were using (automagically). Another useful feature of live555 is that it will handle control protocols such as RTSP to handle PLAY/STOP/PAUSE requests for the stream.

EDIT: since you are searching for code, I've found an open sourced DirectShow filter that uses live555 for you. You should be able to run that with something like GraphStudio or GraphEdit. Eventually, you could learn how to create DirectShow graphs in c++ . This is really not very complicated as tons of examples already exist. The most difficult parts are to create the filters but the decoder and the renderer are already there for you, and I've provided you with the Source filter .

Answer 2

Another example of code using live555 on Windows is available at the video processing project . Like Eric's suggestion, it uses DirectShow (which is the standard way on Windows pre MediaFoundation. The code using live can be found here .

If you're using Windows 7, there is a built in H.264 decoder which will be inserted into the media pipeline when you render the graph. On earlier versions of Windows, you'll have to install your own H.264 decoder filter.

Disclaimer: I am one of the authors of the video processing project.