OpenCV RTP Decoding

Question

Is there any way to read RTP packets (that will be sent from a server over a pre-built tcp connection) and decode them to be able to create a h264 stream to be displayed? The data will be sent/received as a char buffer, but can be easily changed/converted upon receipt. Unfortunately, I have not seen a way to do this in OpenCV yet.

Edit: It is not required to do it in openCV (the reading), but it needs to be able to be received by an openCV data type (Mat/IplImage) once decoded. I remember seeing one about creating a stream, and will try to find it again, but does anyone know a way to create an individual frame from each packet?

Answer 1

you will need an RTP depayloader for your specific stream.

RTP (being targeted at UDP - that is: packet oriented protocols) does not have a 1:1 mapping between packets (as transmitted) and frames. most of the time, a single frame will be transmitted via several packets (an ethernet frame (not to be confused with your image frame) is usually limited to about 1500 bytes, which makes only tiny images...)

splitting and reassambling your payload chunks into/from network packets is the task of the RTP payloader resp. depayloader .

the easiest way to do this is probably via gstreamer , which already provides (de)payloaders for h.264