GStreamer Full HD Video RTP Stream on Raspberry Pi

Question

i am trying to build a video pipeline with GStreamer. In the end it should be a full duplex Full HD video conferencing solution. As the Raspberry Pi 3 is not capable to encode and decode the video stream at the same time i am using a Logitech C920 to encode the video. So the my RPi only needs to encode and decode Audio, and decode the H.264 video. I build GStreamer from source (1.13.0.1) as the 1.4.4 from the raspbian repository had problems with G.722 encoding. Libraries like gst-omx (RaspiVideo), libsrtp (encryption), libwebrtc (echocanceling) (0.1 in repo to old - so manually downloaded 0.3) are build as well. So i do have omxh264dec - but i couldn't find a video sink to play the 1080p video smoothly. So i tried to pipe (with a named pipe) the h.264 stream do hello_video or the omxplayer, which worked fine as long as I stay local. As soon as i try to stream it over RTP hello_video doesn't show anything.

Working pipe: gst-launch-1.0 filesrc location=/opt/vc/src/hello_pi/hello_video/test.h264 ! filesink location=/home/pi/test & /opt/vc/src/hello_pi/hello_video/hello_video.bin /home/pi/test even with stream from camera: gst-launch-1.0 v4l2src ! 'video/x-h264,width=1920,height=1080,framerate=30/1' ! filesink location=/home/pi/test & /opt/vc/src/hello_pi/hello_video/hello_video.bin /home/pi/test

Not working when transmitting over RTP: gst-launch-1.0 v4l2src ! 'video/x-h264,width=1920,height=1080,framerate=30/1' ! h264parse config-interval=1 ! rtph264pay ! udpsink host=10.0.0.129 port=5000 \\ & gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! 'video/x-h264,width=1920,height=1080,framerate=30/1' ! filesink location=/home/pi/test \\ & /opt/vc/src/hello_pi/hello_video/hello_video.bin /home/pi/test

Not working without UDP RTP: gst-launch-1.0 v4l2src ! h264parse config-interval=1 ! rtph264pay ! queue ! rtph264depay ! filesink location=/home/pi/test \\ & /opt/vc/src/hello_pi/hello_video/hello_video.bin /home/pi/test

So it seems that something happens as soon as the video stream gets enclosed in RTP.

I know that the RTP Stream works, as i can receive it on my VM and play it without any trouble. But, of course, on the VM i can use the autovideosink. working receive on ubuntu: gst-launch-1.0 udpsrc port=5000 ! \\ application/x-rtp,\\ encoding-name=H264,payload=96 ! \\ rtph264depay ! h264parse ! avdec_h264 ! \\ autovideosink

Any suggestions are highly appreciated.

Thank You

Best Regards, maunza

Answer 1

Just to add my two cents - all Pi's ranging from the first A, and B revisions up to Zero and 3 are said to be capable of parallel encoding and decoding signals up to 1080p30. And my experience shows they are. I think raspivid once had an option to preview the encoded video that did just that.

Answer 2

I also lost some time trying to figure that out. For those who will look for same thing:

gst-launch-1.0   filesrc location=./test.h264 !   h264parse !  rtph264pay  ! rtph264depay ! h264parse ! video/x-h264, stream-format="byte-stream" ! filesink location=/dev/stdout | ./hello_video.bin /dev/stdin

So, key in:

! video/x-h264, stream-format="byte-stream" !