使用 GStreamer 创建一个简单的 OpenCV 管道

Question

I have a pipeline to stream images over UDP

fps = 20
width = 500
height = 500

out_send = cv2.VideoWriter(
        "appsrc ! videoconvert ! "
        "video/x-raw,format=I420 ! "
        "jpegenc ! rtpjpegpay !"
        "udpsink host=127.0.0.1 port=5000",
        cv2.CAP_GSTREAMER, 0, fps, (width, height), True
    )

while True:
    frame = np.random.randint(255, size=(height, width, 3), dtype=np.uint8)

    out_send.write(frame)

    time.sleep(0.05)

This starts the pipeline but I am unable to receive using the following pipeline. The streaming wont begin, just hangs.

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,media=video,payload=26,clock-rate=90000,encoding-name=JPEG ! rtpjpegdepay ! jpegdec ! videoconvert ! queue ! xvimagesink

However if the frame is captured from webcam as below

cap_send = cv2.VideoCapture(0)

fps = int(cap_send.get(cv2.CAP_PROP_FPS))
width = int(cap_send.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_send.get(cv2.CAP_PROP_FRAME_HEIGHT))

while True:
    ret, frame = cap_send.read()

    if not ret:
        break

    out_send.write(frame)

Then the receiving pipeline starts and streams without any issues.

In both cases I see the following message on the receiving end

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Edit:

I ran the receiving pipeline with GST_DEBUG=3 and see the following output

gst_rtp_jpeg_depay_process:<rtpjpegdepay0> discarding data packets received when we have no header

This only happens if the sending pipeline uses raw images / numpy arrays as above.

What am I doing incorrect here. Very new to gstreamer and need some help

Goal: To stream images generated in opencv, asynchronously.

Answer 1

@Micka pointed me in the right direction.

The issue was with frame sizes. I have tried MJPEG and H264 and both need frame sizes (width/height) that are multiples of 8. Gstreamer will round them up to the closest 8th multiple. If the height is rounded then there is jitter seen at the end of the frame. If the width is rounded then a green screen is seen at the receiver.