如何将 Opencv VideoWriter 与 GStreamer 一起使用？

Question

I am trying to transfer a h264 stream using Opencv VideoWriter to get it on another pc on the network using VideoCapture. However, I am stuck on VideoWriter. Execution of this code returns with an error and out.isOpened () is always false.

    int FOURCC = cv::VideoWriter::fourcc('H', '2', '6', '4');
    cv::VideoWriter out;
    out.open ("appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000",
        cv::CAP_GSTREAMER,
        FOURCC,
        16,
        cv::Size (640, 480),
        true);

    if (!out.isOpened ()) {
        qDebug () << "\ n ***** Failed to open videowriter *****";
        return -1;
    }

[ WARN:0] global D:\Libs\opencv-4.3.0\modules\videoio\src\cap_gstreamer.cpp (1274) cv::CvVideoWriter_GStreamer::close_ OpenCV | GStreamer warning: No source in GStreamer pipeline. Ignore
[ERROR:0] global D:\Libs\opencv-4.3.0\modules\videoio\src\cap.cpp (527) cv::VideoWriter::open VIDEOIO(GSTREAMER): raised OpenCV exception:
OpenCV(4.3.0) D:\Libs\opencv-4.3.0\modules\videoio\src\cap_gstreamer.cpp:144: error: (-215:Assertion failed) ptr in function 'cv::`anonymous-namespace'::GSafePtr<struct _GstElement>::get'

***** Failed open videowriter *****

Even a simple example returns me an error and out.isOpened() false.

    out.open("autovideosrc ! videoconvert ! appsink",
             cv::CAP_GSTREAMER,
             FOURCC,
             16,
             cv::Size(640, 480),
             true);

[ WARN:0] global D:\Libs\opencv-4.3.0\modules\videoio\src\cap_gstreamer.cpp (1500) cv::CvVideoWriter_GStreamer::open    OpenCV | GStreamer warning: OpenCV backend does not support this file type (extension): autovideosrc ! videoconvert !   appsink
[ WARN:0] global D:\Libs\opencv-4.3.0\modules\videoio\src\cap_gstreamer.cpp (1274) cv::CvVideoWriter_GStreamer::close_  OpenCV | GStreamer warning: No source in GStreamer pipeline. Ignore

    ***** Failed to open videowriter *****

The version of opencv 4.3.0 is compiled from source code with gstreamer support. cv::getBuildInformation () says:

    Video I/O:
      DC1394: NO
      FFMPEG: YES (prebuilt binaries)
        avcodec: YES (58.54.100)
        avformat: YES (58.29.100)
        avutil: YES (56.31.100)
        swscale: YES (5.5.100)
        avresample: YES (4.0.0)
      GStreamer: YES (1.16.2)
      DirectShow: YES
      Media Foundation: YES
        DXVA: YES

How can I stream the stream? What parameters should be specified by VideoWriter? I tried various tips from google, but none of them helped me. I would be grateful for a simple example of how to send a stream from VideoWriter on one side and receive it from VideoCapture on the other. I am using Windows 10 x64 and Qt5.13 MSVC2017

Answer 1

You need to feed raw video to appsrc . Setting fourcc to h264 forces VideoWriter to encode video instead of gstreamer pipe. You can set your fourcc to 0 to push raw video. The following should work.

cv::VideoWriter out;
out.open ("appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000",
    cv::CAP_GSTREAMER,
    0,
    16,
    cv::Size (640, 480),
    true);

Answer 2

I believe that this is a bug in OpenCV, which should be fixed by this PR , or maybe this one (or maybe both).

Indeed, it seems like the OpenCV code used to work with some version of GStreamer that would set the error pointer to NULL when the function succeeds, but that is not guaranteed with 1.16.2 in my experience (and from the documentation).

With those PRs, it should properly use the return value instead of relying on the state of the error pointer.