opencv rtsp stream protocol

Question

I want to process and display a network rtsp stream that is created from a raspberry camera. I have this code:

#include <iostream>
#include <functional>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>

int main(int argc, char** argv) {

    cv::VideoCapture * stream = new cv::VideoCapture("rtsp://192.168.55.151:8554/");
    if (!stream->isOpened()) return -1;

    cv::namedWindow("rtsp_stream", CV_WINDOW_AUTOSIZE);
    cv::Mat frame;

    while (true) {

        if (!stream->read(frame)) return -1;

        cv::imshow("rtsp_stream", frame);
        cv::waitKey(15);
    }

    return 1;
}

When the stream is not live, the execution of this results in:

[tcp @ 0xa12480] Connection to tcp://192.168.55.151:8554?timeout=0 failed: Connection refused

Which means that the stream tries to connect with tcp. When the stream is live, the execution results in:

 [rtsp @ 0xb07960] method SETUP failed: 461 Client error

From internet research i found that the problem may be that the stream uses udp. If i change the URL to:

"udp://192.168.55.151:8554/"

Then the execution freezes in the cv::VideoCapture("udp://192.168.55.151:8554/");

VLC is able to open the rtsp stream. As i am given to understand, ffmpeg is used to decode the stream with opencv. When i run:

ffmpeg -rtsp_transport udp -i rtsp://192.168.55.151:8554/ -t 5 test.mp4

the stream decoding and saving is successful. So how can i specify the lower level protocol to be udp in the opencv code? Is there another way to do it with opencv?

EDIT: If i change the ffmpeg command to use tcp, ie:

ffmpeg -rtsp_transport tcp -i rtsp://192.168.55.151:8554/ -t 5 test.mp4

then i get the same exact error with the c++ code, 461 client error

EDIT: When the code uses the udp:// link, after some 15 seconds of freeze, the execution returns with error

Answer 1

It was answered in the github opencv issues. The problem is that opencv forces ffmpeg to use a tcp connection, whereas my server uses rtp protocol(udp transport layer). So in order to resolve the problem i changed the server to an http stream.

Answer 2

根据opencv repo中的合并请求，可以通过setenv("OPENCV_FFMPEG_CAPTURE_OPTIONS", "rtsp_transport;udp", 1);使OpenCV中的FFMPEG使用UDP协议setenv("OPENCV_FFMPEG_CAPTURE_OPTIONS", "rtsp_transport;udp", 1);

Answer 3

just try this code, it easy solve your problem

for(;;) {
    if(!vcap.read(image)) {
        std::cout << "No frame" << std::endl;
        cv::waitKey();
    }
    cv::imshow("Output Window", image);
    if(cv::waitKey(1) >= 0) break;
}