在 OpenCV 中使用 RTSP 相机时，有没有办法避免缓冲图像？

Question

Let's say we have a time consuming process that when we call it on any frame, it takes about 2 seconds for it to complete the operation.

As we capture our frames with Videocapture , frames are stored in a buffer behind the scene(maybe it's occurred in the camera itself), and when process on the nth frame completes, it grabs next frame ((n+1)th frame) , However, I would like to grab the frames in real-time, not in order(ie skip the intermediate frames)

for example you can test below example

cv::Mat frame;
cv::VideoCapture cap("rtsp url");
while (true) {
    cap.read(frame);
    cv::imshow("s",frame);
    cv::waitKey(2000);     //artificial delay
}

Run the above example, and you will see that the frame that shows belongs to the past, not the present. I am wondering how to skip those frames?

Answer 1

• NEW METHOD

After you start videoCapture , you can use the following function in OpenCV.

videoCapture cap(0);
cap.set(CV_CAP_PROP_BUFFERSIZE, 1); //now the opencv buffer just one frame.

• USING THREAD

The task was completed using multi-threaded programming. May I ask what's the point of this question? If you are using a weak processor like raspberrypi or maybe you have a large algorithm that takes a long time to run, you may experience this problem, leading to a large lag in your frames.

This problem was solved in Qt by using the QtConcurrent class that has the ability to run a thread easily and with little coding. Basically, we run our main thread as a processing unit and run a thread to continuously capture frames, so when the main thread finishes processing on one specific frame, it asks for another frame from the second thread. In the second thread, it is very important to capture frames without delay. Therefore, if a processing unit takes two seconds and a frame is captured in 0.2 seconds, we will lose middle frames.

The project is attached as follows

1.main.cpp

#include <opencv2/opencv.hpp>
#include <new_thread.h>     //this is a custom class that required for grab frames(we dedicate a new thread for this class)
#include <QtConcurrent/QtConcurrent>    
using namespace cv;
#include <QThread>     //need for generate artificial delay, which in real situation it will produce by weak processor or etc. 
int main()
{
      Mat frame;
      new_thread th;       //create an object from new_thread class
      QtConcurrent::run(&th,&new_thread::get_frame);   //start a thread with QtConcurrent
      QThread::msleep(2000); //need some time to start(open) camera
      while(true)
      {
          frame=th.return_frame();
          imshow("frame",frame);
          waitKey(20);
          QThread::msleep(2000); //artifical delay, or a big process
      }
}  

2.new_thread.h

#include <opencv2/opencv.hpp>
using namespace cv;
class new_thread
{
public:
    new_thread();             //constructor
    void get_frame(void);     //grab frame
    Mat return_frame();       //return frame to main.cpp
private:
    VideoCapture cap;         
    Mat frame;               //frmae i
};

3.new_thread.cpp

#include <new_thread.h>
#include <QThread>
new_thread::new_thread()
{
    cap.open(0);        //open camera when class is constructing
}

void new_thread::get_frame()       //get frame method 
{
    while(1)                     // while(1) because we want to grab frames continuously 
    {
        cap.read(frame);
    }
}

Mat new_thread::return_frame()
{
    return frame;              //whenever main.cpp need updated frame it can request last frame by usign this method
}