OpenCV显示和处理网络摄像头流

Question

I was looking for process a stream with opencv coming from webcam. My goal is to detect rectangles for each frames. I'm able to display the result but the framerate is really slow, ~1 fps.

Here is an overview of my source code:

int main( int argc, char** argv ) {

CvCapture* cap=cvCreateCameraCapture(0);
cvNamedWindow("LiveFeed",CV_WINDOW_AUTOSIZE);

    while(true) {
    frame=cvQueryFrame(cap);
    if(!frame) 
        printf("\n no");
    else {
        Mat mat_img(frame);
        std::vector<std::vector<cv::Point>> rectangle = findSquaresInImage(mat_img);
        rectangle=filterRectangleSimilaire(rectangle,20.0);
        Mat res=debugSquares(rectangle,mat_img);

        cvShowImage("LiveFeed",new IplImage(res));//new IplImage(res));
    }
    char c=cvWaitKey(33);
    if(c==27)
        break;
}

cvReleaseCapture(&cap);
cvDestroyAllWindows();
return 0;
}

I would like to know if it's possible to make it threaded to gain at least 30fps ?

I'm working on windows 8.1, Visual 2010, c++ & OpenCV 2.4.10

Answer 1

You could use threads before while(true) loop. Of course, reading frames from the camera and displaying them would have to be serialized.

I see you're using C++ std::vector, so I would suggest you use C++11 threads and mutex. The most performatic way I can think of is openning all threads at startup (before the while(true) loop) and make then receive the frames captured by the main thread. All threads would be locked on the same mutex for reading the frame queue and on a second mutex for displaying it. The frame queue may be implemented as:

struct frameQueue{
    std::vector<Mat> frames;
    std::mutex mtx;
};

Depending on the size of your project and side effects, it may even be declared global.

Answer 2

I fixed this issue using another solution found on stackoverflow. Here is my complete code:

Mat mat_img(frame);
Mat gray_mat_img, threshold_mat_img, detected_edges;
std::vector<std::vector<cv::Point>> contours;
vector<Vec4i> hierarchy;

cvtColor(mat_img, gray_mat_img, CV_BGR2GRAY);
blur(gray_mat_img, gray_mat_img, Size(10,10));
cv::threshold(gray_mat_img, threshold_mat_img, 140, 255, cv::THRESH_BINARY | cv::THRESH_OTSU);
Canny(threshold_mat_img, detected_edges, 100, 100, 3);
cv::findContours(detected_edges, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));

RNG rng(12345);
//contours = filterRectangleSimilaire(contours, 10);
//if (contours.size() < 5) {
vector<Rect> boundRect( contours.size() );
vector<Point> approx;


for (size_t i = 0; i < contours.size(); i++) {
    approxPolyDP(Mat(contours[i]), approx, arcLength(Mat(contours[i]), true)*0.02, true);
    if (approx.size() == 4 && fabs(contourArea(Mat(approx))) > 100 && isContourConvex(Mat(approx))) {
        double maxCosine = 0;
        for (int j = 2; j < 5; j++) {
            double cosine = fabs(angle(approx[j%4], approx[j-2], approx[j-1]));
            maxCosine = MAX(maxCosine, cosine);
        }
        if (maxCosine < 0.3) {
            Scalar color = Scalar(rng.uniform(0, 255), rng.uniform(0,255), rng.uniform(0,255));
            boundRect[i] = boundingRect( Mat(approx) );
            rectangle( mat_img, boundRect[i].tl(), boundRect[i].br(), color, 2, 8, 0 );
            drawContours(mat_img, contours, i, color, 2, 8, hierarchy, 0, Point());
        }
    }
}

Here is my reference: OpenCV C++/Obj-C: Detecting a sheet of paper / Square Detection