updating cv::capturevideo frame in a boost::thread safely

Question

I want to render via openGL 2camera in the same time. I want to refresh each frame as soon as possible. To update those frame I am using a thread in an infinite loop.

My actual problem is that my program crash if I put the resolution of those camera to high. With 160:120 there's no problem. But if i put the max resolution (1920:1080) there's just 5 or 6 update of the image before crashing. note: the number of update is not always the same before crashing

I suppose that If the resolution is low enough, frame_L and frame_R are changed quick enough that there's no colision beetween the main loop and the thread.

So I suppose that my mutex isnt doing what it should. How should I do?

(I'm not an expert in thread and variable safety)

My code:

#include <boost/thread/mutex.hpp>
#include <boost/thread/thread.hpp>

#include <opencv2/opencv.hpp>


boost::mutex m; //for an other thread
boost::mutex n;

cv::VideoCapture capture_L(0);
cv::VideoCapture capture_R(1);

cv::Mat frame_L;
cv::Mat frame_R;

void MyThreadFunction()
{
    while (1)
    {

        {
          boost::mutex::scoped_lock lk(n);

          if (capture_L.grab()){
             capture_L.retrieve(frame_L);
             cv::transpose(frame_L, frame_L);
          }

          if (capture_R.grab()){
            capture_R.retrieve(frame_R);
            cv::transpose(frame_R, frame_R);
          }
        }
    }

}

int main()
{

  capture_L.set(CV_CAP_PROP_FRAME_WIDTH, 160);
  capture_L.set(CV_CAP_PROP_FRAME_HEIGHT, 120);
  capture_R.set(CV_CAP_PROP_FRAME_WIDTH, 160);
  capture_R.set(CV_CAP_PROP_FRAME_HEIGHT, 120);

  boost::thread thrd(&MyThreadFunction);

  while(1)
  {
       [  use  frame_L and frame_R  ]  
  }  

}

Answer 1

This is the code that I use for my threaded camera grab. Its part of a camera Object (eg; Each camera has it's own object and own thread for the capturing.

void Camera::setCapture(cv::VideoCapture cap)
{
    pthread_mutex_lock(&latestFrameMutex);
    videoCapture = cap;
    videoCapture.read(latestFrame);
    pthread_mutex_unlock(&latestFrameMutex);
    int iret = pthread_create(&cameraGrabThread,NULL,&Camera::exec,this);
}

void *Camera::exec(void* thr)
{
   reinterpret_cast<Camera *> (thr)->grabFrame();
}

This ensures that a new thread is started when a capture is set for that camera. The following code is run by the exec part of the thread to actually grab the frame.

void *Camera::grabFrame()
{
    while(videoCapture.isOpened())
    {
        pthread_mutex_lock(&latestFrameMutex);
        if(!videoCapture.read(latestFrame))
            std::cout << "Unable to read frame" << std::endl;
        pthread_mutex_unlock(&latestFrameMutex);

        usleep(8000); // Sleep 8ms
    }
}

And finally, the camera needs to be able to return the latest frame;

cv::Mat Camera::getLatestFrame()
{
    while (getMilisecSinceLastCapture() < 35) //Enforce min time of 35 ms between frame requests.
    {
        usleep(5000);
    }
    pthread_mutex_lock(&latestFrameMutex);
    cv::Mat result = latestFrame.clone();
    pthread_mutex_unlock(&latestFrameMutex);
    return result.clone();
}

Changing the size can be done by using

void Camera::setImageSize(const cv::Size& size)
{    
    pthread_mutex_lock(&latestFrameMutex);
    videoCapture.set(CV_CAP_PROP_FRAME_HEIGHT, size.height);
    videoCapture.set(CV_CAP_PROP_FRAME_WIDTH, size.width);
    pthread_mutex_unlock(&latestFrameMutex);
}