Fastest way to grab frames from video file OpenCV

Question

I want to do some openCV Video processing. Im on a Mac working with openCV in Xcode 4 so in fact my code is Objective C++.

I want to access all frames of a video as fast as possible (without displaying it and without drops) and do calculations on them.

my code to get the frames:

CvCapture* capture = cvCaptureFromFile("A MOVIE FILE HERE");

IplImage* frame;

 while(1) {
    frame = cvQueryFrame(capture);

    if (!frame) break;

       // openCV Stuff here...

    char c = cvWaitKey(1); 
    if(c==27) break;
}

I know the speed massively depends on Codec/Resolution/Bitrate - but it seems that I can't read with more than 120% speed... any idea how to grab frames faster?

Answer 1

Actually there's only one thing that slows your program - it's waitKey as Quentin Geissmann already mentioned. And if you say:

tried that already - forgot to mention that. Didn't really speed up things.

Than I don't believe you because I have just tested it on my environment and it speeds up on 30-40%.

Here's benchmark code:

#define WAIT_ON
int main()
{
    cv::Mat frame;
    cv::VideoCapture capture = cv::VideoCapture("video/in.avi");
    int k;

    double benchTime = (double)cv::getTickCount();
    while (1)
    {
        capture >> frame;
        if (!frame.data)
        {
            break;
        }

#ifdef WAIT_ON
        k = cv::waitKey(1);
        if (k == 27)
        {
            break;
        }
#endif
    }

    std::cout << ((double)cv::getTickCount() - benchTime)/cv::getTickFrequency() << std::endl;
}

Video input: 854x480, 24fps, 2:00 .

With WAIT_ON macro: ~11 sec

Without: ~7.3 sec

Update:

To reduce image resolution in videostream set these parameters:

CV_CAP_PROP_FRAME_WIDTH Width of the frames in the video stream.
CV_CAP_PROP_FRAME_HEIGHT Height of the frames in the video stream.

in set method to other (320x240).

Answer 2

I have noticed several times that cv::waitKey() / cvWaitKey() is not accurate for short (<10ms) times. In fact, in my case, it seems to sleep for at least 10ms with any values under 10ms. Maybe someone can bring more precision about this, but I would suggest to remove it from your loop (if you can).

I hope it works, Good luck

Answer 3

The functions

cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 320);
cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, 200);

do not affect the video itself, if you like to change the resolution of the frames, you can use something like:

cv::Size videoSize = cv::Size ((int) 320, (int) 200);
cv::resize(srcFrame,resFrame, videoSize);

This will improve the time you need to process each frame since they will be smaller. Hope it helps