Non-integer frame numbers in OpenCV?

Question

I'm using OpenCV in python on OSX(10.10.5). I'm fairly new to OpenCV. Right now I'm just trying to find the frame number where a certain event happens in a video. I'm navigating the video using a trackbar.

The problem I'm having is that OpenCV is reporting non-integer frame numbers. My program should set the frame number to the trackbar position. The trackbar position is always an integer, but the frame number is not. The problem seems to be worse with longer videos: with shorter videos, the frame number, if not an integer, is something like 253.99999999999994 when the trackbar position is 254. However, with longer videos, the frame number gets further and further from an integer value (eg 11212.20588235294 when the trackbar position is 11212, or 26631.529411764703 when the trackbar position is 26631.) Note that these numbers don't always round or truncate to the same integer as the trackbar position.

The total frame number for each video is also not an integer: 1547.9999999999998 for the shorter video I've been testing with, and 92651.38235294117 for the long video I've been testing. The shorter video is an .mov file and the longer one is .mp4, which I transcoded from .mpg using ffmpeg.

Why is this happening? How can I figure out frame numbers? Thanks for any help! Here's the test code I've been using (note that it's based on the frame grabber from http://giusedroid.blogspot.com/2015/05/python-opencv-frame-grabber.html ):

import numpy as np
import cv2

video_path = '#set video path here'

# grab a VideoCapture object
cap = cv2.VideoCapture(video_path)

#set some shorthand names
current_frame_flag = cv2.cv.CV_CAP_PROP_POS_FRAMES
total_frames_flag = cv2.cv.CV_CAP_PROP_FRAME_COUNT
win_name = "Frameshift calculator"
pos_trackbar='pos_trackbar'

cv2.namedWindow(win_name)


def seek_callback(x): 

    # we want to change the value of the frame variable globally
    global frame
    # by getting the position of the trackbar
    i = cv2.getTrackbarPos(pos_trackbar, win_name)
    # and skipping to the selected frame
    cap.set(current_frame_flag, i)
    _, frame = cap.read()
    # and then update the window
    cv2.imshow(win_name, frame)
    #print out the current frame flag and the trackbar position
    print(cap.get(current_frame_flag), i)

cv2.createTrackbar(pos_trackbar, win_name, 0, int(cap.get(total_frames_flag)), seek_callback)

while True:
    # shows the image
    cv2.imshow(win_name, frame)
    # waits for keystroke
    if cv2.waitKey(0) & 0xFF == ord('q'):
        break
    key = cv2.waitKey(0)

cap.release
cv2.destroyAllWindows()

Some sample (frame number, trackbar position) pairs:

Shorter video (.mov): (1.0, 0) (171.99999999999997, 172) (842.9999999999999, 843) (1141.0, 1141) (1330.0, 1330) (111.99999999999999, 112) (235.99999999999997, 236) (590.9999999999999, 591) (1546.9999999999998, 1547)

Longer video (.mp4): (1.0, 0) (6642.911764705882, 6642) (27496.11764705882, 27496) (49707.529411764706, 49707) (64786.294117647056, 64786) (84065.38235294117, 84065)

Answer 1

This is most likely due to the fact that your frame rate is not an integer. For each video, investigate what the actual frame rate of the video is:

frame_rate = cv2.cv.CV_CAP_PROP_FPS

In fact, for some videos, there is a floating-point frame rate. Usually in television... at least in the North American standards, there is an expected frame rate of 30 FPS. However, you are very likely to have 29.97 FPS instead. This article explains that in more detail: http://theautomaticfilmmaker.com/blog/2009/2/23/about-frame-rates-or-why-2997.html .

However, if I can summarize, back in the 1970s before advanced electronics handled this, before the introduction of colour to television, broadcasting did in fact go at 30 FPS, but because of the extra information required to send colour information, this information was actually interfering with the sound information, and so they had to delay the FPS by 0.03 FPS to compensate and move both the sound information and colour information slightly out of phase.

That's besides the point though. The reason why you're getting floating point frame numbers for one video over another is entirely dependent on the frame rate. As such, by trying to get the frame number at your desired location, there may be a case where the desired frame number you want doesn't exactly coincide with an actual frame in the video, so it will give you the frame number that it bests corresponds to because of the floating point frame rate and that's why you're getting floating point results.

The reason why there is also a drift when you start making the frame index large is again simply because of the frame rate. As you start specifying frame numbers that are higher, remember that for the case of the 30 / 29.97 scenario, you expect that the frame grabbed at an index is what you want, but because at every 30 FPS, you're behind by 0.03, this difference gets exacerbated for larger frame indices, because you're tacking on a difference of 0.03 for every 30 frames.

---

In terms of getting the exact frame number, I don't really have a suggestion for you here. However, you could perhaps extract out one frame before and one frame after the desired set frame index and you can take a look at things from there. Specifying the actual frame index itself (at least from what I have seen) is rarely used in practice and it's for this exact reason why.