How to simulate a video in real time and be able to extract its frames?

Question

I am making a program that identifies a figure in a video. With OpenCV you can extract the frames of a video, as follows (I am working with c++):

cv::VideoCapture vidCap(path/video.mp4);
cv::Mat frame;
while (!endVideo){
     successRead = vidCap.read(frame);
     if (!successRead){
         endVideo = true;
     }
/* work */
}

This is fine for a video, but in the future, I will be working with a camera. Therefore, I want to simulate with this video "path/video.mp4" as if it were a camera, that is, I want it to run in real time and for my program to catch the frames of the video that runs in real time (even if it loses some frame).

The pseudocode would be (I don't care about the library):

video.load(path/video.mp4) //load video
video.run()                //run video
while (!endVideo){
  video.getFrame(frame);  //get frame, even if it's the 10 frame
/* work */
}

Answer 1

One way to do this might be to start a separate thread that reads the video file at its normal speed and stuffs each frame in a global variable. The main thread then just grabs that frame when it's ready.

So, if we use a video that just displays the time and the frame counter, we can grab a frame here and there as the fake camera updates the frame:

#!/usr/bin/env python3

import cv2
import sys
import time
import logging
import numpy as np
import threading, queue

logging.basicConfig(level=logging.DEBUG, format='%(levelname)s %(message)s')

# This is shared between main and the FakeCamera
currentFrame = None

def FakeCamera(Q, filename):
    """Reads the video file at its natural rate, storing the frame in a global called 'currentFrame'"""
    logging.debug(f'[FakeCamera] Generating video stream from {filename}')

    # Open video
    video = cv2.VideoCapture(filename)
    if (video.isOpened()== False):
       logging.critical(f'[FakeCamera] Unable to open video {filename}')
       Q.put('ERROR')
       return

    # Get height, width and framerate so we know how often to read a frame
    h   = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
    w   = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
    fps = video.get(cv2.CAP_PROP_FPS)
    logging.debug(f'[FakeCamera] h={h}, w={w}, fps={fps}')

    # Initialise currentFrame
    global currentFrame
    currentFrame = np.zeros((h,w,3), dtype=np.uint8)

    # Signal main that we are ready
    Q.put('OK')

    while True:
        ret,frame = video.read()
        if ret == False:
            break
        # Store video frame where main can access it
        currentFrame[:] = frame[:]
        # Try and read at appropriate rate
        time.sleep(1.0/fps)

    logging.debug('[FakeCamera] Ending')
    Q.put('DONE')

if __name__ == '__main__':

    #  Create a queue for synchronising and communicating with our fake camera
    Q = queue.Queue()

    # Create a fake camera thread that reads the video in "real-time"
    fc = threading.Thread(target=FakeCamera, args=(Q,'video.mov'))
    fc.start()

    # Wait for fake camera to intialise
    logging.debug(f'[main] Waiting for camera to power up and initialise')
    msg = Q.get()
    if msg != 'OK':
        sys.exit()

    # Main processing loop should go here - we'll just grab a couple frames at different times
    cv2.imshow('Video',currentFrame)
    res = cv2.waitKey(2000)

    cv2.imshow('Video',currentFrame)
    res = cv2.waitKey(5000)

    cv2.imshow('Video',currentFrame)
    res = cv2.waitKey(2000)

    # Wait for buddy to finish
    fc.join()

Sample Output

DEBUG [FakeCamera] Generating video stream from video.mov
DEBUG [main] Waiting for camera to power up and initialise
DEBUG [FakeCamera] h=240, w=320, fps=25.0
DEBUG [FakeCamera] Ending

Keywords : Python, image processing, OpenCV, multi-thread, multi-threaded, multithread, multithreading, video, fake camera.