OpenCV (Python) VideoCapture.read() on missing frames

Question

New to python, new to OpenCV, which I'm gonna use for my master-thesis, and already got some problems using the VideoCapture object of OpenCV.

Situation: I got 2 folders containing corresponding images (taken with RGB and infrared cameras). I want to display them sibe by side in a Window using a while-loop. The problem arises, when there are some images missing from one of the image-sequences (Due to problems while recording or whatever, I don't really know but that should be of no importance). My idea was to use the bool-returnvalue of the.read() function to check wheather there is a frame to be read and if not, replace the image by a black one. This is what I did:

Code:

import cv2
import numpy as np

pathRGB = "Bilder/RGB"
pathIR = "Bilder/IR"
# the paths to the folders containing the images

capRGB = cv2.VideoCapture(pathRGB + "/frame_%06d.jpg")
capIR = cv2.VideoCapture(pathIR + "/frame_%06d.jpg")
# setting up the VideoCapture-elements with the according format

shapeRGB = capRGB.read()[1].shape
shapeIR = capIR.read()[1].shape
# get the shape of the first image in each folder to later create the black
# dummy-image

dtypeRGB = capRGB.read()[1].dtype
dtypeIR = capIR.read()[1].dtype
# get the type of the first image in each folder to later create the black
# dummy-image

if (capRGB.isOpened() is False):
    print("Error opening RGB images")
if (capIR.isOpened() is False):
    print("Error opening IR images")

cv2.namedWindow("frames", cv2.WINDOW_NORMAL)

while capRGB.isOpened() and capIR.isOpened() is True:
    retRGB, imgRGB = capRGB.read()
    retIR, imgIR = capIR.read()
    # read both images

    if retRGB is True and retIR is False:
        imgIR = np.zeros(shapeIR, dtype=dtypeIR)
    # if there is no IR image, crate a dummy one
    if retIR is True and retRGB is False:
        imgRGB = np.zeros(shapeRGB, dtype=dtypeRGB)
    # if there is no RGB image, crate a dummy one
    if retRGB is False and retIR is False:
        break

    imgCombined = np.hstack((imgRGB, imgIR))
    # put both images together
    cv2.imshow("frames", imgCombined)
    k = cv2.waitKey(1)
    if k == ord("q"):
        break

capRGB.release()
capIR.release()
cv2.destroyAllWindows()

Problem: From my understanding, the problem arises as capIR.read() attempts to read a missing image (in my case the 527th) and instead of just returning false/None it attempts to read the same image over and over again. Up to the missing frame, everything works fine, the right "IR" image even turns black but then the videoplayback begins to slow down and while i still can close the window by pressing 'q', spyder IDE freezes and if I wait "too long" i even have to shut it down. Console gives out "[image2 @ 000002a7af8f0480] Could not open file: Bilder/IR/frame_000527.jpg" over and over again, so much that i can't scroll to the top. I guess what I'm asking is: Is there any way to make the.read() function just attempt 1 read and after it fails continue with the next frame?

Best regards and thank you very much in advance!

Answer 1

Assuming that the images in directory1 have the same names as directory2 images, but we know that some image may not be present in both directories...

import glob,os,cv2

path1 = "folder1/"
path2 = "folder2/"

l1 = glob.glob(path1+"*.jpg")
l2 = glob.glob(path2+"*.jpg")#not used 

blackimg = cv2.imread("blackimg.jpg")

for fname in l1:

    #check if image1 exists , then read it .  otherwise im1 = blackimg
    if os.path.isfile(path1+fname):
        im1=cv2.imread(path1+fname)
    else:
        im1=blackimg
    
    #check if image2 exists , then read it .  otherwise im2 = blackimg
    if os.path.isfile(path2+fname):
        im2=cv2.imread(path2+fname)
    else:
        im2=blackimg

    imgCombined = np.hstack((im1, im2))


    cv2.imshow("Combined", imgCombined)
    print("press any key to continue, q to exit")
    k = cv2.waitKey(0) 
    if k == ord("q"):break


cv2.destroyAllWindows()

Answer 2

Simulated for testing with different files and directory names.

Will retrieve the largest frame number from both directories and afterwards iterate over all frame numbers for reading the files from both directories.

import os
import cv2
import re
import glob

image_dir1 = 'test1'
image_dir2 = 'test2'

# retrieve all frames in both directories
frames_cap1 = glob.glob(os.path.join(image_dir1, "frame_*.jpg"))
frames_cap2 = glob.glob(os.path.join(image_dir2, "frame_*.jpg"))

# sort inscending
frames_cap1.sort()
frames_cap2.sort()

# retrieve last frame No for both directories
last_frame_cap1 = frames_cap1[-1]
last_frame_cap2 = frames_cap2[-1]

# extract integer counter
match_cap1 = re.search('frame_(\d+).jpg', last_frame_cap1)
match_cap2 = re.search('frame_(\d+).jpg', last_frame_cap2)
last_frame_no_cap1 = int(match_cap1.group(1))
last_frame_no_cap2 = int(match_cap2.group(1))

# retrieve max frame No
max_frame_no = max(last_frame_no_cap1, last_frame_no_cap2)

for i in range(max_frame_no + 1):
    image_path_cap1 = os.path.join(image_dir1, f"frame_{i:06d}.jpg")
    image_path_cap2 = os.path.join(image_dir2, f"frame_{i:06d}.jpg")
    
    if not os.path.isfile(image_path_cap1):
        print(f"handle missing file: '{image_path_cap1}'")
        # ...
    else:
        img1 = cv2.imread(image_path_cap1)
        # …

    if not os.path.isfile(image_path_cap2):
        print(f"handle missing file: '{image_path_cap2}'")
        # ...
    else:
        img2 = cv2.imread(image_path_cap2)
        # …
# …