Image Processing: Bad Quality of Disparity Image with OpenCV

Question

I want to create a disparity image using two images from low resolution usb cameras. I am using OpenCV 4.0.0. The frames I use are taken from a video. The results I am currently getting are very bad (see below).

Both cameras were calibrated and the calibration data used to undistort the images. Is it because of the low resolution of the left image and right image?

Left:

Right:

To have a better guess there also is an overlay of both images.

Overlay:

The values for the cv2.StereoSGBM_create() function are based on the ones of the example code that comes with OpenCV (located in OpenCV/samples/python/stereo_match.py ).

I would be really thankful for any help or suggestions.

Here is my code:

# convert both image to grayscale
left = cv2.cvtColor(left, cv2.COLOR_BGR2GRAY)
right = cv2.cvtColor(right, cv2.COLOR_BGR2GRAY)

# set the disparity matcher        
window_size = 3
min_disp = 16
num_disp = 112-min_disp
stereo = cv2.StereoSGBM_create(minDisparity = min_disp,
    numDisparities = num_disp,
    blockSize = 16,
    P1 = 8*3*window_size**2,
    P2 = 32*3*window_size**2,
    disp12MaxDiff = 1,
    uniquenessRatio = 10,
    speckleWindowSize = 100,
    speckleRange = 32
)

# compute disparity
dis = stereo.compute(left, right).astype(np.float32) / 16.0

# display the computed disparity image
matploitlib.pyplot.imshow(dis, 'gray')
matploitlib.pyplot.show()

Answer 1

Most stereo algorithms require the input images to be rectified. Rectification transforms images so that corresponding epipolar lines are corresponding horizontal lines in both images. For rectification, you need to know both intrinsic and extrinsic parameters of your cameras.

OpenCV has all the tools required to perform both calibration and rectification. If you need to perform calibration, you need to have a calibration pattern (chessboard) available as well.

In short:

1. Compute intrinsic camera parameters using calibrateCamera() .
2. Use the intrinsic parameters with stereoCalibrate() to perform extrinsic calibration of the stereo pair.
3. Using the paramters from stereoCalibrate() , compute rectification parameters with stereoRectify()
4. Using rectification parameters, calculate maps used for rectification and undistortion with initUndistortRectifyMap()

Now your cameras are calibrated and you can perform rectification and undistortion using remap() for images taken with the camera pair (as long as the cameras do not move relatively to each other). The rectified images calculated by remap() can now be used to calculate disparity images.

Additionally, I recommend checking out some relevant text book on the topic. Learning OpenCV: Computer Vision with the OpenCV Library has a very practical description of the process.