OpenCV error: (-215:Assertion failed) (M0.type() == CV_32F || M0.type() == CV_64F) && M0.rows == 3 && M0.cols == 3 in function 'warpPerspective'

Question

I am trying to perform image registration for two RGB lung mask images of size 128x128. This had worked fine for other images when I was learning image registration but now somehow it throws such error. I am a newbie learning this, any help is appreciated.

I have attached the code of what I am trying to do below, where I have created a registerImage function by following GeeksForGeeks and passed images which I want to register.

import cv2
import numpy as np

def registerImage(img1,img2):
  # Open the image files.
  img1_color = img1  # Image to be aligned.
  img2_color = img2   # Reference image.
  
  # Convert to grayscale.
  img1 = cv2.cvtColor(img1_color, cv2.COLOR_BGR2GRAY)
  img2 = cv2.cvtColor(img2_color, cv2.COLOR_BGR2GRAY)
  height, width = img2.shape
  
  # Create ORB detector with 5000 features.
  ## used to creates keypoints on the reference image
  orb_detector = cv2.ORB_create(5000)
  
  # Find keypoints and descriptors.
  # The first arg is the image, second arg is the mask
  #  (which is not required in this case).
  kp1, d1 = orb_detector.detectAndCompute(img1, None)
  kp2, d2 = orb_detector.detectAndCompute(img2, None)
  
  # Match features between the two images.
  # We create a Brute Force matcher with
  # Hamming distance as measurement mode.

  #Brute-Force matcher is simple. 
  #It takes the descriptor of one feature in first set and is matched with all other features in second set using some distance calculation. And the closest one is returned.
  matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True)
  
  # Match the two sets of descriptors.
  matches = matcher.match(d1, d2)
  
  # Sort matches on the basis of their Hamming distance.
  matches.sort(key = lambda x: x.distance)
  
  # Take the top 90 % matches forward.
  matches = matches[:int(len(matches)*0.9)]
  no_of_matches = len(matches)
  
  # Define empty matrices of shape no_of_matches * 2.
  p1 = np.zeros((no_of_matches, 2))
  p2 = np.zeros((no_of_matches, 2))
  
  for i in range(len(matches)):
    p1[i, :] = kp1[matches[i].queryIdx].pt
    p2[i, :] = kp2[matches[i].trainIdx].pt
  
  # Find the homography matrix.
  homography, mask = cv2.findHomography(p1, p2, cv2.RANSAC)
  
  # Use this matrix to transform the
  # colored image wrt the reference image.
  transformed_img = cv2.warpPerspective(img1_color,
                      homography, (width, height))
  
  # Save the output.
  # cv2.imwrite('output.jpg', transformed_img)

  img1_show = cv2.resize(img1_color,(320,320))
  img2_show = cv2.resize(img2_color,(320,320))
  img3_show = cv2.resize(transformed_img,(320,320))
  img = np.concatenate((img1_show,img2_show,img3_show), axis=1)
  cv2_imshow(img)



ref_path = path + "/mask_0.png"
test_path = path + "/mask_8.png"


from google.colab.patches import cv2_imshow
ref_mask = cv2.imread(ref_path)
cv2_imshow(ref_mask)

test_mask = cv2.imread(test_path)
cv2_imshow(test_mask)

registerImage(ref_mask,test_mask)

############################################################################
Error:
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-18-b7a8933e693e> in <module>()
----> 1 registerImage(ref_mask,test_mask)

<ipython-input-2-3a703c66a8e0> in registerImage(img1, img2)
     54   # colored image wrt the reference image.
     55   transformed_img = cv2.warpPerspective(img1_color,
---> 56                       homography, (width, height))
     57 
     58   # Save the output.

error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/imgwarp.cpp:3167: error: (-215:Assertion failed) (M0.type() == CV_32F || M0.type() == CV_64F) && M0.rows == 3 && M0.cols == 3 in function 'warpPerspective'

Answer 1

Go backwards from where you got the error and trace it back to find what is giving you the error.

You're getting errors at this statement:

  transformed_img = cv2.warpPerspective(img1_color,
                      homography, (width, height))

What I'd suggest is to check the variables themselves that are fed into the function. Here's a crude but simple way to do it.

# Place these statements prior to the error and check their values
print("hemography:",hemography) # In my case this was None, which is why I got an error
print("h:",height,"w:", width) # Check width/height seems correct
cv2_imshow(img1_color) # Check image is loaded properly

Also worth noting if it comes up while others are searching. For similar code I got an error at the matches.sort(..) line due to the type being Tuple and not a list.

To fix it I changed it to the following:

matches = sorted(matches,key=lambda x:x.distance, reverse=False)

Debugging is a skill you'll pickup over time, as are asking/answering questions here. That error isn't simple to understand if you're new to python, which is how I found this in the first place. The documentation will become easier to understand over time also.