Recognizing black and white images with OpenCV

Question

I have this set of images:

The leftmost one is the reference image.

I want to have a value telling me how close is any of the other images to the leftmost one. I experimented with matchShapes(), by calling it for each contour and averaging the values, but I didn't get useful result (the rightmost one had a too high value, for example)

I would also want the matching to work only in the correct orientation.

Answer 1

If they're purely black and white images it would probably be easier to just AND the two pictures together and sum up the total pixels left in the result. Something like this:

import cv2
import numpy as np
x = np.zeros((100,100))
y = np.zeros((100,100))
for i in range(25,75):
    x[i][i] = 255
    y[i][100-i] = 255

cv2.imshow('x', x)
cv2.imshow('y', y)

z = cv2.bitwise_and(x,y)

sum = 0 
for i in range(0,z.shape[0]):
    for j in range(0,z.shape[1]):
        if z[i][j] == 255:
            sum += 1

print(f"Similarity Score: {sum}")


cv2.imshow('z',z)
cv2.waitKey(0)

There probably exists some better library to perform this all in one line but if performance isn't much of a concern perhaps this could work.

Answer 2

It was difficult to not recognize images that were too different. With the methods proposed here, I always got really close values for images that I thought were too different to correspond.

In the end, I did a multistep process:

First I got the contour of the test image like so:

testContours, _ = cv.findContours(testImage, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)

Then, if the contour count between the test image and the original image are not the same, I abort.

If they have the same contour count, I then calculate the average between all shape distances of the contours:

    distances = []
    sd = cv2.createShapeContextDistanceExtractor()
    for i in range(len(testContours)):
        d2 = sd.computeDistance(testContours[i], originalContours[i])
        distances.append(d2)
    value = sum(distances) / len(distances)

Then, I count the number of white pixels after AND-ing the two images, divided by the total number of pixels in the source image (in case the contours match but are not placed correctly)

     exactly_placed_ratio = cv.countNonZero(cv.bitwise_and(testImage, originalImage)) / cv.countNonZero(originalImage)

In the end I have two values, I can use the first one to check if the shapes are close enough, and the second one to check if they are in the right position relative to the whole image.