OpenCV Match Template - Threshold never equal to 1

Question

I have an HTML 5 Web Application, that uses the Canvas element to draw a rich UI for which I need to write some automation tests.

Since there's no API exposed that allows us to tell what has been rendered on the Canvas element, I decided to use OpenCV to continuously scan the rendered content in the browser in order to find out when something has been drawed.

Therefore, I use the following class:

class IMGLocator:

    def __init__(self, path):
        self.reference = cv.imread(path)

    def find(self, image):
        ref  = cv.cvtColor(self.reference, cv.COLOR_BGR2GRAY)
        nparr = numpy.frombuffer(image, numpy.uint8)
        sourceRGB = cv.imdecode(nparr, cv.IMREAD_UNCHANGED)
        sourceGRAY = cv.cvtColor(sourceRGB, cv.COLOR_BGR2GRAY)
        w, h = ref.shape[::-1]
        res = cv.matchTemplate(sourceGRAY, ref, cv.TM_CCOEFF_NORMED)
        threshold = 0.98
        loc = numpy.where(res >= threshold)

        return cv.rectangle(sourceRGB, (loc[1][0], loc[0][0]), (loc[1][0] + w, loc[0][0] + h), (255, 0, 0), 2)

Currently, the class draws a blue rectangle over the area that matches.

And here's the MAIN entry point for the script.

if __name__ == "__main__":
    driver = webdriver.Firefox()
    driver.get(BACKOFFICE_URI)
    locator = IMGLocator("REF.AUTHENTICATION.jpg")

    time.sleep(5)
    data = locator.find(driver.get_screenshot_as_png())

I want an exact match, so I tried setting the threshold to 1, however, no results are yielded then.

How is that possible, and is there a wat to return an exact match?

Answer 1

The formulas used by the various template matching modes are shown in https://docs.opencv.org/master/df/dfb/group__imgproc__object.html

Note the square root the _NORMED denominators. Assuming the pixels are truly identical, you might seeing an artifact of errors creeping in from the inexact representation of real numbers in floating point from that square root. So, if you've decided on one of the two _NORMED mode, you're stuck with a using threshold. Alternatively, consider cv2.TM_SQDIFF and look for the lowest value. (Either way, cv2.minMaxLoc() is your friend.)