python matchtemplate not working properly with mask enabled

Question

I am trying to matchtemplate the same text on different images. the background of the images vary and The brightness of the text varies from gray (150) to pure white (255) but it will have the same pattern so I used TM2.CCOEFF_NORMED. The problem is matchtemplate without mask works so much better than with it and I dont understand any reason behind this. Is there something I might be missing?

Also is there any difference between giving black background with white text image as mask and and giving transparent background as mask, extracting base and alpha channel seperetely like below?

template image with blackbackground with white color, Template1

template image with transparent to extract base and alpha, Template2

image1 image2 image3 image4 image5

the text color for Template2 should be 255,255,255 but i changed it to 159,159,159, so you can see the text import cv2 import numpy as np import matplotlib.pyplot as plt

for i in range(1,6):
# read game image
    img = cv2.imread(f'image{i}.png')

# read bananas image template
    template = cv2.imread('noitem.png', cv2.IMREAD_UNCHANGED)
    hh, ww = template.shape[:2]

    print (template.shape)
    # extract bananas base image and alpha channel and make alpha 3 channels
    base = template[:,:,0:3]
    alpha = template[:,:,3]
    alpha = cv2.merge([alpha,alpha,alpha])

    # do masked template matching and save correlation image
    correlation = cv2.matchTemplate(img, base, cv2.TM_CCOEFF_NORMED, mask=alpha)
    correlation1 = cv2.matchTemplate(img, base, cv2.TM_CCOEFF_NORMED)

    # set threshold and get all matches
    threshhold = 0.9
    loc = np.where(correlation >= threshhold)
    loc1 = np.where(correlation1 >= threshhold)

    # draw matches 
    result = img.copy()
    result1 = img.copy()
    for pt in zip(*loc[::-1]):
        cv2.rectangle(result, pt, (pt[0]+ww, pt[1]+hh), (0,0,255), 1)
        print(pt)

    for pt in zip(*loc1[::-1]):
        cv2.rectangle(result1, pt, (pt[0]+ww, pt[1]+hh), (0,0,255), 1)
        print(pt)

    plt.subplot(1,2,1)
    plt.title('with mask')
    plt.imshow(result[:,:,::-1])

    plt.subplot(1,2,2)
    plt.title('without mask')
    plt.imshow(result1[:,:,::-1])
    plt.show()
    '''
    #cv2.imshow('base',base)
    #cv2.imshow('alpha',alpha)
    cv2.imshow('result',result)
    cv2.waitKey(0)
    cv2.destroyAllWindows()
    '''

result with all the 5 images

Answer 1

Here is one way to do that as I mentioned above in my comments. The template has white text on black background. So copy it as the mask. Find the maximum match location rather than all above a threshold. Use TM_CORR_NORMED not TM_COEFF_NORMED (alternately use TM_SQDIFF or TMSQDIFF_NORMED if your version of OpenCV permits the latter. But then find the minimum location)

Input:

Template and Mask:

import cv2
import numpy as np

# read  image
img = cv2.imread('text_image.png')

# read template
template = cv2.imread('text_template.png')
hh, ww = template.shape[:2]

# copy the template to the mask
mask = template.copy()

# do masked template matching and save correlation image
correlation = cv2.matchTemplate(img, template, cv2.TM_CCORR_NORMED, mask=mask)

# get best match
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(correlation)
max_val_corr = '{:.6f}'.format(max_val)
print("correlation score: " + max_val_corr)
print("match location:", max_loc)

# draw match 
result = img.copy()
cv2.rectangle(result, (max_loc), ( max_loc[0]+ww,  max_loc[1]+hh), (0,0,255), 1)

# save results
cv2.imwrite('text_image_match.png', result)

cv2.imshow('template',template)
cv2.imshow('result',result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Result: