cv2.matchTemplate 在图像中发现错误的模板

Question

I'm trying to create a program that knows what number is on an image with the following function:

def img_in_img(big_picture, small_picture, tamper):
    big_picture = str(big_picture)
    small_picture = str(small_picture)
    if os.path.isfile(big_picture) == False or os.path.isfile(small_picture) == False:
        return "Image does not exist"

    img = cv2.imread(big_picture,0)
    templace_loc = small_picture
    template = cv2.imread(templace_loc,0)
    w, h = template.shape[::-1]
    method = cv2.TM_CCOEFF
    
    tamper = int(tamper)

    res = cv2.matchTemplate(img,template,method)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)

    top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)

    height, width, channels = cv2.imread(templace_loc).shape

    if int(top_left[0]) < int(height + tamper) and int(top_left[0]) > int(height - tamper) and int(top_left[1]) < int(width + tamper) and int(top_left[1]) > int(width - tamper):
        return True
    else:
        return False

But then when I check if 7.png is in img.png with the code

nur = "7"
if img_in_img("2020-01-14-17-36-08.537043/verification_image2.png", "verifynr/" + "old_orange" + "/" + nur + ".png", 25):
    print(Style.BRIGHT + Fore.GREEN + "Color: " + "old_orange" + ", Num: " + nur + Fore.RESET)
else:
    print(Style.BRIGHT + Fore.RED + "Color: " + "old_orange" + ", Num: " + nur + Fore.RESET)

it gives me in RED: Color: old_orange, Num: 7

but then if I check if 6.png is in img.png by changing nur from 7 to 6 it gives me in Green: Color: old_orange, Num: 6 , but that's the wrong image.

I've also tried the following code:

img_rgb = cv2.imread("2020-01-14-17-36-08.537043/verification_image2.png")
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('verifynr/old_orange/7.png',0)
w, h = template.shape[::-1]

res = cv2.matchTemplate(img_gray,template,cv2.TM_SQDIFF)
threshold = 0.8
loc = np.where( res >= threshold)
pt = list(zip(*loc[::-1]))

if len(pt) >= 1:
    print("True")

which prints True , but it does that for every number png I saved.

How do I make my program recognize the 7.png in the img.png without recognizing every single number png?

img.png:

6.png:

7.png:

Answer 1

Template matching not for object detection or pattern recognition, it's function is to find the most similar patch position. For detection use detectors (haar, dnn based ones, ...), for recognition use classifiers (descriptor based or nn based). '

Answer 2

Here is an example of using a template mask to do template matching at one resolution level using Python/OpenCV. The black region of the mask tells the template matching to ignore those regions when matching, so that when the template has a background color different from that in the larger image, it does not degrade the match score. Only the color regions of the template corresponding to the white in the mask contribute to the matching score. Note that masked template matching in OpenCV only works for methods TM_SQDIFF and TM_CCORR_NORMED.

Input:

Template:

Template Mask:

import cv2
import numpy as np

# read image
img = cv2.imread('logo.png')

# read template
tmplt = cv2.imread('hat.png')
hh, ww, cc = tmplt.shape

# read template mask as grayscale
tmplt_mask = cv2.imread('hat_mask.png', cv2.COLOR_BGR2GRAY)

# do template matching
corrimg = cv2.matchTemplate(img,tmplt,cv2.TM_CCORR_NORMED, mask=tmplt_mask)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrimg)
max_val_ncc = '{:.3f}'.format(max_val)
print("correlation match score: " + max_val_ncc)
xx = max_loc[0]
yy = max_loc[1]
print('xmatch =',xx,'ymatch =',yy)

# draw red bounding box to define match location
result = img.copy()
pt1 = (xx,yy)
pt2 = (xx+ww, yy+hh)
cv2.rectangle(result, pt1, pt2, (0,0,255), 1)

cv2.imshow('image', img)
cv2.imshow('template', tmplt)
cv2.imshow('template_mask', tmplt_mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

# save results
cv2.imwrite('logo_hat_match.png', result)

Result:

Textual Information Listed:

correlation match score: 1.000
xmatch = 417 ymatch = 44

(Apologies for borrowing and modifying the ImageMagick logo. I already had that template matching example)

Answer 3

These are some few potential causes:

Scale and rotation difference: If the template image and the image you are searching for the template in have different scales or rotations, cv2.matchTemplate() may not find a match. You can try using techniques such as scaling or rotating the template image to match the image you are searching in.

Lighting and contrast differences: If the lighting or contrast of the template image and the image you are searching in are different, it can affect the accuracy of the match. You can try adjusting the brightness and contrast of the images to make them more similar.

Template size: If the template is too small or too big compared to the area you are searching in, it can affect the accuracy of the match. You should try using a template size that is similar to the expected size of the object in the image.

Similarity threshold: The matchTemplate() method has a threshold value that you can set to control the level of similarity required for a match. If the threshold value is set too high, it may not find any matches even if there are similar templates in the image. If it's set too low, it may return false positives.

Similarity Measurement Algorithm: matchTemplate() uses a specific algorithm for measuring the similarity between the template and the search image. Depending on the image and the template, one algorithm might work better than another.