How to determine pixel coordinates on an image after template matching in python?

Question

I have a 3D film image that has star-like figures in itself. I have a template whose shape is (21,21,1) that helps to find the initial point (rectangle) on the image. I have done the block matching part and be able to determine some matched coordinates which are sometimes correct and incorrect due to the different pixel intensity of the image. The image and template are all gray. The following are my codes , results , and the expected results. Any help (idea) to solve this problem will be acknowledged Codes for template matching

image = cv2.imread('45_gray.jpg', 0 )
template = cv2.imread('45tmpl.jpg', 0)
(tW, tH) = template.shape[::-1]

result = cv2.matchTemplate(image, template, cv2.TM_CCOEFF_NORMED)
#print(result)

threshold = 0.8
(yCoords, xCoords) = np.where(result >= threshold)
clone = image.copy()
#print(yCoords, xCoords)


for (x, y) in zip(xCoords, yCoords):
    # draw the bounding box on the image
    cv2.rectangle(clone, (x, y), (x + tW, y + tH),(255, 247, 263), 3)
    # show our output image *before* applying non-maxima suppression
#cv2.imshow("Before NMS", clone)
#cv2.waitKey(0)
cv2.imwrite('output match.jpg', clone)

Answer 1

I don't think this template matching will work out very well here. As you may notice in the input image there is a gradient from bottom left to top-right, The image is apprearing to fade in that direction. So on the top left side the features are not that pronounced for the template matching to work efficiently. I would recommend to first convert this image to a binary image using adaptiveThreshold technique as:

img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

img_adaptive = cv2.adaptiveThreshold(img_gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 15, 5)

Once you have the bianry image which looks consistent, that is not containing any gradient, you can either follow the template matching path, or in this case I chose to follow an alternate method of detecting the straight lines and then treating their intersection as the point of interest.

Now Detecting the lines could be done using cv2.HoughLinesP , but it has it's own set of parameters which need to be tweaked properly to get it working so I simply counted the number of while pixels present in each row and column respectively and filtered the local maxima points. That is, I chose the rows which had more white pixel count than neighbours and did the same for each column as well.

import cv2
import numpy as np
from scipy.signal import argrelextrema

img = cv2.imread("/home/anmol/Downloads/x5JQF.jpg")
img = cv2.resize(img, None, fx=0.4, fy=0.4)
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

img_adaptive = cv2.adaptiveThreshold(img_gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 11, 5)

cv2.imwrite("./debug.png", img_adaptive)

row_white_pixel_count = np.array([np.count_nonzero(img_adaptive[i,:]) for i in range(img_adaptive.shape[0])])
col_white_pixel_count = np.array([np.count_nonzero(img_adaptive[:,i]) for i in range(img_adaptive.shape[1])])

row_maxima = argrelextrema(row_white_pixel_count, np.greater, order=50)[0]
col_maxima = argrelextrema(col_white_pixel_count, np.greater, order=50)[0]

all_intersection_coords = []
for row_idx in row_maxima:
    for col_idx in col_maxima:
        all_intersection_coords.append((col_idx, row_idx))

for coord in all_intersection_coords:
    img = cv2.circle(img, coord, 10, (0, 240, 0), 2)