Locate second location of a similar image using pyautogui

Question

The problem seems quite simple, I have two similar images on the screen, using pyautogui.LocateOnScreen() it can normally locate the first image, using moveTo() makes me able to move to the image in question. The problem is that I have more than one image, and I want it to detect the first, then move to it, then detect the other and finally move to it. Can anyone help me with this question? Thanks. PS: Both images are equal

Answer 1

Method 1

Using pyautogui.locateAllOnScreen() :

import pyautogui
image = 'TemplateImage.png'
for box in pyautogui.locateAllOnScreen(image):
    print(pyautogui.center(box))

pyautogui.locateAllOnScreen() returns a list of 4-tuples representing (left, top, width, height) of a box surrounding the region in which the template is located. To find the center of this box, we called pyautogui.center()

Method 2

If the template images are always located in the same part of the screen, we can use the region argument to restrict the location in which pyautogui.LocateOnScreen() searches for the template. We search two times, each time in a different region:

box1 = pyautogui.locateOnScreen('TemplateImage.png', region=(0,0, 300, 400))
center1 = pyautogui.center(box1)
box2 = pyautogui.locateOnScreen('TemplateImage.png', region=(300, 400, 300, 400))
center2 = pyautogui.center(box2)

Method 3

Use opencv-python for template matching. After detecting your template image in the screenshot, you can mask the detected region and re-run template matching:

import cv2
import pyscreeze
import pyautogui
import matplotlib.pyplot as plt

#load the image to find
image = 'TemplateImage.png'
templateim = pyscreeze._load_cv2(image,grayscale=False) 
(w, h,_) = templateim.shape

# get screenshot
screenim_color = pyautogui.screenshot() 
screenim_color = cv2.cvtColor(np.array(screenim_color),cv2.COLOR_RGB2BGR)

Nsearch = 2 # number of times to search
center_locations = [] #list to store locations
for k in range(Nsearch):
    result = cv2.matchTemplate(screenim_color, templateim, cv2.TM_CCOEFF_NORMED) # template matching
    
    (_, _, _, maxLoc) = cv2.minMaxLoc(result)  # get the location  where the correlation coefficient is maximum (This is the top-left tip of the box surrounding the template)

    cv2.rectangle(screenim_color, maxLoc, (maxLoc[0] + h, maxLoc[1] + w), (0,0,255), -1) # mask the detected region by drawing a solid rectangle over it

    center_loc = (int(maxLoc[0]+h/2),int(maxLoc[1]+w/2)) # get the center location 
    center_locations.append(center_loc) # append to list

plt.figure()
plt.imshow(screenim_color)

Answer 2

Giving some text examples of the code is great. So, I wouldn't have to ask this silly question. Have you tried the "while" loop?