Python LocateOnScreen not working in fullscreen

Question

Hey guys I'm new to python and I have spent roughly 30 hours trying to figure out how to fix this issue. Sorry that this is so long I tried giving as much info about what I tried already.

I am searching for a cat walking in range of my standing character in a full screen game using pyautogui locateOnScreen(). The problem is that the image is found when I have the shell or command prompt window running the script open in front of my game but if I start the script and focus the game screen immediately, the image is not found at all.

I created a new script with only the cat finding part since the rest of the main script is only keyboard input commands to run to the cat and then stand still.

import pyautogui
import time
import win32api
import keyboard  # in my main script I use hotkeys and keyboard seemed to be the simplest route

def main():
    time.sleep(3)
    print('ok ready')

    while not keyboard.is_pressed('c'):
        if pyautogui.locateOnScreen('catplant.png', region=(260, 450, 300, 250), confidence=0.4) is not None:
            print('cat found')
            keyboard.press('o') # pushes button to start interacting with the cat when found
        else:
            print('cat not found')
        time.sleep(0.1)


if __name__ == '__main__':
    main()

note: the region is 100% correct + extra range since i can find the cat with the command window open and I used win32api.SetCursorPos and a for loop to outline the range and then tested to see when the cat should be in range. Also the confidence has to be 40-50 % as the background seems to be too similar to the cat image and the cat has different animations while walking.

The thing that infuriates me the most that sometimes when only the print('cat found') line is there, it will show that the cat was found, though still inconsistent. As soon as I add any command, mouse click or keyboard input after the cat is found, the script does not find the image at all besides with a command window open. Does not work when print is replaced with any other command either. I tried adding delays as I know the function can be slow. Didn't work.

I've checked that the active window is the game using code just to make sure and the results are still the same.

Also when I run the script and the cat is not found as soon as I alt+tab the cat is found if in range and the cat will also sometimes still be found despite the window now hiding the cat. This does however not work when i hide the cat with windows in other instances like in a picture of the cat in paint or the starting with command prompt window in foreground for example.

I might not be very experienced but I have a feeling it might be an issue with pyautogui. If I can't find a solution I'll have to resort to OpenCV which seems quite complicated and I would not prefer going into that for now. Thank you for taking your time to read and potentially try to solve this problem for me. This is my first post on this website.

Answer 1

Update I finally found a solution reading through an old thread about mouse inputs in full screen games. I run my locate on screen with the game in bordered mode and it works perfectly. For some reason full screen does not work