Python: How do I test an image generation tool?

Question

I've create a small tool to generate images (from other images) using PIL. I'm now wondering how I can create automated tests for this as the test would need to compare the generated image to a template image (which was also generated with this tool)?

I expect if I make changes or one of the underlying libraries make changes the images won't be identical on the bit level even if they look identical to humans. What is the common practice in such scenarios? Use a similarity measure and just fail the test below a certain threshold?

Answer 1

I have an idea but it's certainly not an ideal solution. You can open the generated image on full screen(or specify a region) and the code below would search the template image on your screen which would have the generated image:

pyautogui.locateOnScreen('template_image.png', confidence=0.7, region=...)

This will return None if the image is not found. You can read more about this in the documentation: https://pyautogui.readthedocs.io/en/latest/screenshot.html#the-locate-functions

Answer 2

For now I will use the imagehash library and compare hash of a prebuild and approved image with the one generated in the test. I think this is good enough while developing but in between manual visual inspection will still be needed for know, until I trust the hash will deviations.