Reconciling pixel coordinates on tkinter canvas and PIL image (Python)

Question

I have a form (in .png format) with blank text boxes (usually filled out by hand). I would like to populate the boxes with text.

To do this, I am using tkinter to display the form on the screen, I then use the mouse (with finer positioning using arrow keys) to get the pixel coordinates of a box, then use PIL to write text to that box. A working example is below.

My primary issue is I am struggling to align pixel coordinates in the tkinter canvas and pixel coordinates in the PIL image.

Some additional background. The image is in high resolution and is circa 4961 by 7016 pixels. My screen resolution is 1920 x 1080. I had problems writing text where I needed it to write like this and found greater success if I scaled the image to fit entirely within my screen. I can only assume this is because I was / am confusing screen pixels with picture pixels (and resolved this when I fit the image to the screen to align these two - but understanding the differences here and how to do this task without scaling would be most helpful also).

But I am also having trouble reconciling the pixel coordinates on the tkinter canvas with the PIL picture. For example, the code below is designed to write (x, y) pixel coordinates and then its page relativity {x% across the page, y% up the page} to the box (the reason for this is because it is an input into another process). An example is: (346, 481) >> {49.856, 51.018}

But if (using a scaling factor of 0.14) I click very low to the bottom of the image, I get (209, 986) >> {30.115, -0.407}. The relativities should be bounded between 0 and 100% so should not be negative and I cannot see this on my PIL produced .png file.

If I use a scaling factor of 0.125, I can write text to the tkinter canvas box fine, but the text appears quite a bit lower (ie outside the box) on the PIL .png file which is saved to the drives. So something is clearly not working between these two systems.

How can I reconcile PIL and tkinter pixel coordinates?

As an aside, I have a four separate functions to handle finer key adjustments. Ideally these would be one function, but I could not get the arrow buttons ( etc) to work inside an if elif block (eg, I tried this and some more derivatives of left, right etc)

def mouseMovement(event):
    moveSpeed = 1 
    try: 
        int(event.char)
        moveSpeed = max(1, int(event.char)*5)
        return True
    except ValueError:
        return False

    x, y = pyautogui.position()

    if event.char == '<Left>':
        pyautogui.moveTo(x-moveSpeed, y)     
    elif event.char == '<Right>':
        pyautogui.moveTo(x+moveSpeed, y)          

root.bind('<Key>'  , mouseMovement)

Any help greatly appreciated!

Almost working example below:

from tkinter import *
from PIL import Image, ImageDraw, ImageFont, ImageTk
import pyautogui


# Form

formName = '2013+ MCS4'

# PIL image'

formImage = Image.open(formName+'.png')
wForm, hForm = formImage.size
scale = 0.14
formImage = formImage.resize((int(scale*wForm), int(scale*hForm)), Image.ANTIALIAS)
draw = ImageDraw.Draw(formImage)

font = ImageFont.truetype('arial.ttf', 10)
textColor = (255, 40, 40)

# tkinter canvas

def colorConversion(RGB):
    def hexadecimalScale(RGB):
        hexadecimalSystem = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'A', 'B', 'C', 'D', 'E', 'F')
        return str(hexadecimalSystem[RGB//16]) + str(hexadecimalSystem[RGB%16])
    return '#' + hexadecimalScale(RGB[0]) + hexadecimalScale(RGB[1]) + hexadecimalScale(RGB[2])

fontCanvas = 'arial 7'
textColorCanvas = colorConversion(textColor)


# generate canvas

if __name__ == '__main__':

    root = Tk()

    # set up tkinter canvas with scrollbars

    frame = Frame(root, bd=2, relief=SUNKEN)
    frame.grid_rowconfigure(0, weight=1)
    frame.grid_columnconfigure(0, weight=1)
    xscroll = Scrollbar(frame, orient=HORIZONTAL)
    xscroll.grid(row=1, column=0, sticky=E+W)
    yscroll = Scrollbar(frame)
    yscroll.grid(row=0, column=1, sticky=N+S)
    canvas = Canvas(frame, width=int(scale*wForm), height=int(scale*hForm), bd=0, xscrollcommand=xscroll.set, yscrollcommand=yscroll.set)
    canvas.grid(row=0, column=0, sticky=N+S+E+W)
    xscroll.config(command=canvas.xview)
    yscroll.config(command=canvas.yview)
    frame.pack(fill=BOTH,expand=1)

    # add image

    #img = PhotoImage(file=formName+'.png')
    img = ImageTk.PhotoImage(formImage)
    canvas.create_image(0,0,image=img,anchor="nw")
    canvas.config(scrollregion=canvas.bbox(ALL))

    wForm = img.width()
    hForm = img.height()

    # finer mouse movements

    moveSpeed = 1 

    def setMoveSpeed(event):
        global moveSpeed 
        try: 
            int(event.char)
            moveSpeed = max(1, int(event.char)*5)
            return moveSpeed
        except ValueError:
            return False

    def moveMouseLeft(event):
        x, y = pyautogui.position()
        pyautogui.moveTo(x-moveSpeed, y)

    def moveMouseRight(event):
        x, y = pyautogui.position()
        pyautogui.moveTo(x+moveSpeed, y)

    def moveMouseUp(event):
        x, y = pyautogui.position()
        pyautogui.moveTo(x, y-moveSpeed)

    def moveMouseDown(event):
        x, y = pyautogui.position()
        pyautogui.moveTo(x, y+moveSpeed)

    root.bind('<Key>'  , setMoveSpeed)
    root.bind('<Left>' , moveMouseLeft)
    root.bind('<Right>', moveMouseRight)
    root.bind('<Up>'   , moveMouseUp)
    root.bind('<Down>' , moveMouseDown)


    # print coordinates

    def printCoordinates(event):
        x = event.x    # minor adjustments to correct for differences in tkinter vs PIL methods (investigate further)
        y = event.y    # minor adjustments to correct for differences in tkinter vs PIL methods (investigate further)
        canvas.create_text(x, y-5, fill= textColorCanvas, font= fontCanvas, anchor= 'sw', 
                           text= '{'+str(round(x/wForm*100,3))+', '+str(round((1-y/hForm)*100,3))+'}' )
        draw.text( (x, y-5), '{'+str(round(x/wForm*100,3))+', '+str(round((1-y/hForm)*100,3))+'}' , fill=textColor, font=font)
        print('('+str(x)+', '+str(y)+') >> {'+str(round(x/wForm*100,3))+', '+str(round((1-y/hForm)*100,3))+'}')

    root.bind('<Return>', printCoordinates)

    root.mainloop()

formImage.save('coordinates - '+formName+'.png')

Answer 1

I cannot run your code, so this is just an educated guess.

Since the canvas doesn't typically have focus, and the binding is on the root window, the values for event.x and event.y are possibly relative to the window as a whole rather than the canvas.

This should be easy to determine. You can print out the coordinates in the binding, and then click in the upper left corner of the canvas, as near to 0,0 as possible. The printed coordinates should also be very near to 0,0 if the coordinates are relative to the canvas. If they are off, they might be off by the distance of the top-left corner of the canvas to the top-left corner of the window.