Affine transformation with 3 points and merging images

Question

I am writing a Python program that shows 2 thermal (low res) drone shots next to each other. The user then chooses 3 points on both pictures after which the second picture should be transformed and merged with the first. The following step the program should show the merged picture with the third picture from the flight next to it etc.

At the moment I'm looking at ways to do the transformation and merging of the images.

I was thinking of reading the images as arrays (a list of lists (= rows)), manually calculating the new location of each pixel with the transformation formula (at the moment I have calculated the transformation matrix with OpenCV), creating a new empty list of lists and pasting the pixel value in the corresponding location in the new list of lists. Afterward, I wanted to fill al the empty 'cells' with zero's.

To merge them I would again create a new list of lists and loop through both images and get the maximum or average value for each cell and filling the empty remaining 'cells' with zero's. This resulting merged image I would then show in the window next to the following image from the flight as I mentioned.

I still don't know how I'm going to predict the size of my list of lists before calculating the new coordinates of the pixel values. Also, this approach seems kind of long and inefficient so I was wondering if there is an easier way to do this using Wand or OpenCV (or any other library).

Answer 1

I did everything manually and it works, one problem I still have is diagonal NoData lines through the tranformed image. I suppose this is the result of the rounding of coordinates because I can't use floats (which are the result of the transformation formula) as indices in the lists. (The 3 sets of points weren't chosen super carefully for this example but you get the point.)

EDIT: I solved the diagonal lines by filling in the calculated value in the center pixels and the 8 other pixels bordering the center pixels. It makes the image blurrier but it's the only solution I've got.