Image Registration of Scanned Text Forms

Question

We print 500 bubble surveys, get them back, and scan them in a giant batch giving us 500 PNG images.

Each image has a slight variations in alignment, but identical size and resolution. We need to register the images so they're all perfectly aligned. (With the next step being semi-automated scoring of the bubbles).

If these were 3D-MRI images, I could accomplish this with a single command line utility; But I'm not seeing any such tool for aligning scanned text documents.

I've played around with opencv as described in Image Alignment (Feature Based) using OpenCV , and it produces dynamite results when it works, but it often fails spectacularly. That approach is looking for documents hidden within natural scenes, a much harder problem than our case where the images are just rotated and translated in 2D, not 3.

I've also explored imreg_dft , which runs consistently but does a very poor job -- presumably the dft approach is better on photographs than text documents.

Does a solution for Image Registration of Scanned Forms already exist? If not, what's the correct approach? Opencv, imreg_dft, or something else?

Similar prior question: How to find blank field on scanned document image

Answer 1

What you can try is using the red outline of the answer boxes to create a mask where you can select the outline. I create a sample below. You can also remove the blue letters by creating a mask for the letters, inverting it, then apply it as a mask. I didn't do that, because he image of the publisher is low-res, and it caused issues. I expect your scans to perform better.

When you have the contours of the boxes you can transform/compare them individually (as the boxes have different sizes). Or you can use the biggest contour to create a transform for the entire document.

You can then use minAreaRect to find the cornerpoints of the contours. Threshold the contourArea to exclude noise / non answer area's.

import cv2
import numpy as np
# load image
img = cv2.imread('Untitled.png')
# convert to hsv colorspace
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# define range of image bachground in HSV
lower_val = np.array([0,0,0])
upper_val = np.array([179,255,237])

# Threshold the HSV image 
mask = cv2.inRange(hsv, lower_val, upper_val)

# find external contours in the mask
contours, hier = cv2.findContours(mask, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
# draw contours
for cnt in contours:
    cv2.drawContours(img,[cnt],0,(0,255,0),3)

# display image
cv2.imshow('Result', img)
cv2.waitKey(0)
cv2.destroyAllWindows()