Can't get image registration with OpenCV 4.0.0 and Python 3.7 to work

Question

I'm trying to use the image registration module from OpenCV 4.0.0 from Python 3.7. I define a trivial image m1 and a shifted version m2. Then I try to find the shift both with cv2.reg_MapperGradShift( ) and a combination of cv2.reg_MapperPyramid( ) and cv2.reg_MapperGradShift( )

m1 = numpy.zeros( ( 10, 10 ) )
m1[ 3, 3 ] = 1
m2 = numpy.zeros( ( 10, 10 ) )
m2[ 3, 6 ] = 1

t1 = cv2.reg_MapperGradShift( )
t2 = t1.calculate( m1, m2 )
t3 = cv2.reg.MapTypeCaster_toShift( t2 )
t4 = t3.getShift( )
print( t4 )

t5 = cv2.reg_MapperPyramid( t1 )
t6 = t5.calculate( m1, m2 )
t7 = cv2.reg.MapTypeCaster_toShift( t6 )
t8 = t7.getShift( )
print( t8 )

the results are

[[0.]
 [0.]]
[[9.11379962]
 [6.47128924]]

How can I make it find the right shift?

Also, when I try to run calculate( ) on 2 images with a different size I get

cv2.error: OpenCV(4.0.0) C:\\projects\\opencv-python\\opencv\\modules\\core\\src\\arithm.cpp:663: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'cv::arithm_op'

Shouldn't it be possible to register images with different sizes (to find a smaller image within a larger one)?

Edit: I played around some more and I found that when I define larger images and set a pixel somewhere in the middle, I get the right answer with cv2.reg_MapperPyramid( ), but when I set a pixel near the border, I still get wrong answers. For example

m1 = numpy.zeros( ( 100, 100 ) )
m1[ 30, 30 ] = 1
m2 = numpy.zeros( ( 100, 100 ) )
m2[ 30, 33 ] = 1

t1 = cv2.reg_MapperGradShift( )
t2 = t1.calculate( m1, m2 )
t3 = cv2.reg.MapTypeCaster_toShift( t2 )
t4 = t3.getShift( )
print( t4 )

t5 = cv2.reg_MapperPyramid( t1 )
t6 = t5.calculate( m1, m2 )
t7 = cv2.reg.MapTypeCaster_toShift( t6 )
t8 = t7.getShift( )
print( t8 )

results in

[[0.]
 [0.]]
[[3.02103606]
 [0.        ]]

Could it have something to do with how the registration module extrapolates images (the borderMode or borderType parameters used in other functions)?

Answer 1

You need to change the datatype of the images from int to float32 or float64. It doesn't work if you do not do this.

Answer 2

You should use matchTemplate , it allows you to find a template in an image. For example, if your template is

and your image is

.

The result will be

(The images can be found in the OpenCV documentation)

According to your needs you can choose one method or another, for example, TM_CCORR_NORMED is robust but TM_SQDIFF is needs less calculations.

UPDATE

It will easily work for translation, but for more complicated transformations You should use Features2D and homography to find your template. Here is the OpenCV documentation example:

The algorithm idea is :

1. Extract the key points of each image
2. Compute descriptors for each key points (eg SIFT or SURF)
3. Match the key points thanks to the descriptors
4. Compute the transformation parameters from the matches

to extract key points in each images and then match the descriptors.

Maybe this should also help.