Image interpolation with cv2.resize() changes the integer positon pixel

Question

I'm trying to interpolate up an image with cv2.resize() , while keeping the integer position pixel in new image same as the original image. So I wrote this code to test:

import cv2, scipy
import scipy.misc
import numpy as np

im = cv2.imread('img')
print(im[:,:,0].shape)
print(im[:,:,0])

im = cv2.resize(im, None, fx=2, fy=2, interpolation=cv2.INTER_LANCZOS4)

print(im[::2,::2,0].shape)
print(im[::2,::2,0])

Output:

(128, 128)
[[79 62 64 ... 81 81 79]
 [73 59 57 ... 79 81 79]
 [69 51 51 ... 90 88 87]
 ...
 [40 48 43 ... 79 84 88]
 [45 46 44 ... 84 84 83]
 [48 46 44 ... 80 80 83]]
(128, 128)
[[82 66 63 ... 82 81 80]
 [76 64 57 ... 80 79 79]
 [71 57 50 ... 90 85 85]
 ...
 [38 46 45 ... 77 84 87]
 [43 46 45 ... 83 84 84]
 [48 47 44 ... 81 81 82]]

The above code interpolated the image 2x and ideally the two output should be the same, since im[::2,::2,0] should be the pixel before interpolation.

Is there something wrong in the code or anyway to do it properly?

Answer 1

I found an explanation of the OpenCV coordinate system at some point, but it can't find it back, so this answer is from memory.

OpenCV sees a pixel as a little square, not a point sample in a grid. Interpolation by an integer amount splits up this square into smaller squares. For even-sized interpolation, none of the new square centers will match the original square center.

For odd-sized interpolation, one of the new squares will be at the center, but there will also be other new squares to the top and left of the top-left pixel of the original image. These pixels have been extrapolated!

That is, if you think of an image as a set of samples, like I do, then interpolation in OpenCV will never match your expectations.

The solution is to shift the image by half an original pixel to the left and up while interpolating. Not sure how to implement that in OpenCV though.

Or you could try skimage.transform.rescale in skimage or diplib.Resampling in DIPlib (I'm an author). For the last one, I know it does the right thing out of the box.