Image matching using CNN

Question

There are 2 sets of color images: A and B. For each image in A, I want to find the 10 closest images, that is, 10 highest scores, in B.

Moreover, the size of every image in A as well as those of B are different.

Note that it is not classifying images in A into different categories but finding the most similar images.

Furthermore, a score is needed to account for the similarity. If there are 100 images in B, then there should be 100 scores for each image in A.

I know that CNN can be used to solve this problem but I cannot find any information.

What is the approach of doing it?

Are there any projects or examples to follow?

Thank you very much.

Answer 1

basically you can do the following

train an auto-encoder network on the images (using CNN of course) after a sufficient accuracy achieved, use the encoder part of the network to get a vector representing the image

you can use any similarity method between vectors to get the top-n closest vectors

about your second remark about the different sizes, you will have to normalize the images to one common size or use 2 different networks, for A and B

if you would like to use only one network, that means you need to feed it with images corresponding to the input layer dim, so you will have to deflate inflate the image's pixels to resize the image to the selected size, there are multiple techniques to do it and each one has it's pros and cons (most of them regarding of loss of data)

for instance you can merge each 2X2 block to 1 pixel, you can crop the image to the desired size (loosing it's fringe data) and so on... my suggestion is to use the resize api of your image lib at first and after achieving a stable solution and when tuning the network, experiment with other resizing technics

there are a lot of auto-encoder examples out there, depends on the framework you are using... here's a simple one in keras