OpenCV Python Feature detection examples extension

Question

I'm chasing a little assistance with an idea I'm playing with. I want to take the features located in an image with code similar to the example on

See sample image at bottom of page here Last section/Example is the one I'm talking about

in particular for my issue I wanted to use the matches indicated in the image to find the target in the scene image like illustrated with a seemingly simple addition. I want to draw a bounding box around the target when located in the scene frame

Example of output I'm after

Rather than just putting a bounding box around the features, I would rather have a list of the four contour points that represent the transformed target on the scene frame if that makes sense.

Big picture, I want to take the subsection of the scene image containing my target and crop it out of the scene image, mask the non-target areas out of the image remaining and then use this as my source for a further process.

At this point I've managed to do all it need to with a hard coded set of points to represent the corners of the target image as rotated and transformed in the scene image so everything works I just need an example of how to determine the x,y co-ords of each corner of the target in that scene

I didn't want to post the code as its a bit clunky and its the concept I'm after, not a complete 'do it for me please' fix

Any advice much appreciated, If you could show me using the example code attached how to do this I'd be very grateful, Cheers.

import numpy as np
import cv2
from matplotlib import pyplot as plt

img1 = cv2.imread('box.png',0)          # queryImage
img2 = cv2.imread('box_in_scene.png',0) # trainImage

# Initiate SIFT detector
sift = cv2.SIFT()

# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)

# FLANN parameters
FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks=50)   # or pass empty dictionary

flann = cv2.FlannBasedMatcher(index_params,search_params)

matches = flann.knnMatch(des1,des2,k=2)

# Need to draw only good matches, so create a mask
matchesMask = [[0,0] for i in xrange(len(matches))]

# ratio test as per Lowe's paper
for i,(m,n) in enumerate(matches):
    if m.distance < 0.7*n.distance:
        matchesMask[i]=[1,0]

draw_params = dict(matchColor = (0,255,0),
                   singlePointColor = (255,0,0),
                   matchesMask = matchesMask,
                   flags = 0)

img3 = cv2.drawMatchesKnn(img1,kp1,img2,kp2,matches,None,**draw_params)

plt.imshow(img3,),plt.show()

Answer 1

You need to find the prescriptive transform between the two images.

Create a set of corresponding coordinates according to the matched features.

For example you find that the feature FtI1 in image 1 corresponds to FtJ1 in image 2 so you know that coordinate of FtI1 (xi,yi) corresponds to the coordinate of FtJ1 (xj,yj) and you have this for all the corresponding features. After you have a list of corresponding coordinates between the two images you can calculate the prescriptive transform using opecv getPerspectiveTransform.

Finally use the transformation you found on the 4 coordinates of the enclosing shape in the first image to get the coordinates of the enclosing shape in the second image. The opencv function for that is warpPerspective.

An example of how to do that in opecv is in: http://docs.opencv.org/3.1.0/da/d6e/tutorial_py_geometric_transformations.html