OpenCV: Extract SURF Features from user-defined keypoints

Question

I want to compute SURF Features from keypoints that I specify. I am using the Python wrapper of OpenCV. The following is the code I am trying to use, but I cannot find a working example anywhere.

surf = cv2.SURF()
keypoints, descriptors = surf.detect(np.asarray(image[:,:]),None,useProvidedKeypoints = True)

How can I specify the keypoints to be used by this function?

Similar, unanswered, question: cvExtractSURF don't work when useProvidedKeypoints = true

Documentation

Answer 1

If I understand the source code of the Python bindings correctly, the "keypoints" argument that is present in the C++ interface is never used in the Python bindings. So I hazard that it's not possible to do what you are trying to do with the current bindings. A possible solution would be to write your own binding. I know it's not the answer you were hoping...

Answer 2

Try using cv2.DescriptorMatcher_create for that.

For instance, in the following code I am using pylab, but you can get the message ;)

It computes the keypoints using GFTT, and then uses the SURF descriptor and the Brute force matching. The output of each code part is show as header.

---

%pylab inline
import cv2
import numpy as np

img = cv2.imread('./img/nail.jpg')
gray= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
imshow(gray,  cmap=cm.gray)

Output is something like this http://i.stack.imgur.com/8eOTe.png

(For this example I will cheat and use the same image to get the keypoints and descriptors).

img1 = gray
img2 = gray
detector = cv2.FeatureDetector_create("GFTT")
descriptor = cv2.DescriptorExtractor_create("SURF")
matcher = pt1=(int(k1[m.queryIdx].pt[0]),int(k1[m.queryIdx].pt[1]))("FlannBased")

# detect keypoints
kp1 = detector.detect(img1)
kp2 = detector.detect(img2)

print '#keypoints in image1: %d, image2: %d' % (len(kp1), len(kp2))

keypoints in image1: 1000, image2: 1000

# descriptors
k1, d1 = descriptor.compute(img1, kp1)
k2, d2 = descriptor.compute(img2, kp2)

print '#Descriptors size in image1: %s, image2: %s' % ((d1.shape), (d2.shape))

Descriptors size in image1: (1000, 64), image2: (1000, 64)

# match the keypoints
matches = matcher.match(d1,d2)

# visualize the matches
print '#matches:', len(matches)
dist = [m.distance for m in matches]

print 'distance: min: %.3f' % min(dist)
print 'distance: mean: %.3f' % (sum(dist) / len(dist))
print 'distance: max: %.3f' % max(dist)

matches: 1000

distance: min: 0.000

distance: mean: 0.000

distance: max: 0.000

# threshold: half the mean
thres_dist = (sum(dist) / len(dist)) * 0.5 + 0.5

# keep only the reasonable matches
sel_matches = [m for m in matches if m.distance < thres_dist]

print '#selected matches:', len(sel_matches)

selected matches: 1000

#Plot
h1, w1 = img1.shape[:2]
h2, w2 = img2.shape[:2]
view = zeros((max(h1, h2), w1 + w2, 3), uint8)
view[:h1, :w1, 0] = img1
view[:h2, w1:, 0] = img2
view[:, :, 1] = view[:, :, 0]
view[:, :, 2] = view[:, :, 0]

for m in sel_matches:
    # draw the keypoints
    # print m.queryIdx, m.trainIdx, m.distance
    color = tuple([random.randint(0, 255) for _ in xrange(3)])
    pt1=(int(k1[m.queryIdx].pt[0]),int(k1[m.queryIdx].pt[1]))
    pt2=(int(k2[m.queryIdx].pt[0]+w1),int(k2[m.queryIdx].pt[1]))
    cv2.line(view,pt1,pt2,color)

Output is something like this http://i.stack.imgur.com/8CqrJ.png

Answer 3

Example of how this can be done with the before-mentioned Mahotas :

import mahotas
from mahotas.features import surf
import numpy as np


def process_image(imagename):
    '''Process an image and returns descriptors and keypoints location'''
    # Load the images
    f = mahotas.imread(imagename, as_grey=True)
    f = f.astype(np.uint8)

    spoints = surf.dense(f, spacing=12, include_interest_point=True)
    # spoints includes both the detection information (such as the position
    # and the scale) as well as the descriptor (i.e., what the area around
    # the point looks like). We only want to use the descriptor for
    # clustering. The descriptor starts at position 5:
    desc = spoints[:, 5:]
    kp = spoints[:, :2]

    return kp, desc