Python中的三维规范化互相关

Question

I'm currently doing 2D template matching using OpenCV's MatchTemplate function called from Python. I'm looking to extend my code into 3D but can't find any existing 3D cross-correlation programs. Can anyone help out?

Answer 1

Do you mean that you are currently looking for a known object somewhere in an image, and you are currently only able to handle that object being affine transformed (moved around on a 2D plane), but you want to be able to handle it being perspective transformed?

You could try using a SURF or SIFT algorithm to find features in your reference and unknown images:

def GetSurfPoints(image, mask)
    surfDetector = cv2.FeatureDetector_create("SURF")
    surfExtractor = cv2.DescriptorExtractor_create("SURF")
    keyPoints = surfDetector.detect(image, mask)
    keyPoints, descriptions = surfExtractor.compute(image, keyPoints)
    return keyPoints, descriptions

Then use FLANN to find matching points (this is from one of the cv2 samples):

def MatchFlann(desc1, desc2, r_threshold = 0.6):
    FLANN_INDEX_KDTREE = 1  # bug: flann enums are missing
    flann_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 4)
    flann = cv2.flann_Index(desc2, flann_params)
    idx2, dist = flann.knnSearch(desc1, 2, params = {}) # bug: need to provide empty dict
    mask = dist[:,0] / dist[:,1] < r_threshold
    idx1 = numpy.arange(len(desc1))
    matches = numpy.int32( zip(idx1, idx2[:,0]) )
    return matches[mask]

Now if you want to, you could use FindHomography to find a transformation that aligns the two images:

referencePoints = numpy.array([keyPoints[match[0]].pt for match in matches])
newPoints = numpy.array([keyPoints[match[1]].pt for match in matches])
transformMatrix, mask = cv2.findHomography(newPoints, referencePoints, method = cv2.cv.CV_LMEDS)

You could then use WarpPerspective and that matrix to align the images. Or you could do something else with the set of matched points found earlier.