OpenCV 的 Python ORB 模块的掩码格式是什么？

Question

I've been trying to use ORB to find keypoint/descriptors and I need to mask out part of the image because many features are very similar in two parts of my image. However, I can't determine the correct format of the mask parameter to the detectandcompute function, and the documentation is ambiguous to me. I tried looking at the source code but I am not familiar enough with C++ to understand it. I thought it was just a binary array where 1 = use and 0 = ignore, but every mask I've tried doesn't return any keypoints. Here is some example code:

img1_gray = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
img2_gray = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
#ignore the left half of the first image
mask1 = np.ones(img1_gray.shape)
mask1[:,:mask1.shape[1]/2] = 0
#ignore the right half of the second image
mask2 = np.ones(img2_gray.shape)
mask2[:,mask2.shape[1]/2:] = 0
kp1, des1 =orb.detectAndCompute(img1_gray,mask1)
kp2, des2 =orb.detectAndCompute(img2_gray,mask2)

The documentation is here: http://docs.opencv.org/3.0-beta/modules/features2d/doc/feature_detection_and_description.html

img1 img2

Answer 1

I've used orb in OpenCV C++ and if I recall correctly the mask has to be CV_8UC1 type with values 0 and 255 (at least that's what I used). Have you already tried like this?

Answer 2

ORB is great for detecting pattern on image or finding image duplicates, but for such kind of task, ORB may be not enough informative, because it's designed as fast and lightweight binary descriptor. Try to look at SIFT and SURF

Answer 3

The bug in the above code was that the masks had to be changed to Uint8. Thus, this would change the masks to be of CV_8UC1 type with values 0 to 255. Here is the fully tested working Python code (Version 3) using SIFT features rather than ORB features:

import cv2
import numpy as np 
import matplotlib.pyplot as plt

def showing_features(img1, key_points):
   plt.imshow(cv2.drawKeypoints(img1, key_points, None))
   plt.show() 

img1 = cv2.imread('img1.jpg')
img2 = cv2.imread('img2.jpg')

img1_gray = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
img2_gray = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)

sift= cv2.xfeatures2d.SIFT_create(nfeatures=0,
                            nOctaveLayers=3,
                            contrastThreshold=0.05,
                            edgeThreshold=10.0,
                            sigma=1.6)

"*-------------- Create Masks --------------*"
mask1 = np.ones(img1_gray.shape)
#ignore the left half of the first image
mask1[:,:int(mask1.shape[1]/2)] = 0
#ignore the right half of the second image
mask2 = np.ones(img2_gray.shape)
mask2[:,int(mask2.shape[1]/2):] = 0

"*-------------- Change Masks to Uint8 --------------*"
mask1 = mask1.astype(np.uint8)
mask2 = mask2.astype(np.uint8)

"*-------------- Extract SIFT Features --------------*"
kp1m, des1m =sift.detectAndCompute(img1_gray,mask1)
kp2m, des2m =sift.detectAndCompute(img2_gray,mask2)

"*-------------- Display SIFT features after using MASK --------------*"
showing_features(img1, kp1m)
showing_features(img2, kp2m)