SIFT object detection bounding box

Question

I am trying to track object from video stream using SIFT algorithm. I want to detect the object and track it by drawing a rectangle surrounding it. The problem is, the rectangle gets skewed and not accurately drawn most of the time . I am using the following code to draw the rectangle around the detected object ( videoImage is the frame from video stream).

line(videoImage, sceneCorners[0], sceneCorners[1], Scalar(255, 0, 0), 2);
line(videoImage, sceneCorners[1], sceneCorners[2], Scalar(255, 0, 0), 2);
line(videoImage, sceneCorners[2], sceneCorners[3], Scalar(255, 0, 0), 2);
line(videoImage, sceneCorners[3], sceneCorners[0], Scalar(255, 0, 0), 2);

I also tried the following code ( imgMatches is the image with only the good matches)

line(imgMatches, sceneCorners[0] + Point2f( object.cols, 0), sceneCorners[1] + Point2f( object.cols, 0), Scalar(0, 255, 0), 2);
line(imgMatches, sceneCorners[1] + Point2f( object.cols, 0), sceneCorners[2] + Point2f( object.cols, 0), Scalar(0, 255, 0), 2);
line(imgMatches, sceneCorners[2] + Point2f( object.cols, 0), sceneCorners[3] + Point2f( object.cols, 0), Scalar(0, 255, 0), 2);
line(imgMatches, sceneCorners[3] + Point2f( object.cols, 0), sceneCorners[0] + Point2f( object.cols, 0), Scalar(0, 255, 0), 2);

Both seems give the same result. So, my question is, how do draw a rectangle bounding my tracked object which is consistent to the tracked object? By the way, I am using OpenCV (C++) with Visual Studio 2010 on Windows 7.

Answer 1

The problem is not drawing the rectangle, but detecting the object correctly. It is very common that detections in single images are noisy if you get only some keypoints, even if you filter them with RANSAC and a fundamental matrix or a homography.

If you want a more accurate rectangle around the object, you must write a better detection algorithm. For example, you can try to look for more correspondences when you have a first hint of the position of the object in the image.

Answer 2

Maybe have a look at this question SIFT matches and recognition? . It's about the same question. The solution is a 4D hough space.