如何使用OpenCV从图像中检测旋转的对象？

Question

I have been training OpenCV classifier for recognition of books.The requirement is recognize book from an image. I have used 1000+ images and OpenCV is able to detect books with no rotation. However, when I try to detect books with rotations it does not work properly.So I am wondering if their anyway to detect objects with rotations in images using OpenCV?

Answer 1

What features are you using to detect your books? Are you training a CNN and deploying it with OpenCV? In that case adding rotation image augmentation to your training would make it easy to detect rotated books.

If you are using traditional computer vision techniques instead, you can try to use some rotation invariant feature extractors like SURF, however, the results will not be as good as using CNNs which are now the state of the art for this kind of problems.

Answer 2

Firstly, you should get the main theoritical ideas of pose estimation and image warping .

You should define some important points of the books (some special and strong features that valid for each types of books) and then you can estimate the pose of the book by using this points. After getting the pose angles, you should warp the image to align books. After book alignment you should perform feature extraction so you can improve the success of book detection by this way.

As a summary, pose estimation and warping (alignment) are important for these kinf of rotation problems.

Answer 3

Your problem can be perfectly solved using OpenCV and keypoint matching algorithms such as SURF or ORB. You don't really need a classifier. To my experience, such solution using unmodified openCv can scale up to recognize around 10.000 images.

What I would do is: Offline: Loop over your book images to generate a database of keypoint descriptors matching each descriptor to the id of the book in which it comes from. Online: Compute the keypoints of the query image and try to match (using BF, FLANN, or LSH) each of them to a keypoint of the pre-computed database. Vote for the database book cover which has matched with the most query keypoints. Try to compute an homography matrix between selected db book cover and query image to validate match.

ORB, BRISK, SURF, SIFT feature descriptors are all usable for this task, and rotation invariant. ORB and BRISK are faster and a bit less performant.

See this link for simple example: https://docs.opencv.org/3.3.0/dc/dc3/tutorial_py_matcher.html