Matching 3d models in a 2d scene using pcl or opencv

Question

I have a 3d model obtained with a 3d scanner and I want to match it in a 2d scene (simple 2d video which contains the model).

I know pcl deals only with point clouds and opencv with 2d images, is it possible though to user any of them to extract the keypoints from the 3d model and then use them to find the model in a 2d image?

Answer 1

It depends on the kind of objects. If you look for simple shape objects as boxes, you can detect corners in 3D and in 2D and match its together.

For more complex objects, maybe you will have to mesh your point cloud to find robust interest points. For example, this paper https://hal.inria.fr/hal-00682775/file/squelette-rr.pdf explains a method to extract robust points in a shape, OR a surface, but I don't know if the same keypoints will be extracted in 2D and 3D.

Answer 2

Find all key points and project them on ground plane to get equivalent 2D image. You can use pcl 2d projection techniques also. Possible duplicate of Generate image from an unorganized Point Cloud in PCL