在OpenCV C ++中将3D点集投影到虚拟图像平面中

Question

Anyone know how to project set of 3D points into virtual image plane in opencv c++
Thank you

Answer 1

首先，您需要定义转换矩阵（旋转，平移等），以将3D空间映射到2D虚拟图像平面，然后将3D点坐标（x，y，z）乘以矩阵以获取2D坐标在图像中。

Answer 2

registration (OpenNI 2) or alternative viewPoint capability (openNI 1.5) indeed help to align depth with rgb using a single line of code. The price you pay is that you cannot really restore exact X, Y point locations in 3D space since the row and col are moved after alignment.

Sometimes you need not only Z but also X, Y and want them to be exact; plus you want the alignment of depth and rgb. Then you have to align rgb to depth. Note that this alignment is not supported by Kinect/OpenNI. The price you pay for this - there is no RGB values in the locations where depth is undefined.

If one knows extrinsic parameters that is rotation and translation of the depth camera relative to color one then alignment is just a matter of making an alternative viewpoint: restore 3D from depth, and then look at your point cloud from the point of view of a color camera: that is apply inverse rotation and translation. For example, moving camera to the right is like moving the world (points) to the left. Reproject 3D into 2D and interpolate if needed. This is really easy and is just an inverse of 3d reconstruction; below, Cx is close to w/2 and Cy to h/2;

col = focal*X/Z+Cx row = -focal*Y/Z+Cy // this is because row in the image increases downward

A proper but also more expensive way to get a nice depth map after point cloud rotation is to trace rays from each pixel till it intersects the point cloud or come sufficiently close to one of the points. In this way you will have less holes in your depth map due to sampling artifacts.