Projection of point cloud on 2D image based on mesh information

Question

I have a point cloud and meshes (vertices=points of the point cloud). I want to project the point cloud with a certain virtual camera. Here, since the point cloud is sparse, the rendered result includes the points which should be occluded by foreground objects.

To resolve this issue, I want to use mesh information to identify which points should be occluded. Is there any smart way to do this in python?

Kind advice will be greatly appreciated.

Answer 1

After hours of searching, I conclude that I have to re-implement a novel rendering pipeline to achieve my goal.

So, instead of this, I use a mesh-based renderer to render a depth map. And then I simply project the points of the point cloud with a projection matrix. Here, I use the depth map to check whether the point fits with the depth or not. If the projected point is the one that should be occluded, then the depth of the point would be larger than the depth map value at the corresponding pixel. So, such points should be ignored while rendering.

I know that this is a less elegant and inefficient trick but anyway it works very well:)