Google's Project Tango Get Points In Certain Screen Region

Question

So I'm trying to figure out how to take Tango's point cloud and keep only the points located (by X and Y, depth doesn't matter) in a square area at the center of the DEVICE screen. So imagine a small square reticle that remains at the center of the screen, and when you click the "Capture" button the points ONLY in that square are stored.

I'm still new to Tango's API (Java) and am doing everything I can to learn, but it's a bit overwhelming. I was imagining just iterating through the point cloud and pulling out points that are a certain distance from the screen's center, but again I'm not quite sure on how to achieve this. I could be wrong, but maybe I need to convert the world coordinates (target frame) to the device display coordinates (base frame)?

Any help would be appreciated..

Answer 1

The straightforward solution is projecting all points to a image plane, using the camera instrinsic provided by Tango. In C examples, there's a rgb_depth_sync_example that almost does the exactly same thing you are looking for.