How to calibrate the Kinect xbox one and a projector separately?

Question

I am projecting a game -which has a number of buttons- on a spandex fabric attached to a frame and I want to see which button is touched by the user by monitoring the depth change of that particular area on the fabric. (the fabric is stretched since it's attached to the edges of the frame)

I need to know the positions of the buttons, so I should calibrate both the Kinect and the projector separately with my computer screen.

Can anyone introduce a link or some help that can make me familiar with this process? I do not have a clear idea of first how to do the calibration and second what exactly I would do with the output?

Here there is a photo of the system so that it is more understandable where the Kinect, Projector and the fabric are.

Thank you and I'd appreciate any help.

Answer 1

You can start by taking a look at SLStudio . You would need to get it running and add a camera class for your kinect. But if this is done, it has a nice calibration for your kinect to projector in it. The output will be camera matrices (fx,fy,cx,cy) and distortion coefficients (k1-k6,p1,p2) and R and T of the kinect to projector ( See OpenCV definitions ).

Now if you display a button on the projector, you know the pixel ranges for it. This can now be calculated to the pixels on the kinect. Probably you project them to the color pixels and use the kinect functions to project them on the depth pixels. Select and average the pixels you would like to use to detect the button push.