How to make Android screen see-through using backside camera?

Question

Is there a way I could show what the hind-side camera captures on a full-screen such that it creates an illusion of screen being see-through? It doesn't need to be perfect, just convincing enough, a little lag won't make any difference.

Is it possible to create such an effect using phone camera? If yes, how can the effect be achieved? (as in what transformations to apply etc.)

(I already know how to create a simple Camera Preview )

Edit : Now, I also know it has been done, http://gizmodo.com/5587749/the-samsung-galaxy-s-goes-see+through , But, I still have no clue how to properly do this, I know trial and error is one way, other is calculating what part a user should be seeing if phone wasn't there.

I think there would be some factors involved like -

• viewing distance,
• viewing angle,
• camera zoom range,
• camera focus,
• camera quality,
• phone orientation,
• camera position (where is camera located on phone) etc.

So, I don't feel this problem has a simple enough solution, if it is not so, please clarify with an answer.

Thanks for help,

Shobhit,

Answer 1

You can use standard 3D projection math to project a portion of the backside camera image onto the display; you can manage this by assuming everything the camera sees is at a particular depth from the backside camera, and by assuming a particular viewpoint for the observer

You can improve on this by looking for faces/eyes using the frontside camera. You can get a rough estimate of the the viewing distance from the eye spacing, and assume a viewer position midway between the eyes. Of course, this only works for one viewer at a time (eg, if your face tracker finds multiple faces, you can select one of them).

Also, you can improve the illusion by calibrating the camera and screen so you can match the color and brightness from one to the other.