Augmented Reality, superimposing 3d models

Question

How would I use a library like this https://code.google.com/p/js-handtracking/ with a 3D model in order to superimpose over a hand tracking?

How would you use a 3d model file with such a thing?

What format would the model need to be? I've never dealt with 3D model superimposition.

Answer 1

The 3D object you want to superimpose may be any 3D object (eg obj, 3ds, ply, vrml). Actually the model representation is not an issue as you can convert a 3d representation to any other 3d representation (usually). It is up to your browser/player doing the rendering of the scene.

In order to overlay a 3d model on your detected hand you need to know the position of the hand (relative to your scene of course). The detection algorithm should give you some kind of a transformation matrix which can be translated to rotation, translation and scale. Then you can use these values to place your 3d object at the right position in the 3d scene.

You should first check if there is an API of the handtracking algorithm, or at least how can you intercept the output data (if any). Otherwise you would have to search through the algorithm (source code..) where the detection is made and get the transformation matrix and apply it on your 3d object.