How to calculate 3D coordinates of a point in OpenCV

Question

I have got a calibrated stereo-camera system and pixel positions of the same object in views of both cameras. Is there any easy way to calculate 3D world coordinates of the object using OpenCv (and intrinsic/extrinsic parameters) or do I have to deal with 3D lines generated for both cameras and transformations between them? Does CvReprojectImageTo3D can help? Many Thanks

Answer 1

You can triangulate these points. For every camera, you know that x = IP * EP * X, with x = 2D homogenous screen coordinates, IP = intrinsic camera matrix, EP = extrinsic camera matrix, and X 3D homogenous world coordinates. This formula will yield 2 linear equations per camera, resulting in a linear problem A*X = B. For every camera, append to A:

Ps = IP * EP
A(row 1) = x.x * Ps(row 3) - Ps(row 1)
A(row 2) = x.y * Ps(row 3) - Ps(row 2)
B(row 1 & 2) = 0

There are 4 unknowns. To avoid 0 as a solution, append [0 0 0 1] to A and [ 1 ] to B.

Solving can be done with the Jacobi method , or something similar.

I did not find/know of a method in OpenCV, but implementing above method should be doable if you know what calibration matrices are.