OpenCV project 3D coordinates to 2d camera coordinates

Question

I have the 3D-world coordinates of an object and I want to get its coordinates in the camera-2D-plane. I have already calibrated the camera using cv::calibrateCamera , so that I have the camera matrix and distortionCoeffs .

For projecting the 3D-point to 2d-camera-coordinates, I use cv::projectPoints . Documentation says:

void projectPoints(InputArray objectPoints , InputArray rvec , InputArray tvec , InputArray cameraMatrix , InputArray distCoeffs , OutputArray imagePoints , OutputArray jacobian=noArray() , double aspectRatio=0 )

How do I get rvec/ tvec and is my approach right? calibrateCamera gives me rvecs and tvecs , but they are for each input chessboard-image and I guess, they are the rotation and translation of the chessboard and not of the camera.

Answer 1

As I have the projection matrix ( P ), I can calculate the 2D coordinates ( x ) in the camera plane like this from the 3D-coordinates X (using homogeneous coordinates ):

x (u,v,w) = P * X (x1,x2,x3,w)

I just have a problem with the rectification , so that I do not get the exact projection to my 2d image. See here: OpenCV stereo vision 3D coordinates to 2D camera-plane projection different than triangulating 2D points to 3D