Convert pixel coordinate into world coordinate in OpenCv2

Question

My program is for object tracking. I could object tracking and provide objects x,y with coordinates by moments method.

enter image description here

enter image description here

I want to convert pixel coordinate into world coordinate in OpenCV2. I've already got rotation matrix(3*3) and Translation vector(3*1) by camera calibration and I know focal length of my camera.

Now, I defined as follows.

 CvMat *rotation = (CvMat*)cvLord("Rotation.xml")
 CvMat *translation = (CvMat*)cvLord("Translation.xml")

And this is the part of my program.

void trackFilteredObject(Mat threshold,Mat HSV, Mat &Birds_image){

  vector <Fruit> apples;

  Mat temp;
  threshold.copyTo(temp);

  // these two vectors needed for output of findContours
  vector< vector<Point> > contours;
  vector<Vec4i> hierarchy;

  // find contours of filtered image using OpenCv findCountours function
  findContours(temp,contours,hierarchy,CV_RETR_CCOMP,CV_CHAIN_APPROX_SIMPLE );

  // use moments method to find our filtered object.
    double refArea = 0;
    bool objectFound = false;
    if (hierarchy.size() > 0) {
    int numObjects = hierarchy.size();

    // if number of objects greater than MAX_NUM_OBJECTS, we have a noisy filter.
    if(numObjects<MAX_NUM_OBJECTS){
        for (int index = 0; index >= 0; index = hierarchy[index][0]) {
            Moments moment = moments((cv::Mat)contours[index]);
            double area = moment.m00;

            if(area>MIN_OBJECT_AREA){
                Fruit apple;

                // moments method
                apple.setXPos(moment.m10/area);
                apple.setYPos(moment.m01/area); 
                apples.push_back(apple);

                objectFound = true;

             }else objectFound = false;
        }
        if(objectFound ==true){
            // draw object location on screen
            drawObject(apples,Birds_image);
        }
     }else putText(Birds_image,"TOO MUCH NOISE! ADJUST FILTER",Point(0,50),1,2,Scalar(0,0,255),2);
     }
}

And drawObject(apples,Birds_image) is this.

void drawObject(vector<Fruit> theFruits,Mat &frame){
  for(int i =0; i<theFruits.size(); i++){
  cv::circle(frame,cv::Point(theFruits.at(i).getXPos(),theFruits.at(i).getYPos()),10,cv::Scalar(0,0,255));
  cv::putText(frame,intToString(theFruits.at(i).getXPos())+ " , " + intToString(theFruits.at(i).getYPos()),cv::Point(theFruits.at(i).getXPos(),theFruits.at(i).getYPos()+20),1,1,Scalar(0,255,0));
  }
}

And I use these souse file and header file.

Fruit.h

#pragma once
#include <string>
using namespace std;

class Fruit
{
public:
Fruit(void);
~Fruit(void);

int getXPos();
void setXPos(int x);


int getYPos();
void setYPos(int y);

private:

int xPos, yPos;
string type;

};

Fruit.cpp

#include "Fruit.h"


Fruit::Fruit(void)
{
}


Fruit::~Fruit(void)
{
}

int Fruit::getXPos(){
return Fruit::xPos;
}

void Fruit::setXPos(int x){
Fruit::xPos = x;
xPos = x;
}

int Fruit::getYPos(){
return Fruit::yPos;
}

void Fruit::setYPos(int y){
Fruit::yPos = y;
yPos = y;
}

Could you give me your splendid ideas.

Answer 1

Have a look at the findHomography function in opencv. It helps to find the transformation from one plane to other but only the 2D co ordinates can be found.
This link provides the similar example of converting the coordinates from image plane to object plane. ( Camera pixels to planar world points given 4 known points )