Project Tango点云平移和旋转

Question

I want to use the Tango Device to capture Point Cloud images and save them to a file. From there I can stich the individual Point Clouds together to form a complete 3D image of a building interior. I have modified the PointCloudJava app from (google tango's official github page) to include a button where each time you press it the currently viewable Point Cloud is saved to a file in binary format along with its Pose data. An external app converts the binary Point Clouds into a human readable .PTS format. Here is a snippet of this .PTS file…

…
3.695209841412947 -0.473210369136531 -4.512059939636327
3.688643257302687 -0.479607720759112 -4.520564449562169
3.693362633866713 -0.489787657644946 -4.520674956573583
3.689930121583388 -0.445884992268283 -4.507568610443212
3.685492555779860 -0.462776307847697 -4.512719524635411
3.667657594365523 -0.480809484508235 -4.535319340957738
3.687827806157515 -0.496090938236911 -4.526998413337804
3.673954348248885 -0.508897741224963 -4.541318190826512
3.666625241918013 -0.498062555577952 -4.540336502327062
3.678675393742964 -0.474170316841799 -4.529156101478673
3.684755723161147 -0.485069070842463 -4.528201831115819
3.680454175156996 -0.519459594872195 -4.541313899292089
3.673863212747023 -0.492040087368685 -4.534148228897191
3.683874587697432 -0.502062504079539 -4.534162414802648
3.662268142384932 -0.486682941105563 -4.540854943527318
3.659110169095442 -0.521473725226123 -4.556032908691503
3.673068563622878 -0.526471559789378 -4.549449575676061
…

And here is a snippet of my Pose file…

Snapshot Event ID: 0
Translation: -2.828956167382643 3.18717159105692 0.4915560231473033
Quarternion: 0.7047427543062862 0.6239282881689626 0.23707690815727211 0.2405112909905187
Accuracy: 4.2E-45
Confidence: 1688685432

Snapshot Event ID: 1
Translation: -2.796160158125086 4.193282171082194 0.5346153464487627
Quarternion: 0.4973877606777205 0.4648489837872261 0.515537924119383 0.520328248891662
Accuracy: 4.2E-45
Confidence: 1687633528

Snapshot Event ID: 2
Translation: -3.504446692020165 3.1853833575251675 0.5117019018999412
Quarternion: 0.6256551950823549 0.5778181469917516 -0.3583361921431137 -0.3824616834061323
Accuracy: 4.2E-45
Confidence: 1687858272 

Where snapshot ID is my method of linking the Pose data to a Point Cloud; and Quaternion is in the format WXYZ.

This generally works correct for individual Point Clouds I think and after I swap the Y and Z axis of each point in the Point Cloud I see this…

Original image

Point cloud image (loaded in Cloud Compare)

Which is generally what I expect.

The problem comes when I try to stich multiple PCs together to form a complete 3D interior of a building.

I have translated the Point Clouds by adding the XY and Z coordinates from the corresponding Pose data to each XY and Z point in the Point Cloud accordingly. Not sure if that's the right thing to do. Then I swapped Y and Z as before. Most Point Clouds are dumped on top of each other if I do that, but this is to be expected because I haven't (and am not sure how to) applied rotations yet.

Here is my general method for doing this…

for (File file : files) {
    if (FilenameUtils.getExtension(file.getAbsolutePath()).equals("dat")) {
        PointCloud cloud = PointCloud.newInstance(file.toURI().toURL());
        if (poseMap.containsKey(cloud.getEventID())) {
            Pose pose = poseMap.get(cloud.getEventID());
            //cloud = cloud.rotate(pose);
            cloud = cloud.translate(pose);
        }
        cloud = cloud.invertYandZ();
        //cloud = cloud.invertAllAxis();
        String absFilename = ptsFolderTxt.getText() + '\\' + FilenameUtils.getBaseName(cloud.getFilename()) + ".pts";
        cloud.write(absFilename);

As you can see I have attempted a rotate method but it didn't seem to work so I commented it out.

The rotate method was applying this to each point…

ThreeDPoint rotatePoint(ThreeDPoint oldPoint)
{
    float ox = oldPoint.getxPoint().floatValue();
    float oy = oldPoint.getyPoint().floatValue();
    float oz = oldPoint.getzPoint().floatValue();

    float qx = quaternion.getX().floatValue();
    float qy = quaternion.getY().floatValue();
    float qz = quaternion.getZ().floatValue();
    float qw = quaternion.getW().floatValue();

    float newX = (float)((1-2*qy*qy-2*qz*qz)*ox+(2*qx*qy+2*qw*qz)*oy+(2*qx*qz-2*qw*qy)*oz);
    float newY = (float)((2*qx*qy-2*qw*qz)*ox+(1-2*qx*qx-2*qz*qz)*oy+(2*qy*qz+2*qw*qx)*oz);
    float newZ = (float)((2*qx*qz+2*qw*qy)*ox+(2*qy*qz-2*qw*qx)*oy+(1-2*qx*qx-2*qy*qy)*oz);

    return ThreeDPoint.newInstance(BigDecimal.valueOf(newX), BigDecimal.valueOf(newY), BigDecimal.valueOf(newZ));
}

But as I said, this didn't seem to work right?

Is my method of translation OK?

How do I use the quaternion in the pose file to rotate each PC? Some code examples would be excellent.

What order do I apply these Transforms?

A push in the right direction would be much appreciated.

Answer 1

What happened is that you didn't transform the point cloud in correct frame of reference. Use the solution in this post: Convert device pose to camera pose

In the answer it's assuming OpenGL convention, but you can use similar technic to transform the point to any convention.