Converting an equirectangular depth map into 3d point cloud

Question

I have a 2D equirectangular depth map that is a 1024 x 512 array of floats, each ranging between 0 to 1. Here example (truncated to grayscale):

I want to convert it to a set of 3D points but I am having trouble finding the right formula to do so - it's sort of close - pseudocode here (using a vec3() library):

for(var y = 0; y < array_height; ++y) {

    var lat = (y / array_height) * 180.0 - 90.0;

    var rho = Math.cos(lat * Math.PI / 180.0);

    for(var x = 0; x < array_width; ++x) {

        var lng = (x / array_width) * 360.0 - 180.0;

        var pos = new vec3();
        pos.x = (r * Math.cos(lng * Math.PI / 180.0));
        pos.y = (Math.sin(lat * Math.PI / 180.0));
        pos.z = (r * Math.sin(lng * Math.PI / 180.0));
        pos.norm();

        var depth = parseFloat(depth[(y * array_width) + x] / 255);

        pos.multiply(depth);

        // at this point I can plot pos as an X, Y, Z point
    }
}

What I end up with isn't quite right and I can't tell why not. I am certain the data is correct. Can anyone suggest what I am doing wrong.

Thank you.

Molly.

Answer 1

Well looks like the texture is half-sphere in spherical coordinates:

• x axis is longitude angle a <0,180> [deg]
• y axis is latitude angle b <-45,+45> [deg]
• intensity is radius r <0,1> [-]

So for each pixel simply:

1. linearly convert x,y to a,b

   in degrees:

    a = x*180 / (width -1) b = -45 + ( y* 90 / (height-1) )

   or in radians:

    a = x*M_PI / (width -1) b = -0.25*M_PI + ( 0.5*y*M_PI / (height-1) )

2. apply spherical to cartesian conversion

   x=r*cos(a)*cos(b); y=r*sin(a)*cos(b); z=r* sin(b);

   Looks like you have wrongly coded this conversion as latitude angle should be in all x,y,z not just y !!! Also you should not normalize the resulting position that would corrupt the shape !!!

3. store point into point cloud.

When I put all together in VCL/C++ (sorry do not code in javascript):

List<double> pnt;                   // 3D point list x0,y0,z0,x1,y1,z1,...
void compute()
    {
    int x,y,xs,ys;      // texture positiona and size
    double a,b,r,da,db; // spherical positiona and angle steps
    double xx,yy,zz;    // 3D point
    DWORD *p;           // texture pixel access
    // load and prepare BMP texture
    Graphics::TBitmap *bmp=new Graphics::TBitmap;
    bmp->LoadFromFile("map.bmp");
    bmp->HandleType=bmDIB;
    bmp->PixelFormat=pf32bit;
    xs=bmp->Width;
    ys=bmp->Height;
/*
    // 360x180 deg
    da=2.0*M_PI/double(xs-1);
    db=1.0*M_PI/double(ys-1);
    b=-0.5*M_PI;
*/
    // 180x90 deg
    da=1.0*M_PI/double(xs-1);
    db=0.5*M_PI/double(ys-1);
    b=-0.25*M_PI;

    // proces all its pixels
    pnt.num=0;
    for (                                 y=0; y<ys; y++,b+=db)
     for (p=(DWORD*)bmp->ScanLine[y],a=0.0,x=0; x<xs; x++,a+=da)
        {
        // pixel access
        r=DWORD(p[x]&255);  // obtain intensity from texture <0..255>
        r/=255.0;           // normalize to <0..1>
        // convert to 3D
        xx=r*cos(a)*cos(b);
        yy=r*sin(a)*cos(b);
        zz=r*       sin(b);
        // store to pointcloud
        pnt.add(xx);
        pnt.add(yy);
        pnt.add(zz);
        }
    // clean up
    delete bmp;
    }

Here preview for 180x90 deg:

and preview for 360x180 deg:

Not sure which one is correct (as I do not have any context to your map) but the first option looks more correct to me...

In case its the second just use different numbers (doubled) for the interpolation in bullet #1

Also if you want to remove the background just ignore r==1 pixels:

simply by testing the intensity to max value (before normalization) in my case by adding this line:

if (r==255) continue;

after this one

r=DWORD(p[x]&255);

In your case (you have <0..1> already) you should test r>=0.9999 or something like that instead.