将 3d 点及其 colors 映射到不带循环的 2D 图像

Question

I'm working with point clouds, and I'm trying to render the model on the image plane efficiently w/o looping over the 3d points.

Input: .ply model

Output: 2d image

What I've done so far:

I've constructed the 3x4 projection matrix, using open3d library in python to get the the 3d points of the point cloud and their colors, after loading the model it's just (pcl.points and pcl.colors), I've transformed the 3d points (pcl.points) to homogeneous coords by adding ones as the 4th column the shape is (9729509,4) , then I received my 2d points by multiplying the projection matrix by the transpose of (9729509,4) and getting a new matrix (3, 9729509) and then divide the first row and second row by the third row to normalize and get the 2d pixels.

Now I have a matrix of (3, #of_2d_pixels) and I can even get rid of the third row because it's just ones after normalizing and I have (2, #of_2d_pixels) , define it as P .

The question is: How can I construct a 2d image w/o using for loops and assigning the colors (from pcl.colors) to an empty 2d array where the indices of assignment are the values in the P matrix? Finally I'll show use OpenCV to render this array

Using for loops takes a lot of time and I'm sure there's a way to do it faster, like I did with using the projection matrix on all the points at once w/o using for loops (looping over the 3d points and appending the 2d points I received to a new array takes a lot of time)

Answer 1

My answer assumes you are writing on python.

From a theoretical point of view, we could imagine drawing points as a convolution function, and by transforming it, we can achieve associativity, which will allow us to use concurrent computing. In practice, this does not work so well, since the computation of such a convolution function is expensive and the result will be comparable to a normal loop. Therefore, I decided to use functions with loops at the C levels of the code, for example the standard function map (func, arr). I managed to get a ~ 4x performance gain by vectorizing the index calculation operations using numpy. So that you can understand the context of the code:

import numpy as np
# image width = height
width = 1000
number_of_points = 1000000

# ps = [(x,y,color),(x,y,color),(x,y,color),...]
ps = np.random.randint(width,size=(number_of_points,3))

# color buffer
pixels = np.array([0]*width*width)

• normal loop:

for (x,y,c) in ps:
    pixels[x+y*width] = c

1.24 s ± 37.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

• map + numpy:

# f2(x)  <-> f2([idx,color])
# x[0] = idx
# x[1] = color
def f2(x):
    global all_screen
    pixels[x[0]] = x[1]
# unpack xs,ys,colors
xs,ys,clrs = ps.T
# calculate index in pixel array
idxs = xs + ys*width
# draw all
list(map(f2,zip(idxs,clrs)))
# cast to list use only for evaluate map function, because in default it lazy

306 ms ± 31.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

if you are interested in pure functional style without mutable variables (assignments), we can create a pixel_idx -> color function with a static hash table (dict in python):

# unpack xs,ys,colors
xs,ys,clrs = ps.T
# calculate index on pixel array
idxs = xs + ys*width
ddd = dict(zip(idxs,clrs))

def f3(x):
    return ddd.get(x,0)

# draw all
pixels1 = list(map(f3,np.arange(0,width*width,1)))

809 ms ± 21.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)