如何从 numpy rgb 数组和深度数组创建 rgbd 图像？

Question

I have a numpy array which is the color image "img" of shape (height, width, 3), and also a depth numpy array of shape (height, width). And I want to create an RGBD image and display it, for which I'm doing the following:

o3d.geometry.RGBDImage.create_from_color_and_depth(img, depth)

But I'm getting the error:

TypeError: create_from_color_and_depth(): incompatible function arguments. The following argument types are supported:
    1. (color: open3d.open3d_pybind.geometry.Image, depth: open3d.open3d_pybind.geometry.Image, depth_scale: float = 1000.0, depth_trunc: float = 3.0, convert_rgb_to_intensity: bool = True) -> open3d.open3d_pybind.geometry.RGBDImage

How to fix this? If it requires an Image type, then how to convert the numpy array to an Image type?

Even when I pass in the numpy arrays into the o3d.geometry.Image constructors like so:

o3d.geometry.RGBDImage.create_from_color_and_depth(o3d.geometry.Image(img), o3d.geometry.Image(depth))

I get the error:

TypeError: create_from_color_and_depth(): incompatible function arguments. The following argument types are supported:
    1. (color: open3d.open3d_pybind.geometry.Image, depth: open3d.open3d_pybind.geometry.Image, depth_scale: float = 1000.0, depth_trunc: float = 3.0, convert_rgb_to_intensity: bool = True) -> open3d.open3d_pybind.geometry.RGBDImage

How to fix this and create the RGBD image from the rgb numpy array and the depth numpy array?

Answer 1

Would be great if you shared reproducible example, but I think that creating Image from np.array isn't as simple as calling the ctor, basing on signature . This is not necessarily an array of uint8, right?

Basing on this article , you have to create it as follows:

depth_as_img = o3d.geometry.Image((depth).astype(np.uint8))

and pass further to create_from_color_and_depth . So, you have to explicitly specify that it's an array of uint8s.

Answer 2

You should check the convert_rgb_to_intensity parameter in this function. By default, it will use the greyscale image. In this way, your color image should have only one channel. If you want RGB, set that parameter to false and see if it solves.

In addition, extra comments for from numpy to open3d image: https://github.com/intel-isl/Open3D/issues/957