将 2D 深度图像/热图转换为 3D 高度场？

Question

I was wondering how to convert a 2D depth image/heatmap to a 3D height field. Where the Z values are the values of the 2D image.

I have an RGB image:

I also have a corresponding depth image/heatmap:

I would like to combine them into a height field like this, where the Z values are the values of the depth image/heatmap:

Except keeping the RGB values from the RGB image in the 3D heightfield.

Answer 1

What you are describing is essentially adding another dimension to the dataset. There are a few ways to achieve that, but note that some ways might be confusing to other people. The other program you are exporting to might also have strong expectations of the data's shape, etc, so check that. What follows is just some thoughts about what you're trying to do...

Essentially, you're describing having 2 'attributes' of the data: the colour image (a 3-channel attribute), and the height field (a single 'channel'). Some options:

• Keep them as 2 separate ndarrays. That is, a 3D array for the image and a 2D array for the height field. Name them appropriately and save them in a sensible file structure. If I was in a hurry, this is what I would do (basically, do nothing).
• Put them both into an HDF5 file using h5py . This format is essentially like a little file system, with nested 'folders', and it allows for an arbitrary amount of documentation and metadata. It also provides compression, arbitrary array access, and other useful things. So it's great for organizing lots of (or two) ndarrays, especially if they are large.
• Use xarray . This is basically n-dimensional pandas and would let you store the two arrays in a single data structure. It's not ideal for this use case, but it does at least let you label the axes, so you can be clear about what the 4 'channels' are.
• Make a Python class for your data. This way you can document exactly what the two arrays are, and write methods to do things like plot them together. Class instances can be pickled and saved to disk, or you can write methods to save them in whatever way you like (as HDF5, for example). If I was going to be doing a lot of this and needed a robust, sustainable solution, this is what I would do.

Here's how the class might look:

class BumpImage():
    """
    An image with a 3d height field attached.
    """
    def __init__(self, photo, height):
        if photo.ndim == 2:
            # Make greyscale image into RGB.
            photo = np.repeat(photo[:, :, None], 3, axis=-1)
        self.photo = photo
        if height.shape == photo.shape[:2]:
            self.height = height
        else:
            raise TypeError("Height must be 2d array")

    def size(self):
        return self.height.shape

    def plot(self, ax=None):
        # Make a 3d plot with matplotlib or whatever.
        return ax

    def save(self, fname):
        # Make HDF5 file called fname.
        return

Things I would not do:

• Stack the height channel onto the other 3 channels in NumPy, to make a single 4-channel 'image'. I would say this is a bad idea, since there's no convenient way to document what you did in NumPy.
• Try to render them together in some way, eg as a plot with the height field expressed as hillshade or in a 3D view, etc. It will be very hard to separate the two datasets again.