Segmentation of 3D mesh and feature extraction

Question

I have five pictures of wound models taken from different angles links for the pictures is provided here

I have used SFM for computing a mesh, the picture of the mesh is presented below . I would like to extract only the features associated with the wound region compute volume and depth accordingly.

In order to solve this question, i have used U-Net segmentation to generate a 2D mask of the wound from 2d pictures, an example of 2D mask generated using U-Net is shown below .

I would like to know how i can map this mask onto 3D mesh and extract specific region within the 3D mesh which deals with wound part while removing other regions.

Any other ideas on how to segment the 3D mesh and extract specific region of interest are greatly appreciated, since i don't have different wound models i cannot apply supervised learning using 3D U-Net.

Answer 1

Convert the image and mesh to numpy array and follow the steps:

1. Duplicate the 2D mesh and stack it to make it a 3D mesh. for example, like this:

from skimage.transform import resize
mesh = np.array(mesh)
mesh = resize(mesh, (HEIGHT, WIDTH, 3))
mesh3D = np.array([mesh]*3).reshape(HEIGHT, WIDTH, 3)

2. Convert the pixel value of the mesh to binary (0,1). Set the part of the mesh where the wound is present to 1 and the rest to 0.
3. Multiply the mesh with the image.
4. Part of the mesh where the value is 1, that part of the image will remain as it is and the part of the mesh where the value is 0, that part of the image will be set to 0