3D 体积图像沿三个正交（轴）的二维卷积

Question

Since 3D convolution requires too much computational cost, so I prefer to use 2D conv. My motivation here is using 2D conv for volumetric images to reduce this cost.

I want to apply 2D convolution along three orthogonals to get 3 results, each belongs to one of these orthogonals. More clearly, suppose I have a 3D volumetric image. Instead of apply 3D conv, I want to use 2D conv both xy, xz, yz axis. Then, I expect that 3 different volumetric results. Each result represent three different orthogonals.

Is there way to do that? Thanks for help.

Answer 1

You can permute your images. (Some frameworks such as numpy calls it transpose ).

Assume we use 3 x 3 a convolutional kernel.

# A batch of 16 3 channel images (channels first)
a = tensor(shape=[16,3,1920,1080])

# 2D conv will slide over a `1920 x 1080` image, kernel size is `3 x 3 x 3`
a.shape is (16,3,1920,1080)

# 2D conv will slide over a `3 x 1080` image, kernel size is `1920 x 3 x 3`
a.permute(0,2,1,3)
a.shape is (16,1920,3,1080)

# 2D conv will slide over a `1920 x 3` image, kernel size is `1080 x 3 x 3`
a.permute(0,3,2,1)
a.shape is (16,1080,1920,3)