How to perform 3 channel images voxel wise summation (medical images)

Question

enter image description here As you see the image.

Suppose that three channel images were predicted from the last layer of the 3d CNN model(like nn.net) to the left lung, right lung, and background.

I want to add an auxiliary layer to the.network that performs the voxelwise summarization of only the left and right lung prediction image channels

However, I have only known the concept of image channel as RGB color is 3 channel image and 1 channel image with only light and shade, so I don't know the concept of left and right lung channel and background channel. Can you tell me the principle of this channel concept and tell me how to proceed with the work up there?

        x = self.conv_blocks_context[-1](x)

        for u in range(len(self.tu)):
            x = self.tu[u](x)
            x = torch.cat((x, skips[-(u + 1)]), dim=1)
            x = self.conv_blocks_localization[u](x)
            seg_outputs.append(self.final_nonlin(self.seg_outputs[u](x)))

        if self._deep_supervision and self.do_ds:
            return tuple([seg_outputs[-1]] + [i(j) for i, j in
                                              zip(list(self.upscale_logits_ops)[::-1], seg_outputs[:-1][::-1])])
        else:
            return seg_outputs[-1]

Maybe seg_outputs[-1] is from the last layer of nn.net, so i think i should add auxiliary layer this part

Answer 1

Yes, seg_outputs[-1] is the layer you are looking for. I try to explain the concept with regard to this layer. Assume this layer's shape is B(atch) x C(hannel) x D(epth) x H(eight) x W(idth), the concept you are trying to understand is this C. In your semantic segmentation example, C == 3, where (maybe, depends on your code) c[0] is the background logit, c[1] is left lung logit, and c[2] is right lung logit. Argmax of dimension C gives you the segmentation result. To proceed with the work, your desired output is seg_outputs[-1][:,1:,...].sum(dim=1, keepdim=True), if you still need your background logit, just concat your background channel.