如何将 rgb 转换为标签以进行图像分割

Question

I have around 4000 rgb label images which are masks for some other images. I can use this image label pair in the deep learning encoder-decoder structure (eg:UNet) architecture with the help of regression approach. But I would like to do segmentation approach. For that how can I convert these images?

Sample image: (Above sample image should contain 3 classes. one oval shape part, the remaining red part, and the background white part. This can go upto 7 classes in some other image pairs)

There is supposed to be 7 classes including background for the entire dataset. But when I tried to find the unique values in an RGB label, there are more than 30 unique value pairs coming. Otherwise I would have select the unique rgb pair and do the processing. How to overcome this

Answer 1

Here's one potential way to handle this (in MATLAB, but similar in other situations)

The image you have shared is rather pixelated, and hence quite difficult to handle. If your dataset contains similarly pixelated images, I'd explore some kind of pre-processing to get rid of spurious edge discolorations, as they mess up the clustering. For the sake of demonstration here, I've created a test image with exactly three colors.

% Create a test image - one shared is very pixelated.
I = uint8(zeros(100, 100, 3));
I(10:20, 10:20, 1) = 255;
I(40:50, 40:50, 2) = 255;

If the number of colors here is unknown, but up to 7, here's a quick way to use imsegkmeans and it's 'C' output to find the number of unique centers.

% Specify max clusters
maxNumClusters = 7;

% Run clustering using the max value
[~, C] = imsegkmeans(I, maxNumClusters);
nUniqueClusters = size(unique(C, 'rows'), 1);

'nUniqueClusters' should now contain the 'true' number of clusters in the image. In a way, this is almost like finding the number of unique entries of pixel RGB triplets in the image itself - I think what's affecting your work is noise due to pixelation - which is a separate problem.

[L, C] = imsegkmeans(I, nUniqueClusters);

% Display the labeled image for further verification.
B = labeloverlay(I, L);
figure, imshow(B)

One way to attempt to fix the pixelation problem is to plot a histogram of your image pixels (for one of the three color planes), and then managing the low values somehow - possibly marking all of them with a distinct new color that you know doesn't exist in your dataset otherwise (0, 0, 0), for example - and marking it's label to be 'unknown'. This is slightly outside the scope of your original question - hence just a text description of it here.