显示图像中的红-绿和蓝-黄对立颜色通道

Question

I'm doing an image processing code to split opponent colors Red-Green and Blue-Yellow from an image based on R, G and B channels. I'm doing this in Octave software. Actually I'm based on the following articles related to visual attention modeling: Itti1998 , GaoVasconcelos2007 , Heinen2009 . This code explains how I'm extracting the channels:

clc
rgbNormImage = im2double(imread('sample.jpg'));
Rn = rgbNormImage(:, :, 1); %red normalized channel
Gn = rgbNormImage(:, :, 2); %green normalized channel
Bn = rgbNormImage(:, :, 3); %blue normalized channel

R = Rn - ( Gn + Bn )/2; %tuned Red
G = Gn - ( Rn + Bn )/2; %tuned Green
B = Bn - ( Rn + Gn )/2; %tuned Blue
Y = ( Rn + Gn )/2 - abs( Rn - Gn )/2 - Bn; %Tuned Yellow

RG = R - G; %red-green opponent channel
BY = B - Y; %blue-yellow opponent channel

I verified the code both in Matlab and Octave and it runs well. However, I still don't figure out how to display the RG and BY channels in a figure or plot where I can see the extracted colors. What I want to display and see in the output is something like this:

_{Figures h) and i) show the Red-Green and Blue-Yellow color opponency maps}

Previous image was taken from this article: CORTICAL MECHANISMS OF COLOUR VISION

My plot looks like this:

_{My plot of the opponent channels RG and BY}

I appreciate a lot if someone knows how to display this in the way I'm trying to do it. Thanks!

Answer 1

I'm not 100% sure about my interpretation below, so do take it with a grain of salt.

My understanding is that you are supposed to use the obtained R, G, B, Y channels to define a 3D coordinate system, such that the first dimension differentiates along a black-white axis, the second differentiates along a red-green axis, and the third along a blue-yellow axis. I haven't read the 4 papers you cited above, but I'll assume your derivation of RGBY is correct.

In the code below, I have assumed that the black-white dimension is R+G, which is what I understood from the explanatory image you posted, but even if I got that wrong, it could probably be replaced by any other grayscale image representing 'luminance' (eg using rgb2gray ).

Then, the other two dimensions are specified by RG, and BY in your code. My understanding is that in the RG axis, lower values correspond to 'red colours', and higher values correspond to 'green' colours. Similarly, in the BY axis, my understanding is that lower values correspond to 'blue colours', and higher values correspond to 'yellow colours'.

Therefore, one can visualise the RG layer as a normal truecolour (ie rgb) image, where the Red, Green, and Blue channels are set to RG, 1-RG, and 0 respectively (for an RG that is normalised between 0 and 1), such that when the red channel increases in the truecolor image, the green channel decreases, and vice versa.

Similarly, since 'yellow' is composed of equal values of red and green in a truecolor image, one can visualise the BY layer as the rgb image 1-BY, 1-BY, BY (again for a suitably normalised BY), such that the more 'blue' a pixel is assigned, the less 'yellow' it has, and vice versa.

Here I normalise arbitrarily and liberally throughout the code, even though the exact normalisation that takes place in the brain is unlikely to be the same. But it's good enough for visualisation purposes, and I couldn't simply use the unnormalized values otherwise we'd be trying to plot values that can go into negative ranges. So, while we cannot compare between the different layers in terms of absolute intensities, the visualisation is good enough to get an idea of where a layer shows higher or lower values for the dimension that it represents, allowing a comparison to be made with respect to the original image.

Here is the code:

  clear all; clc;
  pkg load image;

% Get image and RGB channels
  rgbNormImage = im2double(imread('sample.png'));
  Rn = rgbNormImage(:, :, 1); %red   normalized channel
  Gn = rgbNormImage(:, :, 2); %green normalized channel
  Bn = rgbNormImage(:, :, 3); %blue  normalized channel

% Define Red, Green, Blue, and Yellow 'heatmaps'
  Hot_red    = hot(256);
  Hot_green  = hot(256)( :, [3, 1, 2] );
  Hot_blue   = hot(256)( :, [3, 2, 1] );
  Hot_yellow = hot(256)( :, [1, 1, 2] ); 

% Plot Original RGB Channels in their respective 'heatmap'
  figure(1, 'name', 'Original (normalized) RGB channels');
  axes( 'position', [ 0.26, 0.51, 0.48, 0.48 ] ); imagesc( rgbNormImage )            ; axis image off;
  axes( 'position', [ 0.01, 0.01, 0.31, 0.48 ] ); subimage( im2uint8(Rn), Hot_red )  ; axis image off;
  axes( 'position', [ 0.34, 0.01, 0.31, 0.48 ] ); subimage( im2uint8(Gn), Hot_green ); axis image off;
  axes( 'position', [ 0.67, 0.01, 0.31, 0.48 ] ); subimage( im2uint8(Bn), Hot_blue ) ; axis image off;

% Transform from RGB to RGBY
  R = Rn - ( Gn + Bn ) / 2;          %tuned Red
  G = Gn - ( Rn + Bn ) / 2;          %tuned Green
  B = Bn - ( Rn + Gn ) / 2;          %tuned Blue
  Y = (Rn+Gn)/2 - abs(Rn-Gn)/2 - Bn; %Tuned Yellow

% Plot 'tuned' RGBY channels in their respective 'heatmap'
  figure(2, 'name', 'Tuned RGBY channels');
  axes( 'position', [ 0.01, 0.51, 0.48, 0.48 ] ); subimage( im2uint8(mat2gray(R)), Hot_red    ); axis image off;
  axes( 'position', [ 0.51, 0.51, 0.48, 0.48 ] ); subimage( im2uint8(mat2gray(G)), Hot_green  ); axis image off;
  axes( 'position', [ 0.01, 0.01, 0.48, 0.48 ] ); subimage( im2uint8(mat2gray(B)), Hot_blue   ); axis image off;
  axes( 'position', [ 0.51, 0.01, 0.48, 0.48 ] ); subimage( im2uint8(mat2gray(Y)), Hot_yellow ); axis image off;

% Transform RGBY to Black-White, Red-Green, Blue-Yellow channels
  BW = R + G; % black-white opponent channel
  RG = R - G; % red-green   opponent channel
  BY = B - Y; % blue-yellow opponent channel

% Normalise and convert to RGB images for visualisation purposes
  BW = mat2gray( BW );
  RG = mat2gray( RG );
  BY = mat2gray( BY );

  Z = zeros( size( BW ) );
  BW_asRGB = cat( 3, BW    , BW    , BW );
  RG_asRGB = cat( 3, RG    , 1 - RG, Z  );
  BY_asRGB = cat( 3, 1 - BY, 1 - BY, BY );

% Plot BW, RG, BY channels
  figure(3, 'name', 'Black-White, Red-Green, and Blue-Yellow channels')
  axes( 'position', [ 0.26, 0.51, 0.48, 0.48 ] ); imagesc( rgbNormImage ); axis image off;
  axes( 'position', [ 0.01, 0.01, 0.31, 0.48 ] ); imagesc( mat2gray(BW_asRGB)  ); axis image off
  axes( 'position', [ 0.34, 0.01, 0.31, 0.48 ] ); imagesc( mat2gray(RG_asRGB) ); axis image off
  axes( 'position', [ 0.67, 0.01, 0.31, 0.48 ] ); imagesc( mat2gray(BY_asRGB) ); axis image off

And here are the resulting images.
_{(Note: The heatmaps in Fig 1, and Fig 2, all show lower values as darker pixels, and higher values as brighter pixels, but the colours used are otherwise irrelevant. It's just used for nicer color-coding, but they could have just as well been grayscale images).}

_{Fig. 1 - Original image and respective RGB channels (colorized using red, green, and blue heatmaps respectively)}

_{Fig. 2 - 'Tuned' RGBY channels (colorised using red, green, blue, and yello heatmaps respectively)}

_{Black-white, Red-green, and Blue-yellow layers}

The outputs seem to make sense visually. :)

Eg, if we focus on the plate, which is a dark greenish-blue, in the normal RGB case in Fig 1, you can see that the red channel has very low values there, whereas the green has a bit more brightness, and then the blue has even more than the green, ie it has the largest contribution compared to the other two (but note that neither the green nor the blue are that bright themselves, since the plate isn't very high in intensity overall).

In Fig2 you can similarly see that the red and yellow channels do not contribute very much to the plate, (the blue seems to contribute a lot more than the green here, which again is what we might expect... but keep in mind that this large difference may also be due to the normalization applied, so a direct comparison between the layers may not be very appropriate here)

Finally, Fig 3 is our three dimensions of interest. The first dimension shows us that the plate has a low value in the black-white dimension (ie it is of lower luminance). The red-green dimension shows us that the plate is more 'green' than it is 'red' in that dimension. And finally, the blue-yellow dimension shows us that the plate is more 'blue' than it is 'yellow'.