How to correct color of an image based on a standard image

Question

Please see the figure: image1 is the image to be corrected, and image2 is the standard image taken in a black box. There is a triangle in both images with slightly different color, I want to correct image1 through the triangle based on image2 so that the circle and the square in image1 can be also corrected.

How can I do that?

What I have tried:

get the B, G, R mean value of the triangle in image1 and image2, dividing them respectively to get KB, KG, KR, then multiply B, G, R channel of image1 with KB, KG, KR, lastly merge the 3 channel to get the corrected image

Demo code in python with OpenCV:

triangle_image1 = cv2.mean(image1, mask1)[:3]
triangle_image2 = cv2.mean(image2, mask2)[:3]

k_b, k_g, k_r = triangle_image2 / triangle_image1
b, g, r = cv2.split(image1)

corrected = b * k_b, g * k_g, r * k_r
corrected = np.clip(corrected, 0, 255)
corrected = cv2.merge(np.array(corrected, np.uint8))

The result image looks OK but actually not right because the color difference (delta E) of the triangle in the corrected image and image2 is about 6.

Answer 1

I tried executing chromatic adaptation transform, but I have no way telling if the result is correct.

Note that chromatic adaptation corrects the chrominance, but not the luminescence (only the color but not the brightens).
Chromatic adaptation transform is used for color balancing (White Balance), and I don't know if it fits your case.

I reused MATLAB implementation (I did't look for Python examples).

Even if you don't know MATLAB, and the solution is not what you are looking for, you may learn from it (Linearize the RGB value for example).

Here is the code:

T = imread('image.png');       % Load input image (two images side by side).
image1 = T(:, 1:end/2, :);     % Left side
image2 = T(:, end/2+1:end, :); % Right side
I = image1; % Source image is named I

% Use color components in range [0, 1] (colors were found by manual picking).
src_sRGB = [205, 232, 32]/255; %Triangle sRGB color from image 1 "source image" 
dst_sRGB = [13, 133, 38]/255;  %Triangle sRGB color from image 2 "destination image"

%Linearize gamma-corrected RGB values (image values are in sRGB color space, we need to Linearize them).
srcRGB = rgb2lin(src_sRGB)';
dstRGB = rgb2lin(dst_sRGB)';
linI = rgb2lin(double(I)/255); % I in linear RGB color space.


% Color correction by Chromatic Adaptation:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html
% Convert from XYZ D65 color space to Linear sRGB color space.
XYZD65_to_sRGB = [ 3.2404542 -1.5371385 -0.4985314
                  -0.9692660  1.8760108  0.0415560
                   0.0556434 -0.2040259  1.0572252];

% Convert from Linear sRGB color space to XYZ D65 color space.
sRGBtoXYZD65 =   [0.4124564  0.3575761  0.1804375; ...
                  0.2126729  0.7151522  0.0721750; ...
                  0.0193339  0.1191920  0.9503041];

% Convert srcRGB and dstRGB to XYZ color space
srcXYZ = sRGBtoXYZD65 * srcRGB;
dstXYZ = sRGBtoXYZD65 * dstRGB;

% Convert srcXYZ and dstXYZ to xyY color space (get only xy out of xyY - xy applies chromaticity).
xySrc = XYZ2xy(srcXYZ);
xyDst = XYZ2xy(dstXYZ);

xyzSrc = xy2XYZ(xySrc, 1); %normalize Y to 1 so D65 luminance comparable
xyzDst = xy2XYZ(xyDst, 1); %normalize Y to 1 so D65 luminance comparable

% Chromatic adaptation transform
catType = 'bradford'; %Bradford transformation is recommended by Bruce Lindbloom http://www.brucelindbloom.com/index.html?Eqn_ChromAdapt.html
estMAT = cbCAT(xyzSrc, xyzDst, catType);

% Ascale estMAT by XYZD65_to_sRGB before apply color correction 
M = estMAT * XYZD65_to_sRGB;

linI = cbreshape(linI);

% Destination image - apply color correction be multiplying by correction matrix M
linJ = M*linI;

linJ = cbunshape(linJ, size(I));

% Convet J from Linear to sRGB
J = lin2rgb(linJ);


% Convert from double to uint8 (multiply by 255).
J = im2uint8(J);


% Display result
figure;imshow(J);title('Corrected image1');impixelinfo
figure;imshow(image2);title('image2');impixelinfo

% Save result
imwrite(image2, 'image2.png');
imwrite(J, 'J.png');


function xy = XYZ2xy(xyz)
    %xy = XYZ2xy(xyz)
    % Converts CIE XYZ to xy chromaticity.
    X = xyz(1, :);
    Y = xyz(2, :);
    s = sum(xyz);
    xy = [X./s; Y./s];
end


function XYZ = xy2XYZ(xy,Y)
    %XYZ = xy2XYZ(xy,Y)
    % Converts xyY chromaticity to CIE XYZ.
    x = xy(1); y = xy(2);
    XYZ = [Y/y*x; Y; Y/y*(1-x-y)];
end



function outMat = cbCAT(xyz_src, xyz_dst, type)
    %https://web.stanford.edu/~sujason/ColorBalancing/adaptation.html
    %M = cbCAT(xyz_src, xyz_dst, type)
    % Chromatic adaptation transform via von Kries's method.
    %  type     chooses the LMS-like space to apply scaling in, valid options:
    %           'vonKries', 'bradford', 'sharp', 'cmccat2000', 'cat02', 'xyz'
    % See http://www.brucelindbloom.com/index.html?Eqn_ChromAdapt.html


    xyz_src = makecol(xyz_src);
    xyz_dst = makecol(xyz_dst);

    % the following are mostly taken from S. Bianco. "Two New von Kries Based
    % Chromatic Adaptation Transforms Found by Numerical Optimization."
    if strcmpi(type,'vonKries') %Hunt-Pointer-Estevez normalized to D65
        Ma = [0.40024 0.7076 -0.08081; -0.2263 1.16532 0.0457; 0 0 0.91822];
    elseif strcmpi(type,'bradford')
        Ma = [0.8951 0.2664 -0.1614; -0.7502 1.7135 0.0367; 0.0389 -0.0685 1.0296];
    elseif strcmpi(type,'sharp')
        Ma = [1.2694 -0.0988 -0.1706; -0.8364 1.8006 0.0357; 0.0297 -0.0315 1.0018];
    elseif strcmpi(type,'cmccat2000')
        Ma = [0.7982 0.3389 -0.1371; -0.5918 1.5512 0.0406; 0.0008 0.239 0.9753];
    elseif strcmpi(type,'cat02')
        Ma = [0.7328 0.4296 -0.1624; -0.7036 1.6975 0.0061; 0.0030 0.0136 0.9834];
    else
        Ma = eye(3);
    end

    %Chromatic Adaptation Transforms:
    %1. Transform from XYZ into a cone response domain (ro, gamma, beta)
    %2. Scale the vector components by factors dependent upon both the source and destination reference whites.
    %3. Transform from (ro, gamma, beta) back to XYZ using the inverse transform of step 1.

    %D is diagonal matrix marked as inv(Ma)*diag(roD/roS, gammaD/gammaS, betaD/betaS)*Ma.
    %Matrix D applies ratios in "cone response domain".
    D = diag((Ma*xyz_dst)./(Ma*xyz_src));

    %Transform back to XYZ domain:
    M = Ma\D*Ma;

    sRGBtoXYZ =   [0.4124564  0.3575761  0.1804375; ...
                   0.2126729  0.7151522  0.0721750; ...
                   0.0193339  0.1191920  0.9503041];

    outMat = sRGBtoXYZ\M*sRGBtoXYZ;
end



function x = makecol(x)
    %x = makecol(x)
    % returns x as a column vector

    s = size(x);
    if (length(s) == 2) && (s(1) < s(2))
        x = x.';
    end
end


function out = cbreshape(im)
    %out = cbreshape(im)
    % Takes a width x height x 3 RGB image and returns a matrix where each column is an RGB
    % pixel.
    if (size(im, 3) == 3)
        out = reshape(permute(im, [3, 1, 2]), [3, numel(im)/3, 1]);
    else
        out = (im(:))';
    end
end


function out = cbunshape(mat,s)
    %out = cbunshape(im,[height, width])
    % Takes a 3xn matrix of RGB pixels and returns a height x width x 3 RGB
    % image

    height = s(1); width = s(2);

    if (size(mat,1) == 3)
        %In case mat is 3 rows, convert to 3D matrix
        out = reshape(mat,[3,height,width]);
        out = permute(out,[2 3 1]);
    else
        %In case mat is 1 row, convert to 2D matrix
        out = reshape(mat, [height, width]);
    end
end

---

Result:

---

Update:

Same solution with luminescence adjustment:

In case you need to correct the luminescence, add the following code before "Color correction by Chromatic Adaptation":

% Scale the input so the mean of the triangle in image1 and image2 will be the same.
% The scaling is eqivalent to adjusting the exposure level of the camera.
rgb_scale = mean(dstRGB) / mean(srcRGB);
srcRGB = srcRGB*rgb_scale;
linI = linI*rgb_scale;

Result: