opencv blend (overlay) two image with different channels

Question

我想用opencv覆盖两个图像（可能是不同的格式（通道）。最初我使用addWeighted，但是，它在具有不同通道的两个图像上失败了。opencv中有什么方法可以处理这种情况吗？

Answer 1

No , there is no API in OpenCV that offers this feature nativelly. On the other hand, there's nothing stopping you from writing your own code to do that.

A few weeks ago a made some changes to a function I saw somewhere else on the Internet to be able to:

• Pass two input images: the background as BGR and the foreground as BGRA ;
• Blend them together according to a simple transparency rule;

(I don't remember where most of this code came from, sorry... but thank you whoever you are!)

void overlayImage(const cv::Mat &background, const cv::Mat &foreground, cv::Mat &output, cv::Point2i location, double opacity = 1.0)
{
    background.copyTo(output);

    // start at the row indicated by location, or at row 0 if location.y is negative.
    for (int y = std::max(location.y , 0); y < background.rows; ++y) {
        int fY = y - location.y; // because of the translation

        // we are done of we have processed all rows of the foreground image.
        if (fY >= foreground.rows)
            break;

        // start at the column indicated by location, or at column 0 if location.x is negative.
        for (int x = std::max(location.x, 0); x < background.cols; ++x) {
            int fX = x - location.x; // because of the translation.

            // we are done with this row if the column is outside of the foreground image.
            if (fX >= foreground.cols)
                break;

            // determine the opacity of the foregrond pixel, using its fourth (alpha) channel.
            double opacity_level = ((double)foreground.data[fY * foreground.step + fX * foreground.channels() + 3]) / 255.;
            if (opacity >= 0.0 && opacity < 1.0)
                opacity_level *= opacity;

            // and now combine the background and foreground pixel, using the opacity, but only if opacity > 0.
            for (int c = 0; opacity_level > 0 && c < output.channels(); ++c) {
                unsigned char foregroundPx = foreground.data[fY * foreground.step + fX * foreground.channels() + c];
                unsigned char backgroundPx = background.data[y * background.step + x * background.channels() + c];
                output.data[y*output.step + output.channels()*x + c] = backgroundPx * (1.-opacity_level) + foregroundPx * opacity_level;
            }
        }
    }
}

Below you will find the input images used for testing: the left is the background and the image on the right is the foreground .

To copy the foreground completely over the background, just do:

cv::Mat background = cv::imread("road.png");         // 3-chan BGR
cv::Mat foreground= cv::imread("tulip.png", -1);     // 4-chan BGRA
cv::Point location(0, 0);  

cv::Mat output;
overlayImage(input_bkg, input_target, output, location, 1.0);
cv::imwrite("output_alpha1.0.png", output);

and to perform a copy with 50% transparency:

overlayImage(input_bkg, input_target, output, location, 0.5);
cv::imwrite("output_alpha0.5.png", output);

Here are the results:

This implementation is not bulletproof for production purposes, so use it at your own risk.

Answer 2

Thanks for everyone that tried to answer my question. In the end, I took the path that enumerates the conditions of background image and convert it to same channel number and then used weighted sum. Not sure whether this one is correct way to handle it, but it seems work for me, except the transparent background PNG. Any suggestions?

private boolean convertToSameChannels() {
  if (_overlayImage.channels() != _image.channels()) {
    if (_image.channels() == 1) {
      if (_overlayImage.channels() == 3) {
        Imgproc.cvtColor(_overlayImage, _overlayImage, Imgproc.COLOR_BGR2GRAY);
      } else if (_overlayImage.channels() == 4) {
        Imgproc.cvtColor(_overlayImage, _overlayImage, Imgproc.COLOR_BGRA2GRAY);
      }
    } else if (_image.channels() == 3) {
      if (_overlayImage.channels() == 1) {
        Imgproc.cvtColor(_overlayImage, _overlayImage, Imgproc.COLOR_GRAY2BGR);
      } else if (_overlayImage.channels() == 4) {
        Imgproc.cvtColor(_overlayImage, _overlayImage, Imgproc.COLOR_BGRA2BGR);
      }
    } else if (_image.channels() == 4) {
      if (_overlayImage.channels() == 3) {
        Imgproc.cvtColor(_overlayImage, _overlayImage, Imgproc.COLOR_BGR2BGRA);
      } else if (_overlayImage.channels() == 1) {
        Imgproc.cvtColor(_overlayImage, _overlayImage, Imgproc.COLOR_GRAY2BGRA);
      }
    } else {
      System.out.println("overlay image not suppoerted error");
      return false;
    }
  }
  return true;
}