Overlaying an image with another non-rectangular image containing black pixels using OpenCV in Python

Question

I want to programmatically overlay an image, eg the blissfully familiar Windows XP wallpaper:

with another non-rectangular image that contains black pixels, eg a standard large cursor icon:

Copy-pastoring the codes from this and this tutorial which both use OpenCV bitwise masking magic I arrived at:

import cv2 as cv

# Load two images
img1 = cv.imread('bliss.png') # The image I want the overlay to be diplayed on.
img2 = cv.imread('cursor.png') # The image I want to overlay with, containing black pixels.

# I want to put logo on top-left corner, So I create a ROI.
rows, cols, channels = img2.shape
roi = img1[0:rows, 0:cols ]

# Now create a mask of logo and create its inverse mask also.
img2gray = cv.cvtColor(img2, cv.COLOR_BGR2GRAY)
ret, mask = cv.threshold(img2gray, 20, 255, cv.THRESH_BINARY)
mask_inv = cv.bitwise_not(mask)

# Now black-out the area of logo in ROI.
img1_bg = cv.bitwise_and(roi, roi, mask = mask_inv)

# Take only region of logo from logo image.
img2_fg = cv.bitwise_and(img2, img2, mask = mask)

# Put logo in ROI and modify the main image
dst = cv.add(img1_bg, img2_fg)
img1[0:rows, 0:cols ] = dst
cv.imshow('res',img1)
cv.waitKey(0)
cv.destroyAllWindows()

During naive permutations of trying to find the right parameters for cv.threshold (including thres and maxval arguments as well as thresholding types ), I always find that significant number of black pixels present in the original image are missing from the overlaid one. In the zoomed-in picture below on the left you can see the overlaid cursor and on the right is original copied one:

I reckon this loss of pixels is due to the grayscale conversion and/or the inverse (?) masking but could not figure out how or what to change in the code above. In the tutorials I linked above images not containing black pixels were used for the overlay and the results look fine. Is there a way to do the same with images containing black pixels?

Answer 1

The problem here is, that you lose the black pixels in cursor.png at cv.threshold(img2gray, 20, 255, cv.THRESH_BINARY) . The remainder are the white pixels only, thus your mask is just too small. Since cursor.png has transparency information stored in it, you can use its alpha channel for your mask.

Here's your code, modified accordingly (I removed all of your comments; comments show my changes):

import cv2 as cv

img1 = cv.imread('bliss.png')
img2 = cv.imread('cursor.png', cv.IMREAD_UNCHANGED)         # Added cv.IMREAD_UNCHANGED parameter to maintain alpha channel information

alpha = img2[:, :, 3]                                       # Save alpha channel for later use
_, alpha = cv.threshold(alpha, 5, 255, cv.THRESH_BINARY)    # Threshold alpha channel to prevent gradual transparency
img2 = cv.cvtColor(img2, cv.COLOR_BGRA2BGR)                 # Remove alpha channel information, so that code below still works

rows, cols, channels = img2.shape
roi = img1[0:rows, 0:cols ]

                                                            # img2gray no longer needed
mask = alpha                                                # Mask is just the alpha channel saved above
mask_inv = cv.bitwise_not(mask)

img1_bg = cv.bitwise_and(roi, roi, mask = mask_inv)

img2_fg = cv.bitwise_and(img2, img2, mask = mask)

dst = cv.add(img1_bg, img2_fg)
img1[0:rows, 0:cols ] = dst
cv.imshow('res',img1)
cv.waitKey(0)
cv.destroyAllWindows()

Hopefully, the output image then looks like something you expected: