如何使用openCV python用另一个图像替换图像的蒙版部分？

Question

The objective is to replace a part of an image with another image. What I plan to do is to get the segment map of the original photo, and replace the selected part of the original with another image.

For example, This is a photo of the original photo's segment map:

And I want to replace that pink segment mask with another guy model (different person):

How should I do so about it? I am thinking of selecting an ROI based on the pink colour (RGB Values of 192,128,128) and bitwise_and the two images? But I'm not too sure on how to do that, or if that's the best approach.

I am aware that the pink mask and the sample guy are not the exact fit, but I just want to be able to fit the model in the pink segment first, and perhaps scale or transform it afterwards.

Any suggestions will be great! Thank you!

Answer 1

Load libs and read images

from __future__ import division
import cv2
import numpy as np

guy_img = cv2.imread('path')
target_img = cv2.imread('path')

Get bounding boxes for the guy (every pixel that is non-zero) and for the target mask (the pink region).

a = np.where(guy_img > 0)
b = np.where(target_img == 129)  # picked one of the channels in your image
bbox_guy = np.min(a[0]), np.max(a[0]), np.min(a[1]), np.max(a[1])
bbox_mask = np.min(b[0]), np.max(b[0]), np.min(b[1]), np.max(b[1])

Note that when I loaded the target image in the values were different from the ones you provided. There were also some white pixels along the edges of the image. That was likely just due to the image being uploaded to imgur and downloaded. If your values are correct and the rest of your image is completely black except for the pink region you can just get all non zero pixels the same way I did for the guy image with np.where(target_img > 0) .

Now get just the values of the guy and mask regions

guy = guy_img[bbox_guy[0]:bbox_guy[1], bbox_guy[2]:bbox_guy[3],:]
target = target_img[bbox_mask[0]:bbox_mask[1], bbox_mask[2]:bbox_mask[3],:]

Resize the guy to be the same scale / shape as the mask

guy_h, guy_w, _ = guy.shape
mask_h, mask_w, _ = target.shape
fy = mask_h / guy_h
fx = mask_w / guy_w
scaled_guy = cv2.resize(guy, (0,0), fx=fx,fy=fy)

Reassign the masked region of the target image with the guy image values

for i, row in enumerate(range(bbox_mask[0], bbox_mask[1])):
    for j, col in enumerate(range(bbox_mask[2], bbox_mask[3])):
        target_img[row,col,:] = scaled_guy[i,j,:]

cv2.imshow('', target_img)
cv2.waitKey(0)

Answer 2

First you need to consider few things before you go ahead with the implementation.

• Consistent background for mask image? consistent color of masked blob?
• Size of the mask and the object to add image.

I would suggest you to first find the size, color and location of the ROI on the pink segmented image. Scale the dimensions of the Object(Boy image) according to the mask image, so it gets easier to mask the object image to the pink part.

You can then create an image (lets call is step one Image) with dimension similar to mask image. Add the scaled object image to it at location obtained from the pink blob analysis. This way you will have two images with similar dimension and similar scale.

With the step one image and Pink ROI should be very easily overlap-able visually.

if you want exact pink then replace all the pixels on the step one image with the black pixel if the same location pixel is black on the masked image. Otherwise let it be what it is.

if(mask[i][j] == black){ step_one[i][j] = black }

which leaves the pink part masked out of object image.