how to copy a part of image to another with OpenCV?

Question

I have a model that predicts the human face segmentation.

But unfortunately the model is not trained to predict the hair with the face. So now I have the above images as numpy arrays. Is it possible to add the hair from the original photo (The one on the left) to the prediction mask (in the middle) or directly to the result prediction (the one on the right) ?

Basically I just need to deal with the original image to get a little bit from the head above the prediction mask and add it to the prediction so that way at least I'll have a part of the hair in the result.

The code used to create the previous figure:

fn = "images/beard.jpg"
im = cv2.cvtColor(cv2.imread(fn), cv2.COLOR_BGR2RGB)
im = auto_downscaling(im)

# vgg_preprocess: output BGR channel w/ mean substracted.
inp_im = vgg_preprocess(im)

# Predicting the face segmentation
out = model.predict([inp_im])

out_resized = cv2.resize(np.squeeze(out), (im.shape[1],im.shape[0]))
out_resized_clipped = np.clip(out_resized.argmax(axis=2), 0, 1).astype(np.float64)
mask = cv2.GaussianBlur(out_resized_clipped, (7,7), 6)


plt.figure(figsize=(12,6))
plt.subplot("131")
plt.axis('off')
plt.imshow(im)
plt.subplot("132")
plt.axis('off')
plt.imshow(out_resized_clipped, cmap='gray')
plt.subplot("133")
plt.axis('off')
plt.imshow((mask[:,:,np.newaxis]*im.astype(np.float64)).astype(np.uint8))
plt.show()

Answer 1

To demonstrate what you want i tested this. First I create some function to create similar sized images from the inputs.

import cv2
import numpy as np

def resize(img, dim=(300, 300)):
    # perform the actual resizing of the image and show it
    frame = cv2.resize(img, dim, interpolation=cv2.INTER_AREA)
    return frame

Then read and resize the images separately

img1 = resize(cv2.imread('img1.png'))
img2 = resize(cv2.imread('img2.png'))
dst = cv2.addWeighted(img1,0.7,img2,0.3,0)

The cv2.addWeighted function will helps me achieve the result. then you can do as you need with the output

cv2.imwrite('out.png', dst)

So Applying this same principle to your code would be something like. I see that you have similar sized images there.

fn = "images/beard.jpg"
im = cv2.cvtColor(cv2.imread(fn), cv2.COLOR_BGR2RGB)
im = auto_downscaling(im)

# vgg_preprocess: output BGR channel w/ mean substracted.
inp_im = vgg_preprocess(im)

# Predicting the face segmentation
out = model.predict([inp_im])

out_resized = cv2.resize(np.squeeze(out), (im.shape[1],im.shape[0]))
out_resized_clipped = np.clip(out_resized.argmax(axis=2), 0, 1).astype(np.float64)
out_resized_clipped = cv2.GaussianBlur(out_resized_clipped, (7,7), 6)

mask = cv2.addWeighted(out_resized, 0.7, out_resized_clipped,0.3,0)
mask = cv2.GaussianBlur(out_resized_clipped, (7,7), 6)

plt.figure(figsize=(12,6))
plt.subplot("131")
plt.axis('off')
plt.imshow(im)
plt.subplot("132")
plt.axis('off')
plt.imshow(out_resized_clipped, cmap='gray')
plt.subplot("133")
plt.axis('off')
plt.imshow((mask[:,:,np.newaxis]*im.astype(np.float64)).astype(np.uint8))