Using OpenCV and Python to replace a segmented part of an image with other images

Question

I want to replace a part of an image with another image. I am able to segment an image into it's respective class, take for example, in the image links, the picture of an eagle has been segmented using solid colours to differentiate between the object and the background.

Original Photo
Segmented Photo

Is there a suitable algorithm that I can experiment with, which detects object by colour, and performs object replacement? For example, I want to replace this eagle with another type of bird (is there any way to resize the new image to fit the current segmented part of the image?).

I'm very new to this concept, so any feedback will be greatly appreciated! I'm by no means asking for solutions, I'm just looking for a starting point (algorithms, sample code, tutorials, etc) for me to explore further. Thank you so much.

Answer 1

pyimagesearch is a good blog to get idea on many image processing techniques. Here are a collection of few links that may help.

To detect objects by colour

To get a good understanding on background and foreground in an image, and working of face detection in opencv

Very good background subtraction library

Stitching the images, you will find this useful for image replacement

Answer 2

You can use this deeplab v3 model which is trained on thousands of images to correctly segment any common object (in your case, birds). This will give you the mask image.

Now, if you try to crop only the foreground and overlay it on any other background it's very easy to do by simply using opencv's seamless clone . which makes the replacement look natural.

But if you are trying to crop only the background and replace it with another bird it could be a little bit tricky I think. Because you need to 1st remove the bird and replace it with a possible background and then place the other bird.

I hope you can start your journey here and improve along the way.