如何在Python中使用OpenCV提取图像的特定部分？

Question

I am trying to extract a portion of an image by performing Canny edge detection. I have succesfully created a mask of that object. But when I perform a bitwise_and operation with the original image, to extract the foreground section, I am getting the following error .

OpenCV Error: Assertion failed ((mtype == CV_8U || mtype == CV_8S) && _mask.sameSize(*psrc1)) in cv::binary_op, file C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp, line 241
    Traceback (most recent call last):
      File "C:\Users\Boudhayan Dev\Desktop\extraction.py", line 37, in <module>
        new_image = cv2.bitwise_and(img_rgb,img_rgb,mask=mask)
    cv2.error: C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp:241: error: (-215) (mtype == CV_8U || mtype == CV_8S) && _mask.sameSize(*psrc1) in function cv::binary_op

My code is as follows -

import cv2
import numpy as np

img_rgb = cv2.imread("3.jpg")
cv2.namedWindow("Original Image",cv2.WINDOW_NORMAL)

img = cv2.cvtColor(img_rgb,cv2.COLOR_RGB2HSV)
img = cv2.bilateralFilter(img,9,105,105)
r,g,b=cv2.split(img)
equalize1= cv2.equalizeHist(r)
equalize2= cv2.equalizeHist(g)
equalize3= cv2.equalizeHist(b)
equalize=cv2.merge((r,g,b))

equalize = cv2.cvtColor(equalize,cv2.COLOR_RGB2GRAY)

ret,thresh_image = cv2.threshold(equalize,0,255,cv2.THRESH_OTSU+cv2.THRESH_BINARY)
equalize= cv2.equalizeHist(thresh_image)


canny_image = cv2.Canny(equalize,250,255)
canny_image = cv2.convertScaleAbs(canny_image)
kernel = np.ones((3,3), np.uint8)
dilated_image = cv2.dilate(canny_image,kernel,iterations=1)


new,contours, hierarchy = cv2.findContours(dilated_image, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
contours= sorted(contours, key = cv2.contourArea, reverse = True)[:10]
c=contours[0]
print(cv2.contourArea(c))
final = cv2.drawContours(img, [c], -1, (255,0, 0), 3)



mask = np.zeros(img_rgb.shape,np.uint8)
new_image = cv2.drawContours(mask,[c],0,255,-1,)
new_image = cv2.bitwise_and(img_rgb,img_rgb,mask=mask)


cv2.namedWindow("new",cv2.WINDOW_NORMAL)
cv2.imshow("new",new_image)

cv2.imshow("Original Image",img)
cv2.waitKey() 

NOTE :- The code works fine if I try to perform the bitwise_and with a grayscale version of the image. However RGB,HSV or any other color spaces give the error above.

Please help.

---

EDIT 1 - The image in question is this -

EDIT 2-

The following is the result after using Numpy method. As you can see, the extracted image is the same size as the orange but it does not contain the orange instead the mask itself.

EDIT 3- @DanMašek and @lightalchemist , I could finally extract any foreground image.

Thank you

Answer 1

The error is telling you that you are trying to perform the bitwise_and operation on matrices where the entries are not integers. I believe the matrices also have to have the same number of channels. This is why it works with your grayscale image but not with the HSV image.

Instead of using bitwise_and, it is easier and more flexible to just use numpy matrix vectorization to perform the masking like so:

mask = np.zeros_like(img_rgb, dtype=np.uint8)

# I believe you want to draw the filled in contour on the mask
# You code actually assigns the resulting mask to new_image
# But that does not affect things as drawContours modifies mask in place
mask = cv2.drawContours(mask, [c] ,0, 255, -1) 

new_image = img_rgb.copy()
new_image[mask < 255] = 0  # Set values not masked to be 0

Note that if your mask is a single channel matrix instead of a 3 channel matrix, you would have to modify the code to be

new_image[mask < 255, :] = 0

Answer 2

I used the code provided above but only altered the line where the cv2.bitwise_and() is used:

new_image = cv2.bitwise_and(img_rgb, img_rgb, mask = equalize)

This is what I got and what you expected (I guess):

EDIT

I get it, you want to mask your image with an image of contour having the greatest area. In the following additional snippet I have binarized the image containing the contour of greatest area to be used as mask.

new_image = cv2.drawContours(mask,[c], -1, (255,255,255), -1)
new_image_gray = cv2.cvtColor(new_image, cv2.COLOR_BGR2GRAY)
ret, thresh1 = cv2.threshold(new_image_gray, 100, 255, cv2.THRESH_BINARY)
final = cv2.bitwise_and(img_rgb, img_rgb, mask = thresh1)

This is what I got:

Compared to the image above you do not see those holes inside the object of interest.