如何将黑色背景上的白色框架中的间隙与 python opencv 连接起来？

Question

I developed a frame detection algorithm in python using opencv that detects a metallic frame in a rgb image. The output is a white frame on a black background. However, if the frame is partly covered, it contains gaps. How can I fill the gaps, preferably using opencv?

Example output of my algorithm:

Frame with gap

Desired output:

Frame without gap

Edit: This is how I detect the frame currently.

I have an rgb image of a crop canopy and a frame on top of it: Crop canopy image

My code detects bright parts in the image and removes noise such that only the frame should be left (which is the largest brigth object in the image). Below is a reproducible example when setting image path to the example image above:

# import packages
import numpy as np
import cv2

kernel_size = 10
brightness_thres_low = 175
brightness_thres_high = 255
erode_it_1 = 6
dilate_it_1 = 25
erode_it_2 = 20


# read image
img = cv2.imread(image_path)

# find bright pixel values that indicate frame and set to bw
thres = cv2.threshold(cv2.cvtColor(img, cv2.COLOR_BGR2GRAY), brightness_thres_low, brightness_thres_high, cv2.THRESH_BINARY_INV)[1]
thres = np.invert(thres)
        
# adapt image to only contain frame
kernel = np.ones((kernel_size, kernel_size), np.uint8)
img_adapted = cv2.erode(thres,kernel, iterations = erode_it_1)
img_adapted = cv2.dilate(img_adapted,kernel, iterations = dilate_it_1)
img_adapted = cv2.erode(img_adapted, kernel, iterations = erode_it_2)
        

If the frame is fully in the image, it gets detected fine. But as you can see in the example rgb image, part of the frame is covered by a plant. This results in an incomplete detection of the frame. I would like to fill this gap in the frame in a generic way such that it can be applied to any gap of any processed rgb image.

Answer 1

Here is some code you can start with:

import numpy as np
import cv2

kernel_size = 10
erode = 2
thres_low = 175
thres_high = 255

img = cv2.imread("img.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, thres = cv2.threshold(gray, thres_low, thres_high, cv2.THRESH_BINARY)

kernel = np.ones((kernel_size, kernel_size), np.uint8)
thres = cv2.erode(thres, kernel, iterations=erode)   # erode fine contours

contours, _ = cv2.findContours(thres, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)  # find contours
all_contours = np.vstack(contours)   # concatenate all contours
hull = cv2.convexHull(all_contours)  # get convex hull of all contours
peri = cv2.arcLength(hull, True)     # get perimeter length of hull for the next step
approx = cv2.approxPolyDP(hull, 0.05 * peri, True)    # simplify hull
cv2.drawContours(img, [approx], 0, (0, 0, 255), 10)   # draw the contour

cv2.imwrite("out.png", img)

Output image: