如何将 cv2 矩形边界框合并为多边形？ （不是通过重叠/阈值）

Question

I have multiple rectangle bounding boxes that I know are part of the same object (parts of a newspaper article), as in the first image. I'm trying to work out a way to merge these into one polygon bounding box, for the whole article, as in the second image.

I have seen lots of solutions based on merging overlapping bounding boxes, but I don't care if they're overlapping or not - I already know they're part of the same article. In some cases the headline is quite far away (for example above a picture), so solutions based on padding don't work either.

I feel like there should be a cv2 function that does this, but if there is, I am missing it. Any suggestions would be super helpful.

Code to create these two images:

# Individual bounding boxes

image_0 = cv2.imread('63976500-anderson-herald-bulletin-Jun-18-1968-p-64.jpg')
# Black box, to reproduce: image_0 = np.zeros((5000, 6000, 3), dtype = "uint8")

bbox_list = [[195, 3455, 633, 4213], [658, 3427, 1094, 4222], [1120, 3435, 1553, 4473], [295, 3421, 531, 3451], [201, 3313, 1548, 3409]]

for bbox in bbox_list:
    image_0 = cv2.rectangle(image_0, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (0,255,0), 10)

cv2.imwrite("original_bboxes.jpg", image_0)


# Grouped bounding box

image_1 = cv2.imread('63976500-anderson-herald-bulletin-Jun-18-1968-p-64.jpg')
# Black box, to reproduce: image_1 = np.zeros((5000, 6000, 3), dtype = "uint8")

coordinates = np.array([[195,3313],[195,4222],[1120,4222],[1120,4473],[1553,4473],[1553,3313]], np.int32)

image_1 = cv2.polylines(image_1, [coordinates], True, (0,255,0), 10)

cv2.imwrite("grouped_bboxes.jpg", image_1)

Answer 1

You could draw the convex hull of the contour points (this was drawn by hand):

Then keep only the outer contour and try the polygonal approximation. I must admit that I cannot think of a smarter way of getting only vertical and horizontal lines.