Select a bounding box on an image and annotate

Question

I'm working on an project where I would like to take a bounding box already drawn on a subject and select it (via mouse click) so I can have something like a text dialogue box hover above the image, so I can then type in a label. I'm already using OpenCV to detect the object and draw an initial bounding box on it using the Haar Cascade classifier, but so far I can't find the right combination of OpenCV directives to be able to select that bounding box and then annotate it. The relevant code is below.

faces = faceCascade.detectMultiScale(
    gray,
    scaleFactor=1.1,
    minNeighbors=5,
    minSize=(30, 30),
)

# Draw a rectangle around the faces
for (x, y, w, h) in faces:
    cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)

Would appreciate some good pointers. Thanks.

Answer 1

You can take the x/y position of the mouse and compare that to the bounding boxes. The code below descibes how you could do that.

First, to be able to process mouse input, you have to crate a namedWindow. You can then attach a mouseCallback to that window:

# create window
cv2.namedWindow("Frame") 
# attach a callback to the window, that calls 'getFace'
cv2.setMouseCallback("Frame", getFace) 

In the getFace method, you check for button press, then loop through the faces and check if the x/y of the mouse is within the bounds of the bounding box of a face. If so, return the index of the face.

def getFace(event, x,y, flags, param):
        if event == cv2.EVENT_LBUTTONDOWN:
                # if mousepressed
                for i in range(len(faces)): 
                        # loop through faces
                        (face_x,face_y,w,h) = faces[i]
                        # unpack variables
                        if x > face_x and x < face_x + w:
                                # if x is within x-range of face
                                if y > face_y and y < face_y + h:
                                        # if y is also in y-range of face
                                        return i
                                        # then return the index of the face