使用 Python / OpenCV 将圆圈拟合到虹膜照片

Question

I've got a lot of frontal face images, and I want to obtain the exact position of the (center of the) two irises in these pictures. For example:

, ,

I have already run the DLib 68 landmark face annotator to get the position of the two eyes, so for example I would have a picture that looks like this:

, ,

As can be seen, the annotation is able to get the rough position of the eye correct. I have already tried to follow the approach in this article , but this requires the lighting conditions in all pictures to be the same and the threshold to be manually set. Also, the convex polygon around the eye is not always complete, so this won't work, as it will sometimes only have part of the eye inside the mask:

keypoints = [36, 37, 38, 39, 40, 41]
landmarks = image.get_face_landmarks()
points = np.array([landmarks[i] for i in keypoints], dtype=np.int32)

mask = np.zeros(img.shape[:2], dtype=np.uint8)
mask = cv2.fillConvexPoly(mask, points, 255)
    
mask = cv2.dilate(mask, np.ones((9,9)))
eyes = cv2.bitwise_and(img, img, mask=mask)
eyes[(eyes == 0).all(2)] = [255, 255, 255]
eyes_gray = cv2.cvtColor(eyes, cv2.COLOR_BGR2GRAY)

I was thinking of using something like Hough circle transform, but for that I would need the whole iris to be fully visible, not covered partly by an eyelid. As the iris is a partly circular shape, and is surrounded by a very different color inside the eye, I figured it should be possible to fit a circle to the iris, how would I best approach this? This answer suggests using the landmarks and contours, but so far I haven't gotten that working reliably.

Answer 1

The article and paper proposed by Diego Schmaedech in the comments indeed provides a good way to fix this, I have found an implementation here which yields accurate results with good performance