How to know if there is more than one face in front of the camera in mediapipe?

Question

I am running a python script by mediapipe to detect the faces of people, I was able to extract the x coordinate of the nose. Those coordinates will be send serially to a microcontroller to make decisions. A problem arises when there are two faces or more in front of the camera, the x coordinates printed starts oscillating back and forth between each detected face. I want to be able to tell the microcontroller that there are more that 1 faces in front of the camera to know what to do but I can't find a way to extract this information from the data given inside "detection" variable. This is the code I used to track the x position of the nose:

    # face detection

import cv2
import mediapipe as mp
import time

mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils


# capture video
cap = cv2.VideoCapture(0)
prevTime = 0

with mp_face_detection.FaceDetection( model_selection=1,
    min_detection_confidence=0.65) as face_detection:
  while True:
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      break


    #Convert the BGR image to RGB.
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    image.flags.writeable = False
    results = face_detection.process(image)

    # Draw the face detection annotations on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    if results.detections:
      for detection in results.detections:
        mp_drawing.draw_detection(image, detection)
        position = detection.location_data.relative_bounding_box
        x = position.xmin 
        print(x) # x are the coordinates of the nose


    cv2.imshow('BlazeFace Face Detection', image)
    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()

I tried saving the old x and comparing it with the new x and if there is a lot of difference than that means there are two or more faces but that is not a clean way to do it, faces can get close to each other in terms of x coordinates if they are far away from the camera and the threshold for what is considered not normal is hard to tell.

Answer 1

What you want is to be able to track faces. The Face Detection module of mediapipe is not offering such a feature built-in. However, you can use Mediapipe Box Tracking or another tracking algorithm of your choice to track multiple faces in your video stream.

If you can afford the computational cost of face detection at each frame, which depends on your task and hardware, you can implement a simpler naive solution. This could be to assign a certain label to new faces and assign this label to the closest face in the next frame.

Additionally, note that the value you store in x is not the nose coordinates but the minimum x-coordinates of the face bounding box. To access the coordinates of the nose, you can use: mp_face_detection.get_key_point(detection, mp_face_detection.FaceKeyPoint.NOSE_TIP) Take some time to familiarize yourself with the output of the face detection model on Mediapipe documentation .