使用 sklearn svm 进行面部识别的问题

Question

I am working on a school project to make a facial recognition program using Python. I am using the face_recognition and scikit-learn libraries. However, I am facing some issues. Here is my code:

"""
Structure:
        <Data>/
            <person_1>/
                <person_1_face-1>.jpg
                <person_1_face-2>.jpg
                .
                .
                <person_1_face-n>.jpg
           <person_2>/
                <person_2_face-1>.jpg
                <person_2_face-2>.jpg
                .
                .
                <person_2_face-n>.jpg
            .
            .
            <person_n>/
                <person_n_face-1>.jpg
                <person_n_face-2>.jpg
                .
                .
                <person_n_face-n>.jpg
"""
import os

import cv2
import face_recognition
import numpy as np
from sklearn import svm

IMG_DATA_DIR = "Data"
class_names = []
encodings = []
image_dirs = os.listdir(IMG_DATA_DIR)

# Loop through each person in the training directory
for img_dir in image_dirs:
    img_files = os.listdir(f"{IMG_DATA_DIR}/{img_dir}")

    # Loop through each training image for the current person
    for img_file in img_files:
        # Get the face encodings for the face in each image file
        img = face_recognition.load_image_file(f"{IMG_DATA_DIR}/{img_dir}/{img_file}")
        class_names.append(os.path.splitext(img_dir)[0])

        img_encoding = face_recognition.face_encodings(img)[0]
        encodings.append(img_encoding)

clf = svm.SVC(gamma="scale")
clf.fit(encodings, class_names)


# Initializing webcam
camera = cv2.VideoCapture(0)

process_this_frame = True

while True:
    success, img = camera.read()

    if process_this_frame:
        img_small = cv2.resize(img, (0, 0), None, 0.50, 0.50)
        img_small = cv2.cvtColor(img_small, cv2.COLOR_BGR2RGB)

        camera_faces_loc = face_recognition.face_locations(img_small)
        camera_encodings = face_recognition.face_encodings(img_small, camera_faces_loc)

        face_names = []
        for encoding in camera_encodings:
            # loop through each face encodings visible in the camera frame
            # predict the names of the faces currently visible in the frame using clf.predict
            name = clf.predict([encoding])
            print(name)

            face_names.extend(name)

    process_this_frame = not process_this_frame

    for (top, right, bottom, left), name in zip(camera_faces_loc, face_names):
        top *= 2
        right *= 2
        bottom *= 2
        left *= 2

        cv2.rectangle(img, (left, top), (right, bottom), (0, 255, 0), 2)
        cv2.rectangle(
            img, (left, bottom - 35), (right, bottom), (0, 255, 0), cv2.FILLED
        )
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(img, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    cv2.imshow("WebCam", img)

    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

camera.release()
cv2.destroyAllWindows()

As the code above suggests, my aim here is to supply multiple images of the same person to the model so that it gets better over time. So far, I am facing two major issues.

Issue 1: If I have only one picture of the same person in their corresponding directories, the classifier is able to predict the name of the person(s) visible in the camera frame. However, if I add a second image to one of the directories (while keeping the other directories with only one image), the classifier predicts every face in the camera frame to be the person who had two images in his/her directory. For example, if person A has two images under his name in his directory while person B only has one, the classifier will predict person B to be person A (not only person B, the classifier will predict anyone to be person A). What is causing this issue? Having multiple images for the same person is a big reason I am using the svm classifier.

Issue 2: If I show someone's face whose picture was not in the original training data directories, the classifier still randomly predicts this unknown person to be one the known persons. For example, if I have person A to C in my training directories, and I show a completely unknown person D, the classifier, for some reason, randomly predicts the unknown person to be either person A, B, or C. How should I deal with this? How should I get the classifier to notify me in some way that the person currently in the camera frame is not known, so that I can appropriately handle this?

Thanks!

Answer 1

My oppinion is that both your issues are going from here:

face_names = []
for encoding in camera_encodings:
    name = clf.predict([encoding])
    print(name)
    face_names.extend(name)

There is no tolerance treshold being set, so either it founds the proper image as the most similar first.. or then any other too.

So, if you are obliged to use that sklearn.. I've found that

clf = svm.SVC(gamma="scale", tol=0.001)

has parametr tol . Try to set it high or lower then default value (0.001).