使用 opencv dnn 人脸检测器检测检测到的人脸图像中的人脸标志

Question

I trying to detect the 68 facial landmarks of human face. I detected the face using OpenCV dnn face detector as in https://www.pyimagesearch.com/2018/02/26/face-detection-with-opencv-and-deep-learning/ The face detection process done successfully, this is my code:

# import the necessary packages
import numpy as np
import argparse
import cv2
import dlib

ap = argparse.ArgumentParser()
ap.add_argument("-c", "--confidence", type=float, default=0.5,
                help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

# load our serialized model from disk
print("[INFO] loading model...")
net = cv2.dnn.readNetFromCaffe("D:\deep-learning-face-detection\deploy.prototxt.txt",
                               r"D:\deep-learning-face-detection\res10_300x300_ssd_iter_140000.caffemodel")

image = cv2.imread("image\path\jpg")
(h, w) = image.shape[:2]
blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 1.0,
                             (300, 300), (104.0, 177.0, 123.0))

print("[INFO] computing object detections...")
net.setInput(blob)
detections = net.forward()

# loop over the detections
for i in range(0, detections.shape[2]):
    # extract the confidence (i.e., probability) associated with the
    # prediction
    confidence = detections[0, 0, i, 2]

    # filter out weak detections by ensuring the `confidence` is
    # greater than the minimum confidence
    if confidence > args["confidence"]:
        # compute the (x, y)-coordinates of the bounding box for the
        # object
        box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
        (startX, startY, endX, endY) = box.astype("int")

        # draw the bounding box of the face along with the associated
        # probability
        text = "Face#{}".format(i)
        y = startY - 10 if startY - 10 > 10 else startY + 10
        cv2.rectangle(image, (startX, startY), (endX, endY),
                      (0, 0, 255), 2)
        cv2.putText(image, text, (startX, y),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 0, 255), 2)

    # show the output image
cv2.imshow("Output", image)
cv2.waitKey(0)

but when I trying to detect the facial landmarks inside the face, as the following:

predictor = dlib.shape_predictor("D:\shape_predictor_68_face_landmarks.dat")
shape = predictor(image, detections)
vec = []
for i in range(68):
    v = shape.part(i)
    vec.append(v)
print(vec)

I get the following error message

shape = predictor(image, detections) TypeError: call (): incompatible function arguments. The following argument types are supported: 1. (self: dlib.shape_predictor, image: array, box: dlib.rectangle) -> dlib.full_object_detection

Invoked with: , array([[[ 0, 0, 0], [ 0, 0, 0], [ 0, 0, 0], ...,

The error message appeared when I'm using OpenCV dnn face detector, and MTCNN fro dlib face detector, but it cannot be appeared with Haar cascade face detector and the facial landmarks detected successfully. I want to detect the facial landmarks within the OpenCV dnn face detector as the above code due to its accurate where the Haar cascade face detector don't benefit with my face image due to high occlusions consistence. Can anyone please help me.

Answer 1

这将解决问题：

shape = predictor(image,dlib.rectangle(startX, startY, endX, endY))

Answer 2

If you want something like this: example You can use face_recognition.face_landmarks(image) function from face recognition library , is much easier to use than the dnn detector. I hope I've helped.

Answer 3

As the reference shape_predictor , the input should be image and a single box. It seems you are putting more than one.

You can try:

1. Check if size of detections > 0 => if true go to step 2, else there is no face detected.
2. Try

shape = predictor(image, detections[0])

=> get landmarks of the first face

Answer 4

First of all note that for 68 facial landmarks :

https://github.com/davisking/dlib-models

Also note that this model file is designed for use with dlib's HOG face detector. That is, it expects the bounding boxes from the face detector to be aligned a certain way, the way dlib's HOG face detector does it. It won't work as well when used with a face detector that produces differently aligned boxes, such as the CNN based mmod_human_face_detector.dat face detector

For your purpose you need openface/nn4.small2.v1.t7 model instead of shape_predictor_68_face_landmarks.dat

My suggestion :

The drawback of OpenCV dnn face detector is that the input image must be resized to 300*300 so, for small faces the face landmark features diminish and it would be somehow useless to use face feature extractors for face recognition .

For robust and powerful face recognition use dlib's Hog or CNN (if you are in GPU-CUDA business the first one goes fast and the latter needs a poweful Nvidia ) then resize your input video or image not less than 640*480 ( for real-time recognition , preferably not more than that since it will devour your CPU or GPU beyond recognition :) then you can use one of those face features extractors like dlib . See also :

https://openface-api.readthedocs.io/en/latest/openface.html#openface-aligndlib-class