用于面部识别的opencv代码无法在visual c ++中正确预测

Question

This is my code for face recognition in videos. It runs without any error but it's prediction is wrong most of the time.I am using LBPH face recognizer to recognize the faces. I tried using haar cascades but it does not load. so i switched to LBHP.please help me to improve the prediction. I am using gray scale cropped images of size 500 x 500 (pixels) for training the cascade classifier.

#include <opencv2/core/core.hpp>
#include <opencv2/contrib/contrib.hpp
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/objdetect/objdetect.hpp>

#include <iostream>
#include <fstream>
#include <sstream>

using namespace cv;
using namespace std;

static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
    string error_message = "No valid input file was given, please check the given filename.";
    CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
    stringstream liness(line);
    getline(liness, path, separator);
    getline(liness, classlabel);
    if(!path.empty() && !classlabel.empty()) {
        images.push_back(imread(path, 0));
        labels.push_back(atoi(classlabel.c_str()));
    }
}
}

string g_listname_t[]= 
{
"ajay","Aasai","famiz"
};

int main(int argc, const char *argv[]) {
// Check for valid command line arguments, print usage
// if no arguments were given.
//if (argc != 4) {
//   cout << "usage: " << argv[0] << " </path/to/haar_cascade> </path/to/csv.ext> </path/to/device id>"<<endl;
//    cout << "\t </path/to/haar_cascade> -- Path to the Haar Cascade for face detection." << endl;
//    cout << "\t </path/to/csv.ext> -- Path to the CSV file with the face database." << endl;
//    cout << "\t <device id> -- The webcam device id to grab frames from." << endl;
//    exit(1);
//}
//// Get the path to your CSV:
//string fn_haar = string(argv[1]);
//string fn_csv = string(argv[2]);
//int deviceId = atoi(argv[3]);
//// Get the path to your CSV:
// please set the correct path based on your folder
string fn_haar = "lbpcascade_frontalface.xml";
string fn_csv = "reader.ext ";
int deviceId = 0;           // here is my webcam Id. 
// These vectors hold the images and corresponding labels:
vector<Mat> images;
vector<int> labels;
// Read in the data (fails if no valid input filename is given, but you'll get an error message):
try {
    read_csv(fn_csv, images, labels);
} catch (cv::Exception& e) {
    cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;
    // nothing more we can do
    exit(1);
}
// Get the height from the first image. We'll need this
// later in code to reshape the images to their original
// size AND we need to reshape incoming faces to this size:
int im_width = images[0].cols;
int im_height = images[0].rows;
// Create a FaceRecognizer and train it on the given images:
Ptr<FaceRecognizer> model = createLBPHFaceRecognizer();
model->train(images, labels);
cout<<("Facerecognizer created");
// That's it for learning the Face Recognition model. You now
// need to create the classifier for the task of Face Detection.
// We are going to use the haar cascade you have specified in the
// command line arguments:

CascadeClassifier lbp_cascade;

if ( ! lbp_cascade.load(fn_haar) )
{
cout<<("\nlbp cascade not loaded");
} 
else
{
    cout<<("\nlbp cascade loaded");
}

 // Get a handle to the Video device:
 VideoCapture cap(deviceId);
cout<<("\nvideo device is opened");
// Check if we can use this device at all:
if(!cap.isOpened()) {
    cerr << "Capture Device ID " << deviceId << "cannot be opened." << endl;
    return -1;
}


// Holds the current frame from the Video device:
Mat frame;
for(;;) {
    cap >> frame;

    // Clone the current frame:
    Mat original = frame.clone();
    cout<<("\nframe is cloned");
    // Convert the current frame to grayscale:
    Mat gray;
    //gray = imread("G:\Picture\003.jpg",0);
    cvtColor(original, gray, CV_BGR2GRAY);
    imshow("gray image", gray);
    // And display it:
     char key1 = (char) waitKey(50);


    // Find the faces in the frame:
    cout<<("\ncolor converted");
    vector< Rect_<int> > faces;
    cout<<("\ndetecting faces");
    lbp_cascade.detectMultiScale(gray, faces);
    // At this point you have the position of the faces in
    // faces. Now we'll get the faces, make a prediction and
    // annotate it in the video. Cool or what?
    cout<<("\nfaces detected\n");
    cout<<faces.size();
    for(int i = 0; i < faces.size(); i++) 
    {
        // Process face by face:
        cout<<("\nprocessing faces");
        Rect face_i = faces[i];
        // Crop the face from the image. So simple with OpenCV C++:
        Mat face = gray(face_i);
        // Resizing the face is necessary for Eigenfaces and Fisherfaces. You can easily
        // verify this, by reading through the face recognition tutorial coming with OpenCV.
        // Resizing IS NOT NEEDED for Local Binary Patterns Histograms, so preparing the
        // input data really depends on the algorithm used.
        //
        // I strongly encourage you to play around with the algorithms. See which work best
        // in your scenario, LBPH should always be a contender for robust face recognition.
        //
        // Since I am showing the Fisherfaces algorithm here, I also show how to resize the
        // face you have just found:
        /*Mat face_resized;
        cv::resize(face, face_resized, Size(im_width, im_height), 1.0, 1.0, INTER_CUBIC);
        // Now perform the prediction, see how easy that is:
        cout<<("\nface resized");
        imshow("resized face image", face_resized);*/
        int prediction = model->predict(face);
        cout<<("\nface predicted");
        // And finally write all we've found out to the original image!
        // First of all draw a green rectangle around the detected face:
        cout<<("\nnow writing to original");
        rectangle(original, face_i, CV_RGB(0, 255,0), 1);
        // Create the text we will annotate the box with:
        string box_text;
        box_text = format( "Prediction =",prediction);
        // Get stringname
        if ( prediction >= 0 && prediction <=1 )
        {
            box_text.append( g_listname_t[prediction] );
        }
        else box_text.append( "Unknown" );
        // Calculate the position for annotated text (make sure we don't
        // put illegal values in there):
        int pos_x = std::max(face_i.tl().x - 10, 0);
        int pos_y = std::max(face_i.tl().y - 10, 0);
        // And now put it into the image:
        putText(original, box_text, Point(pos_x, pos_y), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0,255,0), 2.0);
    }
    // Show the result:
    imshow("face_recognizer", original);
    // And display it:
    char key = (char) waitKey(50);
    // Exit this loop on escape:
    if(key == 27)
        break;
}
return 0;
}

Answer 1

That is an expected result if you ask me, the code which you showed is the basic one to do recognition, there are some backdrops which we need to take care of before implementing.

1) the quality of training images, how did you crop them ? do they contain any extra information apart from face, if you used haar classifier in our opencv data to crop faces, then, the images tend to contain extra information than the face, as the rectangles are a bit large in size when compared to face.

2) there might be a chance that, even the rotated faces might be trained, so, its tough to classify with the features of rotated faces.

3) how many images, you trained the recognizer with ?, it playes a crucial role.

Answer for the first question, is most likely to be out of opencv, we cant do much about it, as there is very less probability that, we ll find a face detector which is as good and as simple as haar detector, so, we could make this as an exemption, if we can adjust with an accuracy around 70 %.

the second problem could be solved with some preprocessing techniques on training and testing dataset. Like., aligning faces which are being rotated follow this link, very good suggestions for face alignment are being suggested.

How to align face images c++ opencv

the third problem is solved with good number of samples which is not a hard task to achieve, take care of alignment before training, so that correct features could be extracted to classify.

there might be other factors that can improve the accuracy which I might have missed.