How to obtain a “mugshot” from face detection squares?

Question

I'm building an application that will take an image of a single person's whole body and will produce a "mugshot" for that person.

Mugshot meaning an image of the person's whole face, neck, hair and ears at the same general size of another mugshot.

Currently I'm using
http://askernest.com/archive/2008/05/03/face-detection-in-c.aspx
to implement OpenCV and I'm using

harrcascade_frontalface_default.xml  
harrcascade_frontalface_alt.xml  
harrcascade_frontalface_alt2.xml  
harrcascade_frontalface_alt_tree.xml

as my cascades.

I use all of the cascades because a single one will not detect all my faces. After I get all of the faces detected by all of the cascades I find my average square and use that for my final guess of how tall and wide the mugshot should be.

My problem is 3 parts.

• My current process is rather slow. How can I speed up the detection process?
  I'm finding that the processing time is directly related to photo size. 我发现处理时间与照片尺寸直接相关。 Reducing the size of the photos may prove to be helpful.

• A single cascade will not detect all the faces I come across so I'm using all of them. This of course produces many varied squares and a few false positives. What method can I use to identify false positives and leave them out of the average square calculation? ex.
  I'm implementing an average of values within standard deviation. 我正在实现标准偏差范围内的平均值。 Will post code soon.

• I'm not exactly sure of the best way find the mugshot given the square coordinates of the face. Where can I find face to mugshot ratios?
  Solved this one. 解决了这个问题。 Assuming all my heads are ratios of their faces.

   static public Rectangle GetMugshotRectangle(Rectangle rFace) { int y2, x2, w2, h2; //adjust as neccessary double heightRatio = 2; y2 = Convert.ToInt32(rFace.Y - rFace.Height * (heightRatio - 1.0) / 2.0); h2 = Convert.ToInt32(rFace.Height * heightRatio); //height to width ratio is 1.25 : 1 in mugshots w2 = Convert.ToInt32(h2 * 4 / 5); x2 = Convert.ToInt32((rFace.X + rFace.Width / 2) - w2 / 2); return new Rectangle(x2, y2, w2, h2); } 

  
  I just need to get rid of those false positives.

Ok make that 4 issues.

• Our camera that we will be using is currently out of commission so I don't have a method of capturing images at the moment. Where can I find full body images of people that isn't pure pron like google's image search for full body images?
  "Person standing" Makes a good search :) “人站立”做了一个很好的搜索:)

Answer 1

A single cascade could do what all of your cascades do if it is set up this way, plus it does not give you several results to judge from. The cascades you use are maybe different in the collection of teaching pictures they are made of or in some parameters.

A tutorial on how to build an own cascade can be found here . It would be useful to get the pictures used to train the four cascades you use but I don't know if they are publicly available.

Answer 2

I suggest you to use Upper-body Haar cascade file which will return you the rectangle till shoulder. Please find Head and shoulders cascade file at " http://alereimondo.no-ip.org/OpenCV/34 "

Answer 3

Ok, I figured it out but the project is on ice for the moment.
I do not have the source to paste as the VM takes forever to load up.
If someone is really interested, let me know and I'll post.
If you see something that looks like it could be done better let me know.

The steps I took were as follows.

1. Load the image, if greater than 500 pixels in either height or width make a new image of a version scaled down to a maximum of 500px height or width. Save the scale.
2. Run the C# OpenCV implementation at Ask Ernest on all of the frontal face Harrcasscades at Harrtraining .
3. For some reason the rectangles produced by Ask Ernest need to be scaled up 1.3X
4. With the generated rectangles use standard deviation to remove false positives. I did this for each corner of the rectangle finding the distance each rectangle corner was off from the average corner location. I made a note of any rectangle who had a corner out of whack and removed it from the list of candidate rectangles.
5. Get the average rectangle from the remaining rectangles and apply "GetMugshotRectangle" from above in the question.
6. Scale the mugshot rectangle back up using the scale we saved in step 1.
7. Cut out the mugshot from the original image and save to it's new location.

Done!