使用带有OpenCV的haarcascade的手语，可能吗？

Question

I'm doing my project which need to detect/classify some simple sign language. I'm new to opencv, I have try to use contours,hull but it seem very hard to apply... I googled and find the method call "Haarcascade" which seem to be about taking pictures and create .xml file.

So, I decide to do Haarcascade......

Here are some example of the sign language that I want to detect/classify

Set1 : http://www.uppic.org/image-B600_533D7A09.jpg

Set2 : http://www.uppic.org/image-0161_533D7A09.jpg

The result I want here is to classify these 2 set. Any suggestion if I could use haarcascade method with this

*I'm using xcode with my webcam, but soon I'm gonna port them onto iOS device. Is it possible?

Answer 1

First of all: I would not use haar features for learning on whole images.

Let's see how haar features look like:

Let me point out how learning works. We're building a classifier that consists of many 'weak' classifiers. In approximation, every 'weak' classifier is built in such way to find out information about several haar features. To simplify, let's peek one of them to consideration, a first one from edge features. During learning in some way, we compute a threshold value by sliding this feature over the whole input training image, using feature as a mask: we sum pixels 'under' the white part of the feature, sum pixels 'under' black part and subtract one value from other. In our case, threshold value will give an information if vertical edge feature exists on the training image . After training of weak classifier, you repeat process with different haar features. Every weak classifier gives information about different features.

What is important: I summarized how training works to describe what kind of objects are good to be trained in such way. Let's pick the most powerful application - detecting human's face. There's an important feature of face:

1. It has a landmarks which are constrastive (they differ from background - skin)
2. The landmark's locations are correlated to each other in every face (eg distance between them in approximation is some factor of face size)

That makes haar features powerful in that case. As you can see, one can easily point out haar features which are useful for face detection eg first and second of line features are good for detection a nose.

Back to your problem, ask yourself if your problem have features 1. and 2. In case of whole image, there is too much unnecessary data - background, folds on person's shirt and we don't want to noise classifier with it.

Secondly, I would not use haar features from some cropped regions.

I think the difference between palms is too less for haar classifier. You can derive that from above description. The palms are not different so much - the computed threshold levels will be too similar. The most significant features for haar on given palms will be 'edges' between fingers and palm edges. You can;t rely on palm's edges - it depends from the background (walls, clothes etc.) And edges between fingers are carrying too less information. I am claiming that because I have an experience with learning haar classifier for palm. It started to work only if we cropped palm region containing fingers.