to extract lip region and parameter from a face in matlab so as to be used for lip synch

Question

Hello everyone
i am doing my project in Matlab. For this i would be using a lip extraction algorithm which would extract the lip region from the given facial image. For that i have used color based segmentation by which i can obtain the lip region. This gives me the Lip region. However i am not getting the lip parameters. Hence i would have to use a standard lip for lip sync. Ie irrespective of the type of lip a sketch might have,we would have to use the same lips. This technique wouldnt give me an appropriate and accurate result.
What i want to ask is, "Is there any algorithm or method by which i can use the same lip of the sketch and appropriately modify the same lip for each and every viseme(smallest visual element) depending on the sound phoneme"
Thanking you all in advance

Answer 1

I wrote my master thesis on this subject. I used Active Shape Models to determine the exact 'shape' or position of the lips in order to reduce parameters.

http://www.vision.ee.ethz.ch/teaching/sada/sadalink/reports/biwi_00349.pdf

It works fairly well, but still, robustness is a huge problem. I wouldn't rely on color segmentation to extract the lip contours.. it seemed to work only for data sets with high resolution and ideal lighting conditions. Additionally, some people have very thin or skin-toned lips.

Anyway, have a look at the references in the report, they should be helpful.