CNN to first make feature key-points prediction then classify image on the base of these keypoints using Pytorch or TensorFlow

Question

I have already trained an Image Classifier using MobileNet in Pytorch to classify between close eyes image and open eyes image and also deployed it to mobile using Tensorflow

But problem is that the dataset is not big enough and it also doesn't work when the face object is far away or zoomed out.
I want to classify face with predefined key points like following

I want to make a CNN to first make feature key-points prediction and then classify image on the base of these keypoints.

Please guide me to any research paper or guide to predict feature keypoints using CNN and classify keypoints to two classes using Deep Learning techniques. The more Deep Learning used is better

I have already read about unsupervised machine learning but it is not working for me. I want to used deeplearning and pytorch or tensorflow

Answer 1

It seems to me you have enough data. The key is preprocessing. I'll suggest to use MTCNN (implementations: one , two , three ) for lightweight face and eyes detection, crop eyes and pass them through your net. Of course you should learn on cropped eyes (not whole images). You can get more precise eyes keypoints by libs like OpenPose , FAN or Seeta .