Train train_shape_predictor.py file gives an error

Question

I am trying to train the shape predictor of Dlib by using http://dlib.net/train_shape_predictor.py.html

#!/usr/bin/python

import os
import sys
import glob

import dlib
from skimage import io

if len(sys.argv) != 2:
    print(
        "Give the path to the examples/faces directory as the argument to this "
        "program. For example, if you are in the python_examples folder then "
        "execute this program by running:\n"
        "    ./train_shape_predictor.py ../examples/faces")
    exit()
faces_folder = sys.argv[1]

options = dlib.shape_predictor_training_options()

options.oversampling_amount = 300

options.nu = 0.05
options.tree_depth = 2
options.be_verbose = True

training_xml_path = os.path.join(faces_folder, "training_with_face_landmarks.xml")
print(training_xml_path)
dlib.train_shape_predictor(training_xml_path, "predictor.dat", options)

print("\nTraining accuracy: {}".format(
    dlib.test_shape_predictor(training_xml_path, "predictor.dat")))

testing_xml_path = os.path.join(faces_folder, "testing_with_face_landmarks.xml")
print("Testing accuracy: {}".format(
    dlib.test_shape_predictor(testing_xml_path, "predictor.dat")))


predictor = dlib.shape_predictor("predictor.dat")
detector = dlib.get_frontal_face_detector()

print("Showing detections and predictions on the images in the faces folder...")
win = dlib.image_window()
for f in glob.glob(os.path.join(faces_folder, "*.jpg")):
    print("Processing file: {}".format(f))
    img = io.imread(f)

    win.clear_overlay()
    win.set_image(img)

    # Ask the detector to find the bounding boxes of each face. The 1 in the
    # second argument indicates that we should upsample the image 1 time. This
    # will make everything bigger and allow us to detect more faces.
    dets = detector(img, 1)
    print("Number of faces detected: {}".format(len(dets)))
    for k, d in enumerate(dets):
        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(
            k, d.left(), d.top(), d.right(), d.bottom()))
        # Get the landmarks/parts for the face in box d.
        shape = predictor(img, d)
        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),
                                                  shape.part(1)))
        # Draw the face landmarks on the screen.
        win.add_overlay(shape)

    win.add_overlay(dets)
    dlib.hit_enter_to_continue()

Output:

/home/msc/Face_Rec/abc/training_with_face_landmarks.xml
Training with cascade depth: 10
Training with tree depth: 2
Training with 500 trees per cascade level.
Training with nu: 0.05
Training with random seed: 
Training with oversampling amount: 300
Training with feature pool size: 400
Training with feature pool region padding: 0
Training with lambda_param: 0.1
Training with 20 split tests.
Traceback (most recent call last):
  File "train_shape_predictor.py", line 29, in <module>
    dlib.train_shape_predictor(training_xml_path, "predictor.dat", options)
RuntimeError: 

Error detected at line 248.
Error detected in file /tmp/pip_build_root/dlib/dlib/../dlib/image_processing/shape_predictor_trainer.h.
Error detected in function dlib::shape_predictor dlib::shape_predictor_trainer::train(const image_array&, const std::vector<std::vector<dlib::full_object_detection> >&) const [with image_array = dlib::array<dlib::array2d<unsigned char> >].

Failing expression was objects[i][j].num_parts() != 0.
     shape_predictor shape_predictor_trainer::train()
     You can't give objects that don't have any parts to the trainer.

Please someone help me to solve that error. Thanks in advance.

Answer 1

Firstly, lets clear what object and parts mean here.
Let's assume you are training a model to detect the landmarks of a face. So,

• object is the face and
• the parts are it's landmark positions.

So, what this error means is that you are committing one of the following two mistakes:
1. Labelling (or annotating) a dataset with just objects without it's parts or,
2. You are labelling the parts but as boxes.

If it's the former. Don't do it.
If it's the latter, your .xml file must look something like this: feature points annotated as boxes

  <image file='/path/to/image/directory/<filename>.jpg'>
    <box top='145' left='114' width='239' height='257'>

    <box top='178' left='157' width='1' height='1'/>
    <box top='179' left='158' width='1' height='1'/>
    <box top='155' left='211' width='1' height='1'/>
    <box top='152' left='245' width='1' height='1'/>
    <box top='187' left='292' width='1' height='1'/>
    <box top='340' left='343' width='1' height='1'/>
  </image>

While it should look something like this: feature points annotated as parts of the object

  <image file='path/to/image/directory/<filename>.jpg'>
    <box top='185' left='114' width='238' height='261'>
      <label>Hand</label>
      <part name='13' x='192' y='200'/>
      <part name='20' x='219' y='195'/>
      <part name='27' x='277' y='225'/>
      <part name='34' x='345' y='382'/>
      <part name='38' x='192' y='430'/>
      <part name='6' x='123' y='235'/>
    </box>
  </image>

How do we get there?
1. In the command-line type

$ imglab ./<dataset_file>.xml --parts "<label1> <label2> <label3> <label4>"

For eg:

$ imglab ./data-set.xml --parts "head ears right_eye left_eye"

2. When the imglab window opens:

a) Annotate the box around the object of interest

b) Double click the box and right click the location where you wish to the denote the feature point. You'll be prompted by a popup menu with all the possible labels.

3. For more info on imglab, you can go to: About->Help on the imglab panel. Or, imglab -h

Hope this helps. Good Luck mate!