使用张量流训练对象检测模型时，错误索引 [0] = 0 不在 [0, 0) 中

Question

So I am currently attempting to train a custom object-detection model on tensorflow to recognize images of a raspberrypi2. Everything is already set up and running on my hardware,but due to limitations of my gpu I settled for the cloud. I have uploaded my data(train & test records ans csv-files) and my checkpoint model. That is what I get from the logs:

tensorflow:Restoring parameters from /mobilenet/model.ckpt

tensorflow:Starting Session.

tensorflow:Saving checkpoint to path training/model.ckpt

tensorflow:Starting Queues.

tensorflow:Error reported to Coordinator: <class tensorflow.python.framework.errors_impl.InvalidArgumentError'>, indices[0] = 0 is not in [0, 0)

I also have a folder called images with the actual .jpg files and it is also on the cloud, but for some reason I must specify every directory with a preceeding forward slash / and that might be a problem, as I currently do not know whether some of the files are trying to import these images ,but could not find the path because of the missing /. If any of you happens to share a solution I would be really thankful.

EDIT : I fixed it by downloading an older version of the models folder in tensorflow and the model started training, so note to the tf team.

Answer 1

Changing the way I created TF Records worked out for me. Have a look at the following code -

 example = tf.train.Example(
                            features= tf.train.Features(
                                feature={
                                    'image/height': dataset_util.int64_feature(height),
                                    'image/width': dataset_util.int64_feature(width),
                                    'image/filename': dataset_util.bytes_feature(filename_str.encode('utf-8')),
                                    'image/source_id': dataset_util.bytes_feature(filename_str.encode('utf-8')),
                                    'image/format': dataset_util.bytes_feature(image_format),
                                    'image/encoded': dataset_util.bytes_feature(image_data),
                                    'image/object/bbox/xmin': dataset_util.float_list_feature(xmin),
                                    'image/object/bbox/xmax': dataset_util.float_list_feature(xmax),
                                    'image/object/bbox/ymin': dataset_util.float_list_feature(ymin),
                                    'image/object/bbox/ymax': dataset_util.float_list_feature(ymax),
                                    'image/object/class/text': dataset_util.bytes_list_feature(labels_text),
                                    'image/object/class/label': dataset_util.int64_list_feature(labels),
                                }
                            )
                        )

Make sure that the TF Records have the same keys as the ones shown above. This is due to the fact that the model that you use would be expecting keys similar to the ones above. I hope this helps.

Earlier, I had made use of the following, which did not work out-

example = tf.train.Example(
                            features= tf.train.Features(
                                feature={
                                    'image/height': dataset_util.int64_feature(shape[0]),
                                    'image/width': dataset_util.int64_feature(shape[1]),
                                    'image/channels': dataset_util.int64_feature(shape[2]),
                                    'image/shape': dataset_util.int64_list_feature(shape),
                                    'image/object/bbox/xmin': dataset_util.float_list_feature(xmin),
                                    'image/object/bbox/xmax': dataset_util.float_list_feature(xmax),
                                    'image/object/bbox/ymin': dataset_util.float_list_feature(ymin),
                                    'image/object/bbox/ymax': dataset_util.float_list_feature(ymax),
                                    'image/object/bbox/class/label': dataset_util.int64_list_feature(labels),
                                    'image/object/bbox/class/text': dataset_util.bytes_list_feature(labels_text),
                                    'image/object/bbox/difficult': dataset_util.int64_list_feature(difficult),
                                    'image/object/bbox/truncated': dataset_util.int64_list_feature(truncated),
                                    'image/format': dataset_util.bytes_feature(image_format),
                                    'image/encoded': dataset_util.bytes_feature(image_data),
                                    'image/filename': dataset_util.bytes_feature(filename_str.encode('utf-8')),
                                    'image/source_id': dataset_util.bytes_feature(filename_str.encode('utf-8'))
                                }
                            )
                        )

As you observe, I had written image/object/bbox/class/label instead of image/object/class/label. I hope this helps.

A place where you can check this out is the following link - https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/using_your_own_dataset.md

Answer 2

I had the same issue using the centernet_mobilenetv2 model, but I just deleted the num_keypoints parameter in the pipeline.config file and then all was working fine. I don't know what is the problem with that parameter but I was able to run the training without it.