如何對超過1個class進行object檢測model訓練？

Question

鏈接： https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/eager_few_shot_od_training_tf2_colab.ipynb

我已經嘗試使用上面的 google colab 來訓練 object 檢測 model 和 1 class，如示例所示。

我試圖了解如何修改此代碼以便能夠訓練 2 個班級。

在上面的示例中，在我用框注釋圖像后，它運行以下代碼來創建category_index和圖像/框張量。 假設我修改num_classes = 2並將另一個 class 添加到category_index ，那么如何從這里開始？ 例如 - 我相信 one-hot 編碼僅適用於 1 類。 如何修改代碼以使其適用於 2 個類？

# By convention, our non-background classes start counting at 1.  Given
# that we will be predicting just one class, we will therefore assign it a
# `class id` of 1.
duck_class_id = 1
num_classes = 1

category_index = {duck_class_id: {'id': duck_class_id, 'name': 'rubber_ducky'}}

# Convert class labels to one-hot; convert everything to tensors.
# The `label_id_offset` here shifts all classes by a certain number of indices;
# we do this here so that the model receives one-hot labels where non-background
# classes start counting at the zeroth index.  This is ordinarily just handled
# automatically in our training binaries, but we need to reproduce it here.
label_id_offset = 1
train_image_tensors = []
gt_classes_one_hot_tensors = []
gt_box_tensors = []
for (train_image_np, gt_box_np) in zip(
    train_images_np, gt_boxes):
  train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(
      train_image_np, dtype=tf.float32), axis=0))
  gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32))
  zero_indexed_groundtruth_classes = tf.convert_to_tensor(
      np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32) - label_id_offset)
  gt_classes_one_hot_tensors.append(tf.one_hot(
      zero_indexed_groundtruth_classes, num_classes))
print('Done prepping data.')

Answer 1

為了mono-class detection檢測教程： Rubber Ducky detector或Zombie detector 。 將其更改為與multi-class一起使用，需要進行此類更改（兩周后解決方案） ：

• category_index變量必須如下所示。

gt_classes = [1,1,1,1,1,  2,2,2,2,2,2,2,2,  3,3,3,3,3,3,3,3]
# gt_classes = [[1],[1],[1],[1],[1], [2],[2],[2],[2],[2],[2],[2],[2], [3],[3],[3],[3],[3],[3],[3],[3,2]]
zombie_CLASS_ID = 1
cat_CLASS_ID = 2
dog_CLASS_ID = 3
category_index = {zombie_CLASS_ID :
                     {'id'  : zombie_CLASS_ID,'name': 'zombie'},
                cat_CLASS_ID :
                     {'id'  : cat_CLASS_ID,'name': 'cat'},
                dog_CLASS_ID :
                     {'id'  : dog_CLASS_ID,'name': 'dog'}
                  }
NUM_CLASSES = len(category_index)

• np.ones(shape=[gt_box_np.shape[0]], dtype=np.int32)是無意義的，作者發現將 grount true 類變量格式化為張量是一種非常尷尬的方式。 GT_classes條目的格式必須為Tensor("Const:0", shape=(1, NUM_CLASES), dtype=float32)和one_hot 編碼器（float32 很重要） 。
• 為此，必須同時替換為： tf.one_hot和tf.reshape 。 示例創建正確gt_classes_one_hot_tensors ：

label_id_offset = 1 #TF actually starts with 0 
train_image_tensors = []
gt_classes_one_hot_tensors = []
gt_box_tensors = []

for (train_image_np, gt_box_np, gt_class) in zip(list_train_images_np, gt_boxes, gt_classes):
    train_image_tensors.append(tf.expand_dims(tf.convert_to_tensor(train_image_np, dtype=tf.float32), axis=0))
    gt_box_tensors.append(tf.convert_to_tensor(gt_box_np, dtype=tf.float32))

    #HERE the most critical change in gt_classes , tf.reshape to keep format (1, NUM_CLASES) 
    gt_class_hot = tf.one_hot(indices=(gt_class - label_id_offset), depth= NUM_CLASES, dtype=tf.float32)
    gt_classes_one_hot_tensors.append( tf.reshape( gt_class_hot , [-1, NUM_CLASES])   )
print('Done prepping data  Num_loaded : ', len(list_train_images_np) )

如果你是從這些教程Rubber Ducky detector入手，推薦你閱讀： Does tensorflow's object detection api support multi-class multi-label detection?