Tensorflow v1 对象检测api mask_rcnn_inception_v2_coco 模型批量推理

Question

I tried to use mask_rcnn_inception_v2_coco model available in tensorflow object detection api for segmentation task. Here I wanted to run inference for a video.

First I did inference, one frame at a time(batch size =1). Then the performance (frames per seconds) was really low. In order to have better "fps" value I tried to change the code for support for batch inference.

When I run the code with batch_size >1 , I got following error.

I have used the same batch inference code with other models such as ssd_inception_v2_coco (they are object detection models), those ran without any issue.

Does this mean mask_rcnn_inception_v2_coco not support for batch processing? Or is this another issue?

box_ind is deprecated, use box_indices instead
Traceback (most recent call last):
  File "/home/omega/anaconda3/envs/tf_gpu_env/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/home/omega/anaconda3/envs/tf_gpu_env/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "/home/omega/anaconda3/envs/tf_gpu_env/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Can not squeeze dim[0], expected a dimension of 1, got 2
     [[{{node Squeeze_4}}]]
     [[detection_boxes/_333]]
  (1) Invalid argument: Can not squeeze dim[0], expected a dimension of 1, got 2
     [[{{node Squeeze_4}}]]
0 successful operations.
0 derived errors ignored.

Answer 1

I have met the same problem. I understand the reason but have no idea how to solve it for now.

As for the mask part, there is one more section in the default graph which is to reframe the image to its original shape.

But the provided function in tf.utils.op is only accepted single image for now.

You can see the code blew which accepted box_masks as a tensor of size [num_masks, mask_height, mask_width] without the batch.

def reframe_box_masks_to_image_masks(box_masks, boxes, image_height,
                                     image_width):
    """Transforms the box masks back to full image masks.

    Embeds masks in bounding boxes of larger masks whose shapes correspond to
    image shape.

    Args:
      box_masks: A tf.float32 tensor of size [num_masks, mask_height, mask_width].
      boxes: A tf.float32 tensor of size [num_masks, 4] containing the box
             corners. Row i contains [ymin, xmin, ymax, xmax] of the box
             corresponding to mask i. Note that the box corners are in
             normalized coordinates.
      image_height: Image height. The output mask will have the same height as
                    the image height.
      image_width: Image width. The output mask will have the same width as the
                   image width.

    Returns:
      A tf.float32 tensor of size [num_masks, image_height, image_width].
    """

    """# TODO(rathodv): Make this a public function."""

    def reframe_box_masks_to_image_masks_default():
        """The default function when there are more than 0 box masks."""

        def transform_boxes_relative_to_boxes(boxes, reference_boxes):
            boxes = tf.reshape(boxes, [-1, 2, 2])
            min_corner = tf.expand_dims(reference_boxes[:, 0:2], 1)
            max_corner = tf.expand_dims(reference_boxes[:, 2:4], 1)
            transformed_boxes = (boxes - min_corner) / (max_corner - min_corner)
            return tf.reshape(transformed_boxes, [-1, 4])

        box_masks_expanded = tf.expand_dims(box_masks, axis=3)
        num_boxes = tf.shape(box_masks_expanded)[0]
        unit_boxes = tf.concat(
            [tf.zeros([num_boxes, 2]), tf.ones([num_boxes, 2])], axis=1)
        reverse_boxes = transform_boxes_relative_to_boxes(unit_boxes, boxes)
        return tf.image.crop_and_resize(
            image=box_masks_expanded,
            boxes=reverse_boxes,
            box_ind=tf.range(num_boxes),
            crop_size=[image_height, image_width],
            extrapolation_value=0.0)

    image_masks = tf.cond(
        tf.shape(box_masks)[0] > 0,
        reframe_box_masks_to_image_masks_default,
        lambda: tf.zeros([0, image_height, image_width, 1], dtype=tf.float32))
    return tf.squeeze(image_masks, axis=3)