Representing bounding boxes with an angle using TensorFlow object detection

Question

The current version of Tensorflow object detection API supports the representation of bounding boxes without angle - represented by xmin , ymin , xmax , ymax .

I am looking for ideas to represent (and predict) bounding boxes with an angle/orientation.

Like this:

Answer 1

Use backpropagation to identify the pixel contributing the strongest to the activation and put a reasonable threshold to identify which pixels belong to the object.

The default algorithm does this then computes an axis-aligned bounding box of the selected pixels (because it's really simple). You need to run another bounding box algorithm that allows for arbitrary orientation. Wikipedia has some ideas ( link ).

For how to get the interesting pixels you can look inside the tensorflow code to figure it out.

Answer 2

Oriented bounding boxes is a very interesting topic and it has been kind of ignored by the deep learning based object detection approaches and it's hard to find datasets.

A recent paper/dataset/challenge wich I found very interesting (specially because they pay attention to oriented boxes) can be found here:

http://captain.whu.edu.cn/DOTAweb/index.html

They don't share the code (nor give much details in the paper) of their modification of Fater-RCNN to work with oriented bounding boxes but the dataset by itself and the representation discussion are quite usefull.