Modifying loss function faster rcnn detectron

Question

For my thesis I am trying to modify the loss function of faster-rcnn with regards to recognizing table structures.

Currently I am using Facebooks Detectron. Seems to be working great but I am now actively trying to modify the loss function. Debugging my code I notice this is where the loss functions are added fast_rcnn_heads.py:75 :

def add_fast_rcnn_losses(model):
"""Add losses for RoI classification and bounding box regression."""
cls_prob, loss_cls = model.net.SoftmaxWithLoss(
    ['cls_score', 'labels_int32'], ['cls_prob', 'loss_cls'],
    scale=model.GetLossScale()
)
loss_bbox = model.net.SmoothL1Loss(
    [
        'bbox_pred', 'bbox_targets', 'bbox_inside_weights',
        'bbox_outside_weights'
    ],
    'loss_bbox',
    scale=model.GetLossScale()
)
loss_gradients = blob_utils.get_loss_gradients(model, [loss_cls, loss_bbox])
model.Accuracy(['cls_prob', 'labels_int32'], 'accuracy_cls')
model.AddLosses(['loss_cls', 'loss_bbox'])
model.AddMetrics('accuracy_cls')
return loss_gradients

The debugger cant find any declaration or implementation of mode.net.SmoothL1Loss nor SoftmaxWithLoss. Detectron uses caffe, and when I look in the net_builder (which inits the model.net) I see it makes "binds"(dont know the proper word) to caffe2, which on itself is a pylib with a compiled lib behind it.

Am I looking in the wrong place to make a minor adjustment to this loss function, or will I really have to open de source from dcaffe, adjust the loss, recompile the lib?

Greets,

Answer 1

You should implement loss function by yourself. Modifying library source code and recompile it - isn't very good idea :)

You can create python function, that will take GT and predicted data and return loss value.

Also you can create a duplicate of L1-smooth or Cross-entropy, which is currently used and then, when you will make sure, that they are the same, you can modify them. Or you can implement, for example, L2 loss for boxes and use it instead.

More information about custom losses you can find in caffee documentation.