自定义Loss function in Keras（IoU损失函数）和梯度误差？

Question

i'm new to ML and i'm trying to implement my own loss function (an IoU loss function), but getting an error about the gradient ("No gradients provided for any variable"). Note that i'm trying to predict numbers that will represent a row number in a matrix for example(for example y_pred = [1 5 3 9]). In the loss function, i should count the number of the right predicted rows, and then divide it by the overall numbers of elements of the y_true. Since the loss should be minimized, i put 1-IoU at the end of the function.

Here is my function, to be more clear since i don't how to exacxtly explain my problem:

**def** loss_IoU(y_true, y_pred):

  intersection = []
  #roundig the predicted values y_pred, since the values could be floats ex. [1.5 2 8.98 ...]
  Roundy_pred = tf.round(y_pred)   # (none, 6) with shape of none, and the size of the outputs 6.
  intersection = tf.math.count_nonzero( Roundy_pred == y_true, axis=1, keepdims=True)   # (none, 1) count the number of equal values for each row (i.e. for each predicted y and a true y)
  union = y_true.shape[1] # which is 6 here
  iou = intersection/union
  returned_iou = 1 - iou
  return returned_iou

Here is the error i'm getting:

    ValueError                                Traceback (most recent call last)
<ipython-input-176-631f68b50b34> in <module>()
      9 history = model.fit(AS_Training_Set, Label_Training_Set, 
     10                     steps_per_epoch=8, epochs=600, validation_data=
---> 11           (AS_Validation_Set, Label_Validation_Set))

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    971           except Exception as e:  # pylint:disable=broad-except
    972             if hasattr(e, "ag_error_metadata"):
--> 973               raise e.ag_error_metadata.to_exception(e)
    974             else:
    975               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:796 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:789 run_step  **
        outputs = model.train_step(data)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:757 train_step
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:2737 _minimize
        trainable_variables))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:562 _aggregate_gradients
        filtered_grads_and_vars = _filter_grads(grads_and_vars)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:1271 _filter_grads
        ([v.name for _, v in grads_and_vars],))

    ValueError: No gradients provided for any variable: ['conv2d_16/kernel:0', 'conv2d_16/bias:0', 'conv2d_17/kernel:0', 'conv2d_17/bias:0', 'batch_normalization_8/gamma:0', 'batch_normalization_8/beta:0', 'conv2d_18/kernel:0', 'conv2d_18/bias:0', 'conv2d_19/kernel:0', 'conv2d_19/bias:0', 'batch_normalization_9/gamma:0', 'batch_normalization_9/beta:0', 'conv2d_20/kernel:0', 'conv2d_20/bias:0', 'conv2d_21/kernel:0', 'conv2d_21/bias:0', 'batch_normalization_10/gamma:0', 'batch_normalization_10/beta:0', 'conv2d_22/kernel:0', 'conv2d_22/bias:0', 'conv2d_23/kernel:0', 'conv2d_23/bias:0', 'batch_normalization_11/gamma:0', 'batch_normalization_11/beta:0', 'dense_4/kernel:0', 'dense_4/bias:0', 'dense_5/kernel:0', 'dense_5/bias:0'].

Can someone pleas help me with this error, i tried to fix it but without any results, i don't know if the problem is with the rounding function.

Thanks from now.

Answer 1

For IoU loss function, I am using this one for Pascal VOC dataset.

def IoU_loss(y_true, y_pred):
    nb_classes = K.int_shape(y_pred)[-1]
    iou = []
    pred_pixels = K.argmax(y_pred, axis=-1)
    for i in range(0, nb_classes):  # exclude first label (background) and last label (void)
        true_labels = K.equal(y_true[:, :, 0], i)
        pred_labels = K.equal(pred_pixels, i)
        inter = tf.cast(true_labels & pred_labels, dtype=tf.int32)
        union = tf.cast(true_labels | pred_labels, dtype=tf.int32)
        legal_batches = K.sum(tf.cast(true_labels, dtype=tf.int32), axis=1) > 0
        ious = K.sum(inter, axis=1) / K.sum(union, axis=1)
        iou.append(K.mean(ious[legal_batches]))

    iou = tf.stack(iou)
    legal_labels = ~tf.math.is_nan(iou)
    iou = iou[legal_labels]
    return K.mean(iou)

It needs a couple of changes but it will work for you also.