TensorFlow / Keras：如何从我的广义骰子损失函数中获得有意义的损失值？

Question

I am trying to perform semantic segmentation in TensorFlow 1.10's Keras API (using Python) with the generalized dice loss function:

def generalized_dice_loss(onehots_true, logits):
    smooth = tf.constant(1e-17)
    onehots_true, logits = mask(onehots_true, logits) # Not all of my pixels contain ground truth, and I filter those pixels out, which results in shape [num_gt_pixels, num_classes]-shaped labels and logits.
    probabilities = tf.nn.softmax(logits)
    weights = 1.0 / (tf.reduce_sum(onehots_true, axis=0)**2)
    weights = tf.clip_by_value(weights, 1e-17, 1.0 - 1e-7) # Is this the correct way of dealing with inf values (the results of zero divisions)?
    numerator = tf.reduce_sum(onehots_true * probabilities, axis=0)
    numerator = tf.reduce_sum(weights * numerator)

    denominator = tf.reduce_sum(onehots_true + probabilities, axis=0)
    denominator = tf.reduce_sum(weights * denominator)

    loss = 1.0 - 2.0 * (numerator + smooth) / (denominator + smooth)
    return loss

However, I am struggling to get any meaningful loss which isn't always 1. What am I doing wrong here?

After the initial weights (one for each class) are calculated, they contain many inf 's from zero divisions, as typically only a small subset of all classes is present in a sample image. Therefore, I clip the weights to the range [1e-17, 1-1e-17] (is this a good idea?), after which they look like this:

tf.Tensor(
[4.89021e-05 2.21410e-10 5.43187e-11 1.00000e+00 1.00000e+00 4.23855e-07
 5.87461e-09 3.13044e-09 2.95369e-07 1.00000e+00 1.00000e+00 2.22499e-05
 1.00000e+00 1.73611e-03 9.47212e-10 1.12608e-05 2.77563e-09 1.00926e-08
 7.74787e-10 1.00000e+00 1.34570e-07], shape=(21,), dtype=float32)

which seems fine to me, though they are pretty small. The numerators ( tf.reduce_sum(onehots_true * probabilities, axis=0) , prior to their weighting) look like this:

tf.Tensor(
[3.42069e+01 0.00000e+00 9.43506e+03 7.88478e+01 1.50554e-02 0.00000e+00
 1.22765e+01 4.36149e-01 1.75026e+02 0.00000e+00 2.33183e+02 1.81064e-01
 0.00000e+00 1.60128e+02 1.48867e+04 0.00000e+00 3.87697e+00 4.49753e+02
 5.87062e+01 0.00000e+00 0.00000e+00], shape=(21,), dtype=float32)
tf.Tensor(1.0, shape=(), dtype=float32)

which also looks reasonable, since they're basically the labels' respective sizes times the network's certainty about them (which is likely low in the beginning of training). The denominators ( tf.reduce_sum(onehots_true + probabilities, axis=0) , prior to weighting) also look fine:

tf.Tensor(
[ 14053.483   25004.557  250343.36    66548.234    6653.863    3470.502
   5318.3926 164206.19    19914.338    1951.0701   3559.3235   7248.4717
   5984.786    7902.9004 133984.66    41497.473   25010.273   22232.062
  26451.926   66250.39     6497.735 ], shape=(21,), dtype=float32)

These are large, but that is to be expected since the class probabilities of a pixel sum to 1, and therefore the sum of these denominators should more or less equal the amount of pixels with ground truth.

However, summing the numerators gives a very small sum (~0.001, though occasionally it's in a single digit range) while the denominator sums to very large values. This results in my final loss being exclusively 1, or something really close to that. How can I mitigate this effect and obtain stable gradients? I pretty much implemented the exact dice loss formula. What am I missing here?

Answer 1

Apparently I need to omit the weights, then I get a workable loss function. No idea why I can't use weights, and what it would add if I could. Follow-up question: https://stats.stackexchange.com/questions/414107/why-are-weights-being-used-in-generalized-dice-loss-and-why-cant-i