[英]Tensorflow: Sigmoid cross entropy loss does not force network outputs to be 0 or 1
I would like to learn image segmentation in TensorFlow with values in {0.0,1.0}. 我想学习TensorFlow中的图像分割,其值为{0.0,1.0}。 I have two images,
ground_truth
and prediction
and each have shape (120,160)
. 我有两个图像,
ground_truth
和prediction
,每个都有形状(120,160)
。 The ground_truth
image pixels only contain values that are either 0.0 or 1.0. ground_truth
图像像素仅包含0.0或1.0的值。
The prediction image is the output of a decoder and the last two layers of it are a tf.layers.conv2d_transpose
and tf.layers.conv2d
like so: 预测图像是解码器的输出,它的最后两层是
tf.layers.conv2d_transpose
和tf.layers.conv2d
如下所示:
transforms (?,120,160,30) -> (?,120,160,15)
outputs = tf.layers.conv2d_transpose(outputs, filters=15, kernel_size=1, strides=1, padding='same')
# ReLU
outputs = activation(outputs)
# transforms (?,120,160,15) -> (?,120,160,1)
outputs = tf.layers.conv2d(outputs, filters=1, kernel_size=1, strides=1, padding='same')
The last layer does not carry an activation function and thus it's output is unbounded. 最后一层不带有激活功能,因此它的输出是无界的。 I use the following loss function:
我使用以下损失函数:
logits = tf.reshape(predicted, [-1, predicted.get_shape()[1] * predicted.get_shape()[2]])
labels = tf.reshape(ground_truth, [-1, ground_truth.get_shape()[1] * ground_truth.get_shape()[2]])
loss = 0.5 * tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels=labels,logits=logits))
This setup converges nicely. 这种设置收敛得很好。 However, I have realized that the outputs of my last NN layer at validation time seem to be in [-inf, inf].
但是,我已经意识到我在验证时的最后一个NN层的输出似乎在[-inf,inf]中。 If I visualize the output I can see that the segmented object is not segmented since almost all pixels are "activated".
如果我可视化输出,我可以看到分割的对象没有被分割,因为几乎所有像素都被“激活”。 The distributions of values for a single output of the last conv2d layer looks like this:
最后一个conv2d层的单个输出的值分布如下所示:
Do I have to post-process the outputs (crop negative values or run output trough a sigmoid activation etc.)? 我是否必须对输出进行后处理(裁剪负值或通过S形激活等运行输出)? What do I need to do to enforce my output values to be {0,1}?
如何将输出值强制为{0,1},我需要做什么?
Solved it. 解决了它。 The problem was that the
tf.nn.sigmoid_cross_entropy_with_logits
runs the logits through a sigmoid which is of course not used at validation time since the loss operation is only called during train time. 问题是,
tf.nn.sigmoid_cross_entropy_with_logits
通过乙状结肠这当然是在验证时,未使用的,因为损失的操作过程中的列车时刻仅称为运行logits。 The solution therefore is: 因此解决方案是:
make sure to run the network outputs through a tf.nn.sigmoid
at validation/test time like this: 确保在验证/测试时通过
tf.nn.sigmoid
运行网络输出,如下所示:
return output if is_training else tf.nn.sigmoid(output)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.