为什么使用tensorflow的估计器高级API和原始API的mnist分类的交叉熵损失在规模上不同？

Question

I am reading some tensorflow example codes and I find the loss in CNN-using-estimatorAPI and the loss in raw CNN are really different in scale, but they are all the same loss function:

the former is loss_op = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits_train, labels=tf.cast(labels, dtype=tf.int32))) , which use the not-one-hot label.

the latter is loss_op =tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y)) , which use the one-hot vector label.

why the former loss is nearlly 0 ~ 2.39026, and the latter loss is much bigger, why is it?

Answer 1

我知道，这是因为变量初始值设定项有所不同，tf.layers。*的默认值不是tf.random_normal（），对于更大的损失，这是因为softmax_cross_entropy_with_logits中log（0）的内部处理机制认为较低的损失更准确，因为log（1e-5）=-11。