[英]Tensorflow Keras cross entropy loss with linear activation on output
In PyTorch the cross entropy loss function works something like在 PyTorch 中,交叉熵损失 function 类似于
CrossEntropyLoss(x, y) = H(one_hot(y), softmax(x))
CrossEntropyLoss(x, y) = H(one_hot(y), softmax(x))
so you can have a linear output layer.所以你可以有一个线性 output 层。 Is there a way to do that with tf.keras.Sequential?
有没有办法用 tf.keras.Sequential 做到这一点?
I have wirtten this little CNN for MNIST我为 MNIST 写了这个小 CNN
model = tf.keras.Sequential()
model.add(tfkl.Input(shape=(28, 28, 1)))
model.add(tfkl.Conv2D(32, (5, 5), padding="valid", activation=tf.nn.relu))
model.add(tfkl.MaxPool2D((2, 2)))
model.add(tfkl.Conv2D(64, (5, 5), padding="valid", activation=tf.nn.relu))
model.add(tfkl.MaxPool2D((2, 2)))
model.add(tfkl.Flatten())
model.add(tfkl.Dense(1024, activation=tf.nn.relu))
model.add(tfkl.Dense(10, activation=tf.nn.softmax))
model.compile(optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
model.summary()
model.fit(x_train, y_train, epochs=1)
and I would like to have我想拥有
model.add(tfkl.Dense(10))
as the last layer.作为最后一层。
I am trying to implement the ADef algorithm but the entries of the gradient wrt.我正在尝试实现 ADef 算法,但梯度 wrt 的条目。 the input seem to be too small and I guess with a linear output they would be right.
输入似乎太小了,我猜用线性 output 他们会是对的。
I know there is tf.nn.softmax_cross_entropy_with_logits but I don't know how to use it in this context.我知道有 tf.nn.softmax_cross_entropy_with_logits 但我不知道如何在这种情况下使用它。
Edit:编辑:
Changing改变
model.compile(optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
to至
model.compile(optimizer="adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=["accuracy"])
has done the trick.成功了。
Thank you @Moe1234.谢谢@Moe1234。 For the benefit of community providing solution here
为了社区的利益,在这里提供解决方案
Issue was resolved after changing更改后问题已解决
model.compile(optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
to至
model.compile(optimizer="adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=["accuracy"])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.