[英]Tensoflow Keras - Nan loss with sparse_categorical_crossentropy
I am new to tensorflow and deep learning.我是 tensorflow 和深度学习的新手。 I am trying to train a simple network and I get NaN loss on first epoch.
我正在尝试训练一个简单的网络,但在第一个时期我得到了 NaN 损失。 I inspected the weights and they had become nans too.
我检查了重量,它们也变成了 nans。 I tried to reduce the learning rate to 1e-8.
我试图将学习率降低到 1e-8。 Even that doesnt help.
即使那样也无济于事。 Please let me know what I am doing wrong.
请让我知道我做错了什么。
import tensorflow as tf
import numpy as np
a = tf.constant(
np.array([
[ 8, 51, 1, 30, 3, 30],
[ 1, 5, 2, 1, 1, 1],
[11, 29, 1, 1, 1, 1],
[ 1, 43, 1, 44, 27, 45],
[ 1, 1, 1, 1, 1, 19],
])
)
l = tf.constant(np.array([[2], [1], [1], [2], [3]]))
model = tf.keras.Sequential([
tf.keras.layers.Dense(3, activation='softmax', input_shape=[6])
])
optimizer = tf.keras.optimizers.Adam(lr=1e-8)
model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer)
print(model.summary())
history = model.fit(a,l, epochs=1, verbose=2)
The error is here:错误在这里:
l = tf.constant(np.array([[2], [1], [1], [2], [0]])) # change the last index
Your shape of l is not the right shape for categorical cross-entropy.您的 l 形状不是分类交叉熵的正确形状。 Also, your labels must range from 0 to 2 and not from 1 to 3.
此外,您的标签范围必须是 0 到 2,而不是 1 到 3。
import tensorflow as tf
import numpy as np
a = tf.constant(
np.array([
[ 8, 51, 1, 30, 3, 30],
[ 1, 5, 2, 1, 1, 1],
[11, 29, 1, 1, 1, 1],
[ 1, 43, 1, 44, 27, 45],
[ 1, 1, 1, 1, 1, 19],
])
)
l = tf.constant(np.array([1, 0, 0, 1, 2]))
model = tf.keras.Sequential([
tf.keras.layers.Dense(3, activation='softmax', input_shape=[6])
])
optimizer = tf.keras.optimizers.Adam(lr=1e-8)
model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer)
print(model.summary())
history = model.fit(a,l, epochs=3, verbose=2)
Model: "sequential_11"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_39 (Dense) (None, 3) 21
=================================================================
Total params: 21
Trainable params: 21
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/3
1/1 - 0s - loss: 0.2769
Epoch 2/3
1/1 - 0s - loss: 0.2769
Epoch 3/3
1/1 - 0s - loss: 0.2769
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.