[英]Keras Batchnormalization and sample weights
I am trying the the training and evaluation example on the tensorflow website . 我正在tensorflow 网站上尝试训练和评估示例。 Specifically, this part:
具体来说,这部分内容:
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') / 255
x_test = x_test.reshape(10000, 784).astype('float32') / 255
y_train = y_train.astype('float32')
y_test = y_test.astype('float32')
def get_uncompiled_model():
inputs = keras.Input(shape=(784,), name='digits')
x = layers.Dense(64, activation='relu', name='dense_1')(inputs)
x = layers.BatchNormalization()(x)
x = layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = layers.Dense(10, activation='softmax', name='predictions')(x)
model = keras.Model(inputs=inputs, outputs=outputs)
return model
def get_compiled_model():
model = get_uncompiled_model()
model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=1e-3),
loss='sparse_categorical_crossentropy',
metrics=['sparse_categorical_accuracy'])
return model
sample_weight = np.ones(shape=(len(y_train),))
sample_weight[y_train == 5] = 2.
# Create a Dataset that includes sample weights
# (3rd element in the return tuple).
train_dataset = tf.data.Dataset.from_tensor_slices(
(x_train, y_train, sample_weight))
# Shuffle and slice the dataset.
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(64)
model = get_compiled_model()
model.fit(train_dataset, epochs=3)
It appears that if I add the batch normalization layer (this line: x = layers.BatchNormalization()(x)
) I get the following error: 看来,如果添加批处理规范化层(此行:
x = layers.BatchNormalization()(x)
),则会出现以下错误:
InvalidArgumentError: The second input must be a scalar, but it has shape [64] [[{{node batch_normalization_2/cond/ReadVariableOp/Switch}}]]
Any ideas? 有任何想法吗?
The same code works for me. 相同的代码对我有用。
The only lines I changed are : 我更改的唯一行是:
model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=1e-3)
to model.compile(optimizer=keras.optimizers.RMSprop(lr=1e-3)
(which is version specific) model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=1e-3)
到model.compile(optimizer=keras.optimizers.RMSprop(lr=1e-3)
(特定于版本)
Then model.fit(train_dataset, epochs=3)
to model.fit(train_dataset, epochs=3, steps_per_epoch=30)
然后将
model.fit(train_dataset, epochs=3)
更改为model.fit(train_dataset, epochs=3, steps_per_epoch=30)
Reason : When using iterators as input to a model, you should specify the steps_per_epoch
argument 原因:使用迭代器作为模型的输入时,应指定
steps_per_epoch
参数
If you just want to use sample weights, you don't have to use tf.data.Dataset
, you can simply run: 如果只想使用样本权重,则不必使用
tf.data.Dataset
,只需运行:
model.fit(x=x_train, y=y_train, sample_weight=sample_weight, batch_size=64, epochs=3)
and it works for me (when I change learning_rate
to lr
as @ASHu2 mentioned). 并且它对我有用(当我将提及的@ ASHu2更改为
learning_rate
到lr
)。
It gets 97% accuracy after 3 epochs: 3个纪元后,它的准确性达到97%:
...
57408/60000 [===========================>..] - ETA: 0s - loss: 0.1010 - sparse_categorical_accuracy: 0.9709
58816/60000 [============================>.] - ETA: 0s - loss: 0.1011 - sparse_categorical_accuracy: 0.9708
60000/60000 [==============================] - 2s 37us/sample - loss: 0.1007 - sparse_categorical_accuracy: 0.9709
I used TF 1.14.0 on windows. 我在Windows上使用TF 1.14.0。
当我将Tensorflow从版本1.14.1更新到2.0.0-rc1时,此问题已解决。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.