Keras Model无法使用Model API学习

Question

我正在运行以下代码来创建用于分类的keras模型，但由于所有层似乎都很好，我无法弄清楚出了什么问题。

使用的数据集：Cifar-10

宣传：Google colab

码：

# Dataset Props
input_shape = (32, 32, 3)
total_cats = 10

#Code for making model
inputs = Input(shape=(input_shape))

x1 = ZeroPadding2D(padding=(2, 2))(inputs)
x1 = Conv2D(64, kernel_size = (5, 5), strides=2, kernel_initializer = initializers.TruncatedNormal(mean=0, stddev=0.05, seed=None), name = 'conv_layer_1')(x1)
x1 = LeakyReLU(alpha=.001)(x1)

x2 = ZeroPadding2D(padding=(2, 2))(x1)
x2 = Conv2D(128, kernel_size = (5, 5), strides=2, kernel_initializer = initializers.TruncatedNormal(mean=0, stddev=0.05, seed=None),name = 'conv_layer_2')(x2)
x2 = LeakyReLU(alpha=.001)(x2)

x3 = ZeroPadding2D(padding=(2, 2))(x2)
x3 = Conv2D(256, kernel_size = (5, 5), strides=2, kernel_initializer = initializers.TruncatedNormal(mean=0, stddev=0.05, seed=None), name = 'conv_layer_3')(x3)
x3 = LeakyReLU(alpha=.001)(x3)


x4 = ZeroPadding2D(padding=(2, 2))(x3)
x4 = Conv2D(512, kernel_size = (5, 5), strides=2, kernel_initializer = initializers.TruncatedNormal(mean=0, stddev=0.05, seed=None), name = 'conv_layer_4')(x4)
x4 = LeakyReLU(alpha=.001)(x4)

x4_f = Flatten()(x4)

x5 = Dense(2048, name='dense_layer_1')(x4_f)
x5 = LeakyReLU(alpha=.001)(x5)

x6 = Dense(2048, name='dense_layer_2')(x5)
x6 = LeakyReLU(alpha=.001)(x6)

x8 = Dense(128, name='dense_layer_7')(x6)
x9 = Dense(total_cats, name='dense_layer_8', activation='softmax')(x8)

model = Model(inputs, output = x9)

model.compile(loss='categorical_crossentropy', optimizer = Adam(lr=0.001, decay=1e-6), metrics = ['accuracy'])

trained_model = model.fit(x_train[0:10000], y_train_hot[0:10000], epochs=5, batch_size=64)

注意：如果使用Sequential，我不会收到任何错误，并且使用相同的体系结构也可以。 以下是我获得上述代码的输出

Epoch 1/5
10000/10000 [==============================] - 11s 1ms/step - loss: 14.5506 - acc: 0.0972
Epoch 2/5
10000/10000 [==============================] - 10s 980us/step - loss: 14.5482 - acc: 0.0974
Epoch 3/5
10000/10000 [==============================] - 10s 980us/step - loss: 14.5482 - acc: 0.0974
Epoch 4/5
 1216/10000 [==>...........................] - ETA: 8s - loss: 14.8191 - acc: 0.080610000/10000 [==============================] - 10s 985us/step - loss: 14.5482 - acc: 0.0974
Epoch 5/5
10000/10000 [==============================] - 10s 983us/step - loss: 14.5482 - acc: 0.0974

Answer 1

复制评论作为答案：

• 在我看来，截断法线似乎不是一个很好的初始化方法...权重越多，模型的卡住就越少。 我将离开标准的初始化程序。

• 不过，您的模型似乎并未完全陷入困境，因为有时损失可能会有所不同。 它可能太快地推入泄漏的relu的负区域。 将alpha增加到0.01，也可能会帮助您看到更多可变的损失。 （除了做维卡斯建议的事情：降低学习率和培训以获取更多的时代机会）