向 Conv2D 层添加 padding='same' 参数破坏了模型

Question

I created this model

from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPooling2D, Input, Dense
from tensorflow.keras.layers import Reshape, Flatten
from tensorflow.keras import Model


def create_DeepCAPCHA(input_shape=(28,28,1),n_prediction=1,n_class=10,optimizer='adam',
                      show_summary=True):
    inputs = Input(input_shape)
    x = Conv2D(filters=32, kernel_size=3, activation='relu', padding='same')(inputs)
    x = MaxPooling2D(pool_size=2)(x)
    x = Conv2D(filters=48, kernel_size=3, activation='relu', padding='same')(x)
    x = MaxPooling2D(pool_size=2)(x)
    x = Conv2D(filters=64, kernel_size=3, activation='relu', padding='same')(x)
    x = MaxPooling2D(pool_size=2)(x)
    x = Flatten()(x)
    x = Dense(512, activation='relu')(x)
    x = Dense(units=n_prediction*n_class, activation='softmax')(x)
    outputs = Reshape((n_prediction,n_class))(x)
    model = Model(inputs, outputs)
    model.compile(optimizer=optimizer,
                loss='categorical_crossentropy',
                metrics= ['accuracy'])
    if show_summary:
        model.summary()
    return model

I tried the model on MNIST dataset

import tensorflow as tf
import numpy as np

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

inputs = x_train
outputs = tf.keras.utils.to_categorical(y_train, num_classes=10)
outputs = np.expand_dims(outputs,1)

model = create_DeepCAPCHA(input_shape=(28,28,1),n_prediction=1,n_class=10)
model.fit(inputs, outputs, epochs=10, validation_split=0.1)

but it failed to converge (stuck at 10% accuracy => same as random guessing). Yet when I remove the "padding='same'" argument from Conv2D layers, it works flawlessly:

def working_DeepCAPCHA(input_shape=(28,28,1),n_prediction=1,n_class=10,optimizer='adam',
                      show_summary=True):
    inputs = Input(input_shape)
    x = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs)
    x = MaxPooling2D(pool_size=2)(x)
    x = Conv2D(filters=48, kernel_size=3, activation='relu')(x)
    x = MaxPooling2D(pool_size=2)(x)
    x = Conv2D(filters=64, kernel_size=3, activation='relu')(x)
    x = MaxPooling2D(pool_size=2)(x)
    x = Flatten()(x)
    x = Dense(512, activation='relu')(x)
    x = Dense(units=n_prediction*n_class, activation='softmax')(x)
    outputs = Reshape((n_prediction,n_class))(x)
    model = Model(inputs, outputs)
    model.compile(optimizer=optimizer,
                loss='categorical_crossentropy',
                metrics= ['accuracy'])
    if show_summary:
        model.summary()
    return model

Anyone has any idea what problem this is?

Answer 1

Thank you for sharing, it was really interesting to me. So I wrote the code and tested several scenarios. Note that what I'm going to say is just my guest and I'm not sure about it.

My conclusion from those tests is that no padding or valid padding works because it produces (1, 1, 64) output shape for the last conv layer. But if you set the padding to same it will produce (3, 3, 64) and because the next layer is a big Dense layer, it will multiply the number of networks parameters by 9 (I expected to somehow result in overfitting) and it seems to make it much harder for the network to find the good values of the parameters. So I tried some different ways to reduce the output of last conv layer to (1, 1, 64) as below:

• using one more conv layer + maxpooling
• change the last maxpooling to pool_size of 4
• using stride of 2 for one of conv layers
• change the filters of last conv layer to 20

and they all worked well. even changing the dense units from 512 to 64 will help as well (note that even now you may get poor results with a little chance, because of bad initialization I guess).

Then I changed the shape of the last conv layer to (2, 2, 64) and the chance to get a good result (more than 90% accuracy) reduced (alot of time I've got 10% accuracy).

So it seems that a lot of parameters can confuse the model. But if you want to know why the network does not overfit, I have no answer for you.