MNIST的Keras VGG模型：训练和验证准确性之间的差异

Question

I have created the following model with Keras. The dataset is MNIST.

'''
    conv - relu - conv- relu - pool -
    conv - relu - conv- relu - pool -
    conv - relu - conv- relu - pool -
    affine - relu - dropout - affine - dropout - softmax
'''

model = Sequential()
model.add(Conv2D(16, kernel_size=(3, 3),
                 padding='same',
                 input_shape=input_shape)) 
model.add(Activation('relu'))
model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2))) 
model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(50, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Dropout(0.5))
model.add(Activation('softmax'))

The following is the result:

60000/60000 [==============================] - 10s - loss: 1.2707 - acc: 0.5059 - val_loss: 0.0881 - val_acc: 0.9785                    
Epoch 2/20                                                                                                                              
60000/60000 [==============================] - 9s - loss: 0.9694 - acc: 0.5787 - val_loss: 0.0449 - val_acc: 0.9873                                    

...        

Epoch 19/20                                                         
60000/60000 [==============================] - 9s - loss: 0.8530 - acc: 0.6004 - val_loss: 0.0282 - val_acc: 0.9937                     
Epoch 20/20                       
60000/60000 [==============================] - 9s - loss: 0.8564 - acc: 0.5982 - val_loss: 0.0383 - val_acc: 0.9910                     
Test loss: 0.0382921607383        
Test accuracy: 0.991                    

Why is the training accuracy so low, while the validation accururacy is so high?

Answer 1

The dropout on your last Dense layer removes half of your 10 neurons for your classes by random. Your last layer can only by accurate half of the times because in general half of the neurons are missing.

Try to remove that and I assume you get even values.