Validation data performing worse than training data in keras

Question

I am training a CNN on some text data. The sentences are padded and embedded and fed to a CNN. The model architecture is:

model = Sequential()
model.add(Embedding(max_features, embedding_dims, input_length=maxlen))
model.add(Conv1D(128, 5, activation='relu'))
model.add(GlobalMaxPooling1D())

model.add(Dense(50, activation = 'relu'))
model.add(BatchNormalization())

model.add(Dense(50, activation = 'relu'))
model.add(BatchNormalization())

model.add(Dense(25, activation = 'relu'))
#model.add(Dropout(0.2))
model.add(BatchNormalization())

model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

Any help would be appreciated.

Answer 1

You model is over-fitting so the best practice is:

• add layers and preferably that goes in the power of 2

instead of

  model.add(Dense(50, activation = 'relu'))

use

  model.add(Dense(64, activation = 'relu'))

and go with 512 128 64 32 16

• add some dropout layers preferably after two layers.
• train on bigger data.

Answer 2

You can try removing BatchNormalization and adding more Convolutional and Pooling Layer that may increase your accuracy.

You can also check out this -: https://forums.fast.ai/t/batch-normalization-with-a-large-batch-size-breaks-validation-accuracy/7940