我的卷积神经网络过度拟合

Question

Recently, I have built a simple convolutional neural network for hand gesture image recognition using background subtraction to make the hand a white shape on the screen with a black background. It was built using keras Conv2D for the most part. My dataset has 1000 pics for training and 100 pics for validation and testing. The problem oddly occurs immediately after the first epoch, during which the model's loss goes down a great deal. It usually goes down from some big number like 183 to 1 at the start of the second epoch. All the pics from the dataset are from my own hand using cv2, but I only conducted testing with my own hand, so that should not be any problem. In case the dataset was the problem, I have tried to take 3 different datasets, one using cv2's Canny method, which essentially traces a line of the hand and makes the rest of the pic black to see if that made a difference. Regardless, the same thing continued to happen. Furthermore, I have added multiple Dropout layers in different places to see the effect and the same thing always occurs in which the loss drastically decreases and it shows signs of overfitting. I have also implemented EarlyStopping and multiple layers to see if that helped, but the same results seems to always occurs.

model = Sequential()
model.add(Conv2D(32, (3,3), activation = 'relu',
    input_shape = (240, 215, 1)))
model.add(MaxPooling2D((2,2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3,3), activation = 'relu'))
model.add(Conv2D(64, (3,3), activation = 'relu'))
model.add(MaxPooling2D((2,2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, (3,3), activation = 'relu'))
model.add(MaxPooling2D((2,2)))
model.add(Dropout(0.25))
model.add(Conv2D(256, (3,3), activation = 'relu'))
model.add(MaxPooling2D((2,2)))
model.add(Dropout(0.25))
 #model.add(Conv2D(256, (3,3), activation = 'relu'))
 #model.add(MaxPooling2D((2,2)))
 #model.add(Conv2D(128, (3,3), activation = 'relu'))
 #model.add(MaxPooling2D((2,2)))
 #model.add(Conv2D(64, (3,3), activation = 'relu'))
 #model.add(MaxPooling2D((2,2)))
model.add(Flatten())
model.add(Dense(150, activation = 'relu'))
 #model.add(Dropout(0.25))
 #model.add(Dense(1000, activation = 'relu'))
model.add(Dropout(0.75))
model.add(Dense(6, activation = 'softmax'))
model.summary()
model.compile(optimizer = 'adam', loss = 'categorical_crossentropy',
        metrics = ['acc'])
callbacks_list = [EarlyStopping(monitor = 'val_loss', patience = 10),
        ModelCheckpoint(filepath = 'model.h6', monitor = 'val_loss',
        save_best_only = True),]

The commented sections of the code are changes I have tried to implement. I have also varied the Dropout values and positions of them a great deal and nothing significant has changed. Could anyone offer any advice on why my model overfits that quickly?

Answer 1

When dealing with such a massive overfitting phenomenon, a good starting point would be to reduce your number of layers.

Although you add a Dropout after many max-poolings, you still suffer from the overfitting phenomenon.

Here below I present some of my recommendations:

1. Ensure that you have a comprehensive dataset with clean labels. Regardless of how we might want to tune the neural network, if the dataset is not clean, we cannot obtain good results.
2. Add (for the beginning), maximum 3 stacks of convolution + max_pooling + dropout. (32 + 64 + 128) would be a good starting point.
3. Use GlobalAveragePooling2D instead of Dense layers. The latter are not needed in a Convolutional Neural Network, except for the last layer with sigmoid or softmax .
4. Try using SpatialDropout2D . As compared to typical Dropout , which is applied to each element in the feature map, SpatialDropout drops entire feature maps.
5. Try to use Data Augmentation. In this way, you create more artificial examples and your network will be less prone to overfitting.
6. If none of these work, ensure that you use a pre-trained network and you apply transfer learning to your task at hand.

Answer 2

Yes, it is a clear case of overfitting. Here are my suggestions:

1. Try reducing the hidden layers
2. Increase the drop out to 0.5
3. Create more synthetic images or apply transformations on the raw images.