Why is the training accuracy fluctuating?

Question

I'm working with a video classification of 5 classes and using TimeDistributed CNN model in Google Colab platform. The training dataset contains 80 videos containing 5 frames each. The validation dataset contains 20 videos containing 5 frames each. The batch size I used is 64. So, in total, I'm working with 100 videos. I compiled the model using Adam optimizer and categorical cross_entropy loss.

model = Sequential()

input_shape=(5, 128, 128, 3)

model.add(TimeDistributed(Conv2D(32, (3, 3), strides=(1, 1),
            activation='relu', padding='same'), input_shape=input_shape))
model.add(TimeDistributed(MaxPooling2D((2, 2))))
model.add(TimeDistributed(Conv2D(64, (3, 3), strides=(1, 1),
            activation='relu', padding='same')))
model.add(TimeDistributed(Conv2D(128, (3, 3), strides=(1, 1),
            activation='relu', padding='same')))
model.add(TimeDistributed(BatchNormalization()))
model.add(TimeDistributed(MaxPooling2D((2, 2))))
model.add(TimeDistributed(Flatten()))

model.add(GRU(64, return_sequences=False))

model.add(BatchNormalization())

model.add((Dense(128, activation='relu')))
model.add(Dense(5, activation='softmax'))

from tensorflow.keras.optimizers import Adam

model.compile(loss='categorical_crossentropy',
              optimizer=Adam(lr=0.0001),
              metrics=['accuracy'])

But, after fitting this model with the dataset, the training accuracy curve is fluctuating like this:

Can anyone help me out to understand the reason behind this fluctuation?

Answer 1

You can try one or two things to stabilize the training:

1. You can try different batch sizes of 4, 8, 16, 32, 64. You can generate different plots. Have a look at this link . It'll generate mini plots for each batch size.

2. You can also alter the learning rate. You can apply Learning Rate scheduler or Reduce LR on plateau by directly calling keras callbacks. Alternatively, there is Cyclic LR that try to finds out the optimal learning rate. paper Github