Looking for your valuable suggestions on my Loss /accuracy vs epoch curve

Question

My Keras-Tensorflow model is behaving like the below image. Where I can see training and validation loss is performing well, but the training and validation accuracy is quite abnormal. I think that the validation dataset might be very easy than the training set. Hence I am getting high validation accuracy. I am looking forward to your kind suggestions.

Answer 1

The plot that you displayed right here looks normal from the viewpoint of metrics and loss during a training.

It is common to see small spikes, since we are using batch_training. Also, when you see those spikes in loss(the loss increases), the accuracy also decreases.

Therefore, nothing to worry about the plot in itself.

However, your observation with regard to the validation accuracy is indeed sensible: most of the times, this happens due to the fact that the validation dataset is easier.

One way to deal with this issue is to use cross-validation, in order to see if this phenomenon still persists.

Cross-validation is a technique for model validation, in which, at each iteration/fold, there is a different part of your dataset reserved for training and validation. The picture below summaries what I have just written.

Another reason why this phenomenon takes place is due to the regularization technique called Dropout. As you might know, during the training phase, a dropout applied at a certain layer implies the random turn-off/deactivation of certain percentage of neurons. This in turn penalises the performance on the training set, but at the same time the risk of overfitting is mitigated. Therefore, many times when Dropout is used during the training, there may be the case that the validation accuracy is bigger, since during prediction on validation the Dropout is not enabled.