analyze the train-validation accuracy learning curve

Question

I am building a two-layer neural.network from scratch on the Fashion MNIST dataset. In between, using the RELU as activation and on the last layer, I am using softmax cross entropy. I am getting the below learning curve between train and validation accuracy which is wrong obviously. But if you see my loss curve, it's decreasing but my model is not learning. I am not able to my head around where I am going wrong. Could anyone explain these two graphs, like where I could be possibly going wrong?

Answer 1

I don't know exactly what you are doing, and I don't know anything about your architecture, but it's wrong to use ReLU on the last layer.

Usually you leave the last layer as linear (no activation). This will produce the logits that enter the Softmax . The output of the softmax will try to approximate the probability distribution on the classes.

This could be a reason for your results.