What is training accuracy and training loss and why we need to compute them?

Question

I am new to Lstm and machine learning and I am trying to understand some of it's concepts. Below is the code of my Lstm model.

Lstm model:

model = Sequential()
model.add(Embedding(vocab_size, 50, input_length=max_length-1))
model.add(LSTM(50))
model.add(Dropout(0.1))
model.add(Dense(vocab_size, activation='softmax'))

early_stopping = EarlyStopping(monitor='val_loss', patience=42)                                                                                                            
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(X, y, validation_split=0.2, epochs=500, verbose=2,batch_size = 20)

Below is a sample of my output:

And the train/test accuracy and train/test loss diagrams:

My undersanding (and please correct me if I am wrong) is that val_loss and val_accuracy is the loss and accuracy of the test data. My question is, what is the train accuracy and train loss and how these values are computed?. Thank you.

Answer 1

1. loss and val_loss-

In deep learning, the loss is the value that a neural.network is trying to minimize. That is how a neural.network learns by adjusting weights and biases in a manner that reduces the loss.

loss and val_loss differ because the former is applied to the train set , and the latter to the test set . As such, the latter is a good indication of how the model performs on unseen data.

2. accuracy and val_accuracy-

Once again, acc is on the training data , and val_acc is on the validation data . It's best to rely on val_acc for a fair representation of model performance because a good neural.network will end up fitting the training data at 100%, but would perform poorly on unseen data.

Training should be stopped when val_acc stops increasing, otherwise your model will probably overffit. You can use earlystopping callback to stop training.

3. Why do we need train accuracy and loss?

It's not a meaningful evaluation metric because a neural.network with sufficient parameters can essentially memorize the labels of training data and then perform no better than random guessing on previously unseen examples.

However, it can be useful to monitor the accuracy and loss at some fixed interval during training as it may indicate whether the backend is functioning as expected and if the training process needs to be stopped.

Refer here for a detailed explanation about earlystopping.

4. How accuracy and loss are calculated?

Loss and accuracy are calculated as you train, according to the loss and metrics specified in compiling the model. Before you train, you must compile your model to configure the learning process. This allows you to specify the optimizer , loss function , and metrics , which in turn are how the model fit function knows what loss function to use, what metrics to keep track of, etc.

The loss function (like binary cross entropy) documentation can be found here and the metrics (like accuracy ) documentation can be found here .