Pytorch LSTM每个时期从0精度开始

Question

I'm training a LSTM model for time series prediction and at each epoch my accuracy restarts from 0 as if I'm training for the first time .

I attach below the training method snippet:

def train(model, loader, epoch, mini_batch_size, sequence_size):
model.train()
correct = 0
padded_size = 0
size_input = mini_batch_size * sequence_size
for batch_idx, (inputs, labels, agreement_score) in enumerate(loader):


    if(inputs.size(0) == size_input):
        inputs = inputs.clone().reshape(mini_batch_size, sequence_size, inputs.size(1))
        labels = labels.clone().squeeze().reshape(mini_batch_size*sequence_size)
        agreement_score = agreement_score.clone().squeeze().reshape(mini_batch_size*sequence_size)
    else:
        padded_size = size_input - inputs.size(0)
        (inputs, labels, agreement_score) = padd_incomplete_sequences(inputs, labels, agreement_score, mini_batch_size, sequence_size)



    inputs, labels, agreement_score = Variable(inputs.cuda()), Variable(labels.cuda()), Variable(agreement_score.cuda())


    output = model(inputs)


    loss = criterion(output, labels)
    loss = loss * agreement_score
    loss = loss.mean()

    optimizer.zero_grad()
    loss.backward()

    optimizer.step()

    pred = output.data.max(1, keepdim = True)[1]
    correct += pred.eq(labels.data.view_as(pred)).cuda().sum()

    accuracy = 100. * correct / (len(loader.dataset) + padded_size)

    print("Train: Epoch: {}, [{}/{} ({:.0f}%)]\t loss: {:.6f}, Accuracy: {}/{} ({:.0f}%)".format(
        epoch,
        batch_idx * len(output),
        (len(loader.dataset) + padded_size),
        100. * batch_idx / (len(loader.dataset)+padded_size),
        loss.item(),
        correct,
        (len(loader.dataset) + padded_size),
        accuracy))

accuracy = 100. * correct / (len(loader.dataset) + padded_size)

train_accuracy.append(accuracy)
train_epochs.append(epoch)
train_loss.append(loss.item())

According to that my loop looks like:

for epoch in range(1, 10):
    train(audio_lstm_model, train_rnn_audio_loader, epoch, MINI_BATCH_SIZE, SEQUENCE_SIZE_AUDIO)
    evaluation(audio_lstm_model,validation_rnn_audio_loader, epoch, MINI_BATCH_SIZE, SEQUENCE_SIZE_AUDIO)

Consequently, my accuracy and loss restarts at every epoch:

Train: Epoch: 1, [0/1039079 (0%)]    loss: 0.921637, Accuracy: 0/1039079 (0%)
...
Train: Epoch: 1, [10368/1039079 (0%)]    loss: 0.523242, Accuracy: 206010/1039079 (19%)
Test set: loss: 151.4845, Accuracy: 88222/523315 (16%)
Train: Epoch: 2, [0/1039079 (0%)]    loss: 0.921497, Accuracy: 0/1039079 (0%)

If anyone has any clue about it, your help is welcomed! Have a nice day!

Answer 1

The problem turn out to be the fact that the sequence size was too small for the network in order to be able to make some predictions from it. So after increasing the sequence length by some orders of magnitude, I was able to improve my model after each epoch.