遍历pytorch LSTM

Question

I am training a seq2seq model for machine translation in pytorch. I would like to gather the cell state at every time step, while still having the flexibility of multiple layers and bidirectionality, that you can find in the LSTM module of pytorch, for example.

To this end, I have the following encoder and forward method, where I loop over the LSTM module. The problem is, that the model does not train very well. Right after the loop terminates, you can see the normal way to use the LSTM module and with that, the model trains.

So, is the loop not a valid way to do this?

class encoder(nn.Module):
    def __init__(self, input_dim, emb_dim, hid_dim, n_layers, dropout):
        super().__init__()

        self.input_dim = input_dim
        self.emb_dim = emb_dim
        self.hid_dim = hid_dim
        self.n_layers = n_layers
        self.dropout = dropout

        self.embedding = nn.Embedding(input_dim, emb_dim)

        self.rnn = nn.LSTM(emb_dim, hid_dim, n_layers, dropout = dropout)

        self.dropout = nn.Dropout(dropout)

    def forward(self, src):

        #src = [src sent len, batch size]

        embedded = self.dropout(self.embedding(src))

        #embedded = [src sent len, batch size, emb dim]
        hidden_all = []

        for i in range(len(embedded[:,1,1])):
            outputs, hidden = self.rnn(embedded[i,:,:].unsqueeze(0))
            hidden_all.append(hidden)



        #outputs, hidden = self.rnn(embedded)

        #outputs = [src sent len, batch size, hid dim * n directions]
        #hidden = [n layers * n directions, batch size, hid dim]
        #cell = [n layers * n directions, batch size, hid dim]
        None
        #outputs are always from the top hidden layer

        return hidden

Answer 1

好的，修复非常简单，您只需在外部运行第一个时间步，即可将隐藏的元组输入LSTM模块。