使用 Pytorch 的基础 class “nn.Linear” class 实现简单单层 RNN 的困难

Question

While working on making a simple RNN using Pytorch nn.linear function. So firstly I initialized my weights as

self.W_x = nn.Linear(self.input_dim, self.hidden_dim, bias=True)
self.W_h = nn.Linear(self.hidden_dim, self.hidden_dim, bias=True)

Now in the main step where I am getting the result of the current state by using the previous state and the values of the weights using this code statement

h_t = np.tanh((inp * self.W_x) + (prev_h * self.W_h))

So here I am getting the python error as shown below

TypeError: mul(): argument 'other' (position 1) must be Tensor, not Linear

Can anyone help me with his regards...

Answer 1

Your W_x and W_h are not weights, but linear layers, which use a weight and bias (since bias=True ). They need to be called as a function.

Furthermore, you cannot use NumPy operations with PyTorch tensors, but if you convert your tensors to NumPy arrays you can't back propagate through them, since only PyTorch operations are tracked in the computational graph. There is no need for np.tanh anyway, as PyTorch has torch.tanh as well.

h_t = torch.tanh(self.W_x(inp) + self.W_h(prev_h))