too many arguments in a Pytorch custom nn module forward function

Question

I'm trying to make a neural network in pytorch that has a variable number of layers. My problem is that apparently I am passing some sort of iterable with more than one item to a linear layer which can only take one argument. I just can't see why.

So here is some code. First I created my own module and import it to my notebook later

import torch

class NNet(torch.nn.Module):
    def __init__(self, layer_shapes, activation_functions):
        super(NNet, self).__init__()
        assert len(layer_shapes) == len(activation_functions) + 1
        self.layer_shapes = layer_shapes
        self.activation_functions = activation_functions

        linear_functions = list()
        for i in range(len(self.layer_shapes)-1):
            linear_functions.append(torch.nn.Linear(
                    self.layer_shapes[i], self.layer_shapes[i+1]))

        self.linear_functions = linear_functions

    def parameters(self):
        parameters = list()
        for function in self.linear_functions:
            parameters = parameters+list(function.parameters())

        return parameters 

    def forward(self, x):
        assert x.shape[1] == self.layer_shapes[0]
        y = x
        for i in range(len(self.layer_shapes)-1):
            lin = self.linear_functions[i](y)
            y = self.activation_functions[i](lin)
        return y

In the notebook, the error is in the forward function at y = self.activation_functions[i](self.linear_functions[i](y))

Now I try to use the MNIST dataset supplied by torchvision and use my own module.

batch_size = 100
epochs = 500
learning_rate = 0.001




train_set = torchvision.datasets.MNIST(root = 
                                       '../../data',
                                        train=True,
                          transform=torchvision.transforms.ToTensor(),
                                        download=True)

test_set = torchvision.datasets.MNIST(root = 
                                      '../../data',
                                      train=False,                                      
                          transform=torchvision.transforms.ToTensor(),
                                   download=True)

train_loader = torch.utils.data.DataLoader(dataset=train_set, 
                                           batch_size=batch_size, 
                                           shuffle=True)

test_loader = torch.utils.data.DataLoader(dataset=test_set, 
                                          batch_size=batch_size, 
                                          shuffle=False)

model = nnet.NNet([784, 16, 10], [torch.nn.Tanh, 
                                  torch.nn.Softmax(dim=1)])

loss_function = torch.nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

loss_items = list()

for t in range(epochs):
    for i, (images, labels) in enumerate(train_loader):
        images = images.reshape(-1,28*28)

        outputs = model(images)
        loss = loss_function(outputs, labels)
        loss_items.append(loss.item())

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

This last for loop yields the error:

TypeError                      Traceback (most recent call last)
<ipython-input-6-4ccb4b105a41> in <module>()
      5         images = images.reshape(-1,28*28)
      6 
----> 7         outputs = model(images)
      8         loss = loss_function(outputs, labels)
      9         loss_items.append(loss.item())

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in 
__call__(self, *input, **kwargs)
    475             result = self._slow_forward(*input, **kwargs)
    476         else:
--> 477             result = self.forward(*input, **kwargs)
    478         for hook in self._forward_hooks.values():
    479             hook_result = hook(self, input, result)

~/Desktop/Archive/Computing/Projects/Python/ai/neural_network.py in 
forward(self, x)
     28         for i in range(len(self.layer_shapes)-1):
     29             lin = self.linear_functions[i](y)
---> 30             y = self.activation_functions[i](lin)
     31         return y
     32 

TypeError: __init__() takes 1 positional argument but 2 were given

I'm sure someone could tell me why this is happening, but could someone please give me a helpful strategy for debugging here? I'm new to pytorch and I doubt this will be the last trouble that I have. So a strategy for investigating these things would be helpful.

I would appreciate any input.

Answer 1

In the definition of model I forgot the parentheses on the torch.nn.Tanh class. It should be torch.nn.Tanh()

I keep thinking that these are functions and not classes. I still have some things to fix, but I'm glad I saw that. So frustrating. I found it by basically putting assert and print statements all over my code.

Answer 2

You might want to use the Sequential class

import torch.nn as nn

class NNet(nn.Module):
    def __init__(self, idim, hdim, odim, depth):
        super().__init__()
        layers = [nn.Linear(idim, hdim)]
        layers += [nn.Linear(hdim, hdim)
                   for i in range(depth)]
        layers += [nn.Linear(hdim, odim)]
        self.net = nn.Sequential(*layers)

    def forward(self, x):
        return self.net(x)

This takes care of the parameters etc too.