Number of neuron in CNN architecture

Question

I am using a certain CNN architecture, however, I am not sure how to calculate the exact number of neuron I have in it.

        self.conv1 = nn.Conv2d(in_channels=1, out_channels=16,
                               kernel_size=(7, 7), padding=(1, 1),
                               stride=(2, 2))

        self.conv2 = nn.Conv2d(in_channels=16, out_channels=32,
                               kernel_size=(7, 7), padding=(1, 1),
                               stride=(2, 2))

        self.conv3 = nn.Conv2d(in_channels=32, out_channels=64,
                               kernel_size=(3, 3), padding=(2, 2),
                               stride=(2, 2))

        self.fc1 = nn.Linear(64 * 1 * 1, 8)

There is also 2D-maxpooling after each convolution layer with stride of 2 .

I can get the number of parameters and Gmacs in my network, but I am not sure how to get the number of neurons?

Is there a certain way to calculate them?

Thanks.

Answer 1

One quick way to get the total count is to

1. Fetch all parameters with nn.Module.parameters ;
2. Convert the generator to a flattened tensor with torch.nn.utils.parameters_to_vector ;
3. Find the total number of elements with torch.Tensor.numel .

Which corresponds to:

>>> p2v(model.parameters()).numel()
44936

_{Having imported parameters_to_vector from torch.nn.utils as p2v}

---

If you want to count the parameters yourself:

• Convolutions when counting kernels and biases. Given number of input channels in_c , output channels out_c , and kernel size k :

   conv = lambda in_c, out_c, k: k*k*in_c*out_c + out_c

• Fully-connected layers: just a two-dimensional matrix with biases:

   fc = lambda in_c, out_c: in_c*out_c + out_c

• Max-pool layers are non-parametrized layers: 0 parameters.

All in all, this gives you:

>>> conv(1, 16, 7) + conv(16, 32, 7) + conv(32, 64, 3) + fc(64, 8)
44936

---

The word neurons is just an abstraction. If you consider it to be the output dimension for each given layer then:

• For convolution layers, it will depend on the spatial dimension of the input. So given the spatial dimension x , the kernel size k , the padding p , and the stride s :

   conv = lambda x, k, p, s: math.floor((x+2*p - k)/ s + 1)

• For fully connected layers, it corresponds to the number of output features.