PyTorch：如何打印网络中每一层的输出Blob大小？

Question

I can print network structure like (also how to print positional index of each 'simple' layer? because in this example we have 3 for Fire module and it's content ('simple' layers) don't have index):

net = models.squeezenet1_1(pretrained=True)
print(net)



SqueezeNet(
  (features): Sequential(
    (0): Conv2d (3, 64, kernel_size=(3, 3), stride=(2, 2))
    (1): ReLU(inplace)
    (2): MaxPool2d(kernel_size=(3, 3), stride=(2, 2), dilation=(1, 1))
    (3): Fire(
      (squeeze): Conv2d (64, 16, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (16, 64, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (16, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (4): Fire(
      (squeeze): Conv2d (128, 16, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (16, 64, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (16, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (5): MaxPool2d(kernel_size=(3, 3), stride=(2, 2), dilation=(1, 1))
    (6): Fire(
      (squeeze): Conv2d (128, 32, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (32, 128, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (32, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (7): Fire(
      (squeeze): Conv2d (256, 32, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (32, 128, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (32, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (8): MaxPool2d(kernel_size=(3, 3), stride=(2, 2), dilation=(1, 1))
    (9): Fire(
      (squeeze): Conv2d (256, 48, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (48, 192, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (48, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (10): Fire(
      (squeeze): Conv2d (384, 48, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (48, 192, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (48, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (11): Fire(
      (squeeze): Conv2d (384, 64, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (64, 256, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (64, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
    (12): Fire(
      (squeeze): Conv2d (512, 64, kernel_size=(1, 1), stride=(1, 1))
      (squeeze_activation): ReLU(inplace)
      (expand1x1): Conv2d (64, 256, kernel_size=(1, 1), stride=(1, 1))
      (expand1x1_activation): ReLU(inplace)
      (expand3x3): Conv2d (64, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (expand3x3_activation): ReLU(inplace)
    )
  )
  (classifier): Sequential(
    (0): Dropout(p=0.5)
    (1): Conv2d (512, 1000, kernel_size=(1, 1), stride=(1, 1))
    (2): ReLU(inplace)
    (3): AvgPool2d(kernel_size=13, stride=1, padding=0, ceil_mode=False, count_include_pad=True)
  )
)

And I can print weights size like:

for i, weights in enumerate(list(net.parameters())):
    print('i:',i,'weights:',weights.size())



i: 0 weights: torch.Size([64, 3, 3, 3])
i: 1 weights: torch.Size([64])
i: 2 weights: torch.Size([16, 64, 1, 1])
i: 3 weights: torch.Size([16])
i: 4 weights: torch.Size([64, 16, 1, 1])
i: 5 weights: torch.Size([64])
i: 6 weights: torch.Size([64, 16, 3, 3])
i: 7 weights: torch.Size([64])
i: 8 weights: torch.Size([16, 128, 1, 1])
i: 9 weights: torch.Size([16])
i: 10 weights: torch.Size([64, 16, 1, 1])
i: 11 weights: torch.Size([64])
i: 12 weights: torch.Size([64, 16, 3, 3])
i: 13 weights: torch.Size([64])
i: 14 weights: torch.Size([32, 128, 1, 1])
i: 15 weights: torch.Size([32])
i: 16 weights: torch.Size([128, 32, 1, 1])
i: 17 weights: torch.Size([128])
i: 18 weights: torch.Size([128, 32, 3, 3])
i: 19 weights: torch.Size([128])
i: 20 weights: torch.Size([32, 256, 1, 1])
i: 21 weights: torch.Size([32])
i: 22 weights: torch.Size([128, 32, 1, 1])
i: 23 weights: torch.Size([128])
i: 24 weights: torch.Size([128, 32, 3, 3])
i: 25 weights: torch.Size([128])
i: 26 weights: torch.Size([48, 256, 1, 1])
i: 27 weights: torch.Size([48])
i: 28 weights: torch.Size([192, 48, 1, 1])
i: 29 weights: torch.Size([192])
i: 30 weights: torch.Size([192, 48, 3, 3])
i: 31 weights: torch.Size([192])
i: 32 weights: torch.Size([48, 384, 1, 1])
i: 33 weights: torch.Size([48])
i: 34 weights: torch.Size([192, 48, 1, 1])
i: 35 weights: torch.Size([192])
i: 36 weights: torch.Size([192, 48, 3, 3])
i: 37 weights: torch.Size([192])
i: 38 weights: torch.Size([64, 384, 1, 1])
i: 39 weights: torch.Size([64])
i: 40 weights: torch.Size([256, 64, 1, 1])
i: 41 weights: torch.Size([256])
i: 42 weights: torch.Size([256, 64, 3, 3])
i: 43 weights: torch.Size([256])
i: 44 weights: torch.Size([64, 512, 1, 1])
i: 45 weights: torch.Size([64])
i: 46 weights: torch.Size([256, 64, 1, 1])
i: 47 weights: torch.Size([256])
i: 48 weights: torch.Size([256, 64, 3, 3])
i: 49 weights: torch.Size([256])
i: 50 weights: torch.Size([1000, 512, 1, 1])
i: 51 weights: torch.Size([1000])

How to print output blob size of each layer in network?

Answer 1

You can register a hook (callback function) which will print out shapes of input and output tensors like described in the manual: Forward and Backward Function Hooks
Example:

net.register_forward_hook(your_print_blobs_function)

After this you need to do one forward pass against some input tensor.

expected_image_shape = (3, 224, 224)
input_tensor = torch.autograd.Variable(torch.rand(1, *expected_image_shape))
# this call will invoke all registered forward hooks
output_tensor = net(input_tensor)