TensorFlow 和 PyTorch 中的 Conv2D 填充

Question

I am trying to convert TensorFlow model to PyTorch but having trouble with padding . My code for for relevant platforms are as follow:

TensorFlow

conv1 = tf.layers.conv2d(
                inputs=input_layer,
                filters=32,
                kernel_size=[5, 5],
                padding="same",
                activation=tf.nn.relu,
                name = "conv1")

PyTorch

conv1 = nn.Conv2d(1, 32, kernel_size=5, stride=1, padding=2)

I have few questions:

1. Are the above codes equivalent?
2. How many padding s are added on left/right/top/bottom if we use same padding in tensorflow ?
3. How many padding s are added on left/right/top/bottom if we use padding=2 in pytorch ?
4. If the above two code snippets are not equivalent then how can we make the same conv layer?

Thanks in advance.

Answer 1

To answer your questions:

The reason why Pytorch doesn't have padding = 'same' to quite simply put it is due to its dynamic computation graph in comparison to Tensorflow static graph.

1. Both the codes are not equivalent as different padding is used.

2. 'Same' padding tries to pad evenly on the left and right, but if the amount of columns to be added is odd, it will then add an extra column to the right.

3. 'Padding = 2' in Pytorch applies 2 implicit paddings on either side.

4. Pytorch 1.9 has added padding = 'same' for un-strided or stride = 1 convolutions. Which will work for your use case.

But for stride > 2 padding needs to be added manually.

Here is the good implementation to perform 'same' padding:-

https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/layers/padding.py#L28