tf.nn.conv2d_transpose 的填充有什么作用？

Question

As we know, we can calculate the shape of output tensor by padding mode for conv2d , and the algorithm is clear, but I'm very confused about conv2d_transpose , does it pad the input tensor and then invoke conv2d ? And where does it transpose filter or input? How to calculate the shape of output tensor according to the padding mode, SAME or VALID for conv2d_transpose ?

Answer 1

What does Padding mean for conv2d_transpose?

'SAME' means simply multiply the input shape by the strides. For example, if the input shape has a height and width of 7 and the conv2d_transpose has padding=same and strides=3, then the output shape will have a height and width of 7x3 = 21.

'VALID' is nearly the same. Start with 'SAME', then check the kernel_size compared to the strides. If it is larger, then add that amount to the height and width. Why? Because as the kernel is moving across the image for convolution (by strikes amount at a time) the last kernel will hang over the image by the difference. Imagine the above example of a height and width of 7 as the input and this time the padding=valid, strides=3, and kernel=5. The output height and width will be 7x3 + (5-3).

In both cases, if the kernel is less than the strides you'll just get a whole lot of zeros in the output. Why? Consider what stride does...

For a given value of stride, the input image is increased by many times. A stride of 3 makes the input image 3 times wider and higher. The original values occupy every 3rd place and the rest is filled with zeros ! For padding=valid there is that extra we talked about earlier.

The kernel_size is the size of the kernel for convolution that goes over the image, and it is moved over the image by strides. So, if the kernel_size is 1 and the strides is 3 then your output is mostly zeros.

Example SAME

>>> conv2d_tr = tf.keras.layers.Conv2DTranspose(5,kernel_size=3,padding='same',strides=2)
>>> conv2d_tr(np.zeros([3,2,2,4],dtype=np.float32)).numpy().shape

(3, 4, 4, 5)

Example VALID (bigger kernel than strides)

>>> conv2d_tr = tf.keras.layers.Conv2DTranspose(5,kernel_size=3,padding='valid',strides=2)
>>> conv2d_tr(np.zeros([3,10,10,4],dtype=np.float32)).numpy().shape

(3, 21, 21, 5)

Example VALID (kernel equal or less than stride)

>>> conv2d_tr = tf.keras.layers.Conv2DTranspose(5,kernel_size=2,padding='valid',strides=2)
>>> conv2d_tr(np.zeros([3,2,2,4],dtype=np.float32)).numpy().shape

(3, 20, 20, 5)

Example ZEROS for small kernel to stride

>>> conv2d_tr = tf.keras.layers.Conv2DTranspose(1,kernel_size=1,padding='same',strides=2)
>>> conv2d_tr(np.ones([1,2,2,3],dtype=np.float32)).numpy().shape
(1, 4, 4, 1)

>>> conv2d_tr(np.ones([1,2,2,3],dtype=np.float32)).numpy()

array([[[[0.702],[0.   ],[0.702],[0.   ]],
        [[0.   ],[0.   ],[0.   ],[0.   ]],
        [[0.702],[0.   ],[0.702],[0.   ]],
        [[0.   ],[0.   ],[0.   ],[0.   ]]]], dtype=float32)