The input image size in tf.gradients

Question

I'm trying to calculate the gradient at some layer with respect to the input image. The gradient is defined as

feature = g.get_tensor_by_name('inception/conv2d0_pre_relu:0')
gradient = tf.gradients(tf.reduce_max(feature, 3), x)

and my input image has a shape of (299,299), which is the size that inception is trained at

print(img.shape)
# output (299,299,3)

Then the gradient with respect to the input can be calculated as

img_4d=img[np.newaxis]
res = sess.run(gradient, feed_dict={x: img_4d})[0]
print(res.shape)
# output (1,299,299,3)

We see that the gradient has the same shape as the input image, which is expected.

However, it appears that one can use image with any size but still get the gradient. For example, if I have a img_resized with a shape (150,150,3), the gradient with respect to this input will also with a shape of (150,150,3):

img_resized=skimage.transform.resize(img, [150,150], preserve_range=True)
img_4d=img_resized[np.newaxis]
res = sess.run(gradient, feed_dict={x: img_4d})[0]
res.shape
# output (1,150,150,3)

So why does this work? In my naive picture, the dimension of the input image must be fixed at (299,299,3), and the gradient at some layer with respect to the input would always have the shape of (299,299,3). Why is it able to generate a gradient of other sizes?

In other words, what happens in the above code? When we feed an image with shape (150,150,3), does tensorflow resize the image to (299,299,3) and calculate the gradient with shape (299,299,3), and then resize the gradient back to (150,150,3)?

Answer 1

This is an expected phenomena esp. in the case of inception net which can work with any sized input owing to being fully convolutional network. Unlike Alexnet or VGG which rely on Fully Connected layer in later part of the network, Fully Convolutional networks can work on any sized input. Hope this answers your question.