Changing the input shape of pre-trained models along with its weights

Question

I am trying to use transfer learning with a trained [3D] CNN where I have trained model along with its weights.

But the problem is it takes input size of (64,64,7,3) whereas I have an input size of (256,256,256,1) . How could I resolve this issue ?

Answer 1

Here is one solution: once it is reduced to (64,64,64), you could add a 1x1 conv2D layer at the beginning of your network. This 1x1 conv2D will result in a (64,64,n) where n is the number of units you want; in your case 3.

In Tensorflow, you would do it like this:

tf.layers.conv2D(inputs, 3, kernel_size=(1,1), activation=<your_activation>)

The benefits of this approach is that it will learn from the data the transformation to go from (64,64,64) to (64,64,3).

Having said that, I did not really get why you have an image of (256,256,256) or (64,64,64) in the first place. Normally, the last dimension is the number of channels which is often 3 like for 'RGB'.

Edit : In response to your edited question, there are several ways to get the dimensions that you want.

You could for instance apply the same logic as described above: add one or more conv3d layer(s) at the beginning of your network, and choose the proper kernel_size and stride to get the dimensions that you desired. Ex: If you add the following layer:

tf.layers.conv3d(inputs, 3, kernel_size=(193,193,250), strides=(1,1,1))

at the beginning of your network, you will get a Tensor of dim=(None,64,64,7,3) if the inputs is a Tensor of dim=(None,256,256,256,1). You could also use 3d (max/avg/etc) pooling layers.

And of course, You also could use tf.reshape , tf.image.resize_images , tf.transpose , tf.concat , tf.layers.conv2d , etc. to get the dimensions that you desire. Ex:

inputs = tf.placeholder(<tf.type>, shape=(None,256,256,256,1))
a = tf.reshape(inputs, (-1,256,256,256)) # shape=(None,256,256,256)
b = tf.image.resize_images(a, (64,64)) # shape=(None,64,64,256)
c = tf.layers.conv2d(b, 7, kernel_size=(1,1)) # shape=(None,64,64,7)
d = tf.reshape(c, (-1,64,64,7,1)) # shape=(None,64,64,7,1)
e = tf.layers.conv3d(d, 3, kernel_size=(1,1,1)) # shape=(None,64,64,7,3)

Anyway, there are many ways to get the dimensions you want, and depending on your problem you could prefer one approach over another.