I am trying to adapt a Tensorflow r0.12 code (from https://github.com/ZZUTK/Face-Aging-CAAE ) to version 1.2.1 and I am having issues with optimizer.minimize().
In this case I am using GradientDescent, but the following error message is only slightly different (in terms of shapes provided) when I try with different optimizers:
ValueError: Shape must be rank 0 but is rank 1 for 'GradientDescent/update_E_con
v0/w/ApplyGradientDescent' (op: 'ApplyGradientDescent')
with input shapes: [5,5,1,64], [1], [5,5,1,64].
Where [5,5] is my kernels size, 1 is the number of initial channels and 64 is the number of filters in the first convolution. This is the convolutional encoder network it is referring to:
E_conv0: (100, 128, 128, 64)
E_conv1: (100, 64, 64, 128)
E_conv2: (100, 32, 32, 256)
E_conv3: (100, 16, 16, 512)
E_conv4: (100, 8, 8, 1024)
E_conv5: (100, 4, 4, 2048)
...
This is code that's triggering the error:
self.EG_optimizer = tf.train.GradientDescentOptimizer(
learning_rate=EG_learning_rate,
beta1=beta1
).minimize(
loss=self.loss_EG,
global_step=self.EG_global_step,
var_list=self.E_variables + self.G_variables
)
Where:
EG_learning_rate = tf.train.exponential_decay(
learning_rate=learning_rate,
global_step=self.EG_global_step,
decay_steps=size_data / self.size_batch * 2,
decay_rate=decay_rate,
staircase=True
)
self.EG_global_step = tf.get_variable(name='global_step',shape=1, initializer=tf.constant_initializer(0), trainable=False)
And
self.E_variables = [var for var in trainable_variables if 'E_' in var.name]
self.G_variables = [var for var in trainable_variables if 'G_' in var.name]
self.loss_EG = tf.reduce_mean(tf.abs(self.input_image - self.G))
After some debugging I now believe the problem comes from the minimize() method. The error seems to be attributed to the last parameter (var_list) but when I try to comment out the second or third parameter, the error remains the same and is just attributed to the first parameter (loss).
I have changed the code with respect to the one currently on GitHub to adapt it to the new version, so I worked a lot on tf.variable_scope(tf.get_variable_scope(), reuse=True). Could this be the cause?
Thank you so much in advance!
It's tricky to decode, since it comes from an internal op, but this error message points to the cause:
ValueError: Shape must be rank 0 but is rank 1 for 'GradientDescent/update_E_conv0/w/ApplyGradientDescent' (op: 'ApplyGradientDescent') with input shapes: [5,5,1,64], 1 , [5,5,1,64].
One of the inputs to the ApplyGradientDescent
op is a rank 1 tensor (ie a vector) when it should be a rank 0 tensor (ie a scalar). Looking at the definition of the ApplyGradientDescent
op , the only scalar input is alpha
, or the learning rate.
Therefore, it appears that the EG_learning_rate
tensor is a vector when it should be a scalar. A simple fix would be to "slice" a scalar from the EG_learning_rate
tensor when you construct the tf.train.GradientDescentOptimizer
:
scalar_learning_rate = EG_learning_rate[0]
self.EG_optimizer = tf.train.GradientDescentOptimizer(
learning_rate=scalar_learning_rate,
beta1=beta1
).minimize(
loss=self.loss_EG,
global_step=self.EG_global_step,
var_list=self.E_variables + self.G_variables
)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.