How to clip the gradient norm on the grad_and_var tuple in tensorflow-r1.0?

Question

After getting the grad_and_var tuple with compute_gradient :

opt = tf.train.RMSPropOptimizer(learning_rate)
grad_and_var = opt.compute_gradients(losses, params)

I'd like to clip the grad_and_var . But when I do:

clipped_gradients, _ = tf.clip_by_global_norm(grad_and_var, max_gradient_norm)

directly, the resulting clipped_gradients is a list of tensor, that means, the gradient and the variables has been concatenated.

If I do

clipped_gradients = [tf.clip_by_global_norm(x[0], max_gradient_norm)[0] for x in grad_and_var]

I got such an error:

TypeError: t_list should be a sequence

Do you have any idea that how can I fix it? Thanks a lot!

Answer 1

One possible approach that I have seen is to zip clipped_gradients and your variables and to use opt.apply_gradients on the zipped list, like in the code below (taken from here , lines 78-83):

tvars = tf.trainable_variables()
grads, _ = tf.clip_by_global_norm(tf.gradients(self.cost, tvars),
                args.grad_clip)
with tf.name_scope('optimizer'):
    optimizer = tf.train.AdamOptimizer(self.lr)
self.train_op = optimizer.apply_gradients(zip(grads, tvars))

Answer 2

you can clip norm and add gradient noise like this:

opt = tf.train.AdamOptimizer(learning_rate=self.config.learning_rate)
gvs = opt.compute_gradients(self.loss)
gvs = [(tf.clip_by_norm(grad,self.config.max_grad), val) for grad,val in gvs]
gvs = [(tf.add(grad, tf.random_normal(tf.shape(grad),stddev=self.config.grad_noise)), val) for grad,val in gvs]
self.train_op = opt.apply_gradients(gvs)