[英]In tensorflow what is the difference between trainable and stop gradient
I would like to know the difference between the option trainable=False
and the tf.stop_gradient()
. 我想知道选项
tf.stop_gradient()
trainable=False
和tf.stop_gradient()
之间的区别。 If I make the trainable
option False
will my optimizer not consider the variable for training? 如果我使
trainable
选项False
我的优化器不会考虑变量进行训练吗? Does this option make the it a constant value throughout the training? 在整个培训过程中,此选项是否使其成为恒定值?
trainable=False
可训练=假
Here the variable value will be constant throughout the training. 这里变量值在整个训练过程中是恒定的。 Optimizer won't consider this variable for training, no gradient update op.
优化器不会将此变量视为训练,也不会考虑梯度更新操作。
stop_gradient
stop_gradient
In certain situations, you want to calculate the gradient of a op with respect to some variable keeping a few other variables constant; 在某些情况下,您希望根据某个变量计算op的梯度,同时保持一些其他变量不变; but for other ops you may use those variables also to calculate gradient.
但对于其他操作,您也可以使用这些变量来计算梯度。 So here you can't use
trinable=False
, as you need those variable for training with other ops. 所以在这里你不能使用
trinable=False
,因为你需要那些变量用于训练其他操作。
stop_gradient
is very useful for ops; stop_gradient
对于操作非常有用; you can selectively optimize a op with respect to select few variables while keeping other constant. 您可以选择性地优化操作,选择少数变量,同时保持其他常数。
y1 = tf.stop_gradient(W1x+b1)
y2 = W2y1+b2
cost = cost_function(y2, y)
# this following op wont optimize the cost with respect to W1 and b1
train_op_w2_b2 = tf.train.MomentumOptimizer(0.001, 0.9).minimize(cost)
W1 = tf.get_variable('w1', trainable=False)
y1 = W1x+b1
y2 = W2y1+b2
cost = cost_function(y2, y)
# this following op wont optimize the cost with respect to W1
train_op = tf.train.MomentumOptimizer(0.001, 0.9).minimize(cost)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.