[英]Tensorflow gradient always gives None when using GradientTape
I was playing around and trying to implement my own loss function in TensorFlow but I always get None
gradients.我正在玩耍并试图在 TensorFlow 中实现我自己的损失 function 但我总是得到None
梯度。 To reproduce the problem I've now reduced my program to a minimal example.为了重现这个问题,我现在将我的程序简化为一个最小的例子。 I define a very simple model:我定义了一个非常简单的 model:
import tensorflow as tf
model = tf.keras.Sequential(
[
tf.keras.Input(shape=(3,), name="input"),
tf.keras.layers.Dense(64, activation="relu", name="layer2"),
tf.keras.layers.Dense(3, activation="softmax", name="output"),
]
)
and then define a very simple (but probably useless) loss function:然后定义一个非常简单(但可能没用)的损失 function:
def dummy_loss(x):
return tf.reduce_sum(x)
def train(model, inputs, learning_rate):
outputs = model(inputs)
with tf.GradientTape() as t:
current_loss = dummy_loss(outputs)
temp = t.gradient(current_loss, model.trainable_weights)
train(model, tf.random.normal((10, 3)), learning_rate=0.001)
but t.gradient(current_loss, model.trainable_weights)
gives me only a list of None
values, ie [None, None, None, None]
.但是t.gradient(current_loss, model.trainable_weights)
只给了我一个None
值的列表,即[None, None, None, None]
。 Why is this the case?为什么会这样? What am I doing wrong?我究竟做错了什么? Might there be a misconception on my side about how TensorFlow works?我对 TensorFlow 的工作原理可能有误解吗?
You need to run (ie forward pass) the computation graph or model within the context of GradientTape
so that all the operations in the model could be recorded:您需要在GradientTape
的上下文中运行(即前向传递)计算图或 model,以便可以记录 model 中的所有操作:
with tf.GradientTape() as t:
outputs = model(inputs) # This line should be within context manager
current_loss = dummy_loss(outputs)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.