類型錯誤：調用 tensorflow 優化器 `apply_gradients` 時第一個參數必須是可調用的

Question

我希望有人能幫我解決這個讓我發瘋好幾天的問題。 我正在構建一些受此keras 示例啟發的東西。 我正在嘗試手動計算 a.network 的梯度，但我無法弄清楚我做錯了什么。 這是 model 的定義

inputs = layers.Input(shape=(state_dim,))
layer1 = layers.Dense(l1_dim, activation="relu")(inputs)
ayer2 = layers.Dense(l2_dim, activation="relu")(layer1)
action = layers.Dense(num_actions, activation="softmax")(layer2)
critic = layers.Dense(1, activation=None)(layer2)

model = keras.Model(inputs=inputs, outputs=[critic, action])
# model.compile(optimizer=keras.optimizers.Adam(learning_rate=learning_rate))
optimizer = keras.optimizers.Adam(learning_rate=learning_rate)

然后我有我的訓練循環，給定集合（狀態，動作，獎勵，終端，state_）：

state = tf.convert_to_tensor([state], dtype=tf.float32)
state_ = tf.convert_to_tensor([state_], dtype=tf.float32)
reward = tf.convert_to_tensor(reward, dtype=tf.float32)  # not fed to NN
with tf.GradientTape(persistent=True) as tape:
     state_value, probs = model(state)
     state_value_, _ = model(state_)
     state_value = tf.squeeze(state_value)
     state_value_ = tf.squeeze(state_value_)

     action_probs = tfp.distributions.Categorical(probs=probs)
     log_prob = action_probs.log_prob(action)

     delta = reward + self.gamma * state_value_ * (1 - int(terminal)) - state_value
     actor_loss = -log_prob * delta
     critic_loss = delta ** 2
     total_loss = actor_loss + critic_loss

gradient = tape.gradient(total_loss, model.trainable_variables)
optimizer.apply_gradients(zip(gradient, model.trainable_variables))

但是在我的最后一行代碼中，調用optimizer.apply_gradients時出現以下錯誤：

Traceback (most recent call last):
  File "/Users/maccheroni/.virtualenvs/rl_gym/lib/python3.9/site-packages/keras/optimizer_v2/optimizer_v2.py", line 639, in apply_gradients
    self._create_all_weights(var_list)
  File "/Users/maccheroni/.virtualenvs/rl_gym/lib/python3.9/site-packages/keras/optimizer_v2/optimizer_v2.py", line 829, in _create_all_weights
    self._create_hypers()
  File "/Users/maccheroni/.virtualenvs/rl_gym/lib/python3.9/site-packages/keras/optimizer_v2/optimizer_v2.py", line 977, in _create_hypers
    self._hyper[name] = self.add_weight(
  File "/Users/maccheroni/.virtualenvs/rl_gym/lib/python3.9/site-packages/keras/optimizer_v2/optimizer_v2.py", line 1192, in add_weight
    variable = self._add_variable_with_custom_getter(
  File "/Users/maccheroni/.virtualenvs/rl_gym/lib/python3.9/site-packages/tensorflow/python/training/tracking/base.py", line 816, in _add_variable_with_custom_getter
    new_variable = getter(
  File "/Users/maccheroni/.virtualenvs/rl_gym/lib/python3.9/site-packages/keras/engine/base_layer_utils.py", line 106, in make_variable
    init_val = functools.partial(initializer, shape, dtype=dtype)
TypeError: the first argument must be callable

我真的不明白為什么，因為我已經閱讀了很多教程，遵循了很多示例，他們似乎都以這種方式使用這個 function。

Answer 1

我也有同樣的錯誤並找到了解決方案。 在我的例子中，優化器的初始化：

optimizer = keras.optimizers.Adam(learning_rate=learning_rate)

使用的變量learning_rate是None 。 用數字或簡單地初始化：

optimizer = keras.optimizers.Adam()

解決了這個問題。 在你的情況下，不清楚learning_rate是什么，但你可以檢查一下。