目标评论家初始化不佳

Question

What is a good way to handle the first round of off policy training with Deep Deterministic Policy Gradients?

Here is my problem: I initialize all weights with Xavier Initialization and zeros for bias. However when computing the critic loss I'm getting an infinite MSE since the difference between Q_target and Q_eval is so large. Is it a bad idea to just clip this to a very large value?

Q_target_i = r_i + discount * Q_target(i+1)
critic_loss = MSE(Q_target_i, Q_eval_i)

Answer 1

我通过将评估网络初始化为与目标网络相同来解决此问题。