安装 keras model 时，文件“mtrand.pyx”中出现 Numpy 错误

Question

I am using:

• keras-rl2: 1.0.4
• tensorflow: 2.4.1
• numpy: 1.19.5
• gym 0.18.0

For the training of a DQN model for a reinforcement learning project.

My action space contains 60 dicrete values:

self.action_space = Discrete(60)

and I am getting this error after x steps:

1901/10000 [====>.........................] - ETA: 1:02 - reward: 6.1348Traceback (most recent call last):
  File "D:/GitHub/networth-simulation/rebalancing_simple_discrete.py", line 203, in <module>
    dqn.fit(env, nb_steps=5000, visualize=False, verbose=1)
  File "D:\GitHub\networth-simulation\venv\lib\site-packages\rl\core.py", line 169, in fit
    action = self.forward(observation)
  File "D:\GitHub\networth-simulation\venv\lib\site-packages\rl\agents\dqn.py", line 227, in forward
    action = self.policy.select_action(q_values=q_values)
  File "D:\GitHub\networth-simulation\venv\lib\site-packages\rl\policy.py", line 227, in select_action
    action = np.random.choice(range(nb_actions), p=probs)
  File "mtrand.pyx", line 928, in numpy.random.mtrand.RandomState.choice
ValueError: probabilities contain NaN

When I use a lower number of discrete actions (<10) it doesn't happen every time.

I found that fix but I don't understand how to apply it. I can't find any file "numpy/random/mtrand/mtrand.pyx"

Has anyone found a way to fix that error?

Answer 1

This error apparently pops up when the values of environment class attributes become too high (positively or negatively).

I solved the problem with limitations in my step() method.