StableBaselines 创建 model 分段错误

Question

尝试在 CartPole-v1 OpenAI Gym 环境中创建 stable_baselines3 PPO model 时出现分段错误。

到目前为止，我已经尝试在 Python 3.10 和 Python 3.9 上运行一个简短的示例代码。 我在 Conda 环境中运行 python 脚本。 我所做的是使用 pip 安装 stable-baselines[extra]。我还使用 conda 安装了 openAI gym 库。

使用 Python 3.10，我在pthreading.py文件中遇到分段错误，并调用wait function 调用。 使用 Python 3.9 我在_IntegerInterval.check方法中的constraints.py文件中遇到不同的错误

这是示例代码：

import sys
import gym
from stable_baselines3 import PPO, A2C
from stable_baselines3.common.env_util import make_vec_env

import faulthandler

faulthandler.enable()

def main(): 
  print("Going to create model")
  env = gym.make("CartPole-v1")

  model = PPO("MlpPolicy", env, verbose=1)
  print("Model created")
  model.learn(total_timesteps=25000)
  model.save("ppo_cartpole")

if __name__ == '__main__':
    main()
    sys.settrace(None)

这是终端 output：

(py39) ilijas-mbp:Doom_DQN_GC ilijavuk$  cd /Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC ; /usr/bin/env /Users/ilijavuk/opt/anaconda3/envs/py39/bin/python /Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 50143 -- /Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py 
Going to create model
Backend MacOSX is interactive backend. Turning interactive mode on.
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Fatal Python error: Segmentation fault

Thread 0x0000700007a7b000 (most recent call first):
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/threading.py", line 316 in wait
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/threading.py", line 581 in wait
  File "/Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 261 in _on_run
  File "/Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
(py39) ilijas-mbp:Doom_DQN_GC ilijavuk$ 

所以看起来 model 创建在 DummyVecEnv 包装步骤的某处失败了

编辑：我设法用 gdb 运行测试脚本。这是我得到的当前跟踪：

Starting program: /Users/ilijavuk/opt/anaconda3/envs/py39/bin/python StableBaselinesTest.py
[New Thread 0x1603 of process 2803]
[New Thread 0x2003 of process 2803]
warning: unhandled dyld version (17)
Going to create model
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Model created
[New Thread 0x1807 of process 2803]
[New Thread 0x2103 of process 2803]
[New Thread 0x2203 of process 2803]
[New Thread 0x2303 of process 2803]
[New Thread 0x2403 of process 2803]
[New Thread 0x2503 of process 2803]
[New Thread 0x2603 of process 2803]
[New Thread 0x2703 of process 2803]
[New Thread 0x2803 of process 2803]
[New Thread 0x2903 of process 2803]
[New Thread 0x2a03 of process 2803]
[New Thread 0x2b03 of process 2803]
[New Thread 0x3e03 of process 2803]
[New Thread 0x3f03 of process 2803]

Thread 3 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x1807 of process 2803]
0x0000000000000000 in ?? ()
(gdb) backtrace
#0  0x0000000000000000 in ?? ()
#1  0x0000000103a48538 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb)

退出 gdb 后我也明白了。 但是，这似乎与model.learn调用有关，因为当我评论 model.learn 和 model.save out 时，此错误消失

(gdb) q
A debugging session is active

        Inferior 1 [process 2803] will be killed.

Quit anyway? (y or n) y
Fatal Python error: Segmentation fault

Thread 0x00007ff847bb84c0 (most recent call first):
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 114 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/container.py", line 204 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/torch_layers.py", line 263 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/policies.py", line 627 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 166 in collect_rollouts
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 248 in learn
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/ppo/ppo.py", line 307 in learn
  File "/Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py", line 16 in main
  File "/Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py", line 20 in <module>

Answer 1

尝试恢复到 Python 的早期版本（例如 3.7.0）