[英]Tf-Agents ParallelPyEnvironment fails silently
我已经编写了一个自定义环境,因此我可以玩增强学习(PPO)和TF代理。 如果我将env(继承自py_environment.PyEnvironment)包装在TfPyEnvironment
,则此方法TfPyEnvironment
,但是如果我尝试将其包装到ParallelPyEnvironment
,则失败。 我尝试使用ParallelPyEnvironment
所有关键字参数,但是代码只运行到该行,然后什么也没发生-没有异常,该程序不会终止,等等。
这是我的代码初始化环境并展示eval_env
的工作变体:
train_env = tf_py_environment.TFPyEnvironment(
ParallelPyEnvironment(
[CardGameEnv()] * hparams['parallel_environments']
)
)
# this works perfectly:
eval_env = tf_py_environment.TFPyEnvironment(CardGameEnv(debug=True))
如果我通过CTRL+C
终止脚本,则正在输出:
Traceback (most recent call last):
Traceback (most recent call last):
File "E:\Users\tmp\Documents\Programming\Neural Nets\Poker_AI\poker_logic\train.py", line 229, in <module>
File "<string>", line 1, in <module>
train(model_num=3)
File "C:\Python37\lib\multiprocessing\spawn.py", line 105, in spawn_main
File "E:\Users\tmp\Documents\Programming\Neural Nets\Poker_AI\poker_logic\train.py", line 64, in train
[CardGameEnv()] * hparams['parallel_environments']
exitcode = _main(fd)
File "E:\Users\tmp\AppData\Roaming\Python\Python37\site-packages\gin\config.py", line 1009, in wrapper
File "C:\Python37\lib\multiprocessing\spawn.py", line 113, in _main
preparation_data = reduction.pickle.load(from_parent)
KeyboardInterrupt
return fn(*new_args, **new_kwargs)
File "C:\Python37\lib\site-packages\tf_agents\environments\parallel_py_environment.py", line 70, in __init__
self.start()
File "C:\Python37\lib\site-packages\tf_agents\environments\parallel_py_environment.py", line 83, in start
env.start(wait_to_start=self._start_serially)
File "C:\Python37\lib\site-packages\tf_agents\environments\parallel_py_environment.py", line 223, in start
self._process.start()
File "C:\Python37\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Python37\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Python37\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Python37\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
reduction.dump(process_obj, to_child)
File "C:\Python37\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Python37\lib\site-packages\tf_agents\environments\parallel_py_environment.py", line 264, in __getattr__
return self._receive()
File "C:\Python37\lib\site-packages\tf_agents\environments\parallel_py_environment.py", line 333, in _receive
message, payload = self._conn.recv()
File "C:\Python37\lib\multiprocessing\connection.py", line 250, in recv
buf = self._recv_bytes()
File "C:\Python37\lib\multiprocessing\connection.py", line 306, in _recv_bytes
[ov.event], False, INFINITE)
KeyboardInterrupt
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "C:\Python37\lib\site-packages\tf_agents\environments\parallel_py_environment.py", line 289, in close
self._process.join(5)
File "C:\Python37\lib\multiprocessing\process.py", line 139, in join
assert self._popen is not None, 'can only join a started process'
AssertionError: can only join a started process
由此得出的结论是, ParallelPyEnvironment
正在尝试启动的线程并不能做到这一点,但是由于我对Python线程的使用经验不是很丰富,所以我不知道从这里开始应该走什么路,特别是如何解决这个问题。 当前的培训需要很长时间,并且根本没有使用我的PC的功能(使用了3GB的32GB RAM,处理器使用了3%,GPU几乎无法工作,但VRAM已满),因此这将大大缩短培训时间。
解决方案是传递可调用对象,而不是环境,因此ParallelPyEnvironment
可以自己构造它们:
train_env = tf_py_environment.TFPyEnvironment(
ParallelPyEnvironment(
[CardGameEnv] * hparams['parallel_environments']
)
)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.