[英]Loguru error when using dask with a multiprocessing scheduler
I'm getting a RuntimeError
when trying to log from a dask.delayed
function using a logger handler with enqueue
enabled.尝试使用启用了
enqueue
的记录器处理程序从dask.delayed
function 记录时,我收到了RuntimeError
。
Here's a minimal reproducible example这是一个最小的可重现示例
import sys, dask
from loguru import logger
logger.add(sys.stderr, enqueue=True)
@dask.delayed
def log():
logger.info("Logging!")
dask.compute(*[log() for i in range(10)], scheduler="processes")
Which produces the following error:这会产生以下错误:
RuntimeError: SimpleQueue objects should only be shared between processes through inheritance
The solution is simple.解决方案很简单。 Import the logger from within your
delayed
function:从
delayed
的 function 中导入记录器:
In [1]: import sys, dask
...: from loguru import logger
...:
...: logger.add(sys.stderr, enqueue=True)
...:
...: @dask.delayed
...: def log():
...: from loguru import logger
...: logger.info("Logging!")
...:
...: dask.compute(*[log() for i in range(10)], scheduler="processes")
2022-09-13 21:42:36.001 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.001 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.001 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.001 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.002 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.002 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.004 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.004 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.004 | INFO | __main__:log:9 - Logging!
2022-09-13 21:42:36.004 | INFO | __main__:log:9 - Logging!
Out[1]: (None, None, None, None, None, None, None, None, None, None)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.