[英]Printing to stdout in IPython parallel processes
I'm new to IPython and would like to print intermediate results to stdout while running IPython parallel cluster functions. 我是IPython的新手,想在运行IPython并行集群功能时将中间结果打印到stdout。 (I'm aware that with multiple processes, this might mangle the output, but that's fine--it's just for testing/debugging, and the processes I'd be running are long enough that such a collision is unlikely.) I checked the documentation for IPython but can't find an example where the parallelized function prints.
(我知道有多个进程,这可能会破坏输出,但这很好 - 它只是用于测试/调试,而我正在运行的进程足够长,以至于不太可能发生此类冲突。)我检查了IPython的文档,但找不到并行化函数打印的示例。 Basically, I'm looking for a way to redirect the print output of the subprocesses to the main stdout, the IPython equivalent of
基本上,我正在寻找一种方法将子进程的打印输出重定向到主stdout,IPython相当于
subprocess.Popen( ... , stdout=...)
Printing inside the process doesn't work: 在流程内打印不起作用:
rc = Client()
dview = rc()
def ff(x):
print(x)
return x**2
sync = dview.map_sync(ff,[1,2,3,4])
print('sync res=%s'%repr(sync))
async = dview.map_async(ff,[1,2,3,4])
print('async res=%s'%repr(async))
print(async.display_outputs())
returns 回报
sync res=[1, 4, 9, 16]
async res=[1, 4, 9, 16]
So the computation executes correctly, but the print statement in the function ff is never printed, not even when all the processes have returned. 因此计算正确执行,但函数ff中的print语句永远不会打印,即使返回所有进程也是如此。 What am I doing wrong?
我究竟做错了什么? How do I get "print" to work?
如何让“打印”工作?
It's actually more similar to subprocess.Popen( ... , stdout=PIPE)
than you seem to be expecting. 它实际上更像是
subprocess.Popen( ... , stdout=PIPE)
不是你想象的那样。 Just like the Popen
object has a stdout
attribute, which you can read to see the stdout of the subprocess, An AsyncResult has a stdout
attribute that contains the stdout captured from the engines. 就像
Popen
对象有一个stdout
属性,您可以读取该属性来查看子进程的stdout
,AsyncResult有一个stdout
属性,其中包含从引擎捕获的标准输出。 It does differ in that AsyncResult.stdout
is a list of strings , where each item in the list is the stdout of a single engine as a string. 它的不同之处在于
AsyncResult.stdout
是一个字符串列表 ,其中列表中的每个项目都是单个引擎的标准输出作为字符串。
So, to start out: 所以,开始:
rc = parallel.Client()
dview = rc[:]
def ff(x):
print(x)
return x**2
sync = dview.map_sync(ff,[1,2,3,4])
print('sync res=%r' % sync)
async = dview.map_async(ff,[1,2,3,4])
print('async res=%r' % async)
async.get()
gives 给
sync res=[1, 4, 9, 16]
async res=<AsyncMapResult: ff>
We can see the AsyncResult.stdout
list of strings: 我们可以看到
AsyncResult.stdout
字符串列表:
print(async.stdout)
['1\n2\n', '3\n4\n']
We can see the stdout of the async result: 我们可以看到异步结果的标准输出:
print('async output:')
async.display_outputs()
which prints: 打印:
async output:
[stdout:0]
1
2
[stdout:1]
3
4
And here is a notebook with all of this demonstrated. 这是一个笔记本 ,所有这些都证明了这一点。
Some things to note, based on your question: 根据您的问题需要注意的一些事项:
async.get()
) async.get()
) display_outputs()
does not return anything - it actually does the printing/displaying itself, so print(async.display_outputs())
doesn't make sense. display_outputs()
不返回任何内容 - 它实际上是打印/显示本身,因此print(async.display_outputs())
没有意义。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.