[英]Multiple inputs and outputs in python subprocess communicate
I need to do something like this post , but I need to create a subprocess that can be given input and give output many times.我需要做类似这篇文章的事情,但我需要创建一个可以多次给出输入和输出的子流程。 The accepted answer of that post has good code...该帖子的接受答案有很好的代码......
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input=b'one\ntwo\nthree\nfour\nfive\nsix\n')[0]
print(grep_stdout.decode())
# four
# five
...that I would like to continue like this: ...我想继续这样:
grep_stdout2 = p.communicate(input=b'spam\neggs\nfrench fries\nbacon\nspam\nspam\n')[0]
print(grep_stdout2.decode())
# french fries
But alas, I get the following error:但是,唉,我收到以下错误:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 928, in communicate
raise ValueError("Cannot send input after starting communication")
ValueError: Cannot send input after starting communication
The proc.stdin.write() method not enable you to collect output, if I understand correctly.如果我理解正确的话,proc.stdin.write() 方法不能让你收集输出。 What is the simplest way to keep the lines open for ongoing input/output?为正在进行的输入/输出保持线路开放的最简单方法是什么?
Edit: ====================编辑:====================
It looks like pexpect
is a useful library for what I am trying to do, but I am having trouble getting it to work.看起来pexpect
对我正在尝试做的事情pexpect
是一个有用的库,但我无法让它工作。 Here is a more complete explanation of my actual task.这是对我的实际任务的更完整的解释。 I am using hfst
to get grammar analyses of individual (Russian) words.我正在使用hfst
来获取单个(俄语)单词的语法分析。 The following demonstrates its behavior in a bash shell:下面演示了它在 bash shell 中的行为:
$ hfst-lookup analyser-gt-desc.hfstol
> слово
слово слово+N+Neu+Inan+Sg+Acc 0.000000
слово слово+N+Neu+Inan+Sg+Nom 0.000000
> сработай
сработай сработать+V+Perf+IV+Imp+Sg2 0.000000
сработай сработать+V+Perf+TV+Imp+Sg2 0.000000
>
I want my script to be able to get the analyses of one form at a time.我希望我的脚本能够一次获得一种形式的分析。 I tried code like this, but it is not working.我试过这样的代码,但它不起作用。
import pexpect
analyzer = pexpect.spawnu('hfst-lookup analyser-gt-desc.hfstol')
for newWord in ['слово','сработай'] :
print('Trying', newWord, '...')
analyzer.expect('> ')
analyzer.sendline( newWord )
print(analyzer.before)
# trying слово ...
#
# trying сработай ...
# слово
# слово слово+N+Neu+Inan+Sg+Acc 0.000000
# слово слово+N+Neu+Inan+Sg+Nom 0.000000
#
#
I obviously have misunderstood what pexpect.before
does.我显然误解了pexpect.before
作用。 How can I get the output for each word, one at a time?如何获得每个单词的输出,一次一个?
Popen.communicate()
is a helper method that does a one-time write of data to stdin
and creates threads to pull data from stdout
and stderr
. Popen.communicate()
是一种辅助方法,它一次性将数据写入stdin
并创建线程以从stdout
和stderr
提取数据。 It closes stdin
when its done writing data and reads stdout
and stderr
until those pipes close.它在完成写入数据后关闭stdin
并读取stdout
和stderr
直到这些管道关闭。 You can't do a second communicate
because the child has already exited by the time it returns.您无法进行第二次communicate
因为孩子在返回时已经退出。
An interactive session with a child process is quite a bit more complicated.与子进程的交互式会话要复杂得多。
One problem is whether the child process even recognizes that it should be interactive.一个问题是子进程是否甚至认识到它应该是交互式的。 In the C libraries that most command line programs use for interaction, programs run from terminals (eg, a linux console or "pty" pseudo-terminal) are interactive and flush their output frequently, but those run from other programs via PIPES are non-interactive and flush their output infrequently.在大多数命令行程序用于交互的 C 库中,从终端(例如,linux 控制台或“pty”伪终端)运行的程序是交互式的并经常刷新它们的输出,但那些通过 PIPES 从其他程序运行的程序是非交互并很少刷新它们的输出。
Another is how you should read and process stdout
and stderr
without deadlocking.另一个是您应该如何在不死锁的情况下读取和处理stdout
和stderr
。 For instance, if you block reading stdout
, but stderr
fills its pipe, the child will halt and you are stuck.例如,如果您阻止读取stdout
,但stderr
填充了它的管道,则孩子将停止并且您被卡住。 You can use threads to pull both into internal buffers.您可以使用线程将两者都拉入内部缓冲区。
Yet another is how you deal with a child that exits unexpectedly.另一个是你如何处理一个意外退出的孩子。
For "unixy" systems like linux and OSX, the pexpect
module is written to handle the complexities of an interactive child process.对于像 linux 和 OSX 这样的“unixy”系统,编写pexpect
模块来处理交互式子进程的复杂性。 For Windows, there is no good tool that I know of to do it.对于 Windows,我所知道的没有好的工具可以做到这一点。
This answer should be attributed to @JFSebastian.这个答案应该归功于@JFSebastian。 Thanks for the comments!感谢您的评论!
The following code got my expected behavior:以下代码得到了我的预期行为:
import pexpect
analyzer = pexpect.spawn('hfst-lookup analyser-gt-desc.hfstol', encoding='utf-8')
analyzer.expect('> ')
for word in ['слово', 'сработай']:
print('Trying', word, '...')
analyzer.sendline(word)
analyzer.expect('> ')
print(analyzer.before)
Whenever you want to send input to the process, use proc.stdin.write()
.每当您想向进程发送输入时,请使用proc.stdin.write()
。 Whenever you want to get output from the process, use proc.stdout.read()
.每当您想从进程中获取输出时,请使用proc.stdout.read()
。 Both stdin
and stdout
arguments to the constructor need to be set to PIPE
.构造函数的stdin
和stdout
参数都需要设置为PIPE
。
HFST has Python bindings: https://pypi.python.org/pypi/hfst HFST 具有 Python 绑定: https ://pypi.python.org/pypi/hfst
Using those should avoid the whole flushing issue, and will give you a cleaner API to work with than parsing the string output from pexpect.使用这些应该可以避免整个刷新问题,并且会为您提供一个比解析 pexpect 的字符串输出更清晰的 API。
From the Python REPL, you can get some doc's on the bindings with从 Python REPL,你可以得到一些关于绑定的文档
dir(hfst)
help(hfst.HfstTransducer)
or read https://hfst.github.io/python/3.12.2/QuickStart.html或阅读https://hfst.github.io/python/3.12.2/QuickStart.html
Snatching the relevant parts of the docs:抢夺文档的相关部分:
istr = hfst.HfstInputStream('hfst-lookup analyser-gt-desc.hfstol')
transducers = []
while not (istr.is_eof()):
transducers.append(istr.read())
istr.close()
print("Read %i transducers in total." % len(transducers))
if len(transducers) == 1:
out = transducers[0].lookup_optimize("слово")
print("got %s" % (out,))
else:
pass # or handle >1 fst in the file, though I'm guessing you don't use that feature
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.