[英]Check on the stdout of a running subprocess in python
If need to periodically check the stdout
of a running process. 如果需要定期检查正在运行的进程的
stdout
。 For example, the process is tail -f /tmp/file
, which is spawned in the python script. 例如,进程是
tail -f /tmp/file
,它是在python脚本中生成的。 Then every x seconds, the stdout of that subprocess is written to a string and further processed. 然后每隔x秒,该子进程的stdout被写入字符串并进一步处理。 The subprocess is eventually stopped by the script.
脚本最终会停止子进程。
To parse the stdout of a subprocess, if used check_output
until now, which doesn't seem to work, as the process is still running and doesn't produce a definite output. 要解析子
check_output
的stdout,如果直到现在使用check_output
,这似乎不起作用,因为进程仍在运行并且不会产生确定的输出。
>>> from subprocess import check_output
>>> out = check_output(["tail", "-f", "/tmp/file"])
#(waiting for tail to finish)
It should be possible to use threads for the subprocesses, so that the output of multiple subprocesses may be processed (eg tail -f /tmp/file1, tail -f /tmp/file2). 应该可以为子进程使用线程,以便可以处理多个子进程的输出(例如tail -f / tmp / file1,tail -f / tmp / file2)。
How can I start a subprocess, periodically check and process its stdout and eventually stop the subprocess in a multithreading friendly way? 如何启动子进程,定期检查和处理其标准输出并最终以多线程友好的方式停止子进程? The python script runs on a Linux system.
python脚本在Linux系统上运行。
The goal is not to continuously read a file, the tail command is an example, as it behaves exactly like the actual command used. 目标不是连续读取文件,tail命令是一个示例,因为它的行为与使用的实际命令完全相同。
edit: I didn't think this through, the file did not exist. 编辑:我没想到这个,文件不存在。
check_output
now simply waits for the process to finish. check_output
现在只是等待进程完成。
edit2: An alternative method, with Popen
and PIPE
appears to result in the same issue. edit2:另一种方法,使用
Popen
和PIPE
似乎会导致同样的问题。 It waits for tail
to finish. 它等待
tail
完成。
>>> from subprocess import Popen, PIPE, STDOUT
>>> cmd = 'tail -f /tmp/file'
>>> p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
>>> output = p.stdout.read()
#(waiting for tail to finish)
Your second attempt is 90% correct. 你的第二次尝试是正确的90%。 The only issue is that you are attempting to read all of
tail
's stdout at the same time once it's finished. 唯一的问题是你试图在它完成后同时阅读所有
tail
的标准输出。 However, tail
is intended to run (indefinitely?) in the background, so you really want to read stdout from it line-by-line: 但是,
tail
应该在后台运行(无限期?),所以你真的想逐行读取stdout:
from subprocess import Popen, PIPE, STDOUT
p = Popen(["tail", "-f", "/tmp/file"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
for line in p.stdout:
print(line)
I have removed the shell=True
and close_fds=True
arguments. 我删除了
shell=True
和close_fds=True
参数。 The first is unnecessary and potentially dangerous, while the second is just the default. 第一个是不必要的,有潜在危险,而第二个是默认值。
Remember that file objects are iterable over their lines in Python. 请记住,文件对象可以在Python中通过它们的行进行迭代。 The
for
loop will run until tail
dies, but it will process each line as it appears, as opposed to read
, which will block until tail
dies. for
循环将一直运行,直到tail
死亡,但它将处理每条线,而不是read
,这将阻塞直到tail
死亡。
If I create an empty file in /tmp/file
, start this program and begin echoing lines into the file using another shell, the program will echo those lines. 如果我在
/tmp/file
创建一个空文件,启动该程序并开始使用另一个shell将行回显到文件中,程序将回显这些行。 You should probably replace print
with something a bit more useful. 您可能应该用更有用的东西替换
print
。
Here is an example of commands I typed after starting the code above: 以下是我在启动上述代码后输入的命令示例:
Command line 命令行
$ echo a > /tmp/file
$ echo b > /tmp/file
$ echo c >> /tmp/file
Program Output (From Python in a different shell) 程序输出(来自不同shell中的Python)
b'a\n'
b'tail: /tmp/file: file truncated\n'
b'b\n'
b'c\n'
In the case that you want your main program be responsive while you respond to the output of tail
, start the loop in a separate thread. 如果您希望主程序在响应
tail
的输出时响应,请在单独的线程中启动循环。 You should make this thread a daemon so that it does not prevent your program from exiting even if tail
is not finished. 您应该将此线程设置为守护程序,以便即使
tail
未完成也不会阻止程序退出。 You can have the thread open the sub-process or you can just pass in the standard output to it. 您可以让线程打开子流程,也可以将标准输出传递给它。 I prefer the latter approach since it gives you more control in the main thread:
我更喜欢后一种方法,因为它可以让你在主线程中有更多的控制权:
def deal_with_stdout():
for line in p.stdout:
print(line)
from subprocess import Popen, PIPE, STDOUT
from threading import Thread
p = Popen(["tail", "-f", "/tmp/file"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
t = Thread(target=deal_with_stdout, daemon=True)
t.start()
t.join()
The code here is nearly identical, with the addition of a new thread. 这里的代码几乎相同,添加了一个新线程。 I added a
join()
at the end so the program would behave well as an example ( join
waits for the thread to die before returning). 我在最后添加了一个
join()
,所以程序作为一个例子表现得很好( join
在返回之前等待线程死掉)。 You probably want to replace that with whatever processing code you would normally be running. 您可能希望用通常运行的任何处理代码替换它。
If your thread is complex enough, you may also want to inherit from Thread
and override the run
method instead of passing in a simple target
. 如果您的线程足够复杂,您可能还希望从
Thread
继承并覆盖run
方法而不是传入一个简单的target
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.