[英]Linux FIFO not returning EOF when I expect it to
Let us consider the following Python code, to be executed by cpython on a Linux system (warning: it will try to create or overwrite files in /tmp/first
, /tmp/second
and /tmp/third
).让我们考虑以下 Python 代码,由 cpython 在 Linux 系统上执行(警告:它会尝试创建或覆盖
/tmp/first
、 /tmp/second
和/tmp/third
)。
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
import subprocess
import os
import sys
import threading
class ThreadizedPopen(threading.Thread):
def __init__(self, command, stdin_name, stdout_name):
super(ThreadizedPopen, self).__init__()
self.command = command
self.stdin_name = stdin_name
self.stdout_name = stdout_name
self.returncode = None
def run(self):
with open(self.stdin_name, 'rb') as fin:
with open(self.stdout_name, 'wb') as fout:
popen = subprocess.Popen(self.command, stdin=fin, stdout=fout, stderr=None)
popen.communicate()
self.returncode = popen.returncode
def main():
os.system('mkfifo /tmp/first')
os.system('mkfifo /tmp/second')
os.system('mkfifo /tmp/third')
popen1 = ThreadizedPopen(['cat'], '/tmp/first', '/tmp/second')
popen2 = ThreadizedPopen(['cat'], '/tmp/second', '/tmp/third')
popen1.start()
popen2.start()
with open('/tmp/third') as fin:
print fin.read()
popen1.join()
popen2.join()
if __name__ == '__main__':
main()
I execute it then, on another shell, I write something in /tmp/first
(say with echo test > /tmp/first
).然后我执行它,在另一个 shell 上,我在
/tmp/first
写了一些东西(比如echo test > /tmp/first
)。 I would expect the Python program to quickly exit and print the same thing I fed to the first FIFO.我希望 Python 程序能够快速退出并打印我提供给第一个 FIFO 的相同内容。
In theory it should happen that the string I wrote in /tmp/first
gets copied over by the two cat
processes spawned by my program to the other two FIFOs and then picked up by the main Python program to be wrote on its stdout.从理论上讲,我在
/tmp/first
写入的字符串应该会被我的程序产生的两个cat
进程复制到其他两个 FIFO,然后被主 Python 程序拾取以写入其标准输出。 As soon as every cat
process finished, it should close its end of the writing FIFO, making the corresponding reading end return EOF and triggering the termination of the following cat
process.每一个
cat
进程完成后,就应该关闭它的写FIFO端,使对应的读端返回EOF并触发后续cat
进程的终止。 Looking at the program with strace
reveals that the test string is copied correctly through all the three FIFOs and is read by the main Python program.查看带有
strace
的程序显示测试字符串通过所有三个 FIFO 正确复制,并由主 Python 程序读取。 The first FIFO is also correctly closed (and the first cat
process exits, together with its manager Python thread).第一个 FIFO 也正确关闭(第一个
cat
进程与其管理器 Python 线程一起退出)。 However the second cat
process is stuck in a read()
call, expecting data from its reading FIFO.然而,第二个
cat
进程卡在read()
调用中,需要从其读取 FIFO 中获取数据。
I do not understand why this happens.我不明白为什么会这样。 From the
pipe(t)
man page (which, I understand, covers also this kind of FIFOs) it seems that a read on a FIFO is returned EOF as soon as the writing end (and all its duplicates) are closed.从
pipe(t)
手册页(据我所知,它也涵盖了这种 FIFO),似乎只要写入结束(及其所有重复项)关闭,对 FIFO 的读取就会返回 EOF。 According to strace
this appears to be the trace (in particular, the cat
process is dead, thus all its file descriptors are closed; its managing thread has closed its descriptors as well, I can see it in the strace
output).根据
strace
这似乎是跟踪(特别是, cat
进程已死,因此其所有文件描述符都已关闭;其管理线程也已关闭其描述符,我可以在strace
输出中看到它)。
Can you suggest me why that happens?你能告诉我为什么会这样吗? I can post the
strace
output if it can be useful.如果有用,我可以发布
strace
输出。
I found this question and simply added close_fds=True
to your subprocess
call.我发现了这个问题,并简单地将
close_fds=True
添加到您的subprocess
close_fds=True
调用中。 Your code now reads:您的代码现在显示为:
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
import subprocess
import os
import sys
import threading
class ThreadizedPopen(threading.Thread):
def __init__(self, command, stdin_name, stdout_name):
super(ThreadizedPopen, self).__init__()
self.command = command
self.stdin_name = stdin_name
self.stdout_name = stdout_name
self.returncode = None
def run(self):
with open(self.stdin_name, 'rb') as fin:
with open(self.stdout_name, 'wb') as fout:
popen = subprocess.Popen(self.command, stdin=fin, stdout=fout, stderr=None, close_fds=True)
popen.communicate()
self.returncode = popen.returncode
def main():
os.system('mkfifo /tmp/first')
os.system('mkfifo /tmp/second')
os.system('mkfifo /tmp/third')
popen1 = ThreadizedPopen(['cat'], '/tmp/first', '/tmp/second')
popen2 = ThreadizedPopen(['cat'], '/tmp/second', '/tmp/third')
popen1.start()
popen2.start()
with open('/tmp/third') as fin:
print fin.read()
popen1.join()
popen2.join()
if __name__ == '__main__':
main()
I placed your code in a script called fifo_issue.py
and ran it in a terminal.我将您的代码放在一个名为
fifo_issue.py
的脚本中,并在终端中运行它。 The script was idling as you'd expect (ignore mkfifo: cannot create fifo
):脚本如您所料地处于空闲状态(忽略
mkfifo: cannot create fifo
):
$ python fifo_issue.py
mkfifo: cannot create fifo ‘/tmp/first’: File exists
mkfifo: cannot create fifo ‘/tmp/second’: File exists
mkfifo: cannot create fifo ‘/tmp/third’: File exists
Then, in a second terminal, I typed:然后,在第二个终端中,我输入:
$ echo "I was echoed to /tmp/first!" > /tmp/first
Back to the first terminal that's still running your idling threads:回到第一个仍在运行空闲线程的终端:
$ python fifo_issue.py
mkfifo: cannot create fifo ‘/tmp/first’: File exists
mkfifo: cannot create fifo ‘/tmp/second’: File exists
mkfifo: cannot create fifo ‘/tmp/third’: File exists
I was echoed to /tmp/first!
After which python exited correctly之后python正确退出
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.