[英]Resource temporarily unavailable error with subprocess module in Python
In Python, I spawn a gnuplot
process to generate gif images from a data set. 在Python中,我生成了一个
gnuplot
进程,用于从数据集生成gif图像。
from subprocess import Popen, PIPE
def gnuplotter(...)
p = Popen([GNUPLOT], shell=False, stdin=PIPE, stdout=PIPE)
p.stdin.write(r'set terminal gif;')
...
p.stdin.write(contents)
p.stdout.close()
It works fine when I use gnuplotter()
one time, but when I launch the process multiple times, I got Resource temporarily unavailable
error. 当我使用
gnuplotter()
一次时它工作正常,但是当我多次启动进程时,我得到Resource temporarily unavailable
错误。
for i in range(54):
gnuplotter(i, ...
File "/Users/smcho/code/PycharmProjects/contextAggregator/aggregation_analyzer/aggregation_analyzer/gnuplotter.py", line 48, in gnuplotter
p = Popen([GNUPLOT], shell=False, stdin=PIPE, stdout=PIPE)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1205, in _execute_child
self.pid = os.fork()
OSError: [Errno 35] Resource temporarily unavailable
What's wrong, and how can I close gnuplot process before spewing another one? 怎么了,怎么能在喷出另一个之前关闭gnuplot进程呢?
pid numbers, open file descriptors, memory are limited resources. pid数字,打开文件描述符,内存是有限的资源。
fork(2) manual says when errno.EAGAIN
should happen: fork(2)manual说当
errno.EAGAIN
应该发生时:
[EAGAIN] The system-imposed limit on the total number of processes under execution would be exceeded. This limit is configuration-dependent. [EAGAIN] The system-imposed limit MAXUPRC () on the total number of processes under execution by a single user would be exceeded.
To reproduce the error more easily, you could add at the start of your program: 要更轻松地重现错误,您可以在程序开头添加:
import resource
resource.setrlimit(resource.RLIMIT_NPROC, (20, 20))
The issue might be that all child processes are alive because you haven't called p.stdin.close()
and gnuplot's stdin might be fully buffered when redirected to a pipe ie, gnuplot
processes might be stuck awaiting input. 问题可能是所有子进程都处于活动状态,因为您没有调用
p.stdin.close()
并且当重定向到管道时,gnuplot的stdin可能会被完全缓冲,即gnuplot
进程可能会卡在等待输入。 And/or your application uses too many file descriptors (file descriptors are inherited by child processes by default on Python 2.7) without releasing them. 和/或您的应用程序使用太多文件描述符(文件描述符在Python 2.7上默认由子进程继承)而不释放它们。
If input doesn't depend on the output and the input is limited in size then use .communicate()
: 如果输入不依赖于输出并且输入的大小有限,那么使用
.communicate()
:
from subprocess import Popen, PIPE, STDOUT
p = Popen("gnuplot", stdin=PIPE, stdout=PIPE, stderr=PIPE,
close_fds=True, # to avoid running out of file descriptors
bufsize=-1, # fully buffered (use zero (default) if no p.communicate())
universal_newlines=True) # translate newlines, encode/decode text
out, err = p.communicate("\n".join(['set terminal gif;', contents]))
.communicate()
writes all input and reads all output (concurrently, so there is no deadlock) then closes p.stdin, p.stdout, p.stderr (even if input is small and gnuplot's side is fully buffered; EOF flushes the buffer) and waits for the process to finish (no zombies). .communicate()
写入所有输入并读取所有输出(并发,因此没有死锁)然后关闭p.stdin,p.stdout,p.stderr(即使输入很小并且gnuplot的端完全缓冲; EOF刷新缓冲区)并等待过程完成(没有僵尸)。
Popen
calls _cleanup()
in its constructor that polls exit status of all known subprocesses ie, even if you won't call p.wait()
there shouldn't be many zombie processes (dead but with unread status). Popen
调用_cleanup()
在其构造是所有已知的子流程投票退出状态 ,即,即使你不会打电话p.wait()
不应该有很多僵尸进程(死了但是未读状态)。
You need to should call p.wait()
to wait for the subprocess to finish, and then collect it, after you are done communicating with it. 你
需要调用p.wait()
来等待子p.wait()
完成,然后在完成与它的通信后收集它。
If you have special situations (where you want to start N and wait for them later), p.poll()
will let you check whether one has finished. 如果您有特殊情况(您想要启动N并稍后等待它们),
p.poll()
将让您检查是否已完成。
Since you have pipes set up, you should be using p.communicate()
to avoid deadlocks. 由于您已设置管道,因此您应该使用
p.communicate()
来避免死锁。 See the documentation . 请参阅文档 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.