[英]Bash style process substitution with Python's Popen
In Bash you can easily redirect the output of a process to a temporary file descriptor and it is all automagically handled by bash like this: 在Bash中,您可以轻松地将进程的输出重定向到临时文件描述符,并且bash会自动将其全部处理,如下所示:
$ mydaemon --config-file <(echo "autostart: True \n daemonize: True")
or like this: 或像这样:
$ wc -l <(ls)
15 /dev/fd/63
see how it is not stdin redirection: 看看怎么不是stdin重定向:
$ vim <(echo "Hello World")
vim opens a text file containing "Hello world"
$ echo "Hello World" | vim
Vim: Warning: Input is not from a terminal
You can see in the second example how bash automatically creates a file descriptor and allows you to pass the output of a program to another program. 您可以在第二个示例中看到bash如何自动创建文件描述符,并允许您将程序的输出传递给另一个程序。
Now onto my question: How can I do the same thing with Python, using Popen in the subprocess module? 现在,我的问题是:如何在子流程模块中使用Popen使用Python做同样的事情?
I have been using a normal file of kmers and just reading it in, but my program now generates a specific list of kmers at runtime based on user parameters. 我一直在使用kmers的普通文件并只是读入其中,但是我的程序现在会在运行时根据用户参数生成kmers的特定列表。 I'd like to avoid writing to a temporary file manually because dealing with file permissions could cause problems for my primitive users.
我想避免手动写入临时文件,因为处理文件权限可能会对原始用户造成问题。
Here is my code to run my program and capture the stdout with an actual file "kmer_file" 这是我的代码,用于运行程序并使用实际文件“ kmer_file”捕获标准输出
input_file = Popen(["pram_axdnull", str(kmer), input_file, kmer_file], stdout=PIPE)
I created a function called generate_kmers which returns a string that can be written out to a file easily (includes newlines) or to a StringIO. 我创建了一个称为generate_kmers的函数,该函数返回一个可以轻松写到文件(包括换行符)或StringIO的字符串。 I also have a python script that is standalone to do the same thing
我也有一个独立执行相同操作的python脚本
So now I want to pass it in as my 3rd parameter: 所以现在我想将其作为我的第三个参数传递:
This doesn't work: 这不起作用:
kmer_file = stringIO(generate_kmers(3))
input_file = Popen(["pram_axdnull", str(kmer), input_file, kmer_file], stdout=PIPE)
Nor does this: 这也不是:
kmer_file = Popen(["generate_kmers", str(kmer)], stdout=PIPE)
input_file = Popen(["pram_axdnull", str(kmer), input_file, kmer_file.stdout], stdout=PIPE)
So I am out of ideas. 所以我没主意。
Does anyone know of a good way to resolve this? 有人知道解决此问题的好方法吗? I was thinking using the shell=True option and using the actual bashism of <() but I haven't figured that out.
我在考虑使用shell = True选项并使用<()的实际bashism,但我还没有弄清楚。
Thank you! 谢谢!
If pram_axdnull
understands "-"
convention to mean: "read from stdin" then you could: 如果
pram_axdnull
"-"
约定理解为:“从stdin读取”,则可以:
p = Popen(["pram_axdnull", str(kmer), input_filename, "-"],
stdin=PIPE, stdout=PIPE)
output = p.communicate(generate_kmers(3))[0]
If the input is generated by external process: 如果输入是由外部过程生成的:
kmer_proc = Popen(["generate_kmers", str(kmer)], stdout=PIPE)
p = Popen(["pram_axdnull", str(kmer), input_filename, "-"],
stdin=kmer_proc.stdout, stdout=PIPE)
kmer_proc.stdout.close()
output = p.communicate()[0]
If pram_axdnull
doesn't understand "-"
convention: 如果
pram_axdnull
无法理解"-"
约定:
import os
import tempfile
from subprocess import check_output
with tempfile.NamedTemporaryFile() as file:
file.write(generate_kmers(3))
file.delete = False
try:
p = Popen(["pram_axdnull", str(kmer), input_filename, file.name],
stdout=PIPE)
output = p.communicate()[0]
# or
# output = check_output(["pram_axdnull", str(kmer), input_filename,
file.name])
finally:
os.remove(file.name)
To generate temporary file using external process: 要使用外部过程生成临时文件:
from subprocess import check_call
with tempfile.NamedTemporaryFile() as file:
check_call(["generate_kmers", str(kmer)], stdout=file)
file.delete = False
To avoid waiting for all kmers to be generated ie, to write/read kmers simultaneously, you could use os.mkfifo()
on Unix (suggested by @cdarke): 为了避免等待所有kmers生成,即同时写入/读取kmers,可以在Unix上使用
os.mkfifo()
(由@cdarke建议):
import os
import shutil
import tempfile
from contextlib import contextmanager
from subprocess import Popen, PIPE
@contextmanager
def named_pipe():
dirname = tempfile.mkdtemp()
try:
path = os.path.join(dirname, 'named_pipe')
os.mkfifo(path)
yield path
finally:
shutil.rmtree(dirname)
with named_pipe() as path:
p = Popen(["pram_axdnull", str(kmer), input_filename, path],
stdout=PIPE) # read from path
with open(path, 'wb') as wpipe:
kmer_proc = Popen(["generate_kmers", str(kmer)],
stdout=wpipe) # write to path
output = p.communicate()[0]
kmer_proc.wait()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.