python中的Gzip和子进程的标准输出

Question

I'm using python 2.6.4 and discovered that I can't use gzip with subprocess the way I might hope. This illustrates the problem:

May 17 18:05:36> python
Python 2.6.4 (r264:75706, Mar 10 2010, 14:41:19)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.

>>> import gzip
>>> import subprocess
>>> fh = gzip.open("tmp","wb")
>>> subprocess.Popen("echo HI", shell=True, stdout=fh).wait()
0
>>> fh.close()
>>>
[2]+  Stopped                 python
May 17 18:17:49> file tmp
tmp: data
May 17 18:17:53> less tmp
"tmp" may be a binary file.  See it anyway?
May 17 18:17:58> zcat tmp

zcat: tmp: not in gzip format

Here's what it looks like inside less

HI
^_<8B>^H^Hh<C0><F1>K^B<FF>tmp^@^C^@^@^@^@^@^@^@^@^@

which looks like it put in the stdout as text and then put in an empty gzip file. Indeed, if I remove the "Hi\\n", then I get this:

May 17 18:22:34> file tmp
tmp: gzip compressed data, was "tmp", last modified: Mon May 17 18:17:12 2010, max compression

What is going on here?

UPDATE: This earlier question is asking the same thing: Can I use an opened gzip file with Popen in Python?

Answer 1

You can't use file-likes with subprocess , only real files. The fileno() method of GzipFile returns the FD of the underlying file, so that's what the echo redirects to. The GzipFile then closes, writing an empty gzip file.

Answer 2

just pipe that sucker

from subprocess import Popen,PIPE
GZ = Popen("gzip > outfile.gz",stdin=PIPE,shell=True)
P = Popen("echo HI",stdout=GZ.stdin,shell=True)
# these next three must be in order
P.wait()
GZ.stdin.close()
GZ.wait()

Answer 3

我不完全确定为什么这不起作用（也许输出重定向没有调用python的write，这就是gzip的工作原理？），但是这可行：

>>> fh.write(subprocess.Popen("echo Hi", shell=True, stdout=subprocess.PIPE).stdout.read())

Answer 4

You don't need to use subprocess to write to the gzip.GzipFile . Instead, write to it like any other file-like object. The result is automagically gzipped!

import gzip
with gzip.open("tmp.gz", "wb") as fh:
    fh.write('echo HI')