python的子进程模块无法运行bash命令，例如“ <cmd1> | <cmd2> ”

Question

The following python script:

#!/usr/bin/env python

import os
cmd = "echo Hello world | cut -d' ' -f1"
test=os.system(cmd)
print(test)

it runs ok (the output is Hello ). But when I use subprocess module this one:

#!/usr/bin/env python

import subprocess
cmd = "echo Hello world | cut -d' ' -f1"
process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
test = process.communicate()[0]
print (test)

is not ok. The output is Hello world | cut -d' ' -f1 Hello world | cut -d' ' -f1 and I expect to be only Hello . How can I correct it?

I saw that in general subprocess module will fail when I'm using a bash command like:

<cmd1> | <cmd2> 

Answer 1

This:

echo Hello world | cut -d' ' -f1

… is not actually a command, it's a fragment of shell script. So you need to have the shell execute it.

You can do this just by adding shell=True to the Popen constructor.

---

The documentation explains how this works. It also explains better ways to do the same thing without the shell . For example:

p1 = Popen(['echo', 'Hello', 'world'], stdout=PIPE)
p2 = Popen(['cut', "-d' '", '-f1'], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()
test = p2.communicate()[0]

---

Meanwhile, you almost never want to use split on a command line—and in fact, your example shows exactly why you don't want to:

>>> cmd = "echo Hello world | cut -d' ' -f1"
>>> cmd.split()
['echo', 'Hello', 'world', '|', 'cut', "-d'", "'", '-f1']

Notice that it split -d' ' into two arguments, -d' and ' .

If you're using shell=True , don't try to split the arguments at all; just pass a string as your cmd :

process = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)

If you're not using the shell, the right way to do this is with the shlex module:

>>> shlex.split(cmd)
['echo', 'Hello', 'world', '|', 'cut', '-d ', '-f1']

Notice that the "-d' '" turned into "-d " this time. That may seem odd at first glance, but it's in fact exactly what the shell would do, and what you want; the cut program will get a space as its d option. (In other words, the quotes are for the shell, not for the program the shell runs.)

(The shlex module also has a handle quote function that you can use for the exact opposite purpose: building a command line from a list of arguments for shell=True .)

However, it's usually better to just create an list of arguments in the first place, instead of trying to figure out how to create a string that, when run through shlex.split() , will give you the list you wanted.