executing shell script using subprocess.Popen in Python?

Question

I am trying to execute shell script from the Python program. And instead of using subprocess.call , I am using subprocess.Popen as I want to see the output of the shell script and error if any while executing the shell script in a variable.

#!/usr/bin/python

import subprocess
import json
import socket
import os

jsonStr = '{"script":"#!/bin/bash\\necho Hello world\\n"}'
j = json.loads(jsonStr)

shell_script = j['script']

print shell_script

print "start"
proc = subprocess.Popen(shell_script, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(stdout, stderr) = proc.communicate()
if stderr:
   print "Shell script gave some error"
   print stderr
else:
   print stdout
   print "end" # Shell script ran fine.

But the above code whenever I am running, I am always getting error like this -

Traceback (most recent call last):
  File "hello.py", line 29, in <module>
    proc = subprocess.Popen(shell_script, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Any idea what wrong I am doing here?

Answer 1

To execute an arbitrary shell script given as a string, just add shell=True parameter.

#!/usr/bin/env python
from subprocess import call
from textwrap import dedent

call(dedent("""\
    #!/bin/bash
    echo Hello world
    """), shell=True)

Answer 2

You can execute it with shell=True (you can leave out the shebang, too).

proc = subprocess.Popen(j['script'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate()

Or, you could just do:

proc = subprocess.Popen(['echo', 'Hello world'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

Or, you could write the script to file, then invoke it:

inf = open('test.sh', 'wb')
inf.write(j['script'])
inf.close()

print "start"
proc = subprocess.Popen(['sh', 'test.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(stdout, stderr) = proc.communicate()