How to execute a string containing a script using python subprocess module

Question

I receive a script in a string and trying to find a way to execute it using the subprocess module of python. eg

import subprocess
script = """#!/usr/bin/python
            print "inside script"
         """
subprocess.Popen(script)

This throws an OSError: saying there is no such file or directory, which is fair enough as I think its trying to look for command name.

I then tried subprocess.Popen(script, shell=True) which works only for shell commands and totally ignores the #! at the top of the script. Is there any way to make suprocess run my script stored in string like its an executable file ?

Answer 1

As you can read in the Popen Python Documentation the parameter to Popen is an executable to launch, not the contents of some script to perform. You cannot use Popen in this manner.

If you really want to you could write the script to a file and then pass that filename to Popen, but that doesn't seem to be an elegant solution to whatever your problem is.

Seeing as all you want is simply to execute some Python code from a string, you'll want to use the exec builtin function

Answer 2

This solution is based on -c (compile) Python interpreter cmdline argument. It ignores using shell properties entirely.

Popen may be created in similar way. subprocess.call was used to show that script is in fact executed, and python interpreter return code is changed by code executed.

import subprocess

executable_code = """
print 'inside script'
print 123
"""

code_with_exception = """
raise Exception()
"""

ok_rc = subprocess.call(['python', '-c', executable_code])
assert ok_rc == 0

bad_rc = subprocess.call(['python', '-c', code_with_exception])
assert bad_rc == 1

Additional improvement would be to skip 'python' hardcoded string and use sys.executable .

A string giving the absolute path of the executable binary for the Python interpreter, on systems where this makes sense. If Python is unable to retrieve the real path to its executable, sys.executable will be an empty string or None.

import sys
import subprocesss
code = "raise Exception()"
bad_rc = subprocess.call([sys.executable, '-c', code])
assert bad_rc == 1

Answer 3

You can run scripts by pumping them into a child python's stdin . This may be a preferred solution to larger scripts:

import sys
import subprocess as subp
import shutil

script = """\
import sys
print "hello", sys.argv
"""

p = subp.Popen([sys.executable, '-', 'arg1', 'arg2'], stdin=subp.PIPE)
p.stdin.write(script)
p.stdin.close()
p.wait()

I use this trick across ssh connections to control remote servers.

Answer 4

To execute a string as a Python script using subprocess module:

>>> import subprocess, sys, textwrap
>>> subprocess.call(textwrap.dedent('''
...    print "Hello, world!"
... '''), shell=True, executable=sys.executable)
Hello, world!
0

Though as @wich suggested you could use Python builtin function exec() -- it is a statement on Python 2:

>>> exec 'print "Hello, exec!"'
Hello, exec!

If you want to run Python code in a different process, you could use multiprocessing or concurrent.futures modules :

#!/urs/bin/env python2
from multiprocessing import Process, freeze_support

def run(code):
    exec code # Python 2

if __name__ == "__main__":
    freeze_support()
    Process(target=run, args=['print "Hello, multiprocessing"']).start()