How to run a shell script in python from memory?

Question

The application I'm writing retrieves a shell script through HTTP from Network, I want to run this script in python however I don't want to physically save it to the hard drive because I have its content already in memory, and I would like to just execute it. I have tried something like this:

import subprocess

script = retrieve_script()
popen = subprocess.Popen(scrpit, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdOut, stdErr = popen.communicate()

def retrieve_script_content():
    # in reality I retrieve a shell script content from network,
    # but for testing purposes I will just hardcode some test content here
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

This snippet will not work because subprocess.Popen expects you to provide only one command at a time.

Are there any alternatives to run a shell script from memory?

Answer 1

This snippet will not work because subprocess.Popen expects you to provide only one command at a time.

That is not the case. Instead, the reason why it doesn't work is:

1. The declaration of retrieve_script has to come before the call
2. You call it retrieve_script_content instead of retrieve_script
3. You misspelled script as scrpit

Just fix those and it's fine:

import subprocess

def retrieve_script():
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

script = retrieve_script()
popen = subprocess.Popen(script, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdOut, stdErr = popen.communicate()
print(stdOut);

Result:

$ python foo.py
command1
command2
command3

However, note that this will ignore the shebang (if any) and run the script with the system's sh every time.

Answer 2

Are you using a Unix-like OS? If so, you should be able to use a virtual filesystem to make an in-memory file-like object at which you could point subprocess.Popen :

import subprocess
import tempfile
import os
import stat

def retrieve_script_content():
    # in reality I retrieve a shell script content from network,
    # but for testing purposes I will just hardcode some test content here
    return "echo command1" + "\n" + "echo command2" + " \n" + "echo command3"

content = retrieve_script_content()
with tempfile.NamedTemporaryFile(mode='w', delete=False, dir='/dev/shm') as f:
    f.write(content)
    os.chmod(f.name, stat.S_IRUSR | stat.S_IXUSR)
    # print(f.name)
popen = subprocess.Popen(f.name, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
                         shell=True)
stdOut, stdErr = popen.communicate()

print(stdOut.decode('ascii'))
# os.unlink(f.name)

prints

command1
command2
command3

Above I used /dev/shm as the virtual filesystem since Linux systems based on Glibc always have a tmpfs mounted on /dev/shm . If security is a concern you may wish to setup a ramfs .

---

One reason why you might want to use a virtual file instead of passing the script contents directly to subprocess.Popen is that the maximum size for a single string argument is limited to 131071 bytes .

Answer 3

You can execute multi command script with Popen. Popen only restricts you to one-command string when shell flag is False, yet it is possible to pass a list of commands. Popen 's flag shell=True allows for multi-command scripts (it is considered unsecure, though what you are doing - executing scripts from the web - is already very risky).