是否有任何模块可以读取Shell脚本？

Question

I'm making a python script right now, and I need to use some environment variables which are set in a bash shell script.

The bash script is something like:

#! /bin/sh

#sets some names:
export DISTRO="unified"
#export DISTRO="other"

#number of parallel builds
export BB_NUM_THREADS=2

#set build dir
export BUILDDIR=$PWD

Normally, I would just source this script in bash, then go do my builds. I'm trying to wrap python around the whole process to do some management of the output so I want to remove the manual source ./this_script.sh step.

What I want to do is read this script from python and then use os.environ to set up the variables within it. (I know this will not affect the parent, but only the current running Python instance and that's fine)

So to make my work easier, I'm trying to find out are there any modules which can "parse" the bash script and make use of the environment variables found within? Currently I'm doing this by hand and it's a bit of a pain.

If no such module exists to do exactly what I want, is there a more pythonic (read: easier/shorter) way of manually parsing a file in general, right now I'm doing this:

def parse_bash_script(fn):
  with open(fn) as f:
    for line in f:
      if not line[:1] == '#':   #ignore comments
        if "export" in line:
          line = line.replace(" ","").strip()
          var = line[6:line.find("=")]
          val = line[line.find("=")+1:len(line)]
          if "\"" in val:
            val = val[1:-1]
          os.environ[var]=val

Answer 1

There is no module to do exactly what you want, but shlex will do a lot of what you want. In particular, it will get the quoting , etc. right without you having to worry about it (which is the hardest part of this), as well as skipping comments, etc. The only thing it won't do is handle the export keywords.

The easy way around that is to preprocess:

with open(fn) as f:
    processed = f.read().replace('export ', '')
for line in shlex.split(processed):
    var, _, value = line.partition('=')
    os.environ[var] = val

---

It's a bit hackier, but you can also do it a bit less verbosely by post-processing. In particular, shlex will treat export foo="bar spam eggs" as two values: export and foo="bar spam eggs" , and you can just skip the ones that == 'export' , or where the partition finds nothing, or… For example:

with open(fn) as f:
    for line in shlex.split(f.read()):
        var, eq, value = line.partition('=')
        if eq:
            os.environ[var] = val

---

If you want to get fancier, you can construct a shlex object and (a) drive the parser directly from the file, and (b) control the parsing at a finer-grained level. However, I don't think that's necessary here.

---

Meanwhile, if you want to handle environment substitution (as the BUILDDIR=$PWD implies), this won't magically take care of that for you. You can make configparser do that for you with its ExtendedInterpolation feature , but then you'll need to trick configparser into handling shlex syntax, at which point… why bother.

You can of course do it manually by writing your own interpolator, but that's hard to get right. You need to know the shell's rules for why $PWD-foo is the same as ${PWD}-foo , but $PWD_foo is the same as ${PWD_foo} , etc.

A better solution at this point—assuming the script is actually safe to run—would be to actually use a shell to do it for you. For example:

with open('script.sh') as f:
    script = f.read()
script += b'\nenv'
with subprocess.Popen(['sh'], stdin=subprocess.PIPE, stdout=subprocess.PIPE) as p:
    result = p.communicate(script)
for line in result.splitlines():
    var, _, value = line.partition('=')
    os.environ[var] = value

Of course this will also override things like _=/usr/bin/env , but probably not anything you care about.

Answer 2

def parse_bash_script(fn):
  with open(fn) as f:
    for line in f:
      if not line.startswith('#'):   #ignore comments
        if "export" in line:
          var, _, val = line.partition('=')
          var = var.lstrip()
          val = val.rstrip()
          if val.startswith('"'):
            vals = val.rpartition('"')
            val = vals[0][1]+vals[2]
          os.environ[var]=val

Answer 3

I had the same problem, and based on the advice from abarnert, I decided to implement the solution as a subprocess call to a restricted bash shell, combined with shlex.

import shlex
import subprocess

filename = '/path/to/file.conf'
o, e = subprocess.Popen(
    ['/bin/bash', '--restricted', '--noprofile', '--init-file', 
     filename, '-i', '-c', 'declare'],
    env={'PATH': ''},
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE).communicate()

if e:
    raise StandardError('conf error in {}: {}'.format(filename, e))

for token in shlex.split(o):
    parts = token.split('=', 1)
    if len(parts) == 2:
        os.environ[parts[0]] = parts[1]

The advantage to the restricted shell is that it blocks many of the undesirable or malicious side effects that may otherwise happen when executing a shell script. From the bash documentation:

A restricted shell is used to set up an environment more controlled than the standard shell. It behaves identically to bash with the exception that the following are disallowed or not performed:

• changing directories with cd
• setting or unsetting the values of SHELL, PATH, ENV, or BASH_ENV
• specifying command names containing /
• specifying a file name containing a / as an argument to the . builtin command
• Specifying a filename containing a slash as an argument to the -p option to the hash builtin command
• importing function definitions from the shell environment at startup
• parsing the value of SHELLOPTS from the shell environment at startup
• redirecting output using the >, >|, <>, >&, &>, and >> redirection operators
• using the exec builtin command to replace the shell with another command
• adding or deleting builtin commands with the -f and -d options to the enable builtin command
• Using the enable builtin command to enable disabled shell builtins
• specifying the -p option to the command builtin command
• turning off restricted mode with set +r or set +o restricted.