Import bash variables from a python script

Question

I have seen plenty examples of running a python script from inside a bash script and either passing in variables as arguments or using export to give the child shell access, I am trying to do the opposite here though.

I am running a python script and have a separate file, lets call it myGlobalVariables.bash

myGlobalVariables.bash:

foo_1="var1"    
foo_2="var2"   
foo_3="var3"  

My python script needs to use these variables.

For a very simple example:

myPythonScript.py:

print "foo_1: {}".format(foo_1)

Is there a way I can import them directly? Also, I do not want to alter the bash script if possible since it is a common file referenced many times elsewhere.

Answer 1

If your .bash file is formatted as you indicated - you might be able to just import it direct as a Python module via the imp module.

import imp
bash_module = imp.load_source("bash_module, "/path/to/myGlobalVariables.bash")
print bash_module.foo_1

Answer 2

You can also use os.environ:

Bash:

#!/bin/bash
# works without export as well
export testtest=one

Python:

#!/usr/bin/python
import os
os.environ['testtest']  # 'one'

Answer 3

I am very new to python, so I would welcome suggestions for more idiomatic ways to do this, but the following code uses bash itself to tell us which values get set by first calling bash with an empty environment ( env -i bash ) to tell us what variables are set as a baseline, then I call it again and tell bash to source your "variables" file, and then tell us what variables are now set. After removing some false-positives and an apparently-blank line, I loop through the "additional" output, looking for variables that were not in the baseline. Newly-seen variables get split (carefully) and put into the bash dictionary. I've left here (but commented-out) my previous idea for using exec to set the variables natively in python, but I ran into quoting/escaping issues, so I switched gears to using a dict.

If the exact call (path, etc) to your "variables" file is different than mine, then you'll need to change all of the instances of that value -- in the subprocess.check_output() call, in the list.remove() calls.

Here's the sample variable file I was using, just to demonstrate some of the things that could happen:

foo_1="var1"
foo_2="var2"
foo_3="var3"
if [[ -z $foo_3 ]]; then
    foo_4="test"
else
    foo_4="testing"
fi
foo_5="O'Neil"
foo_6='I love" quotes'
foo_7="embedded
newline"

... and here's the python script:

#!/usr/bin/env python

import subprocess

output = subprocess.check_output(['env', '-i', 'bash', '-c', 'set'])
baseline = output.split("\n")

output = subprocess.check_output(['env', '-i', 'bash', '-c', '. myGlobalVariables.bash; set'])
additional = output.split("\n")

# these get set when ". myGlobal..." runs and so are false positives
additional.remove("BASH_EXECUTION_STRING='. myGlobalVariables.bash; set'")
additional.remove('PIPESTATUS=([0]="0")')
additional.remove('_=myGlobalVariables.bash')
# I get an empty item at the end (blank line from subprocess?)
additional.remove('')

bash = {}
for assign in additional:
        if not assign in baseline:
                name, value = assign.split("=", 1)
                bash[name]=value
                #exec(name + '="' + value + '"')

print "New values:"
for key in bash:
  print "Key: ", key, " = ", bash[key]

Another way to do it:

Inspired by Marat 's answer, I came up with this two-stage hack. Start with a python program, let's call it "stage 1", which uses subprocess to call bash to source the variable file, as my above answer does, but it then tells bash to export all of the variables, and then exec the rest of your python program, which is in "stage 2".

Stage 1 python program:

#!/usr/bin/env python

import subprocess

status = subprocess.call(
  ['bash', '-c',
  '. myGlobalVariables.bash; export $(compgen -v); exec ./stage2.py'
  ]);

Stage 2 python program:

#!/usr/bin/env python
# anything you want! for example,
import os
for key in os.environ:
  print key, " = ", os.environ[key]

Answer 4

As stated in @theorifice answer, the trick here may be that such formatted file may be interpreted by both as bash and as python code. But his answer is outdated. imp module is deprecated in favour of importlib .

As your file has extension other than ".py", you can use the following approach:

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader 

spec = spec_from_loader("foobar", SourceFileLoader("foobar", "myGlobalVariables.bash"))
foobar = module_from_spec(spec)
spec.loader.exec_module(foobar)

I do not completely understand how this code works (where there are these foobar parameters), however, it worked for me. Found it here .