running separate python programs with subprocess

Question

I am trying to create a script that will run my other python programs. I am new to subprocess module so this is a bit confusing to me.

project structure

 /qe-functional /qe /tests cron_functional.py test_web_events.py setup.sh 

cron_functional.py

print(os.getcwd())
# print(subprocess.check_output('ls'))
runtag = "daily_run_" + datetime.today().strftime("%m_%d_%y")
testrun = "source ../../setup.sh; ./test_web_events.py -n 10 -t prf -E ctg-businessevent -p post {}".format(runtag)
cmd = testrun.split()
print(cmd)
subprocess.check_output(cmd)

output

$ python cron_functional.py 
/Users/bli1/Development/QE/qe-functional/qe/tests
['source', '../../setup.sh;', './test_web_events.py', '-n', '10', '-t', 'prf', '-E', 'ctg-businessevent', '-p', 'post', 'daily_run_05_26_15']
Traceback (most recent call last):
  File "cron_functional.py", line 11, in <module>
    subprocess.check_output(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 566, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 709, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1326, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Answer 1

source is an internal shell command, not an executable. What you want is not to run one source command with 11 arguments, but a one-liner shell script. You need to pass the whole script as one string to be interpreted by the shell.

subprocess.check_output(testrun, shell=True)

You haven't said what setup.sh does. If it's setting up environment variables and changing the working directory, consider doing that within Python instead. Then you can run

subprocess.check_output(['./test_web_events.py', '-n', '10', …, '-p', 'post', runtag])

… without involving the shell.