Running bash commands from python script in certain directory

Question

I am trying to write a Python script to run recursively a program from bash in multiple directories and save(now just display) output in log file.

The problem is that when I try to run that app from home directory just giving the full path of input files it crashes, so I usually run it directly in directory containing input files. I have tried to do this with os.popen() and pexpect but I think most suitable here will be subprocess.

My code:

import subprocess,
from subprocess import Popen, PIPE 

subprocess.Popen("$fp "+pcr+" "+dat, cwd=path+dirs[0], stdout=PIPE)
print output.stdout.read()

where pcr and dat are the names of input files.

As I try to run it I obtain following error:

Traceback (most recent call last):
File "file_ed.py", line 47, in <module>
subprocess.Popen("$fp "+pcr+" "+dat, cwd=retval, stdout=PIPE)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
 File "/usr/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

What could be the cause of this problem and how to fix it?

Answer 1

I think your problem is that you want to pass pcr and dat as arguments to the program and you think that the whitespace will do that for you – but Python is not bash. Just use an array and it should work:

subprocess.Popen([fp, pcr, dat], cwd=path+dirs[0], stdout=PIPE)

The reason you got "no such file or directory": Python was looking for a program named "$fp "+pcr+" "+dat , ie it assumed that the whitespace (and the dollar sign, unless you meant it as kind of a pseudo-code placeholder) is part of the program name. The error tells you that there is no program with such a name.

However it is hard to guess what you mean because your snippet is incomplete. I don't really know what $fp and pcr and dat mean.

Also have a look at the examples in the documentation of the subprocess module.