如何在Mac OS上使用python subprocess.check_output linux命令

Question

I am trying to run a linux executable on Max OS X 10.11.6 via python2.7

I would like to use subprocess.check_output.

The command, which works via the terminal is:

mosel -c "exec PATH/TO/SCRIPT arg1='value1', arg2='value2'"

However, when I try:

subprocess.check_output(['mosel','-c',cmd])

where

cmd="exec PATH/TO/SCRIPT arg1='value1', arg2='value2'"'

I get:

File "/usr/local/lib/python2.7/site-packages/subprocess32.py", line 629, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/local/lib/python2.7/site-packages/subprocess32.py", line 825, in __init__
restore_signals, start_new_session)
File "/usr/local/lib/python2.7/site-packages/subprocess32.py", line 1574, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 2] No such file or directory: 'mosel'

I have been able to get it to "echo" the command to an output file, but I cannot run "which mosel" via python, which leads me to believe that it has to do with check_output using "bin/sh"as the executable.

So, do I need to use "Popen" instead and set

executable=path/to/mosel

?

If so, how do use Python to get the user's path to mosel (ie get the output of "which mosel")?

Thanks!

UPDATE:

PyCharm was not seeing the system paths, which I fixed using this answer: Setting environment variables in OS X?

Now, it appears that

subprocess.check_output(['mosel','-c',cmd])

Is sending the square brackets to the command line, because it now returns:

dyld: Library not loaded: libxprm_mc.dylib
Referenced from: /usr/local/opt/xpress/bin/mosel
Reason: image not found
Traceback (most recent call last):
File "/Users/nlaws/projects/sunlamp/sunlamp-ss/RunScenarios/run.py", line 70, in <module>
run(1)
File "/Users/nlaws/projects/sunlamp/sunlamp-ss/RunScenarios/run.py", line 44, in run
out = check_output(['mosel', '-c', cmd])
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 219, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['mosel', '-c', cmd]' returned non-zero exit status -5

Or is there still a path issue?! (I can run mosel -c cmd via the mac terminal, but not in pycharm via python, nor in the mac terminal via python).

Answer 1

The problem is you're using check_output 's arguments incorrectly. Unless you pass it shell=True , check_output expects a list of parameters as its input , in the form: check_call(['binary_name','arg1','arg2','arg3', ....])

So in this case, you should do:

subprocess.check_call(['mosel', '-c', "exec PATH/TO/SCRIPT arg1='value1', arg2='value2'"])

Answer 2

The root of the issue turns out to be the DYLD_LIBRARY_PATH :

The new OS X release 10.11 "El Capitan" has a "security" feature that prevents passing DYLD_LIBRARY_PATH to child processes. Somehow, that variable is stripped from the environment. - (quoted from https://www.postgresql.org/message-id/20151103113612.GW11897@awork2.anarazel.de )

The security feature is called SIP or "System Integrity Protection". Unfortunately, it seems that no one has come up with a solution to this issue (other than work-arounds that must be tailored to each situation).

Here is another example of this issue: https://github.com/soumith/cudnn.torch/issues/111

Google "mac os inherit dyld_library_path" and you will find many other examples.