Syntax for subprocess.call (Win7 x64)

Question

I am trying to call an .exe file that's not in my local Python directory using subprocess.call() . The command (as I type it into cmd.exe) is exactly as follows: "C:\\Program Files\\R\\R-2.15.2\\bin\\Rscript.exe" --vanilla C:\\python\\buyback_parse_guide.r

The script runs, does what I need to do, and I have confirmed the output is correct.

Here's my python code, which I thought would do the exact same thing:

## Set Rcmd
Rcmd = r'"C:\Program Files\R\R-2.15.2\bin\Rscript.exe"'
## Set Rargs
Rargs = r'--vanilla C:\python\buyback_parse_guide.r'

retval = subprocess.call([Rcmd,Rargs],shell=True)

When I call retval in my Python console, it returns 1 and the .R script doesn't run, but I get no errors. I'm pretty sure this is a really simple syntax error... help? Much thanks!

Answer 1

To quote the docs :

If shell is True, it is recommended to pass args as a string rather than as a sequence.

Splitting it up (either manually, or via shlex ) just so subprocess can recombine them so the shell can split them again is silly.

I'm not sure why you think you need shell=True here. (If you don't have a good reason, you generally don't want it…) But even without shell=True :

On Windows, if args is a sequence, it will be converted to a string in a manner described in Converting an argument sequence to a string on Windows. This is because the underlying CreateProcess() operates on strings.

So, just give the shell the command line:

Rcmd = r'"C:\Program Files\R\R-2.15.2\bin\Rscript.exe" --vanilla C:\python\buyback_parse_guide.r'
retval = subprocess.call(Rcmd, shell=True)

Answer 2

According to the docs , Rscript:

… is an alternative front end for use in #! scripts and other scripting applications.

… is convenient for writing #! scripts… (The standard Windows command line has no concept of #! scripts, but Cygwin shells do.)

… is only supported on systems with the execv system call.

So, it is not the way to run R scripts from another program under Windows.

This answer says:

Rscript.exe is your friend for batch scripts… For everything else, there's R.exe

So, unless you have some good reason to be using Rscript outside of a batch script, you should switch to R.exe.

You may wonder why it works under cmd.exe, but not from Python. I don't know the answer to that, and I don't think it's worth digging through code or experimenting to find out, but I can make some guesses.

One possibility is that when you're running from the command line, that's a cmd.exe that controls a terminal, while when you're running from subprocess.call(shell=True) or os.system , that's a headless cmd.exe . Running a .bat/.cmd batch file gets you a non-headless cmd , but running cmd directly from another app does not. R has historically had all kinds of complexities dealing with the Windows terminal, which is why they used to have separate Rterm.exe and Rcmd.exe tools. Nowadays, those are both merged into R.exe, and it should work just fine either way. But if you try doing things the docs say not to do, that may not be tested, it's perfectly reasonable that it may not work.

At any rate, it doesn't really matter why it works in some situations even though it's not documented to. That certainly doesn't mean it should work in other situations it's not documented to work in, or that you should try to force it to do so. Just do the right thing and run R.exe instead of Rscript.exe .

Unless you have some information that contradicts everything I've found in the documentation and everywhere else I can find, I'm placing my money on Rscript.exe itself being the problem.

You'll have to read the documentation on the invocation differences between Rscript.exe and R.exe , but they're not identical. According to the intro docs ,:

If you just want to run a file foo.R of R commands, the recommended way is to use R CMD BATCH foo.R

According to your comment above:

When I type "C:\\R\\R-2.15.2\\bin\\i386\\R.exe" CMD BATCH C:\\python\\buyback_parse_guide.r into cmd.exe, the .R script runs successfully. What's the proper syntax for passing this into python?

That depends on the platform. On Windows, a list of arguments gets turned into a string, so you're better off just using a string so you don't have to debug the joining; on Unix, a string gets split into a list of arguments, so you're better off using a list so you don't have to debug the joining.

Since there are no spaces in the path, I'd take the quotes out.

So:

rcmd = r'C:\R\R-2.15.2\bin\i386\R.exe CMD BATCH C:\python\buyback_parse_guide.r'
retval = subprocess.call(rcmd)