Python subprocess.checkoutput() exception using 'svn log'

Question

I have run in to an issue that I havent been able to resolve.

I am trying to run a subversion command through python to process the results. I had this code running my dev system but when I run it on target system I get an exception in the subprocess module.

Why would this code produce different results on my two systems?

command = "svn log --username " + __SVNUSERNAME + " --password " + __SVNPASSWORD + " --no-auth-cache --non-interactive -v -r {2022-09-26}:{2022-09-27} --xml " + __SVNTRUNK
bytes = subprocess.check_output([command], stderr=subprocess.STDOUT,shell=True)

I get the following exception:

  File "E:\5_scripts\svn.py", line 89, in __run
    bytes = subprocess.check_output([command], stderr=subprocess.STDOUT,shell=True)
  File "C:\tools\Python3\lib\subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\tools\Python3\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['{command}']' returned non-zero exit status 1.

1. I tried to run the same subversion command on the command line and it executes as I expect. When I echo %ERRORLEVEL% it returns a status of 0. So Im lost as to what is happening differently on my target system.

2. I tried investigating the python docs for the checkoutput() function and there appears to be no major updates between 3.6 and 3.9

3. I tried investigating the Apache subversion release notes and didnt find any changes to the 'log' subcommand

Dev System:

• Linux 3.10
• python 3.6.8
• svn 1.9.9

Target System:

• Windows Server
• python 3.9.6
• svn 1.7.4

Answer 1

This is because your target system is Windows. Specifically, the shell=True doesn't work, or doesn't work the same way, on Windows as on Linux. This makes some sense because Windows and Linux have different shells (historically).

You can try the following code, which should work on both platforms:

   command = [
      "svn", "log",
      "--username", __SVNUSERNAME,
      "--password",  __SVNPASSWORD,
      "--no-auth-cache", "--non-interactive", "-v", 
      "-r", "{2022-09-26}:{2022-09-27}",
      "--xml",
      __SVNTRUNK
    ]
    
    bytes = subprocess.check_output(command, stderr=subprocess.STDOUT)

In short, this version no longer relies on the shell to tokenize the command string into a command array, but instead explicitly builds that array. Notice that shell=True has been removed from the call to check_output() .

This change can have gotchas, notably that bash or command.com won't do any shell-unique processing of the string (eg transliterating environmental variables). It doesn't look like your previous code was relying on shell behavior (other than tokenizing the command line) So this is a good change since you are supporting two different environments.