How can I catch errors from subprocess in a python cgi script?

Question

I'm trying to write a Python CGI script that will call sox (an audio processing program) using subprocess. The problem is that when I get errors from the sox call, everything crashes and I get a "malformed header" error from Apache.

The relevant bits:

def downsample(in_file, in_location, sox, out_location):
  """ run sox """
...
  sox = shlex.split(sox)
  retcode = subprocess.check_call(sox)
  if not retcode == 0:
    print '<b>something went wrong in sox: returned error code ' +\
          retcode + ', but we are continuing anyway...</b>'
  """p = subprocess.Popen(sox)
  if p.stdout:
    print '<b>something happened in sox: returned ' +\
          p.stdout.read() + ', but we will keep going...</b>'
  if p.stderr:
    print '<b>something happened in sox: returned ' +\
          p.stderr.read() + ', but we will keep going...</b>'"""
...

def main():
  print "Content-Type: text/html\n\n"

...
    downsample(in_file, in_location, command, out_location)
...

if __name__ == '__main__':
  main()

I'm using check_call to allow the cgi error handler to print the stack trace right now (to avoid the 500 page), but I'd really like to catch the error, handle it myself, and move on with the script. I tried doing that by wrapping the check_call in a try: except CalledProcessError: statement, but that just led to the 500 page again. That part that's commented out doesn't work for me, either.

And from /var/www/apache2/error_log:

Wed Apr 13 10:08:21 2011] [error] [client ::1] sox FAIL formats: can't open input file `/tmp/wavs/haha/f.wav': WAVE: RIFF header not found, referer: http://localhost/~Valkyrie_savage/
[Wed Apr 13 10:08:21 2011] [error] [client ::1] malformed header from script. Bad header=\x1f\x8b\b: downsample.py, referer: http://localhost/~Valkyrie_savage/

I can't understand why it seems to be running the sox command before the header prints. Or, if it is, why does it say the header is malformed?

Answer 1

import subprocess
p = subprocess.Popen(cmd)  
   p.wait()  
   if p.returncode:  
      print "failed with code: %s" % str(p.returncode) 

Answer 2

It sounds like you are using buffered output.

If that is the case, the output from the command is printed before the HTTP headers, so the web browser gets confused.

You can call sys.stdout.flush() before doing the system call, to be sure all the headers and html in the output buffer is actually printed.

Hope this helps.

Answer 3

You can use subprocess.check_call and catch the CalledProcessError exception.

try:
    retcode = subprocess.check_call(sox)
except CalledProcessError:
    print '<b>something went wrong in sox: returned error code ' +\
      retcode + ', but we are continuing anyway...</b>'