Calling subprocess.call hangs when I set the stdout

Question

I have a function which I call a progarm, with some args and want to get the result.

When I use the following

proc = subprocess.call(["fetch.py", "--cookies=/tmp/tmp-cookies"], stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
return stdout

The app just hangs. But if I run

return subprocess.call(["fetch.py", "--cookies=/tmp/tmp-cookies"])

then I get the output on my screen and the app works fine, however I need to get the output into a function.

I am using python 2.6.1, and unable to use check_output

Answer 1

As the spec says,

Do not use stdout=PIPE or stderr=PIPE with this function. As the pipes are not being read in the current process, the child process may block if it generates enough output to a pipe to fill up the OS pipe buffer.

What you need instead, is subprocess.Popen :

proc = subprocess.Popen(["fetch.py", "--cookies=/tmp/tmp-cookies"], 
                       stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
return stdout

(Also, subprocess.call does not return the process object, only exit status)

Answer 2

Your subprocess must read its stdin before the communication is complete, and the main process can continue. The workaround would be write out in a thread.