当使用python脚本作为CGI时，Subprocess Popen无法运行后台进程

Question

I am trying to run another Python script in background from a CGI python script and want this script to run the process in background without waiting for the other script to complete. Somehow when I am running the same from Linux shell, I can run the other python script in background. But when I tried doing the same through CGI, The front end keeps on loading until the other script completes and not just make it run in background.

I have tried running the same on the Linux Shell and it works. When I shifted to CGI that is when the script waits for the other process to complete.

python1.py:

command = [sys.executable,'python2.py', str(senddata)]
proc=subprocess.Popen(command,shell=False,stdin=None,stdout=None,stderr=None,close_fds=True)

print("Content-type: text/html\r\n\r\n")

print("The script is running in background! Expect an email in 10 minutes.")

python2.py: This script takes 2-5 minutes to execute and then sends an email to the group.

The expected output is to have this message:

The script is running in background! Expect an email in 10 minutes.

And run python2.py in background without waiting for it to complete.

Answer 1

The webserver will keep the client response active (causing the client to stay in a "loading" state) until all of the output from the CGI program has been collected and forwarded to the client. The way the webserver knows that all output has been collected is that it sees the standard output stream of the CGI process being closed. That normally happens when the CGI process exits.

The reason why you're having this problem is that when subprocess.Popen is told to execute a program with stdout=None , the spawned program will share its parent's standard output stream. Here that means that your background program shares the CGI process's standard output. That means that from the webserver's point of view that stream remains open until both of the processes exit.

To fix, launch the background process with stdout=subprocess.PIPE . If the background process misbehaves if its stdout gets closed when the CGI process dies, try launching it with stdout=open('/dev/null') instead.

The background process's stdin and stderr will have the same issue; they will be shared with the CGI process. AFAIK sharing those will not cause trouble as long as the background process does not attempt to read from its standard input, but if it does do that (or if you just want to be cautious) you can treat them the same way as you treat stdout and either set them to subprocess.PIPE or associate them with /dev/null .

More detail is at https://docs.python.org/2/library/subprocess.html#popen-constructor