Subprocess.Popen wait untill the child finish

Question

I am developing a web tool based on CGI, Apache2, python (2.76). The web server is on Ubuntu virtual machine. After users submit a job, I want the server return a web page immediately, showing the link to the upcoming results. Because the calculation will take some time, so I used Subprocess.Popen to call a function, which do all calculation. Below show the flow of my codes. I tested on my own (Mac) computer, it can return a web page immediately, before the calculation finish. But on the Ubuntu virtual machine, it return the page untill the calculation finish. Doesn't Subprocess.Popen work on Ubuntu?

Main()
   preprocess 
   return_a_wen_page
   Subprocess.Popen(function_do_calculation)

Answer 1

subprocess.Popen works on Ubuntu. The issue might be that you are calling it from Apache. You could try to redirect the standard streams:

with open(os.devnull, 'r+b', 0) as DEVNULL:
    subprocess.Popen(cmd, stdin=DEVNULL, stdout=DEVNULL, stderr=DEVNULL,
                     close_fds=True)

If it doesn't help, add also preexec_fn=os.setsid ( start_new_session=True analog). Or just send a message ((unix) socket, named pipe) to independently running daemon on the machine that can call subprocess.Popen freely instead of the cgi script (consider if something like celery may streamline it for you).

Answer 2

It's the first time for me to do a web application, I don't know mod_wsgi.

with open(os.devnull, 'r+b', 0) as DEVNULL:
      subprocess.Popen(cmd, stdin=DEVNULL, stdout=DEVNULL, stderr=DEVNULL,close_fds=True)