如何从运行在 ubuntu 上的 Apache 提供的 python 脚本生成一个长时间运行的 python 进程？

Question

Everything I've found so far is good ONLY IF if it's not being used in a web environment.

I have a simple python 'admin tool' web app running in Apache, and I'm using the simple configuration in the vhost :

AddHandler cgi-script .py.

There's no need for high performance, there's no python framework. Users select certain criteria which is sent via a GET request to the server & a python script then gets called by apache and the report is compiled. Problem is that the report complexity is growing, and the length of time it takes to compile this report can be > 20 minutes so I want to spawn a new totally independent process to generate the report asynchronously, allowing the server to return quickly to the client.

My issue is that I've tried absolutely everything I can find to be able to do this, and everything fails in the same way; when I spawn a new process the parent script will not return to the client until the new process has completed.

All of these do exactly the same - they start a new process BUT the main script waits for the child process to finish before returning to the client. I want the main script to return straight back to the client, and leave the newly spawned process running to compile the report.

os.system("""
cd /the_directory
python -m web.wait &
""")

command = ['python', '/the_directory/wait.py &']
Popen(command, shell=True, start_new_session=True)

L = ['python', '/the_directory/wait.py']
os.spawnvp(os.P_NOWAIT, 'python', L)

L = ['test.sh']
os.spawnvpe(os.P_NOWAIT, '/the_directory/test.sh', L, os.environ)


where test.sh contains: 

#!/bin/sh
/the_directory/wait.py &

jobs = []
for i in range(5):
    p = multiprocessing.Process(target=wait.dostuff, )
    jobs.append(p)
    p.start()

I want the simplest possible solution to this - it's an admin tool maintained only by me for a very few users.

If there's really no way to accomplish this then I guess I will have to use cron running once a minute and use a simple queue, maybe with redis or something, but this is not ideal.

I used to do this kind of thing relatively easily in php, I can't believe it's not possible in python.

Answer 1

You could use Popen with shell=False

example:

Popen(["bash", "test.sh"], shell=False)