Python WSGI deployment on Windows for CPU-bound application

Question

What options do I have for the deployment of a CPU bound Python-WSGI application on Windows?

The application benefits greatly from multiple CPUs (image manipulation/encoding) but the GIL prevents it from using them.

My understanding is:

• mod_wsgi has no support for WSGIDaemonProcess on Windows and Apache itself only runs with one process

• all fork based solutions (flup, spawning, gunicorn) only work on unix

Are there any other deployment options I'm missing?

PS: I asked that on serverfault but someone suggested to ask here.

Answer 1

I have successfully used isapi-wsgi to deploy WSGI web apps on Windows IIS (I assume that since you're deploying on Windows, IIS is an option).

Create an IIS Application Pool to host your application in and configure it as a Web Garden (Properties | Performance | Maximum number of worker processes).

Disclaimer: I have never used this feature myself (I have always used the default App Pool configuration, where Max. number of worker processes is 1). But it is my understanding that this will spin up more processes to handle requests.

Answer 2

It would be a bit of a mess, but you could use the subprocess module to fire off worker processes yourself. I'm pretty sure that Popen.wait() and/or Popen.communicate() ought to release the GIL. You've still got the process creation overhead though, so you might not gain a lot/anything over standard CGI.

Another option is to have separate server/worker processes running the whole time and use some form of IPC, although this isn't going to be an easy option. Have a look at the multiprocessing module, and potentially also Pyro.