Can I use uwsgi + (tornado, gevent, etc) at the same time?

Question

why? because I have a django project that capture data from user and consume many webservices displaying the results to the user in order to compare information, something like aggregators websites who search flights tickets via airlines webservices and show the result in real time in order to compare tickets.

nowaday im doing this in a "waiting page", where celery hits webservices while jquery is asking every 5 seconds if all results are ready, so when ready redirect to the results page.

what I want to do is not to use this "waiting page", I want to feed the results page in real time as the results are comming, and I want to make it following the best practices, I mean I dont want to jquery get the results each X seconds to feed the table.

I think some coroutine-based python library can help me with this, but I want to learn more about your experience first and see some examples, I am confused because this part of the project was designed to run asynchronously, I mean, consuming webservices with celery-chords, but not designed for dispatching the information in real time through the app server.

actual architecture: python 2.7, django 1.3, postgresql 9, celery 3 + redis, uwsgi, nginx, all hosted on aws.

thank you in advance.

Answer 1

uWSGI+gevent is a solid combo, while there is currently no-way to run uWSGI with the tornado api (and as uWSGI dropped support in 1.9 for callback based approach, i think that we will never see that combo working).

The problem you need to solve before starting working with gevent, is ensuring that all of your pieces are gevent friendly (redis and celery are ok, you need to check your database adapter). After that simply add --gevent to your uWSGI instance, where is the maximum number of concurrent requests per worker.

Answer 2

I don't know about uWSGI+gevent, but you can use tornado with uWSGI. Tornado basically gives you an inbuilt WSGI support in tornado.wsgi.WSGIContainer module to make it compactable with other WSGI servers like uWSGI and gunicorn.But it depends on your use and I think it's not a good idea to use an Asynchronous framework with a synchronous server(like uWSGI). Tornado has this warning for this.

WSGI is a synchronous interface, while Tornado's concurrency model is based on single-threaded asynchronous execution. This means that running a WSGI app with Tornado's WSGIContainer is less scalable than running the same app in a multi-threaded WSGI server like gunicorn or uwsgi. Use WSGIContainer only when there are benefits to combining Tornado and WSGI in the same process that outweigh the reduced scalability.