Python: Sharing data between instances

Question

I am currently running a python application using Tornado Webserver. I use Supervisord to launch 4 Tornado instances on different ports. In front of these I am running Nginx for load balancing between the servers.

The problem is that I have implemented long-polling which means that I need for data to be shared between the different Tornado instances.

As far as I understand, when a user sends a request he gets connected to one of the tornado instances, but the callback is only saved in that single instance and data getting pushed will only be pushed to callbacks on a single instance.

How do I share the callbacks between the instances? I have been playing around with Redis, the problem here is how I save the callbacks in the key-store? Is there an easier way? Or how do I do it in Redis?

I have been looking at this thread but they don't really give me the full answer:

Sharing data between multiple tornado instances

Thanks!

Answer 1

You can't put the callbacks in redis. Instead, you put the data in redis, and each process reads that data and passes it along to its own callbacks.