dynamic routing , tornado deployment in production

Question

I have 4 python tornado threads running on different ports on different machines. I used nginx to route and load balance. The code is the same for all of them. It is a asynchronous code. I also have a local file, lets say function.py on each machine that gets called by the python thread, does some computation and returns the answer.

My requirement is that I may need to periodically update the function.py file. However, I do not want the server to be stopped to reload the function since I don't want to drop any incoming request . I am open to changing nginx to something else if required. Any suggestions will be appreciated. Thanks!

Edit: Could there be a way to modify/configure the nginx in such a way that it will redirect to certain servers(say port 8011-8014) only when they are up ? In that case I can modify the main python threads and then gracefully shut down port 8011-8014. But is this type of configuration feasible ?

Answer 1

When updating your python code you can start a new set of python processes on different ports (say 8001-8004 for the previous set and 8011-8014 for the new set) then modify you nginx config to redirect to 8011-8014 instead 8001-8004 and run service nginx reload (or equivalent for your OS).

This way, nginx will redirect new requests to the new processes without dropping any request and finishing pending ones on the previous processes. When you know that all pending requests to the old set of python processes are finished (that might be non trivial) you can stop them.