Fabric in Django .How to solve view waiting for SSH to complete

Question

I have a simple view:

def test(request):
 os.system('fab remote_uname -i /path/to/keyfile -H hostname')
 return HttpResposnse('Complete')

In fabfile.py :

from fabric.api import *
env.user='ubuntu'
def remote_uname():
 run('uname -a')

When I hit the url it waits long to deliver the response. This may confuse user to hit refresh executing remote_uname() again. How can I deliver the view and let the operations complete later? How can I implement concurrency in this case?

Answer 1

for simple cases you can use Threads, however you have to deal with failures after delivering the view. That might be a bit tricky, are you interested in the return value at all?

def ssh_connect():
 os.system('fab remote_uname -i /path/to/keyfile -H hostname')    

def test(request):
 threading.Thread(target=ssh_connect, args=()).start()
 return HttpResposnse('Complete')

Answer 2

Threads/processes concurrency might need throtlling

You have several options, like concurrency on a thread/process level eg with Python's multiprocessing module. But going this route I fear you might be exposed to spawning more threads/processes than your hardware can handle, unless you throttle your Django view or something to limit the rate.

It gets the job done, very simple and your view will return immediately, but you risk being overloaded, and you manage results and errors manually.

Task queues

Otherwise, you can implement this as a queued task, like Celery * eg, which will queue up the tasks, run them in one by one or at a concurrency level you set. Another plus is the result in case of successs, and failures are logged, so you can return them back to the user if you wish to.

_{* A task queue like Celery might look complicated to setup but it's not, and has solid support for use from within Django too.}