Is there any way to make an asynchronous function call from Python [Django]?

Question

I am creating a Django application that does various long computations with uploaded files. I don't want to make the user wait for the file to be handled - I just want to show the user a page reading something like 'file is being parsed'.

How can I make an asynchronous function call from a view?

Something that may look like that:

def view(request):
    ...
    if form.is_valid():
        form.save()
        async_call(handle_file)
    return render_to_response(...)

Answer 1

Rather than trying to manage this via subprocesses or threads, I recommend you separate it out completely. There are two approaches: the first is to set a flag in a database table somewhere, and have a cron job running regularly that checks the flag and performs the required operation.

The second option is to use a message queue. Your file upload process sends a message on the queue, and a separate listener receives the message and does what's needed. I've used RabbitMQ for this sort of thing, but others are available.

Either way, your user doesn't have to wait for the process to finish, and you don't have to worry about managing subprocesses.

Answer 2

I have tried to do the same and failed after multiple attempt due of the nature of django and other asynchronous call.

The solution I have come up which could be a bit over the top for you is to have another asynchronous server in the background processing messages queues from the web request and throwing some chunked javascript which get parsed directly from the browser in an asynchronous way (ie: ajax).

Everything is made transparent for the end user via mod_proxy setting.

Answer 3

Unless you specifically need to use a separate process, which seems to be the gist of the other questions S.Lott is indicating as duplicate of yours, the threading module from the Python standard library (documented here ) may offer the simplest solution. Just make sure that handle_file is not accessing any globals that might get modified, nor especially modifying any globals itself; ideally it should communicate with the rest of your process only through Queue instances; etc, etc, all the usual recommendations about threading;-).

Answer 4

threading will break runserver if I'm not mistaken. I've had good luck with multiprocess in request handlers with mod_wsgi and runserver. Maybe someone can enlighten me as to why this is bad:

def _bulk_action(action, objs):
    # mean ponies here

def bulk_action(request, t):

    ...
    objs = model.objects.filter(pk__in=pks)

    if request.method == 'POST':
        objs.update(is_processing=True)

        from multiprocessing import Process
        p = Process(target=_bulk_action,args=(action,objs))
        p.start()

        return HttpResponseRedirect(next_url)

    context = {'t': t, 'action': action, 'objs': objs, 'model': model}
    return render_to_response(...)

http://docs.python.org/library/multiprocessing.html

New in 2.6