Twisted: how to wrap a function that takes a call back?

Question

I am trying to use twisted framework for some async programming task recently. One thing I do not quite understand is how to wrap a function which takes a call back function and make it an function that return a deferred object?

For example if I have a function like below:

def registerCallbackForData(callback):
   pass # this is a function that I do not control, some library code

And now the way I use it is to just register a callback. But I want to be able to incorporate this into the twisted framework, returning a deferred object probably and use reactor.run() later. Is this possible?

Answer 1

def convert_callback_to_deferred(f):
    def g():
        d = Deferred()
        d.addCallback(callback)
        f(d.callback)
        return d
    return g

from somelib import registerCallbackForData

getSomeDeferredForData = convert_callback_to_deferred(registerCallbackForData)

d = getSomeDeferredForData()
d.addCallback(...)
...

However, bear in mind that a Deferred can produce at most one result. If registerCallbackForData(cb) will result in cb being called more than once, there is nowhere for the 2nd and following calls' data to go. Only if you have an at-most-once event source does it make sense to convert it to the Deferred interface.

Answer 2

defer.maybeDeferred will wrap a blocking function call to return a deferred. If you want the call to be non-blocking you can instead use threads.deferToThread .

You can see the difference by switching which call is commented out:

import time
from twisted.internet import reactor, defer, threads


def foo():
    print "going to sleep"
    time.sleep(1)
    print "woke up"
    return "a result"


def show_result_and_stop_reactor(result):
    print "got result: %s, stopping the reactor" % result
    reactor.stop()


print "making the deferred"
d = defer.maybeDeferred(foo)
# d = threads.deferToThread(foo)

print "adding the callback"
d.addCallback(show_result_and_stop_reactor)

print "running the reactor"
reactor.run()