Interactive Python Client/Server with Twisted

Question

I've been trying to wrap my mind around how to get Twisted to perform, for lack of a better word, "interactive" client/server behavior.

I managed to put together a pair of Protocol and ClientFactory classes that do connect to a service, and perform an immediate query/response (see: connectionMade -> self.queryStatus). This succeeds as expected and prints the server's response from the Factory class.

My problem now is that I'll have outside events that must cause data to be sent, while always listening for potential incoming data. But once the reactor.run() loop is going, I'm not sure how the rest of my application is meant to trigger a data send.

I've tried a few different approaches since, but this is the simplest approach that did handle the recv part as described:

class myListenerProtocol(LineReceiver):
    delimiter = '\n'

    def connectionMade(self):
        print("Connected to: %s" % self.transport.getPeer())
        self.queryStatus(1)

    def dataReceived(self, data):
        print("Receiving Data from %s" % self.transport.getPeer())
        ...
        self.commandReceived(self.myData)

    def commandReceived(self, myData):
        self.factory.commandReceived(myData)

    def connectionLost(self, reason):
        print("Disconnected.")

    def queryStatus(self, CommandValue):
        ...
        strSend = CommandValue # or some such
        self.transport.write(strSend)

class mySocketFactory(ClientFactory):
    protocol = myListenerProtocol

    def __init__(self):
        pass

    def buildProtocol(self, address):
        proto = ClientFactory.buildProtocol(self, address)
        return proto

    def commandReceived(self, myData):
        print myData
        reactor.stop() # It won't normally stop after recv

    def clientConnectionFailed(self, connector, reason):
        print("Connection failed.")
        reactor.stop()


def main():
    f = mySocketFactory()
    reactor.connectTCP("10.10.10.1", 1234, f)
    reactor.run()

I imagine this is pretty straight-forward, but countless hours into numerous examples and documentation have left me without a good understanding of how I'm meant to deal with this scenario.

Answer 1

My problem now is that I'll have outside events that must cause data to be sent, while always listening for potential incoming data. But once the reactor.run() loop is going, I'm not sure how the rest of my application is meant to trigger a data send.

"Outside events"? Like what? Data arriving on a connection? Great, having the reactor running means you'll actually be able to handle that data.

Or maybe someone is clicking a button in a GUI? Try one of the GUI integration reactors - again, you can't handle those events until you have a reactor running.

You're probably getting stuck because you think your main function should do reactor.run() and then go on to do other things . This isn't how it works. When you write an event-driven program, you define all of your event sources and then let the event loop call your handlers when events arrive on those sources.

Answer 2

Well, there are many approaches to that, and the best one really depends on the context of your application, so I won't detail you one way of doing this here, but rather link you to a reading I had recently on hacker's news:

• http://www.devmusings.com/blog/2013/05/23/python-concurrency/

and good use-case example, though it may not apply to what you're working on (or you may have read it):

• http://eflorenzano.com/blog/2008/11/17/writing-markov-chain-irc-bot-twisted-and-python/

BTW, you may also have a look at gevent or tornado that are good at handling that kind of things.

If your other "events" are from a GUI toolkit (like GTK or QT) be really careful of the GIL , and even if you just want command line events you'll need threads and still be careful of that.

Finally, if you want to make more interaction, you may as well write different kind of "peers" for your server, that interacts with the different use cases you're working on (one client that connects to a GUI, another with a CLI, another with a database, another with a SAAS' API etc..).

In other words, if your design is not working, try changing your perspective!