Non-Blocking Server Apache Thrift Python

Question

In one Python Module AI am doing some stuff. In the middle of doing that stuff I am creating a Thrift connection. The problem is after that connection starts, the program gets stuck in the network logic. (ie blocking).

In module AI have:

stuff = "do some stuff"
network.ConnectionManager(host, port, ...)
stuff = "do more stuff" # not getting to this point

In network...

ConnectionManager.start_service_handler()
def start_service_handler(self):
        handler = ServiceHandler(self)
        processor = Service.Processor(handler)
        transport = TSocket.TServerSocket(port=self.port)
        tfactory = TTransport.TBufferedTransportFactory()
        pfactory = TBinaryProtocol.TBinaryProtocolFactory()
        # server = TServer.TThreadedServer(processor, transport, tfactory, pfactory)
        server = TNonblockingServer(processor, transport, tfactory, pfactory)
        logger().info('starting server...')
        server.serve()

I try this, but yet the code in module A does not continue as soon as the connection code starts.

I thought TNonblockingServer would do the trick, but unfortunately did not.

Answer 1

The code blocks at server.serve() which is by design, across all target languages supported by Thrift. The usual use case is to run a server like this (pseudo code):

init server
setup thrift protocol/tramsport stack
server.serve()
shutdown code

The "nonblocking" does not refer to the server.serve() call, rather to the code taking the actual client call. With a TSimpleServer , the server can only handle one call at a time. In contrast, the TNonblockingServer is designed to accept a number of connections in parallel .

Conclusion: If you want to run a Thrift server and also have some other work to do in parallel, or need to start and stop the server on the fly during program run, you will need another thread to achieve that.