具有流式传输功能的Python JSON RPC服务器

Question

I have come across several guides and packages on implementing a python JSON RPC server, eg:

• http://json-rpc.org/wiki/python-json-rpc
• http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/552751
• http://pythonpaste.org/webob/jsonrpc-example.html

They all do a good job in the sense that the server/application implementation is very simple, you just return the python object as a result and the framework takes care of serializing it. However, this is not suitable for my needs mainly because I am looking forward to serializing possibly thousands of records from database and such a solution would require me to create a single python object containing all the records and return that as the result.

The ideal solution I am looking for would involve a framework that would provide the application a stream to write the response to and a JSON encoder that could encode an iterator (in this case a cursor from pyodbc) on the fly, something like this:

def process(self, request, response):
  // retrieve parameters from request.

  cursor = self.conn.cursor()
  cursor.execute(sql) // etc.

  // Dump the column descriptions and the results (an iterator)
  json.dump(response.getOut(), [cursor.description, cursor])

Can someone point me to a server framework that can provide me a stream to write to and a json serialization framework that can handle an iterable such as the pyodbc cursor and serialize it on the fly.

Answer 1

如果典型的JSON-RPC框架不允许你有效地转储如此庞大的数据，为什么不只是使用HTTP服务器并返回json数据，这样你就可以流式传输和读取流式数据，好的一点是你甚至可以为它进行gzip更快的传输，你也可以使用许多标准服务器，例如apache。