組合龍卷風和zmq ioloop：通過對等異常重置連接

Question

當我嘗試按照pyzmq官方文檔中所述將Tornado和pyzmq ioloops組合在一起時，我遇到了一個煩人的問題（盡管不是很關鍵）。

我有一個運行龍卷風（T）服務器的進程，該進程接受來自客戶端（C）的REST API請求，並通過ZMQ傳輸到實際工作的另一個進程（Z）來代理它們。

C <-> T <-> Z

如果C 在 Z回復T 之前關閉了連接，則Z（龍卷風）會輸出一堆異常跟蹤（請參閱底部）。 想象下面的例子：

import tornado.ioloop
from tornado.web import Application, RequestHandler, asynchronous
from zmq.eventloop import ioloop
import time

def time_consuming_task():
    time.sleep(5)

class TestHandler(RequestHandler):
    def get(self, arg):
        print "Test arg", arg
        time_consuming_task()
        print "Ok, time to reply"
        self.write("Reply")

if __name__ == "__main__":
    app = tornado.web.Application(
        [
            (r"/test/([0-9]+)", TestHandler) 
        ])

    ioloop.install()
    app.listen(8080)
    tornado.ioloop.IOLoop.instance().start()

這個示例實際上並沒有與任何ZMQ對等方交談，它只是將pyzmq ioloop附加到龍卷風的ioloop中。 雖然，足以說明問題。

從控制台一台運行服務器：

% python example.py

從控制台兩個運行客戶端，並在服務器回復之前中斷客戶端（即在5秒鍾內）：

% curl -is http://localhost:8080/test/1
^C 

服務器的輸出為：

Test arg 1
Ok, time to reply
WARNING:root:Read error on 24: [Errno 54] Connection reset by peer
ERROR:root:Uncaught exception GET /test/1 (::1)
HTTPRequest(protocol='http', host='localhost:8080', method='GET', uri='/test/1', version='HTTP/1.1', remote_ip='::1', body='', headers={'Host': 'localhost:8080', 'Accept': '*/*', 'User-Agent': 'curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5'})
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1023, in _execute
    self.finish()
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 701, in finish
    self.request.finish()
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 433, in finish
    self.connection.finish()
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 187, in finish
    self._finish_request()
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 223, in _finish_request
    self.stream.read_until(b("\r\n\r\n"), self._header_callback)
  File "/Library/Python/2.7/site-packages/tornado/iostream.py", line 153, in read_until
    self._try_inline_read()
  File "/Library/Python/2.7/site-packages/tornado/iostream.py", line 386, in _try_inline_read
    if self._read_to_buffer() == 0:
  File "/Library/Python/2.7/site-packages/tornado/iostream.py", line 421, in _read_to_buffer
    chunk = self._read_from_socket()
  File "/Library/Python/2.7/site-packages/tornado/iostream.py", line 402, in _read_from_socket
    chunk = self.socket.recv(self.read_chunk_size)
error: [Errno 54] Connection reset by peer
ERROR:root:Cannot send error response after headers written
ERROR:root:Uncaught exception, closing connection.
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/tornado/iostream.py", line 304, in wrapper
    callback(*args)
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 262, in _on_headers
    self.request_callback(self._request)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1412, in __call__
    handler._execute(transforms, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1025, in _execute
    self._handle_request_exception(e)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1065, in _handle_request_exception
    self.send_error(500, exc_info=sys.exc_info())
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 720, in send_error
    self.finish()
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 700, in finish
    self.flush(include_footers=True)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 660, in flush
    self.request.write(headers + chunk, callback=callback)
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 429, in write
    self.connection.write(chunk, callback=callback)
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 177, in write
    assert self._request, "Request closed"
AssertionError: Request closed
ERROR:root:Exception in callback 
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pyzmq-2.2.0-py2.7-macosx-10.7-intel.egg/zmq/eventloop/ioloop.py", line 434, in _run_callback
    callback()
  File "/Library/Python/2.7/site-packages/tornado/iostream.py", line 304, in wrapper
    callback(*args)
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 262, in _on_headers
    self.request_callback(self._request)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1412, in __call__
    handler._execute(transforms, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1025, in _execute
    self._handle_request_exception(e)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 1065, in _handle_request_exception
    self.send_error(500, exc_info=sys.exc_info())
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 720, in send_error
    self.finish()
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 700, in finish
    self.flush(include_footers=True)
  File "/Library/Python/2.7/site-packages/tornado/web.py", line 660, in flush
    self.request.write(headers + chunk, callback=callback)
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 429, in write
    self.connection.write(chunk, callback=callback)
  File "/Library/Python/2.7/site-packages/tornado/httpserver.py", line 177, in write
    assert self._request, "Request closed"
AssertionError: Request closed

注意：似乎是與pyzmq相關的問題，因為排除pyzmq ioloop后消失了。

服務器不會死，它可以被其他客戶端使用，因此問題並不嚴重。 但是，在日志文件中找到這些巨大的混亂痕跡非常煩人。

那么，有沒有知名的方法可以解決這個問題呢？ 謝謝。

Answer 1

這不是ZMQ問題。 可以出於超時以外的原因關閉請求。 ZMQ唯一的問題是它們引發了AssertionError ，這很常見，而不是更具體的異常。

如果您確定您不想在日志文件中包含這些異常，請執行以下操作：

try: 
    time_consuming_task()
except AssertionError as e:
    if e.message == 'Request closed': 
        logging.info('Bad, annoying client, came to us again!') 
    else:
        raise e