Tweepy OpenSSL.SSL.WantReadError

Question

蟒蛇 3.6。 我使用 tweepy 的流光来获取推文。 它运作良好。 但有时，如果我让它打开超过 24 小时，就会出现此错误

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 277, in recv_into
return self.connection.recv_into(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1547, in recv_into
self._raise_ssl_error(self._ssl, result)
    File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1353, in _raise_ssl_error
raise WantReadError()
  OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
 File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 277, in recv_into
return self.connection.recv_into(*args, **kwargs)
    File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1547, in recv_into
self._raise_ssl_error(self._ssl, result)
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1370, in _raise_ssl_error
raise SysCallError(errno, errorcode.get(errno))
OpenSSL.SSL.SysCallError: (10054, 'WSAECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 302, in _error_catcher
yield
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 384, in read
data = self._fp.read(amt)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 449, in read
n = self.readinto(b)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 483, in readinto
return self._readinto_chunked(b)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 578, in _readinto_chunked
chunk_left = self._get_chunk_left()
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 546, in _get_chunk_left
chunk_left = self._read_next_chunk_size()
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 506, in _read_next_chunk_size
line = self.fp.readline(_MAXLINE + 1)
  File "C:\ProgramData\Anaconda3\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 293, in recv_into
return self.recv_into(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 282, in recv_into
raise SocketError(str(e))
OSError: (10054, 'WSAECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
 File "twitter_aspi_v0.8.py", line 179, in _init_stream
tweepy.Stream(auth, listener).userstream()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 396, in userstream
self._start(async)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 363, in _start
self._run()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 296, in _run
raise exception
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 265, in _run
self._read_loop(resp)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 315, in _read_loop
line = buf.read_line().strip()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 180, in read_line
self._buffer += self._stream.read(self._chunk_size)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 401, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 100, in __exit__
self.gen.throw(type, value, traceback)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 320, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: OSError("(10054, \'WSAECONNRESET\')",)', OSError("(10054, 'WSAECONNRESET')",))

我的代码很长，关于错误，它似乎来自 urllib3、OpenSSL 和访问 Twitter API 的 tweepy 方式。 所以我可以在启动流媒体之前try处理这个问题，但我想知道是否有更好的解决方法来理解和避免这种情况？ 谢谢！

Answer 1

这看起来更像是Tweepy不处理的临时连接超时，因此您应该只编写一个包装器并捕获异常并重新启动它。 我不认为可以避免异常，因为您连接到外部站点，有时它可能会超时

您应该查看此http://docs.tweepy.org/en/v3.5.0/streaming_how_to.html#handling-errors的错误处理部分，并查看在发生连接超时时是否on_error

class MyStreamListener(tweepy.StreamListener):

    def on_error(self, status_code):
        if status_code == 420:
            #returning False in on_data disconnects the stream
            return False

如果这没有帮助，则使用包装器方法

Answer 2

根据 Twitter 开发人员文档：速率限制预计会在您超过使用限制时导致连接重置/失败。

请求/15 分钟窗口（用户身份验证）= 900

请求数 / 15 分钟窗口（应用程序验证）= 1500

它也清楚地说明如下。

如果初始重新连接尝试不成功，您的客户端应继续尝试使用指数退避模式重新连接，直到成功重新连接。

（更新）

无论您的客户端如何断开连接，您都应该将应用程序配置为立即重新连接。 如果您的第一次重新连接尝试不成功，我们建议您的应用在随后的重新连接尝试中实现指数退避模式（例如，等待 1 秒，然后是 2 秒，然后是 4、8、16 等），并有一些合理的上限。 如果达到此上限，您应该将您的客户端配置为通知您的团队，以便您可以进一步调查。

标准（免费）Twitter API，即 Tweepy API 由一个 REST API 和一个流 API 组成。 Streaming API 提供对推文的低延迟访问。 Ads API 在列入白名单时还有其他限制。

REST API 限制

客户端可以通过 user_timeline REST API 方法的 page 和 count 参数访问理论上最多 3,200 个状态。 其他时间线方法理论上最多有 800 个状态。 超过限制的请求将导致状态代码为 200 的回复和请求格式的空结果。 Twitter 仍然维护着一个包含用户发送的所有推文的数据库。 但是，为了确保性能，对 API 调用实施了此限制。

可能是用户不应该发送垃圾邮件的简单原因，即强制执行此操作。

解决方案：

您可以捕获exception并重新建立与 Twitter 的连接并继续阅读推文。

不幸的是，目前别无选择，只能从 Twitter 获得更好的使用许可。

Answer 3

Python 3.6。 我使用tweepy的流光来获取推文。 它运作良好。 但是有时候，如果我将其打开超过24小时，则会出现此错误

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 277, in recv_into
return self.connection.recv_into(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1547, in recv_into
self._raise_ssl_error(self._ssl, result)
    File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1353, in _raise_ssl_error
raise WantReadError()
  OpenSSL.SSL.WantReadError

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
 File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 277, in recv_into
return self.connection.recv_into(*args, **kwargs)
    File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1547, in recv_into
self._raise_ssl_error(self._ssl, result)
  File "C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py", line 1370, in _raise_ssl_error
raise SysCallError(errno, errorcode.get(errno))
OpenSSL.SSL.SysCallError: (10054, 'WSAECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 302, in _error_catcher
yield
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 384, in read
data = self._fp.read(amt)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 449, in read
n = self.readinto(b)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 483, in readinto
return self._readinto_chunked(b)
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 578, in _readinto_chunked
chunk_left = self._get_chunk_left()
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 546, in _get_chunk_left
chunk_left = self._read_next_chunk_size()
  File "C:\ProgramData\Anaconda3\lib\http\client.py", line 506, in _read_next_chunk_size
line = self.fp.readline(_MAXLINE + 1)
  File "C:\ProgramData\Anaconda3\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 293, in recv_into
return self.recv_into(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\contrib\pyopenssl.py", line 282, in recv_into
raise SocketError(str(e))
OSError: (10054, 'WSAECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
 File "twitter_aspi_v0.8.py", line 179, in _init_stream
tweepy.Stream(auth, listener).userstream()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 396, in userstream
self._start(async)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 363, in _start
self._run()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 296, in _run
raise exception
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 265, in _run
self._read_loop(resp)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 315, in _read_loop
line = buf.read_line().strip()
  File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\streaming.py", line 180, in read_line
self._buffer += self._stream.read(self._chunk_size)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 401, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 100, in __exit__
self.gen.throw(type, value, traceback)
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\packages\urllib3\response.py", line 320, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: OSError("(10054, \'WSAECONNRESET\')",)', OSError("(10054, 'WSAECONNRESET')",))

我的代码很长，关于错误，它似乎来自urllib3，OpenSSL和访问Twitter API的扭曲方式。 因此，在启动拖缆之前，我可以try ，但是我想知道是否有更好的解决方法来理解和避免这种情况？ 谢谢！