[英]Connection reset by Peer pymongo
I have some documents which I have to fetch from mongodb and set it to memcache. 我有一些文件,我必须从mongodb获取并将其设置为memcache。 Here is the code
这是代码
import memcache
from pymongo import MongoClient
db = mongo_client.job_db.JobParsedData
jobs = db.find().sort("JobId", 1)
def set_to_memcache_raw(jobs):
print("Setting raw message to memcache")
count = 0
for item in jobs:
job_id = item.get('JobId')
job_details = item.get('JobDetails')
if job_id.strip():
count += 1
memcache_obj.set(job_id, job_details, time=72000)
if count % 1000 == 0:
print("Inserted {} keys in memcache".format(count))
if count >= 1000000:
break
But, after some odd number of iterations the code throw this error - 但是,经过一些奇数次的迭代,代码抛出了这个错误 -
Traceback (most recent call last):
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 450, in receive_message
self.sock, operation, request_id, self.max_message_size)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/network.py", line 137, in receive_message
header = _receive_data_on_socket(sock, 16)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/network.py", line 164, in _receive_data_on_socket
chunk = sock.recv(length)
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "memcache-poc.py", line 56, in <module>
elapsed = time.time() - t0
File "memcache-poc.py", line 52, in main
jobs = db.find(query)
File "memcache-poc.py", line 17, in set_to_memcache_raw
print("Setting raw message to memcache")
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/cursor.py", line 1114, in next
if len(self.__data) or self._refresh():
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/cursor.py", line 1056, in _refresh
self.__max_await_time_ms))
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/cursor.py", line 873, in __send_message
**kwargs)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/mongo_client.py", line 905, in _send_message_with_response
exhaust)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/mongo_client.py", line 916, in _reset_on_error
return func(*args, **kwargs)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/server.py", line 136, in send_message_with_response
response_data = sock_info.receive_message(1, request_id)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 452, in receive_message
self._raise_connection_failure(error)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 550, in _raise_connection_failure
_raise_connection_failure(self.address, error)
File "/home/dimension/.virtualenvs/docparser/lib/python3.5/site-packages/pymongo/pool.py", line 211, in _raise_connection_failure
raise AutoReconnect(msg)
pymongo.errors.AutoReconnect: xxx.xxx.xxx.xxx:27017: [Errno 104] Connection reset by peer
I have gone through links such as 我已经通过了诸如此类的链接
mongodb-TCP keep-alive mongodb-TCP keep-alive
why-does-pymongo-throw-autoreconnect 为什么此结果pymongo掷,autoReconnect的
Their is no question of socket inactivity in the above code as my jobs object is an iterator and every time next() is called on this object it will fetch the next document (from mongo itself) 它们不是上面代码中的套接字不活动的问题,因为我的jobs对象是一个迭代器,并且每次在这个对象上调用next()时它将获取下一个文件(来自mongo本身)
I have mongodb installed on Azure cloud and my TCP keep alive for the instance is 7200 seconds. 我在Azure云上安装了mongodb,我的TCP保持活动为7200秒。 I get this figure by firing this command
我通过触发此命令获得此数字
sysctl net.ipv4.tcp_keepalive_time
7200
Do having a try cacth block over the for loop helps in this case 在这种情况下,在for循环上尝试cacth块有帮助
According to the information below for the issue from the pymongo
api document here , the exception need to be handled manually. 根据下面对从问题的信息
pymongo
API文档在这里 ,除了需要手动处理。
exception pymongo.errors.AutoReconnect(message='', errors=None)
Raised when a connection to the database is lost and an attempt to auto-reconnect will be made.
在与数据库的连接丢失并尝试自动重新连接时引发。
In order to auto-reconnect you must handle this exception, recognizing that the operation which caused it has not necessarily succeeded.
为了自动重新连接,您必须处理此异常,并认识到导致它的操作未必成功。 Future operations will attempt to open a new connection to the database (and will continue to raise this exception until the first successful connection is made).
将来的操作将尝试打开与数据库的新连接(并将继续引发此异常,直到第一次成功建立连接)。
I searched some existing solution for the issue, please see below. 我搜索了一些现有的解决方案,请看下面的内容。
AutoReconnect
exception. AutoReconnect
异常。 MongoDBProxy
to automatically handle AutoReconnect
exception. MongoDBProxy
的中间件自动处理AutoReconnect
异常。 Hope it helps. 希望能帮助到你。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.