I have my mongodb server sitting on machine with fedora31. Most of my clients are running from fedora27.
I see that those client are much slower in fetching data. Example code:
from pymongo import MongoClient
MONGO_HOST = 'XXX'
client = MongoClient(MONGO_HOST, 27017, connect=True)
collection = client.my_database.my_collection
result = collection.find_one(my_key)
Running from fedora31 to fedora31:
> time python3 slow_mongo_db_debug.py
real 0m0.798s
user 0m0.257s
sys 0m0.078s
Running from fedora27 to fedora31:
> time python3 slow_mongo_db_debug.py
real 0m12.633s
user 0m0.270s
sys 0m0.082s
What can be the reason for this massive difference in execution speed?
EDIT I've added some monitoring . Updated code:
from pymongo import MongoClient, monitoring
import logging
logging.basicConfig(format='%(asctime)s %(levelname)s:%(message)s', level=logging.DEBUG)
class CommandLogger(monitoring.CommandListener):
def started(self, event):
logging.info("Command {0.command_name} with request id "
"{0.request_id} started on server "
"{0.connection_id}".format(event))
def succeeded(self, event):
logging.info("Command {0.command_name} with request id "
"{0.request_id} on server {0.connection_id} "
"succeeded in {0.duration_micros} "
"microseconds".format(event))
def failed(self, event):
logging.info("Command {0.command_name} with request id "
"{0.request_id} on server {0.connection_id} "
"failed in {0.duration_micros} "
"microseconds".format(event))
class ServerLogger(monitoring.ServerListener):
def opened(self, event):
logging.info("Server {0.server_address} added to topology "
"{0.topology_id}".format(event))
def description_changed(self, event):
previous_server_type = event.previous_description.server_type
new_server_type = event.new_description.server_type
if new_server_type != previous_server_type:
# server_type_name was added in PyMongo 3.4
logging.info(
"Server {0.server_address} changed type from "
"{0.previous_description.server_type_name} to "
"{0.new_description.server_type_name}".format(event))
def closed(self, event):
logging.warning("Server {0.server_address} removed from topology "
"{0.topology_id}".format(event))
class TopologyLogger(monitoring.TopologyListener):
def opened(self, event):
logging.info("Topology with id {0.topology_id} "
"opened".format(event))
def description_changed(self, event):
logging.info("Topology description updated for "
"topology id {0.topology_id}".format(event))
previous_topology_type = event.previous_description.topology_type
new_topology_type = event.new_description.topology_type
if new_topology_type != previous_topology_type:
# topology_type_name was added in PyMongo 3.4
logging.info(
"Topology {0.topology_id} changed type from "
"{0.previous_description.topology_type_name} to "
"{0.new_description.topology_type_name}".format(event))
# The has_writable_server and has_readable_server methods
# were added in PyMongo 3.4.
if not event.new_description.has_writable_server():
logging.warning("No writable servers available.")
if not event.new_description.has_readable_server():
logging.warning("No readable servers available.")
def closed(self, event):
logging.info("Topology with id {0.topology_id} "
"closed".format(event))
MONGO_HOST = 'XXX'
logging.info('Start test')
client = MongoClient(MONGO_HOST, 27017, connect=True, event_listeners=[CommandLogger(), ServerLogger(), TopologyLogger()])
my_collection = client.my_db.my_collection
result = my_collection.find_one(my_key)
Here's the result: Fast:
$ time python3 slow_mongo_db_debug.py
2020-08-24 12:07:43,562 INFO:Start test
2020-08-24 12:07:43,564 INFO:Topology with id 5f43835f98b10083af58e8d7 opened
2020-08-24 12:07:43,565 INFO:Topology description updated for topology id 5f43835f98b10083af58e8d7
2020-08-24 12:07:43,565 INFO:Topology 5f43835f98b10083af58e8d7 changed type from Unknown to Single
2020-08-24 12:07:43,565 WARNING:No writable servers available.
2020-08-24 12:07:43,565 WARNING:No readable servers available.
2020-08-24 12:07:43,566 INFO:Server ('XXX', 27017) added to topology 5f43835f98b10083af58e8d7
2020-08-24 12:07:43,574 INFO:Command find with request id 1681692777 started on server ('XXX', 27017)
2020-08-24 12:07:43,684 INFO:Command find with request id 1681692777 on server ('XXX', 27017) succeeded in 109556 microseconds
real 0m0.776s
user 0m0.269s
sys 0m0.057s
Slow:
$ time python3 slow_mongo_db_debug.py
2020-08-24 12:07:59,315 INFO:Start test
2020-08-24 12:07:59,316 INFO:Topology with id 5f43836f85d65e47d21ad426 opened
2020-08-24 12:07:59,316 INFO:Topology description updated for topology id 5f43836f85d65e47d21ad426
2020-08-24 12:07:59,317 INFO:Topology 5f43836f85d65e47d21ad426 changed type from Unknown to Single
2020-08-24 12:07:59,317 WARNING:No writable servers available.
2020-08-24 12:07:59,317 WARNING:No readable servers available.
2020-08-24 12:07:59,318 INFO:Server ('XXX', 27017) added to topology 5f43836f85d65e47d21ad426
2020-08-24 12:08:13,336 INFO:Server ('XXX', 27017) changed type from Unknown to Standalone
2020-08-24 12:08:13,337 INFO:Topology description updated for topology id 5f43836f85d65e47d21ad426
2020-08-24 12:08:13,337 INFO:Topology description updated for topology id 5f43836f85d65e47d21ad426
2020-08-24 12:08:18,387 INFO:Command find with request id 1714636915 started on server ('XXX', 27017)
2020-08-24 12:08:18,478 INFO:Command find with request id 1714636915 on server ('XXX', 27017) succeeded in 90880 microseconds
real 0m19.762s
user 0m0.192s
sys 0m0.076s
EDIT
Added heartbeat monitoring:
$ time python3 slow_mongo_db_debug.py
2020-08-26 12:06:14,801 INFO:Start test
2020-08-26 12:06:14,802 INFO:Topology with id 5f462606670b6643437d2022 opened
2020-08-26 12:06:14,803 INFO:Topology description updated for topology id 5f462606670b6643437d2022
2020-08-26 12:06:14,804 INFO:Topology 5f462606670b6643437d2022 changed type from Unknown to Single
2020-08-26 12:06:14,804 WARNING:No writable servers available.
2020-08-26 12:06:14,804 WARNING:No readable servers available.
2020-08-26 12:06:14,804 INFO:Server ('XXX', 27017) added to topology 5f462606670b6643437d2022
2020-08-26 12:06:14,804 INFO:Heartbeat sent to server ('XXX', 27017)
2020-08-26 12:06:19,747 INFO:Heartbeat to server ('XXX', 27017) succeeded with reply {'ismaster': True, 'maxBsonObjectSize': 16777216, 'maxMessageSizeBytes': 48000000, 'maxWriteBatchSize': 100000, 'localTime': datetime.datetime(2020, 8, 26, 9, 6, 19, 747000), 'logicalSessionTimeoutMinutes': 30, 'connectionId': 9503401, 'minWireVersion': 0, 'maxWireVersion': 8, 'readOnly': False, 'ok': 1.0}
2020-08-26 12:06:19,810 INFO:Server ('XXX', 27017) changed type from Unknown to Standalone
2020-08-26 12:06:19,810 INFO:Topology description updated for topology id 5f462606670b6643437d2022
2020-08-26 12:06:20,248 INFO:Heartbeat sent to server ('XXX', 27017)
2020-08-26 12:06:20,250 INFO:Heartbeat to server ('XXX', 27017) succeeded with reply {'ismaster': True, 'maxBsonObjectSize': 16777216, 'maxMessageSizeBytes': 48000000, 'maxWriteBatchSize': 100000, 'localTime': datetime.datetime(2020, 8, 26, 9, 6, 20, 250000), 'logicalSessionTimeoutMinutes': 30, 'connectionId': 9503401, 'minWireVersion': 0, 'maxWireVersion': 8, 'readOnly': False, 'ok': 1.0}
2020-08-26 12:06:20,812 INFO:Topology description updated for topology id 5f462606670b6643437d2022
2020-08-26 12:06:25,765 INFO:Command find with request id 1714636915 started on server ('XXX', 27017)
2020-08-26 12:06:25,766 INFO:Command find with request id 1714636915 on server ('XXX', 27017) succeeded in 1094 microseconds
real 0m11.732s
user 0m0.126s
sys 0m0.064s
Use monitoring to see what the driver is doing.
find
is published the same amount of time from process start in both scenarios. This will tell you whether time is spent on server discovery or the operation itself. You should also be receiving other command started events for auth & such.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.