简体   繁体   中英

MongoDB, Mongoose - Slow query when fetching 10k+ documents

I have a MongoDB database with 10-12k documents in a collection and I'm experiencing really slow queries when attempting to fetch all documents, like this:

Sales.find()
    .where('author').equals(author)
    .where('date').gt(startDate.unix()).lt(endDate.unix())
    .exec(function(err, results) {
        callback();
    });

This query fetches around 10.5k documents and it takes 1000-1300ms to execute. I tried removing the "where" conditions - it only makes it slower (more documents fetched?).

Does the problem come from Mongoose, MongoDB, JavaScript or Node? I used to run PHP/MySQL database and it was 10-20 times faster in similar conditions, like fetching 10k+ rows of data. What am I doing wrong?

EDIT

Sales schema:

var salesSchema = new Schema({
    author: String,
    kind: String,
    productID: String,
    description: String,
    date: String,
    amount: String,
    transactionID: {
        type: String,
        unique : true
    }
});

Query result from the RoboMongo desktop client:

db.getCollection('sales').find({}).explain()

executionTimeMillis: 46
nReturned: 10359

The problem came from Mongoose. By default, find() will return documents as Mongoose Documents, which costs a lot. By adding lean() to the query, documents are returned as plain JavaScript objects and for this case of 10k+ returned documents, the query time got reduced 3-5 times .

Sales.find()
    .where('author').equals(author)
    .where('date').gt(startDate.unix()).lt(endDate.unix())
    .lean()
    .exec(function(err, results) {
        callback();
    });

Read more here: http://www.tothenew.com/blog/high-performance-find-query-using-lean-in-mongoose-2/

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM