How to speed up query mongdb with mongoose in nodejs

Question

I'm a newbie in MongoDB and Node.js.

I wrote an example app to measure how fast a get request to MongoDB using Mongoose.

I have a collection with about 200000 records. In my code, I want to get the first 100000 rows by query:

var query = db.myCollection.find().limit(100000);
query.exec(function(err, data){
      // ....
});

and it took about 99s, I think that speed was too slow. Does anyone have ideas on how to speed up the query?

Thank you so much!

Answer 1

With mongodb, you can index some of your keys to improve the performance when you're querying. But in this case, this will not work.

Your 99s are probably caused by your PC which just can't handle this kind of heavy load of data in a single time (that's perfectly understandable).

That is absolutely not in correlation with mongodb but with your testing machine memory.

But you maybe can improve it by piping the result :

// use our lame formatter
var format = new ArrayFormatter;

// first pipe the querystream to the formatter
myCollection.find().stream().pipe(format);

// then pipe the formatter to the response
// (node 0.4x style pipe non-chaining)
format.pipe(res);

// In node 0.6 we can P.find().stream().pipe(format).pipe(res);

With this gist