How to improve the performance of this MongoDB / Mongoose query?

Question

I have a GET all products endpoint which is taking an extremely long time to return responses:

Product.find(find, function(err, _products) {
  if (err) {
    res.status(400).json({ error: err })
    return
  }

  res.json({ data: _products })

}).sort( [['_id', -1]] ).populate([
  { path: 'colors', model: 'Color' },
  { path: 'size', model: 'Size' },
  { path: 'price', model: 'Price' }
]).lean()

This query is taking up to 4 seconds , despite there only being 60 documents in the products collection.

This query came from a previous developer, and I'm not so familiar with Mongoose.

What are the performance consequences of sort and populate ? I assume populate is to blame here? I am not really sure what populate is doing, so I'm unclear how to either avoid it or index at a DB level to improve performance.

Answer 1

From the Mongoose docs, "Population is the process of automatically replacing the specified paths in the document with document(s) from other collection(s)"

So your ObjectId reference on your model gets replaced by an entire Mongoose document . Doing so on multiple paths in one query will therefore slow down your app. If you want to keep the same code structure, you can use select to specify what fields of the document that should be populated, ie { path: 'colors', model: 'Color', select: 'name' } . So instead of returning all the data of the Color document here, you just get the name.

You can also call cursor() to stream query results from MongoDB :

var cursor = Person.find().cursor();
cursor.on('data', function(doc) {
  // Called once for every document
});
cursor.on('close', function() {
  // Called when done
});

You can read more about the cursor function in the Mongoose documentation here .

In general, try to only use populate for specific tasks like getting the name of a color for only one product.

sort will not cause any major performance issues until you reach much larger databases.

Hope it helps!