MongoDB - Populate document field recursively

Question

I've got a Page :

const PageSchema = new Schema({
  children: [{type: mongoose.Schema.Types.ObjectId, ref: 'Page'}]
});

As you can see each Page has an array of children which is also a Page .

Now what I need to do is fetch all "main" pages and populate their children array, easy enough except for the fact that I need to do this recursively since a Page contains an array of Page s.

MongoDB doesn't have any out of the box support for this, it only supports a 2 level deep population.

Here's my current query (removed all extra stuff for readability) without using the current .populate method (since it's not gonna work anyway):

Page.find(query)
  .exec((err, pages) => {

    if (err) {
      return next(err);
    }

    res.json(pages);
  });

I looked at this question which is similar but not exactly what I need:

mongoose recursive populate

That seems to use a parent to populate recursively and it also starts from just 1 document, rather than my scenario which uses an array of documents since I'm using .find and not .findOne for example.

How can I create my own deep recursive populate function for this?

Sidenote:

I am aware that the solution I need isn't recommended due to performance but I've come to the conclusion that it is the only solution that is going to work for me. I need to do recursive fetching regardless if it's in the frontend or backend, and doing it right in the backend will simplify things massively. Also the number of pages won't be big enough to cause performance issues.

Answer 1

The answer actually lied in one of the answers from the previous questions, although a bit vague. Here's what I ended up with and it works really well:

Page.find(query)
  .or({label: new RegExp(config.query, 'i')})
  .sort(config.sortBy)
  .limit(config.limit)
  .skip(config.offset)
  .exec((err, pages) => {

    if (err) {
      return next(err);
    }

    // takes a collection and a document id and returns this document fully nested with its children
    const populateChildren = (coll, id) => {

      return coll.findOne({_id: id})
        .then((page) => {

          if (!page.children || !page.children.length) {
            return page;
          }

          return Promise.all(page.children.map(childId => populateChildren(coll, childId)))
            .then(children => Object.assign(page, {children}))
        });
    }

    Promise.all(pages.map((page) => {
      return populateChildren(Page, page._id);
    })).then((pages) => {

      res.json({
        error: null,
        data: pages,
        total: total,
        results: pages.length
      });
    });
  });

The function itself should be refactored into a utils function that can be used anywhere and also it should be a bit more general so it can be used for other deep populations as well.

I hope this helps someone else in the future :)

Answer 2

You can recursively populate a field also like:

User.findOne({ name: 'Joe' })
   .populate({
      path: 'blogPosts',
      populate: {
         path: 'comments',
         model: 'comment',
         populate: {
            path: 'user',
            model: 'user'
         }
      }
   })
   .then((user) => {});

Please note that for first population, you don't need to specify model attribute, as it is already defined in your model 's schema, but for next nested populations, you need to do that.