How to perform server-side pagination from mongodb using nodejs?

Question

I am working on a project that deals with a large amount of data. Fetching of all data at once from mongodb is not an option since it results in bad user experience. I am working on creating an infinite loading setup and with each scroll, I want a fix number of data that is fetched from mongodb and will concatenate the newly fetched data with the previously fetched data to show results on my webpage. How to do pagination in mongodb using nodejs?

Answer 1

The mongodb node.js driver allows you to use the pagination through the limit and the skip attributes.

// You can first start by counting the number of entities in your collection 
  collection.countDocuments().then((count) => {
      var step = 1000;
      var offset = 0;
      var limit = step;
//then exploiting your offset and limit variables
//you can limit the number of results you get in each query (a page of results)
      while (offset < count) {
        process(offset, limit);
        offset += step;
      }
    });
  })
  .catch((error) => console.error(error));


 async function process(offset, limit) {
      var entities = collection.find(
        {},
        {
          limit: limit,
          skip: offset,
        }
      );
      for await (const entity of entities) {
       // do what you want
        
       
      }
    }

You can find more details on the MongoDB documentation page. https://www.mongodb.com/docs/drivers/node/current/fundamentals/crud/read-operations/limit/