使用Java在mongodb上分页的最佳方法是什么

Question

I am trying to create a simple pagination in mongodb by below code.

collection.find().skip(n).limit(n);

but doesn't it looks like there will be a performance issue if we see that in java terms first find will return all the records consider i have 2 million records, then it will pass it to skip method then it will be passed to limit method. it means every time this query will be fetching all the db records or mongodb drivers work differently, what i missed?

Answer 1

When talking about pagination in MongoDB, it is easily to write this code:

collection.find().skip(pageSize*(pageNum-1)).limit(pageSize);

Above is the native solution supported by MongoDB, but this is not efficient if there are huge documents in the collection. Suppose you have 100M documents, and you want to get the data from the middle offset(50Mth). MongoDB has to build up the full dataset and walk from the beginning to the specified offset , this will be low performance. As your offset increases, the performance keeps degrade.

The root cause is the skip() command which is not efficient and can not take big benifit from index.

---

Below is another solution to improve performance on large data pagination:

The typical usage scenario of pagination is that there is a table or list to show data of specified page, and also a 'Previous Page' & 'Next Page' button to load data of previous or next page.

If you got the '_id' of the last document in current page, you can use find() instead of skip() . Use _id > currentPage_LastDocument._id as one of the criteria to find next page data. Here is pseudocode:

//Page 1
collection.find().limit(pageSize);
//Get the _id of the last document in this page
last_id = ...

//Page 2
users = collection.find({'_id': {$gt: last_id}}).limit(pageSize);
//Update the last id with the _id of the last document in this page
last_id = ...

This will avoid MongoDB to walk through large data when using skip() .