Flutter pagination with firestore stream

Question

How to properly implement pagination with firestore stream on flutter (in this case flutter web)? my current approach with bloc which is most likely wrong is like this

function called on bloc when load next page, notice that i increased the lastPage variable of the state by 1 each time the function is called:

Stream<JobPostingState> _loadNextPage() async* {
yield state.copyWith(isLoading: true);
try {
  service
      .getAllDataByClassPage(state.lastPage+1)
      .listen((List<Future<DataJob>> listDataJob) async {
    List<DataJob?> listData = [];
    await Future.forEach(listDataJob, (dynamic element) async {
      DataJob data= await element;
      listData.add(data);
    });
    bool isHasMoreData = state.listJobPostBlock.length!=listData.length;
    //Update data on state here
  });
} on Exception catch (e, s) {
  yield StateFailure(error: e.toString());
}}

function called to get the stream data

Stream<List<Future<DataJob>>> getAllDataByClassPage(
      String className, int page) {
    Stream<QuerySnapshot> stream;
    if (className.isNotEmpty)
      stream = collection
          .orderBy('timestamp', "desc")
          .where('class', "==", className).limit(page*20)
          .onSnapshot;
    else
      stream = collection.onSnapshot;

    return stream.map((QuerySnapshot query) {
      return query.docs.map((e) async {
        return DataJob.fromMap(e.data());
      }).toList();
    });
  }

With this approach it works as intended where the data loaded increased when i load next page and still listening to the stream, but i dont know if this is proper approach since it replace the stream could it possibly read the data twice and end up making my read count on firestore much more than without using pagination. Any advice is really appreciated, thanks.

Answer 1

Your approach is not very the best possible indeed, and as you scale you going to be more costly. What I would do in your shoes would be to create a global variable that represents your stream so you can manipulate it. I can't see all of your code so I am going to be as generic as possible so you can apply this to your code.

First let's declare the stream controller as a global variable that can hold the value of your stream:

StreamController<List<DocumentSnapshot>> streamController = 
    StreamController<List<DocumentSnapshot>>();

After that we need to change your getAllDataByClassPage function to the following:

async getAllDataByClassPage(String className) {
    Stream stream = streamController.stream;
    //taking out of the code your className logic
    ...
    if(stream.isEmpty){
         QuerySnapshot snap = await collection.orderBy('timestamp', "desc")
                                              .where('class', "==", className)
                                              .limit(20) 
                                              .onSnapshot
         streamController.add(snap.docs);
    }else{
         DocumentSnapshot lastDoc = stream.last;
         QuerySnapshot snap = await collection.orderBy('timestamp', "desc")
                                              .where('class', "==", className)
                                              .startAfterDocument(lastDoc)
                                              .limit(20)
                                              .onSnapshot;
         streamController.add(snap.docs);
    }
  }

After that all you need to do in order to get the stream is invoke streamController.stream;

NOTE : I did not test this code but this is the general ideal of what you should try to do.

Answer 2

You can keep track of last document and if has more data on the list using startAfterDocument method. something like this

 final data = await db
      .collection(collection)
      .where(field, arrayContains: value)
      .limit(limit)
      .startAfterDocument(lastDoc)
      .get()
      .then((snapshots) => {
            'lastDoc': snapshots.docs[snapshots.size - 1],
            'docs': snapshots.docs.map((e) => e.data()).toList(),
            'hasMore': snapshots.docs.length == limit,
          });