findAll() - there is limit for amount of data fetched from mongo?

Question

I have not so big collection and we want to just fetch all documents. We're using findAll() and it works for collection with 33.5k(45MB) documents, but we're getting error with 130k (50MB) collection.

I know that I can get all elements with for example chunking (skip/limit), but I'm curious:

This limitation of findAll() is in spring-data, or just mongo?
How big is this limit? It's based on data MB, or documents count?
Can I change this limit somehow?

We're using spring-data-mongo 2.2.5.RELEASE and mongo 3.6.17

Answer 1

Taking a look into MongoDB documentation and my summaries of Mongo University, neither mongo shell nor mongo compass have a findAll method and if you want to iterate over them you have to use the "it" operator.

So I'm assuming the limitation comes from your mongo connector who is the responsable of fetching data.

Answer 2

I have two offer for you:

first: Stream

// Repository
@Query(value="{name: ?0}",
        fields = "{ _id: 1 }")
Stream<Item> getLatestApprovedIdByName(String name);

// Service
default Item getLatestApprovedIdByName(String name) {
    return getLatestApprovedIdByName(name).stream().findFirst().orElse(null);
}

second: Pageable

// Repository
@Query(value = "{name: ?0,fields = "{ _id: 1 }")
Page<Item> getLatestApprovedIdByName(String name, Pageable pageable);

// Service
default Item getLatestApprovedIdByName(String name) {
    PageRequest request = new PageRequest(0, 1, new Sort(Sort.Direction.DESC, "approval.approvedDate"));
    return getLatestApprovedIdByName(name, request).getContent().get(0);
}

Answer 3

Just use toArray() to list all your documents.