How to fix limit and orderBy in Mongodb using Java

Question

I am trying to fetch a list of documents using specified filters. I want to limit the output to 10 and orderBy a column of type long(totalTransactionCount that is found in the sample data posted below). However, the output I get is just 8 instead of 10, there are over 100 records in the collection that meets the requirement. gged the size of the arraylist, it returned 10. This implies that the issue is not coming from the arrayList rather from my loop.

Here is the code

 Bson filter = Filters.and(Filters.eq("type", "CASH_OUT"),
                Filters.exists("successfulTransactionInfo.totalTransactionAmount"),
                AnalyticFilters.agentsFilter(queryParams));
      

        Bson orderBySort = orderBy(ascending("successfulTransactionInfo.totalTransactionCount"));

        ArrayList<TransactionAnalytic> transactionAnalytics = transactionAnalyticCollection.find(filter).
                sort(orderBySort).limit(10).into(new ArrayList<>());

 Set<String> agentIds = new HashSet<>();

    transactionAnalytics.forEach(transaction ->{
        log.info("Here is the list of agentIds fetchec {}", agentIds);
        agentIds.add(transaction.getAgentID());
        try {
            BulkAgentInfoResponse agentPhoneNumber = getAgentPhoneNumber(agentIds, token);
            agentPhoneNumber.getData().forEach(tranx ->{
                if(tranx.getAgentID().equalsIgnoreCase(transaction.getAgentID())){
                    String name = tranx.getFirstName() + " " + tranx.getLastName();
                    result.put(transaction.getAgentID(), new AgentPerformanceDto(name,
                            transaction.getAgentID(), tranx.getPhoneNumber(), transaction.getSuccessfulTransactionInfo().getTotalTransactionAmount(),
                            transaction.getSuccessfulTransactionInfo().getTransactionCount()));
                }
            });

        } catch (ExecutionException | IOException | InterruptedException e) {
            throw new RuntimeException(e);
        }

    });
    return result;

I am currently not passing anything into the query params therefore it is using the default parameter. I have tried to skip 2 (skip(2) but I still get 8 records instead of 10.

This is a sample of the response data. I am ordering by totalTransactionCount. The one with totalTransaction is meant to be at the top and it should follow suit.

"data": [
        {
            "agentName": "job o ajari",
            "agentID": "0001381",
            "phoneNumber": "01223445",
            "cashOutAmount": 51466.0,
            "cashOutCount": 7
        },
        {
            "agentName": "mabel ekporo",
            "agentID": "0001437",
            "phoneNumber": "01223445",
            "cashOutAmount": 459899.25,
            "cashOutCount": 7
        },
        {
            "agentName": "zion son",
            "agentID": "0000578",
            "phoneNumber": "01223445",
            "cashOutAmount": 159317.5,
            "cashOutCount": 6
        },
        {
            "agentName": "tukur saidu",
            "agentID": "0001304",
            "phoneNumber": "01223445",
            "cashOutAmount": 5050.0,
            "cashOutCount": 1
        },
        {
            "agentName": "obed eneh",
            "agentID": "0000885",
            "phoneNumber": "01223445",
            "cashOutAmount": 44080.75,
            "cashOutCount": 6
        },
        {
            "agentName": "bolanle olarotimi",
            "agentID": "0000995",
            "phoneNumber": "01223445",
            "cashOutAmount": 41892.5,
            "cashOutCount": 8
        },
        {
            "agentName": "Blessing Nana Mathew",
            "agentID": "0001487",
            "phoneNumber": "01223445",
            "cashOutAmount": 10000.0,
            "cashOutCount": 1
        },
        {
            "agentName": "anene chinonso",
            "agentID": "0001585",
            "phoneNumber": "01223445",
            "cashOutAmount": 10300.0,
            "cashOutCount": 1
        }
    ]

This is the input filter here. Like I stated earlier, I am not passing anything so, it's going to use the default value which is the current date minus 1. The collection contains data from August 1st - 22nd. Thus, there are over 100 records that meets the requirement.

public static Bson agentsFilter(MultivaluedMap<String, String> queryParams) throws ParseException{
        Bson filter = new Document();
        String startDate = queryParams.getFirst("startDate");

        String endDate = queryParams.getFirst("endDate");

        if (Objects.nonNull(startDate) && Objects.nonNull(endDate)) {
            Date date1 = new SimpleDateFormat(DATE_FORMAT).parse(startDate);
            Date date2 = DateUtils.addHours(new SimpleDateFormat(DATE_FORMAT).parse(endDate), 24);
            filter = Filters.and(
                    filter,
                    Filters.and(
                            Filters.gte(CREATED_AT, date1),
                            Filters.lte(CREATED_AT, date2)
                    )
            );
        } else{
            Date defaultStartDate =  new SimpleDateFormat(DATE_FORMAT)
                    .parse(LocalDate.now().minusMonths(1).toString() );
            filter = Filters.and(filter, Filters.gte(CREATED_AT, defaultStartDate)
            );
        }

        return filter;
    }

The orderBy is meant to start with the largest value but it's just random. How do I fix this problem? It is expedient to state that I am using MongoJack and Dropwizard as the framework.

Answer 1

I figured out what the issue is. I was using a non-existent field in the filter "successfulTransactionInfo.totalTransactionCount". The actual field is "successfulTransactionInfo.transactionCount". This gave me the same outcome as the query from the db directly.