带有特定条件计数的 Mongo DB 聚合并按输出投影的日期范围过滤无法按预期工作

Question

Please consider that I'm a beginner to MongoDB and I need to retrieve data from MongoDB database in somewhat complex query format. I've referred several Questions and Answers published in the community but my expected query was much complex due to some complex counts operations for certain conditions etc. However I was able manage to retrieve the data to a very similar query result which I'm expecting. But still I was unable to get the expected query result with my own. If anyone who can help to find a solution on this matter is much appreciable.

ProjectionOperation projection = project("createdAt", "status")
            .and(DateOperators.DateToString.dateOf("createdAt").toString("%Y-%m-%d")).as("otpDate")
            .and(ConditionalOperators.when(ComparisonOperators.Eq.valueOf("status").equalToValue("VERIFIED")).then(1).otherwise(0)).as("verifyStatus");

// Group by otpDate created in projection to get total otp count by date-wise
GroupOperation totalCount = group("otpDate").count().as("totalCount");

// Group by verifyStatus  created in projection to get total verified OTPs by status "VERIFIED"
GroupOperation loggedInCount = group( "verifyStatus").sum("verifyStatus").as("loggedIn");

// Filter data for given specific date range
MatchOperation match = match(Criteria.where("createdAt").gte(from).lte(to));

// Sort the result to ascending order by _id 
SortOperation sortOperation = sort(Sort.Direction.ASC, "_id");

final TypedAggregation<Otp> aggregation = newAggregation(
            Otp.class, match, projection, totalCount, loggedInCount, sortOperation);

mongoTemplate.aggregate(aggregation, OtpProjDto.class).getMappedResults();

Please find the Projection Dto, Expected and the Actual Results I've mentioned below with this code.

OtpProjDto.java

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.io.Serializable;
import java.math.BigInteger;
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class OtpProjDto implements Serializable {

    private String createdAt;
    private BigInteger totalCount;
    private BigInteger loggedIn;
}

Expected Result :

db.otp.aggregate([
{
    $match: {

        "createdAt": {
            $gte: new ISODate("2021-03-10"),
            $lte: new ISODate("2021-03-31")
        }
    }
},
{
    $group: {
        _id: {
            $dateToString: {
                format: "%Y-%m-%d",
                date: "$createdAt"
            }
        },
        "totalCount": {
            "$sum": 1
        },
        "logged_in": {
            "$sum": {
                "$cond": [{"$eq": ["$status", "VERIFIED"]}, 1, 0]
            }
        }
    }
},
{
    $sort: {
        _id: 1
    }
}
]);

Actual Result :

When I run the exact same code I've mentioned above with the two group it gives me an error like this:

But if I run the same code with one grouping at one time it works perfectly fine:

1. By commenting out this line from the code GroupOperation loggedInCount = group("verifyStatus").sum("verifyStatus").as("loggedIn"); result is:

    { "aggregate": "__collection__", "pipeline": [{ "$match": { "createdAt": { "$gte": {"$java": 2021 - 03 - 10}, "$lte": {"$java": 2021 - 03 - 31} } } }, { "$project": { "createdAt": 1, "status": 1, "otpDate": {"$dateToString": {"format": "%Y-%m-%d", "date": "$createdAt"}}, "verifyStatus": {"$cond": {"if": {"$eq": ["$status", "VERIFIED"]}, "then": 1, "else": 0}} } }, { "$group": { "_id": "$otpDate", "totalCount": {"$sum": 1} } }, { "$sort": {"_id": 1} }] }

2. By commenting out this line from the code GroupOperation totalCount = group("otpDate").count().as("totalCount") result is:

    { "aggregate" : "__collection__", "pipeline": [{ "$match": { "createdAt": { "$gte": {"$java": 2021 - 03 - 10}, "$lte": {"$java": 2021 - 03 - 31} } } }, { "$project": { "createdAt": 1, "status": 1, "otpDate": {"$dateToString": {"format": "%Y-%m-%d", "date": "$createdAt"}}, "verifyStatus": {"$cond": {"if": {"$eq": ["$status", "VERIFIED"]}, "then": 1, "else": 0}} } }, { "$group": { "_id": "$verifyStatus", "loggedIn": {"$sum": "$verifyStatus"} } }, { "$sort": {"_id": 1} }]; }

I think the issue is with the multiple grouping. If anyone who can help to find a solution on this matter is much appreciable.

Answer 1

After doing some researches by referring documentations and articles and spending some time with the above code I was able to find the solution for the above problem.

The problem was the exact thing what I suspected and it was the issue with trying to add two groups (more than one) grouping by different columns. So, I fixed this issue mainly by joining the second group count result using push() method with the first group I've used. So, this fixed my issue.

ProjectionOperation projection = project("createdAt", "status")
            .and(DateOperators.DateToString.dateOf("createdAt").toString("%Y-%m-%d")).as("otpDate")
            .and(AccumulatorOperators.Sum.sumOf(ConditionalOperators.when(ComparisonOperators.Eq.valueOf("status")
                    .equalToValue("VERIFIED")).then(1).otherwise(0))).as("verifyStatus"); // <--- (1) This Line Changed

// Joined the two group result into one result
GroupOperation totalCount = group("otpDate").count().as("totalCount")
            .push("verifyStatus").as("loggedIn"); // <--- (2) This Line Changed

// Filter-data for given specific-dates-range
MatchOperation match = match(Criteria.where("createdAt").gte(from).lte(to));

// Sort the result to ascending order by _id
SortOperation sortOperation = sort(Sort.Direction.ASC, "_id");

final TypedAggregation<Otp> aggregation = newAggregation(
            Otp.class, match, projection, totalCount, sortOperation);

mongoTemplate.aggregate(aggregation, OtpGenerationDataDto.class).getMappedResults()

Current Actual Result :

    {
    "aggregate": "__collection__",
    "pipeline": [{
        "$match": {
            "createdAt": {
                "$gte": {"$java": 2021 - 03 - 10},
                "$lte": {"$java": 2021 - 03 - 31}
            }
        }
    }, {
        "$project": {
            "createdAt": 1,
            "status": 1,
            "otpDate": {"$dateToString": {"format": "%Y-%m-%d", "date": "$createdAt"}},
            "verifyStatus": {"$sum": {"$cond": {"if": {"$eq": ["$status", "VERIFIED"]}, "then": 1, "else": 0}}}
        }
    }, {
        "$group": {
            "_id": "$otpDate",
            "totalCount": {"$sum": 1},
            "loggedIn": {"$push": "$verifyStatus"}
        }
    }, {
        "$sort": {"_id": 1}
    }];
}