MongoDB聚合查询（组）花费太多时间

Question

The Group stage takes 5 min to excute and i have 100000000 records in my collection . i am using mongodb 4.2 and i have 8 cpu with 32gb RAM. Is there any better way to optimised query or index?

db.getCollection("text").explain("executionStats").aggregate(
[
    { 
        "$match" : { 
            "CreatedDate" : { 
                "$gte" : ISODate("2021-01-01T15:43:50.325+0000"), 
                "$lte" : ISODate("2021-03-29T15:43:50.325+0000")
            }
        }
    }, 
    { 
        "$project" : { 
            "TX_DATE" : { 
                "$dateToString" : { 
                    "format" : "%Y-%m", 
                    "date" : "$CreatedDate"
                }
            }, 
            "Exp_Count" : 1.0
        }
    }, 
    { 
        "$group" : { 
            "_id" : { 
                "TX_DATE_Month" : "$TX_DATE"
            }, 
            "Exp_Count" : { 
                "$sum" : "$Exp_Count"
            }
        }
    }, 
    { 
        "$project" : { 
            "_id" : 0.0, 
            "TX_DATE" : "$_id.TX_DATE_Month", 
            "Exp_Count" : 1.0
        }
    }, 

    { 
        "$sort" : { 
            "TX_DATE" : 1.0
        }
    }
], 
{ 
    "allowDiskUse" : false
}

);

Answer 1

$match stage: can help you with this,

The matching stage is used to select the required documents only. This matching will reduce our aggregation process to the required documents. It's more similar to the where clause that we use in a MySQL query. Matching helps us to use the indexing that we had created in the collection. With the usage of indexed keys in the matching stage, it becomes easy to find and group required documents in a collection. For example, To group the data of students by gender with age 13 in a school's data with age indexed. The command for the aggregation by gender:

db.SchoolData.aggregate([{’$match’:{’age’:13}},{’$group’:{’_id’:’$gender’}}])

This will reduce our focus to documents with an age 13 and with indexing on the same key this becomes much more efficient.

Note that,

db.SchoolData.aggregate([{’$match’:{’age’:13}},{’$group’:{’_id’:’$gender’}}])

and

db.SchoolData.aggregate([{’$group’:{’_id’:’$gender’}},{’$match’:{’age’:13}}])

will have an entirely different execution time since in the first command it performs the aggregation only on the documents with age 13 and in the second case, it does aggregation on all the documents and returns the results having age 13.