Mongodb query execution take too much time

Question

Iam working on the Go project and I am using mongodb to store my data. But suddenly the mongodb query execution took too much time to get data. I have a collection named "cars" with around 25000 documents and each document containing around 200 fields (4.385KB). I have an aggregate query like this:

db.cars.aggregate([
    {
        $lookup:
        {
            from: "users",
            localField: "uid",
            foreignField: "_id",
            as: "customer_info"
        }
    },{
        $unwind: "$customer_info"
    },{
        $lookup:
        {
            from: "user_addresses",
            localField: "uid",
            foreignField: "_id",
            as: "address"
        }
    },{
        $unwind: "$address"
    },{
    $lookup:
        {
            from: "models",
            localField: "_id",
            foreignField: "car_id",
            as: "model_info"
        }
    },{
    $match:{
        purchased_on:{$gt:1538392491}, 
        status:{$in:[1,2,3,4]}, 
        "customer_info.status":{$ne:9}, 
        "model_info.status":{$ne:9},
        }
    },{
        $sort:{
            arrival_time:1
        }
    },{
        $skip:0
    },{
        $limit:5
    }
])

My document structure is like: https://drive.google.com/file/d/1hM-lPwvE45_213rQDYaYuYYbt3LRTgF0/view .

Now, If run this query with out indexing then it take around 10 mins to load the data. Can anyone suggest me how can I reduce its execution time ?

Answer 1

There are many things to do to optimize your query. What I would try :

• As Anthony Winzlet said in comments, use as possible $match stage as first stage. This way, you can reduce number of documents passed to the following stages, and use indexes.

• Assuming you use at least 3.6 mongo version, change your lookup stages using the 'let/pipeline' syntax ( see here ). This way, you can integrate your 'external filters' ( "customer_info.status":{$ne:9}, "model_info.status":{$ne:9} ) in a $match stage in your lookups pipeline. With indexes on right fields / collections, you will gain some time / memory in your $lookup stages.

• Do your unwind stages as late as possible, to restrict number of documents passed to the following stages.

It's important to understand how works aggregation pipeline : each stage receive data, do its stuff, and pass data to next stage. So the less data is passed to the pipeline, the faster will be your query.