In MongoDB aggregation pipeline $lookup returning empty array

Question

I am trying to apply $lookup aggregation on two of my collections, but it is returning an empty array each time. I have checked the data type for the local and foreign field and they are same. I am using pymongo library to implement it in a Python program.

This is the schema of paid collection:

{
"_id":{"$oid":"60aba58c5eff5afbc6e7c9f3"},
"created":{"$date":{"$numberLong":"1621861772297"}},
"questionId: {"$oid":"60aba53d485bfcf514e0bc1d"},
"userId":"gctqXxSIrAe8O8HidhOhxzYIDjR2"
}

This is the schema for the question collection:

{
"_id":{"$oid":"60ab8e0c485bfcf514e0bc18"},
"created":{"$date":{"$numberLong":"1621855756763"}},
"clientId":"ksCBoGLIr7MPNcod2FqXNPNcgZj2",
"topic":"Engineering"
}

Now I want to join these using questionId of paid collection and _id of question collection so I can then group by userId and topic and get the count of questions of each topic that each userId is following

I am using $lookup in the following way:

 pipeline = [{
       '$lookup':
         {
           'from': 'coll',
           'localField': 'questionId',
           'foreignField': '_id',
           'as': 'output'
         }
 }]

records = list(paid.aggregate(pipeline))

This is returning an empty array. I am fetching the collections from MongoDB Atlas.
Please someone help me out here as I am fairly new to Mongo and am using these aggregation functions for the first time.

Answer 1

This thing finally got fixed.
I was fetching the collections from MongoDB Atlas using collection1 = db['question] and collection2 = db['paid'] and then using lookup as:

 pipeline = [{
       '$lookup':
         {
           'from': 'collection1',
           'localField': 'questionId',
           'foreignField': '_id',
           'as': 'output'
         }
 }]

records = list(collection2.aggregate(pipeline))

But, instead of this we have to use the original collection name in the from clause instead of the name we give after fetching the data from Atlas to Python. So the code will change to:

pipeline = [{
           '$lookup':
             {
               'from': 'question',
               'localField': 'questionId',
               'foreignField': '_id',
               'as': 'output'
             }
     }]
    
records = list(collection2.aggregate(pipeline))

Hopefully this helps someone who does the same mistake while using Pymongo and MongoDB Atlas