Removing duplicates from MongoDB but MongoDB db.collection.find() return empty result

Question

How to remove duplicates from MongoDB? But MongoDB db.collection.find() looks not working at all.

Code

start_date = "2020-05-00"
end_date = "2020-07-00"

collection.create_index([('text', 'text')])
df = pd.DataFrame(list(collection.find({"$text": 
                                        {"$search": "ACTIVE - ACT 06"},
                                        "datetime": {
                                            "$gte": start_date+"T00:00:00.000Z",
                                            "$lt": end_date+"T00:00:00.000Z"
                                        }},
                                       {"_id":1, "datetime":1,"name":1})))

Sample duplicate data. I want to make sure name is unique for each date (ignore time).

{
    "datetime": "2020-05-03 06:43:52",
    "name": "ACTIVE - ACT 06"
},
{
    "datetime": "2020-05-03 06:44:01",
    "name": "ACTIVE - ACT 05"
},
{
    "datetime": "2020-05-03 07:43:52",
    "name": "ACTIVE - ACT 06"
},
{
    "datetime": "2020-05-03 07:44:01",
    "name": "ACTIVE - ACT 05"
},
{
    "datetime": "2020-05-03 08:43:52",
    "name": "ACTIVE - ACT 06"
},
{
    "datetime": "2020-05-03 08:44:01",
    "name": "ACTIVE - ACT 05"
}

After removal. Save only for the earlier records.

{
    "datetime": "2020-05-03 06:43:52",
    "name": "ACTIVE - ACT 06"
},
{
    "datetime": "2020-05-03 06:44:01",
    "name": "ACTIVE - ACT 05"
}

Answer 1

You can use aggregate framework like this:

1. First List the duplicate data.
2. Then, remove it one by one.

db.collection.aggregate([
      {
        $group: {
          _id: "$name",
          dups: {
            $push: "$_id"
          },
          count: {
            $sum: 1
          }
        }
      },
      {
        $match: {
          count: {
            $gt: 1
          }
        }
      }
    ]).forEach(function(doc){
      db.collection.remove({
        _id: {
          $in: doc.dups
        }
      });
    })