MongoDB collection update with $set and aggregate

Question

I need to change the type of timestamp_ms filed from string to double and create FixedDate field which is based on this new timestamp_ms field. Than, change timestamp info into ISO date in NewDate field.

I used this code:

collection.update({
    "FixedDate": {"$exists": False}
},[{
    "$set":
        {"FixedDate":
            {"$convert":
                {"input": "$timestamp_ms",
                "to": "double"
                }
            }
        }
    },
    {"$set":
        {"NewDate":
                {"$toDate": "$FixedDate"
                }
            }
    }
], multi=True)

It gives this error message:

TypeError: document must be an instance of dict, bson.son.SON, or other type that inherits from collections.Mapping

My data looks like this:

{'_id': ObjectId('5afea920d326051990a7f337'), 'created_at': 'Fri May 18 10:21:07 +0000 2018', 'timestamp_ms': '1526638867739'}
{'_id': ObjectId('5afea920d326051990a7f339'), 'created_at': 'Fri May 18 10:21:08 +0000 2018', 'timestamp_ms': '1526638868310'}
{'_id': ObjectId('5afea972d326051c5c05bc11'), 'created_at': 'Fri May 18 10:22:30 +0000 2018', 'timestamp_ms': '1526638950799'}
{'_id': ObjectId('5afea974d326051c5c05bc16'), 'created_at': 'Fri May 18 10:22:32 +0000 2018', 'timestamp_ms': '1526638952160'}
{'_id': ObjectId('5afea974d326051c5c05bc17'), 'created_at': 'Fri May 18 10:22:32 +0000 2018', 'timestamp_ms': '1526638952841'}

Answer 1

It looks like you have a couple of issues, the first is you are not using the latest driver for 4.2 (it's still in beta so if you just did pip upgrade you would still be on 3.8 and you need 3.9). Install the latest with python -m pip install https://github.com/mongodb/mongo-python-driver/archive/3.9.0b1.tar.gz at least until it's GA.

Second, you should use update_one or update_many rather than update with multi flag.

Trying your code with latest version, this works:

collection.update_many(
   {"FixedDate": {"$exists": False}},
   [{"$set":{
      "FixedDate":{"$toDouble":"$timestamp_ms"}
  }},
  {"$set":{
      "NewDate": {"$toDate": "$FixedDate"}
  }}
])

Answer 2

I have tried your problem with aggregation framework,

let query = [
    {
        $match: {
            "FixedDate": { "$exists": false }
        }
    }, {
        $addFields: {
            "FixedDate":
                {
                    "$convert":
                        {
                            "input": "$timestamp_ms",
                            "to": "double"
                        }
                },

        },
    }, {
        $addFields: {
            "NewDate":
                {
                    "$toDate": "$FixedDate"
                }
        }
    }]

db.collection.aggregate(query)

here you need to use two times $addFields

Result

/* 1 createdAt:18/05/2018, 15:51:20*/
{
    "_id" : ObjectId("5afea920d326051990a7f337"),
    "created_at" : "Fri May 18 10:21:07 +0000 2018",
    "timestamp_ms" : "1526638867739",
    "FixedDate" : 1526638867739,
    "NewDate" : ISODate("2018-05-18T15:51:07.739+05:30")
},

/* 2 createdAt:18/05/2018, 15:51:20*/
{
    "_id" : ObjectId("5afea920d326051990a7f339"),
    "created_at" : "Fri May 18 10:21:08 +0000 2018",
    "timestamp_ms" : "1526638868310",
    "FixedDate" : 1526638868310,
    "NewDate" : ISODate("2018-05-18T15:51:08.310+05:30")
},

/* 3 createdAt:18/05/2018, 15:52:42*/
{
    "_id" : ObjectId("5afea972d326051c5c05bc11"),
    "created_at" : "Fri May 18 10:22:30 +0000 2018",
    "timestamp_ms" : "1526638950799",
    "FixedDate" : 1526638950799,
    "NewDate" : ISODate("2018-05-18T15:52:30.799+05:30")
},

/* 4 createdAt:18/05/2018, 15:52:44*/
{
    "_id" : ObjectId("5afea974d326051c5c05bc16"),
    "created_at" : "Fri May 18 10:22:32 +0000 2018",
    "timestamp_ms" : "1526638952160",
    "FixedDate" : 1526638952160,
    "NewDate" : ISODate("2018-05-18T15:52:32.160+05:30")
},

/* 5 createdAt:18/05/2018, 15:52:44*/
{
    "_id" : ObjectId("5afea974d326051c5c05bc17"),
    "created_at" : "Fri May 18 10:22:32 +0000 2018",
    "timestamp_ms" : "1526638952841",
    "FixedDate" : 1526638952841,
    "NewDate" : ISODate("2018-05-18T15:52:32.841+05:30")
}