mongodb update structure of database, super slow

Question

If I need to update the structure of 70,000 documents. Looping through the cursor and updating each one is super super slow. It does about 10/s

Any tips on how to update the document structure (essentially updating some values in every document in my database)

import json
import pymongo
from bson.objectid import ObjectId

for document in applications.find({}):
    document["name"] = {"name": document["name"], "level": "999"}
    applications.update_one({}, {"$set": document})

The database is hosted on a decent enough server, and I feel like its the code which is limiting the speed. Not the server.

https://gyazo.com/f8227e0cd24c3f5c9c0cb67e52e5a9ca

Answer 1

Your current attempt is iterating the find() result set at js level and fire individual update back to database, which creates a lot of overhead and slow.

With MongoDB v4.2+, you can update with an aggregation pipeline.

db.collection.update({},
[
  {
    $addFields: {
      name: {
        name: "$name",
        level: "999"
      }
    }
  }
],
{
  multi: true
})

Here is the Mongo playground for your reference.