Upsert an array of items with MongoDB

Question

I've got an array of objects I'd like to insert into a MongoDB collection, however I'd like to check if each item already exists in the collection first. If it doesn't exist then insert it, if it does exist then don't insert it.

After some Googling it looks like I'm best using the update method with the upsert property set to true.

This seems to work fine if I pass it a single item, as well as if I iterate over my array and insert the items one at a time, however I'm not sure whether this is the correct way to do this, or if it's the most efficient. Here's what I'm doing at the moment:

var data = [{}, {}, ... {}];

for (var i = 0; i < data.length; i++) {

  var item = data[i];

  collection.update(
    {userId: item.id},
    {$setOnInsert: item},
    {upsert: true},
    function(err, result) {

      if (err) {
        // Handle errors here
      }

  });

}

Is iterating over my array and inserting them one-by-one the most efficient way to do this, or is there a better way?

Answer 1

I can see two options here, depending your exact needs:

• If you need to insert or update use an upsert query . In that case, if the object is already present you will update it -- assuming your document will have some content, this means that you will possibly overwrite the previous value of some fields. As you noticed, using $setOnInsert will mitigate that issue.
• If you need to insert or ignore add an unique index on the related fields and let the insert fail on duplicate key. As you have several documents to insert, you might send them all in an array using an unordered insert (ie: setting the ordered option to false ) so MongoDB will continue to insert the remaining documents in case of error with one of them.

As about performances, with the second option MongoDB will use the index to look up for a possible duplicate. So chances are good this would perform better.