Azure MongoDB API Bulk Insert

Question

I currently make use of Azure Mongo API for documentDB. However this has been causing us many headaches such as inserting documents into the db and trying to ignore the duplicate key errors during an InsertMany() . If there is a document within the batch that is a duplicate it immediately throws and error and returns even though I would require it to carry on. I am aware why we are getting these as I am busy moving data across from a non partitioned collection to a partitioned collection and there is a good chance they already have been migrated.

Apparently during the insert many you can pass isOrdered = false within the insertmanyoptions() and it should do this, it did not.

After eventually giving up, I attempted to try this via a BulkWriteAsync whilst making use of the IsUpsert = true Sadly I get a funny error:

Value cannot be null. Parameter name : requests

Here is my code below.

    public async Task BulkInsertDeviceConfigData(List<DeviceEventConfigure> Elements)
    {
        var models = new List<WriteModel<BsonDocument>>();

        // use ReplaceOneModel with property IsUpsert set to true to upsert whole documents
        foreach (var element in Elements)
        {
            var bsonDoc = element.ToBsonDocument();
            models.Add(new ReplaceOneModel<BsonDocument>(new BsonDocument("_id", element.Id), bsonDoc) { IsUpsert = true });
        };

        await BulkWriteAsync(models as IEnumerable<WriteModel<DeviceEventData>>);
    }

    public virtual async Task BulkWriteAsync(IEnumerable<WriteModel<T>> models)
    {
        await Collection.BulkWriteAsync(models);
    }

This has been hurting me as we have been increasing our RUs a lot lately to do this even though sadly have been unsuccessful.

Is this an Azure Mongo API issue or is it me?

Answer 1

Maybe your cast

models as IEnumerable<WriteModel<DeviceEventData>> 

isn't working resulting in a null value for the parameter requests. I don't think casting to a BsonDocument is necessary, so it can be written as:

public async Task BulkInsertDeviceConfigData(List<DeviceEventConfigure> Elements)
{
    var models = new List<WriteModel<DeviceEventConfigure>>();

    // use ReplaceOneModel with property IsUpsert set to true to upsert whole documents
    foreach (var element in Elements)
    {
        models.Add(new ReplaceOneModel<DeviceEventConfigure>(new BsonDocument("_id", element.Id), element) { IsUpsert = true });
    };

    await BulkWriteAsync(models as IEnumerable<WriteModel<DeviceEventConfigure>>);
}

public virtual async Task BulkWriteAsync(IEnumerable<WriteModel<T>> models)
{
    await Collection.BulkWriteAsync(models);
}