Can't backfill array to larger than 1500000 elements

Question

I have a mongodb document named collection:

{
    _id: ObjectId('53e9dd54c784717558c46997'),
    bloks: [ /* subdocument array */ ],
    sections: [ /* subdocument array */ ],
    chapters: [ /* subdocument array */ ]
} 

and each of the subdocuments have id and state fields as strings among some other fields.

When using mongoose, I can update the bloks and return the collection using:

update = { bloks: [ /*some changed array*/ ] };

Collection.findByIdAndUpdate(collection._id, update, function (err, collection) {
    if (err) {
        // return an error
    } else {
       // return the collection
    }
});

But when I try to update also a specific section and chapter state in the other arrays:

update = { 
    bloks: [ /*some changed array*/ ],
    'sections.140439739188823467.state': 'some state',
    'chapters.1404397391757313579.state': 'some state'
};

I get an error:

Can't backfill array to larger than 1500000 elements

How can I update the collection document with the bloks, sections and chapters data and have it's current value?

Please note, I'm using .findByIdAndUpdate() because it is more efficient and I had problems making the .update() method do the actual save.

Answer 1

Thanks to Leonid Beschastny comment in the question, I figured out that I was using the subdocuments IDs as indices in the arrays, so I modifed the code to figure out the correct ones.

    sindex = _.findIndex(collection.sections, function (section) {
        return sid === section.id;
    });

    if (-1 < sindex) {
        update['sections.' + sindex + '.state'] = 'state';
    }



    hindex = _.findIndex(collection.chapters, function (chapter) {
        return hid === chapter.id;
    });
    if (-1 < hindex) {
        update['chapters.' + hindex + '.state'] = 'state';
    }

Answer 2

Actually, there is a way to update sub-document's field by its id, using positional operator $ :

db.collection.update({
    _id: collection._id,
    'sections.id': '140439739188823467'
}, {
    'sections.$.state': 'some state'
});

The only problem with this approach is that you can't update multiple sub-documents using single query. So, it'll require two requests to update both sections and chapters .

---

You should also consider using mongoose Sub Docs feature . The only thing you'll need to change in your existing schema is the primary id field of you sub docs, because mongoose always use _id field as a primary identifier.

It'll allow you to use MongooseDocumentArray::id helper , especially designed for your case:

sindex = collection.sections.id(sid);
hindex = collection.sections.id(hid);

---

And another thing.

Since you're already fetching your document, there is no need for you to issue an findAndModify operation, because when you call .save() method on mongoose document it issues an update operation, sending to MongoDB only updated fields:

collcetion[sindex].state = 'new state';
collcetion[hindex].state = 'new state';
collcetion.save(next);
// Mongoose sends update operation, setting only this two fields

Additionally, mongoose is using versioning to ensure that the order of documents in sub-documents array haven't changed, thus protecting you from updating wrong sub-document.