简体   繁体   English

流星集合(MongoDB)更新性能很差

[英]Meteor collection (MongoDB) update very poor performance

I'm having issues optimising update performance for a Meteor collection. 我在优化Meteor集合的更新性能时遇到问题。

I take a document from one collection (CollectionA), modify it, cherry-pick some of the content, and then update a near duplicate of it in another collection (CollectionB). 我从一个集合(CollectionA)中获取文档,对其进行修改,挑选其中的一些内容,然后在另一个集合(CollectionB)中对其进行近乎重复的更新。 I also upsert in case new documents have been added to CollectionA 如果新文档已添加到CollectionA,我还会提出建议

Everything happens in a few milliseconds, but the update can take anywhere from 10–30+ seconds depending on the size of the document being updated. 一切都在几毫秒内发生,但是更新可能需要10到30+秒,这取决于要更新的​​文档的大小。 The documents are typically around 30kb... 这些文件通常约为30kb ...

I have tried without Meteor.defer, with writeConcern : 0, and on both local and cloud replica set clusters. 我尝试过在没有Meteor.defer的情况下使用writeConcern:0,并且在本地和云副本集群集上进行了尝试。 Have also tried using insert instead of update. 也尝试过使用插入而不是更新。 Nothing makes a noticeable difference. 没有什么明显的不同。

cronTask(){

    CollectionA.find({isPublished : true, "approval.status" : { $gte : 1 } }).forEach((doc)=>{

        let newDoc = {
            parentId : doc._id,
            slug : doc.slug, // these never change, only the content array changes...
            title : doc.title,
            description: doc.description,
            tags : doc.tags,
            category : doc.category,
            createdAt : new Date(),
            content: [...,...,...] // the content of the new document is cherrypicked from the parents before saving
        }

        while(doc.content.length) {
            // cherry-picking and pushing to newDoc.content
            // super quick, a couple of MS
        }

        Meteor.defer(()=>{
            CollectionB.update({parentId : doc._id}, {$set : newDoc}, {upsert : true}, (err,res)=>{
                if (!err) {
                    console.log(`Saved child for ${doc.title}`);
                } else {
                    console.log(`Error saving child for ${doc.title}: ${err}`);
                }
            });
        });

    });
}

Turned out that the issue was not in fact the update, it was schema validation (using https://github.com/aldeed/meteor-simple-schema ). 原来,问题实际上不是更新,而是架构验证(使用https://github.com/aldeed/meteor-simple-schema )。

I disabled schema checking for the objects in the array (the method is on the server so it's safe to not validate in this case). 我为阵列中的对象禁用了模式检查(该方法在服务器上,因此在这种情况下不进行验证是安全的)。 Now it takes <1ms to update all 30 documents. 现在只需不到1毫秒即可更新所有30个文档。

Not sure why the schema validation was so slow. 不知道为什么模式验证这么慢。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM