[英]Mongodb aggregation - sort makes the query very slow
Mongodb 3.2,安裝在centos 6上,有足夠的RAM和磁盤。 我有一個包含以下結構的10K文檔的集合:
{
"id":5752034,
"score":7.6,
"name":"ASUS X551 15.6-inch Laptop",
"categoryId":"803",
"positiveAspects":[{
"id":30030525,
"name":"price",
"score":9.8,
"frequency":139,
"rank":100098
},
{
"id":30028399,
"name":"use",
"score":9.9,
"frequency":99,
"rank":100099
}
.
.
]
}
對於每個文檔,嵌套數組positiveAspects只有幾百個元素。
collectoin有以下索引:
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "positiveAspects.id" : 1.0, "positiveAspects.score" : 1.0 }, "name" : "positiveAspects.id_1_positiveAspects.score_1", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "categoryId" : 1.0, "score" : 1.0 }, "name" : "categoryId_1_score_1", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "rank" : -1.0 }, "name" : "rank_-1", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "positiveAspects.rank" : -1.0 }, "name" : "positiveAspects.rank_-1", "ns" : "proddb.product_trees" }
我想運行以下聚合,大約需要40秒:
{
aggregate:"product_trees",
pipeline:[
{
$match:{
categoryId:"803",
score:{
$gte:8.0
}
}
},
{
$unwind:"$positiveAspects"
},
{
$match:{
positiveAspects.id:30030525,
positiveAspects.score:{
$gte:9.0
}
}
},
{
$sort:{
positiveAspects.rank:-1
}
},
{
$project:{
_id:0,
score:1,
id:1,
name:1,
positiveAspects:1
}
},
{
$limit:10
}
]
}
通過以下說明:
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Beginning planning...
=============================
Options = NO_BLOCKING_SORT INDEX_INTERSECTION
Canonical query:
ns=proddb.product_treesTree: $and
categoryId == "803"
score $gte 8.0
Sort: {}
Proj: {}
=============================
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Index 0 is kp: { _id: 1 } unique name: '_id_' io: { v: 1, key: { _id: 1 }, name: "_id_", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Index 1 is kp: { positiveAspects.id: 1.0, positiveAspects.score: 1.0 } multikey name: 'positiveAspects.id_1_positiveAspects.score_1' io: { v: 1, key: { positiveAspects.id: 1.0, positiveAspects.score: 1.0 }, name: "positiveAspects.id_1_positiveAspects.score_1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Index 2 is kp: { categoryId: 1.0, score: 1.0 } name: 'categoryId_1_score_1' io: { v: 1, key: { categoryId: 1.0, score: 1.0 }, name: "categoryId_1_score_1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Index 3 is kp: { rank: -1.0 } name: 'rank_-1' io: { v: 1, key: { rank: -1.0 }, name: "rank_-1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Index 4 is kp: { positiveAspects.rank: -1.0 } multikey name: 'positiveAspects.rank_-1' io: { v: 1, key: { positiveAspects.rank: -1.0 }, name: "positiveAspects.rank_-1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Predicate over field 'score'
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Predicate over field 'categoryId'
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Relevant index 0 is kp: { categoryId: 1.0, score: 1.0 } name: 'categoryId_1_score_1' io: { v: 1, key: { categoryId: 1.0, score: 1.0 }, name: "categoryId_1_score_1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Rated tree:
$and
categoryId == "803" || First: 0 notFirst: full path: categoryId
score $gte 8.0 || First: notFirst: 0 full path: score
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Tagging memoID 1
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Enumerator: memo just before moving:
2016-06-01T16:10:49.140-0500 D QUERY [conn47] About to build solntree from tagged tree:
$and
categoryId == "803" || Selected Index #0 pos 0
score $gte 8.0 || Selected Index #0 pos 1
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Planner: adding solution:
FETCH
---fetched = 1
---sortedByDiskLoc = 0
---getSort = [{ categoryId: 1 }, { categoryId: 1, score: 1 }, { score: 1 }, ]
---Child:
------IXSCAN
---------keyPattern = { categoryId: 1.0, score: 1.0 }
---------direction = 1
---------bounds = field #0['categoryId']: ["803", "803"], field #1['score']: [8.0, inf.0]
---------fetched = 0
---------sortedByDiskLoc = 0
---------getSort = [{ categoryId: 1 }, { categoryId: 1, score: 1 }, { score: 1 }, ]
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Planner: outputted 1 indexed solutions.
2016-06-01T16:10:49.140-0500 D QUERY [conn47] Only one plan is available; it will be run but will not be cached. query: { categoryId: "803", score: { $gte: 8.0 } } sort: {} projection: {}, planSummary: IXSCAN { categoryId: 1.0, score: 1.0 }
2016-06-01T16:11:27.170-0500 I COMMAND [conn47] command proddb.product_trees command: aggregate { aggregate: "product_trees", pipeline: [ { $match: { categoryId: "803", score: { $gte: 8.0 } } }, { $unwind: "$positiveAspects" }, { $match: { positiveAspects.id: 30030525, positiveAspects.score: { $gte: 9.0 } } }, { $sort: { positiveAspects.rank: -1 } }, { $project: { _id: 0, score: 1, id: 1, name: 1, positiveAspects: 1 } }, { $limit: 10 } ], cursor: {} } keyUpdates:0 writeConflicts:0 numYields:226 reslen:7459 locks:{ Global: { acquireCount: { r: 906 } }, Database: { acquireCount: { r: 453 } }, Collection: { acquireCount: { r: 453 } } } protocol:op_query 38030ms
取出$sort
,查詢在2秒內運行。
你能解釋一下為什么$sort
導致這樣的性能下降,考慮它有可以使用的索引嗎? 我錯過了一個索引可以做什么來修復?
謝謝!
Mongodb aggregarion - sort使查詢非常慢
這是因為$sort
在聚合框架的早期階段沒有使用時沒有使用索引。 要利用索引,必須使用$ sort或$ match作為第一階段。
請參閱管道運營商和索引
這是因為$ unwind stage.Your查詢性能很慢,因為查詢不考慮$ unwind階段之后的索引。檢查解釋。你會知道。這發生是因為$ unwind整個文檔將改變后變為存儲在RAM中用於索引目的的不同。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.