[英]Postgres does not use an index even less than 5% rows are returned
我有一個具有以下結構的postgres表:
+---------+-------------+-------------+----------+---------+---------+
| id | timestamp | numvalues | text1 | text2 | text3 |
+---------+-------------+-------------+----------+---------+---------+
|abcd12344| 4124135 |[1,2,53,1241]| apple | banana | papaya |
+---------+-------------+-------------+----------+---------+---------+
id - random alphanumeric value<br>
timestamp - epoch timestamp<br>
numvalues - array of integers<br>
text(n) - text values
該表約有1.5億行。
我進行內部查詢,以獲取按數組值之一排序的數據的第n個百分點。 然后,我需要從數組中獲取多個值的平均值。 數組的長度約為31。
SELECT AVG(COALESCE(numvalues[2], 0))::NUMERIC(10,0), AVG(COALESCE(numvalues[3], 0))::NUMERIC(10,0)...AVG(COALESCE(numvalues[12], 0))::NUMERIC(10,0)
FROM (SELECT timestamp, numvalues, ntile(100)
OVER (ORDER BY numvalues[1]) pval FROM tablename WHERE timestamp >= somevalue and timestamp <= somevalue) innertable
WHERE pval >= x and pval <= y;
這將返回1.5億行中的約700萬行,約占表的5%。 但是,它不使用索引。 即使將enable_seqscan設置為off,運行EXPLAIN ANALYZE也會顯示它使用Seq掃描代替。
但是,類似的查詢:
SELECT text1, count(distinct(id))
FROM (SELECT timestamp, id, text1, numvalues, ntile(100)
OVER (ORDER BY numvalues[1]) pval FROM tablename WHERE timestamp >= somevalue and timestamp <= somevalue) innertable
WHERE pval >= x and pval <= y GROUP BY text1;
確實使用索引。
索引在時間戳列上
EXPLAIN ANALYZE的結果:
explain analyze select text1, count(distinct(id))
from (select timestamp, text1, numvalues, id, ntile(100) over (order by numvalues[1]) pval from table where timestamp >= 1431100800 and timestamp <= 1431108000 and numvalues[1] NOTNULL) innertable
where pval >= 90 and pval <= 90 group by text1;
QUERY PLAN
--------------------------------------------------------------------------------
GroupAggregate (cost=4554118.06..4554641.66 rows=1 width=28) (actual time=218641.221..219051.984 rows=20 loops=1)
-> Sort (cost=4554118.06..4554292.59 rows=69812 width=28) (actual time=218640.546..218728.294 rows=71441 loops=1)
Sort Key: innertable.text1
Sort Method: quicksort Memory: 8654kB
-> Subquery Scan on innertable (cost=4094722.75..4548501.27 rows=69812 width=28) (actual time=216502.946..218521.666 rows=71441 loops=1)
Filter: ((innertable.pval >= 90) AND (innertable.pval <= 90))
Rows Removed by Filter: 7072674
-> WindowAgg (cost=4094722.75..4339065.03 rows=13962416 width=118) (actual time=202276.333..211374.235 rows=7144115 loops=1)
-> Sort (cost=4094722.75..4129628.79 rows=13962416 width=118) (actual time=164912.487..190272.316 rows=7144115 loops=1)
Sort Key: (table.numvalues[9])
Sort Method: external merge Disk: 1387704kB
-> Index Scan using table_time_idx on table (cost=0.57..1578710.87 rows=13962416 width=118) (actual time=0.124..141014.505 rows=7144115 loops=1)
Index Cond: (("timestamp" >= 1431100800) AND ("timestamp" <= 1431108000))
Filter: (numvalues[1] IS NOT NULL)
Rows Removed by Filter: 7090075
Total runtime: 219340.709 ms
(16 rows)
explain analyze select avg(coalesce(numvalues[9], 0))::NUMERIC(10,0) mean9, avg(coalesce(numvalues[30],0))::NUMERIC(10,0), avg(coalesce(numvalues[8],0))::NUMERIC(10,0) - avg(coalesce(numvalues[30], 0))::NUMERIC(10,0) mean0, avg(coalesce(numvalues[10],0))::NUMERIC(10,0) - avg(coalesce(numvalues[8], 0))::NUMERIC(10,0) mean1, avg(coalesce(numvalues[14],0))::NUMERIC(10,0) - avg(coalesce(numvalues[10], 0))::NUMERIC(10,0) mean2, avg(coalesce(numvalues[13],0))::NUMERIC(10,0) - avg(coalesce(numvalues[14], 0))::NUMERIC(10,0) mean3, avg(coalesce(numvalues[9],0))::NUMERIC(10,0) - avg(coalesce(numvalues[13], 0))::NUMERIC(10,0) mean4
from (select timestamp, id, numvalues, ntile(100) over (order by numvalues[1] ) pval from table where timestamp >= 1431093600.00 and timestamp <= 1431100800.00 and numvalues[9] NOTNULL) innerTable
where pval >= 90.00 and pval <= 90.00 ;
QUERY PLAN
--------------------------------------------------------------------------------
Aggregate (cost=12662077.32..12662077.37 rows=1 width=82) (actual time=650343.769..650343.770 rows=1 loops=1)
-> Subquery Scan on innertable (cost=12634854.54..12661968.84 rows=3615 width=82) (actual time=647745.962..650232.725 rows=71441 loops=1)
Filter: (((innertable.pval)::numeric >= 90.00) AND ((innertable.pval)::numeric <= 90.00))
Rows Removed by Filter: 7072674
-> WindowAgg (cost=12634854.54..12647507.88 rows=723048 width=248) (actual time=632388.293..642338.237 rows=7144115 loops=1)
-> Sort (cost=12634854.54..12636662.16 rows=723048 width=248) (actual time=599893.771..617413.102 rows=7144115 loops=1)
Sort Key: (table.numvalues[9])
Sort Method: external merge Disk: 3214248kB
-> Seq Scan on req_p0swajch2t (cost=0.00..12480460.48 rows=723048 width=248) (actual time=0.041..575423.062 rows=7144115 loops=1)
Filter: ((numvalues[1] IS NOT NULL) AND (("timestamp")::numeric >= 1431100800.00) AND (("timestamp")::numeric <= 1431108000.00))
Rows Removed by Filter: 138191935
Total runtime: 650785.126 ms
(12 rows)
誰能為我提供幫助,為什么Postgres會對一個查詢而不是另一個查詢使用索引? 運行VACUUM ANALYZE也無濟於事。
有什么辦法可以加快查詢速度嗎? 整個表的查詢大約需要20-30分鍾! 分區沒有太大區別,因為查詢跨越多個分區,並且當跨越的分區數量變大時,它僅顯示了幾分鍾的改進。我有一個具有以下結構的postgres表:
+---------+-------------+-------------+----------+---------+---------+
| id | timestamp | numvalues | text1 | text2 | text3 |
+---------+-------------+-------------+----------+---------+---------+
|abcd12344| 4124135 |[1,2,53,1241]| apple | banana | papaya |
+---------+-------------+-------------+----------+---------+---------+
id - random alphanumeric value<br>
timestamp - epoch timestamp<br>
numvalues - array of integers<br>
text(n) - text values
該表約有1.5億行。
我進行內部查詢,以獲取按數組值之一排序的數據的第n個百分點。 然后,我需要從數組中獲取多個值的平均值。 數組的長度約為31。
SELECT AVG(COALESCE(numvalues[2], 0))::NUMERIC(10,0), AVG(COALESCE(numvalues[3], 0))::NUMERIC(10,0)...AVG(COALESCE(numvalues[12], 0))::NUMERIC(10,0)
FROM (SELECT timestamp, numvalues, ntile(100)
OVER (ORDER BY numvalues[1]) pval FROM tablename WHERE timestamp >= somevalue and timestamp <= somevalue) innertable
WHERE pval >= x and pval <= y;
這將返回1.5億行中的約700萬行,約占表的5%。 但是,它不使用索引。 即使將enable_seqscan設置為off,運行EXPLAIN ANALYZE也會顯示它使用Seq掃描代替。
但是,類似的查詢:
SELECT text1, count(distinct(id))
FROM (SELECT timestamp, id, text1, numvalues, ntile(100)
OVER (ORDER BY numvalues[1]) pval FROM tablename WHERE timestamp >= somevalue and timestamp <= somevalue) innertable
WHERE pval >= x and pval <= y GROUP BY text1;
確實使用索引。
索引在時間戳列上
EXPLAIN ANALYZE的結果:
explain analyze select text1, count(distinct(id))
from (select timestamp, text1, numvalues, id, ntile(100) over (order by numvalues[1]) pval from table where timestamp >= 1431100800 and timestamp <= 1431108000 and numvalues[1] NOTNULL) innertable
where pval >= 90 and pval <= 90 group by text1;
QUERY PLAN
--------------------------------------------------------------------------------
GroupAggregate (cost=4554118.06..4554641.66 rows=1 width=28) (actual time=218641.221..219051.984 rows=20 loops=1)
-> Sort (cost=4554118.06..4554292.59 rows=69812 width=28) (actual time=218640.546..218728.294 rows=71441 loops=1)
Sort Key: innertable.text1
Sort Method: quicksort Memory: 8654kB
-> Subquery Scan on innertable (cost=4094722.75..4548501.27 rows=69812 width=28) (actual time=216502.946..218521.666 rows=71441 loops=1)
Filter: ((innertable.pval >= 90) AND (innertable.pval <= 90))
Rows Removed by Filter: 7072674
-> WindowAgg (cost=4094722.75..4339065.03 rows=13962416 width=118) (actual time=202276.333..211374.235 rows=7144115 loops=1)
-> Sort (cost=4094722.75..4129628.79 rows=13962416 width=118) (actual time=164912.487..190272.316 rows=7144115 loops=1)
Sort Key: (table.numvalues[9])
Sort Method: external merge Disk: 1387704kB
-> Index Scan using table_time_idx on table (cost=0.57..1578710.87 rows=13962416 width=118) (actual time=0.124..141014.505 rows=7144115 loops=1)
Index Cond: (("timestamp" >= 1431100800) AND ("timestamp" <= 1431108000))
Filter: (numvalues[1] IS NOT NULL)
Rows Removed by Filter: 7090075
Total runtime: 219340.709 ms
(16 rows)
explain analyze select avg(coalesce(numvalues[9], 0))::NUMERIC(10,0) mean9, avg(coalesce(numvalues[30],0))::NUMERIC(10,0), avg(coalesce(numvalues[8],0))::NUMERIC(10,0) - avg(coalesce(numvalues[30], 0))::NUMERIC(10,0) mean0, avg(coalesce(numvalues[10],0))::NUMERIC(10,0) - avg(coalesce(numvalues[8], 0))::NUMERIC(10,0) mean1, avg(coalesce(numvalues[14],0))::NUMERIC(10,0) - avg(coalesce(numvalues[10], 0))::NUMERIC(10,0) mean2, avg(coalesce(numvalues[13],0))::NUMERIC(10,0) - avg(coalesce(numvalues[14], 0))::NUMERIC(10,0) mean3, avg(coalesce(numvalues[9],0))::NUMERIC(10,0) - avg(coalesce(numvalues[13], 0))::NUMERIC(10,0) mean4
from (select timestamp, id, numvalues, ntile(100) over (order by numvalues[1] ) pval from table where timestamp >= 1431093600.00 and timestamp <= 1431100800.00 and numvalues[9] NOTNULL) innerTable
where pval >= 90.00 and pval <= 90.00 ;
QUERY PLAN
--------------------------------------------------------------------------------
Aggregate (cost=12662077.32..12662077.37 rows=1 width=82) (actual time=650343.769..650343.770 rows=1 loops=1)
-> Subquery Scan on innertable (cost=12634854.54..12661968.84 rows=3615 width=82) (actual time=647745.962..650232.725 rows=71441 loops=1)
Filter: (((innertable.pval)::numeric >= 90.00) AND ((innertable.pval)::numeric <= 90.00))
Rows Removed by Filter: 7072674
-> WindowAgg (cost=12634854.54..12647507.88 rows=723048 width=248) (actual time=632388.293..642338.237 rows=7144115 loops=1)
-> Sort (cost=12634854.54..12636662.16 rows=723048 width=248) (actual time=599893.771..617413.102 rows=7144115 loops=1)
Sort Key: (table.numvalues[9])
Sort Method: external merge Disk: 3214248kB
-> Seq Scan on req_p0swajch2t (cost=0.00..12480460.48 rows=723048 width=248) (actual time=0.041..575423.062 rows=7144115 loops=1)
Filter: ((numvalues[1] IS NOT NULL) AND (("timestamp")::numeric >= 1431100800.00) AND (("timestamp")::numeric <= 1431108000.00))
Rows Removed by Filter: 138191935
Total runtime: 650785.126 ms
(12 rows)
誰能為我提供幫助,為什么Postgres會對一個查詢而不是另一個查詢使用索引? 運行VACUUM ANALYZE也無濟於事。
有什么辦法可以加快查詢速度嗎? 整個表的查詢大約需要20-30分鍾! 分區沒有太大區別,因為查詢跨越多個分區,並且當跨越的分區數量變大時,它僅顯示了幾分鍾的改進。
我想您無法在評論中回復,因此我必須發布答案。
Explain analyze
表明,您的時間戳列與數值timestamp >= 1431093600.00 and timestamp <= 1431100800.00
,因此它們被timestamp >= 1431093600.00 and timestamp <= 1431100800.00
為數字:
Filter: ((numvalues[1] IS NOT NULL) AND (("timestamp")::numeric >= 1431100800.00) AND (("timestamp")::numeric <= 1431108000.00))
調查發生這種情況的原因並嘗試解決。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.