为什么 postgres 查询需要这么长时间
[英]Why postgres query is taking so long
问题描述
I have a problematic query on a view which is taking more than 5 mins to run in aws rds production.
我对在 aws rds 生产中运行需要超过 5 分钟的视图有一个有问题的查询。 The query is simple and there are only 85000 records.查询很简单，只有85000条记录。 Not able to understand why it is taking so long.无法理解为什么要花这么长时间。 If I run the query from postgres client it takes 300ms, but the same query when its running from application, in New Relic it shows 5 mins or more.如果我从 postgres 客户端运行查询需要 300 毫秒，但是当它从应用程序运行时相同的查询，在 New Relic 中它显示 5 分钟或更长时间。 The query is :查询是：
select
esnpartvie0_.esn_id as col_0_0_,
esnpartvie0_.esn as col_1_0_,
esnpartvie0_.quarter_point as col_2_0_,
esnpartvie0_.work_order_number as col_3_0_,
esnpartvie0_.site as col_4_0_,
sum(esnpartvie0_.critical) as col_5_0_,
sum(esnpartvie0_.numshort) as col_6_0_,
sum(esnpartvie0_.wa) as col_7_0_,
esnpartvie0_.customer as col_8_0_,
esnpartvie0_.adj_accum_date as col_9_0_,
esnpartvie0_.g2_otr as col_10_0_,
esnpartvie0_.induct_date as col_11_0_,
min(esnpartvie0_.delta) as col_12_0_,
esnpartvie0_.fiscal_week_bucket_date as col_13_0_
from
moa.esn_part_view esnpartvie0_
where
esnpartvie0_.site_id =8
and esnpartvie0_.esn_id =1234
group by
esnpartvie0_.esn_id ,
esnpartvie0_.esn ,
esnpartvie0_.quarter_point ,
esnpartvie0_.work_order_number ,
esnpartvie0_.site ,
esnpartvie0_.customer ,
esnpartvie0_.adj_accum_date ,
esnpartvie0_.g2_otr ,
esnpartvie0_.induct_date ,
esnpartvie0_.fiscal_week_bucket_date
I also have the EXPLAIN ANALYZE BUFFERS FORMAT TEXT plan from pgadmin4 ...though couldnt understand much of it我也有来自 pgadmin4 的 EXPLAIN ANALYZE BUFFERS FORMAT TEXT 计划……虽然我看不懂很多
"GroupAggregate (cost=68482.59..68482.64 rows=1 width=82) (actual time=663.095..663.171 rows=1 loops=1)"
" Group Key: esnpartvie0_.esn_id, esnpartvie0_.esn, esnpartvie0_.quarter_point, esnpartvie0_.work_order_number, esnpartvie0_.site, esnpartvie0_.customer, esnpartvie0_.adj_accum_date, esnpartvie0_.g2_otr, esnpartvie0_.induct_date, esnpartvie0_.fiscal_week_bucket_date"
" Buffers: shared hit=19908, temp read=27667 written=6782"
" -> Sort (cost=68482.59..68482.60 rows=1 width=70) (actual time=663.087..663.163 rows=8 loops=1)"
" Sort Key: esnpartvie0_.esn, esnpartvie0_.quarter_point, esnpartvie0_.work_order_number, esnpartvie0_.site, esnpartvie0_.customer, esnpartvie0_.adj_accum_date, esnpartvie0_.g2_otr, esnpartvie0_.induct_date, esnpartvie0_.fiscal_week_bucket_date"
" Sort Method: quicksort Memory: 26kB"
" Buffers: shared hit=19908, temp read=27667 written=6782"
" -> Subquery Scan on esnpartvie0_ (cost=68482.49..68482.58 rows=1 width=70) (actual time=663.024..663.110 rows=8 loops=1)"
" Buffers: shared hit=19897, temp read=27667 written=6782"
" -> GroupAggregate (cost=68482.49..68482.57 rows=1 width=2016) (actual time=663.023..663.107 rows=8 loops=1)"
" Group Key: e.esn_id, w.number, ed.adj_accum_date, (COALESCE(ed.gate_2_otr, 0)), ed.gate_0_start, ed.gate_1_stop, p.part_id, st.name, mat.name, so.name, dr.name, hpc.hpc_status_name, module.module_name, c.customer_id, m.model_id, ef.engine_family_id, s.site_id, ws.name, ic.comment"
" Buffers: shared hit=19897, temp read=27667 written=6782"
" CTE indexed_comments"
" -> WindowAgg (cost=39839.28..44263.80 rows=221226 width=118) (actual time=165.462..272.574 rows=215274 loops=1)"
" Buffers: shared hit=5135, temp read=3293 written=3301"
" -> Sort (cost=39839.28..40392.34 rows=221226 width=110) (actual time=165.454..197.793 rows=215274 loops=1)"
" Sort Key: part_comment.part_id, part_comment.created_at DESC"
" Sort Method: external merge Disk: 26344kB"
" Buffers: shared hit=5135, temp read=3293 written=3301"
" -> Seq Scan on part_comment (cost=0.00..7344.26 rows=221226 width=110) (actual time=0.019..35.400 rows=215274 loops=1)"
" Buffers: shared hit=5132"
" -> Sort (cost=24218.69..24218.70 rows=1 width=717) (actual time=663.017..663.090 rows=8 loops=1)"
" Sort Key: w.number, ed.adj_accum_date, (COALESCE(ed.gate_2_otr, 0)), ed.gate_0_start, ed.gate_1_stop, p.part_id, st.name, mat.name, so.name, dr.name, hpc.hpc_status_name, module.module_name, c.customer_id, m.model_id, ef.engine_family_id, s.site_id, ws.name, ic.comment"
" Sort Method: quicksort Memory: 27kB"
" Buffers: shared hit=19897, temp read=27667 written=6782"
" -> Nested Loop (cost=1010.23..24218.68 rows=1 width=717) (actual time=412.426..663.017 rows=8 loops=1)"
" Buffers: shared hit=19894, temp read=27667 written=6782"
" -> Nested Loop Left Join (cost=1009.94..24210.36 rows=1 width=697) (actual time=412.389..662.811 rows=8 loops=1)"
" Join Filter: (ic.part_id = p.part_id)"
" Rows Removed by Join Filter: 467615"
" Buffers: shared hit=19870, temp read=27667 written=6782"
" -> Nested Loop Left Join (cost=1009.94..19218.95 rows=1 width=181) (actual time=56.592..57.839 rows=8 loops=1)"
" Buffers: shared hit=14735"
" -> Nested Loop Left Join (cost=1009.81..19218.35 rows=1 width=183) (actual time=56.583..57.812 rows=8 loops=1)"
" Buffers: shared hit=14735"
" -> Nested Loop Left Join (cost=1009.67..19217.66 rows=1 width=181) (actual time=56.576..57.790 rows=8 loops=1)"
" Buffers: shared hit=14735"
" -> Nested Loop Left Join (cost=1009.55..19216.82 rows=1 width=179) (actual time=56.567..57.759 rows=8 loops=1)"
" Buffers: shared hit=14735"
" -> Nested Loop Left Join (cost=1009.42..19216.04 rows=1 width=178) (actual time=56.552..57.699 rows=8 loops=1)"
" Buffers: shared hit=14719"
" -> Nested Loop Left Join (cost=1009.28..19215.37 rows=1 width=175) (actual time=56.536..57.638 rows=8 loops=1)"
" Buffers: shared hit=14703"
" -> Nested Loop Left Join (cost=1009.14..19214.77 rows=1 width=170) (actual time=56.514..57.556 rows=8 loops=1)"
" Join Filter: (e.work_scope_id = ws.work_scope_id)"
" Buffers: shared hit=14687"
" -> Nested Loop Left Join (cost=1009.14..19213.54 rows=1 width=166) (actual time=56.487..57.481 rows=8 loops=1)"
" Join Filter: (e.esn_id = p.esn_id)"
" Buffers: shared hit=14679"
" -> Nested Loop (cost=9.14..31.40 rows=1 width=125) (actual time=0.084..0.134 rows=1 loops=1)"
" Buffers: shared hit=15"
" -> Nested Loop (cost=8.87..23.08 rows=1 width=118) (actual time=0.072..0.122 rows=1 loops=1)"
" Buffers: shared hit=12"
" -> Nested Loop (cost=8.73..21.86 rows=1 width=108) (actual time=0.063..0.111 rows=1 loops=1)"
" Buffers: shared hit=10"
" -> Nested Loop (cost=8.60..21.65 rows=1 width=46) (actual time=0.052..0.100 rows=1 loops=1)"
" Buffers: shared hit=8"
" -> Hash Join (cost=8.31..13.34 rows=1 width=41) (actual time=0.043..0.089 rows=1 loops=1)"
" Hash Cond: (m.model_id = e.model_id)"
" Buffers: shared hit=5"
" -> Seq Scan on model m (cost=0.00..4.39 rows=239 width=17) (actual time=0.011..0.039 rows=240 loops=1)"
" Buffers: shared hit=2"
" -> Hash (cost=8.30..8.30 rows=1 width=28) (actual time=0.011..0.012 rows=1 loops=1)"
" Buckets: 1024 Batches: 1 Memory Usage: 9kB"
" Buffers: shared hit=3"
" -> Index Scan using esn_pkey on esn e (cost=0.29..8.30 rows=1 width=28) (actual time=0.008..0.008 rows=1 loops=1)"
" Index Cond: (esn_id = 140339)"
" Filter: active"
" Buffers: shared hit=3"
" -> Index Scan using work_order_pkey on work_order w (cost=0.29..8.30 rows=1 width=13) (actual time=0.009..0.009 rows=1 loops=1)"
" Index Cond: (work_order_id = e.work_order_id)"
" Buffers: shared hit=3"
" -> Index Scan using engine_family_pkey on engine_family ef (cost=0.14..0.20 rows=1 width=66) (actual time=0.010..0.010 rows=1 loops=1)"
" Index Cond: (engine_family_id = m.engine_family_id)"
" Buffers: shared hit=2"
" -> Index Scan using site_pkey on site s (cost=0.14..1.15 rows=1 width=14) (actual time=0.009..0.009 rows=1 loops=1)"
" Index Cond: (site_id = ef.site_id)"
" Buffers: shared hit=2"
" -> Index Scan using customer_pkey on customer c (cost=0.27..8.29 rows=1 width=11) (actual time=0.011..0.011 rows=1 loops=1)"
" Index Cond: (customer_id = e.customer_id)"
" Buffers: shared hit=3"
" -> Gather (cost=1000.00..19181.78 rows=29 width=41) (actual time=56.400..57.333 rows=8 loops=1)"
" Workers Planned: 2"
" Workers Launched: 2"
" Buffers: shared hit=14664"
" -> Parallel Seq Scan on part p (cost=0.00..18178.88 rows=12 width=41) (actual time=51.721..52.608 rows=3 loops=3)"
" Filter: (active AND (esn_id = 140339))"
" Rows Removed by Filter: 224748"
" Buffers: shared hit=14664"
" -> Seq Scan on work_scope ws (cost=0.00..1.10 rows=10 width=12) (actual time=0.006..0.006 rows=1 loops=8)"
" Buffers: shared hit=8"
" -> Index Scan using source_pkey on source so (cost=0.14..0.57 rows=1 width=13) (actual time=0.006..0.006 rows=1 loops=8)"
" Index Cond: (p.source_id = source_id)"
" Buffers: shared hit=16"
" -> Index Scan using status_pkey on status st (cost=0.13..0.56 rows=1 width=11) (actual time=0.004..0.004 rows=1 loops=8)"
" Index Cond: (p.status_id = status_id)"
" Buffers: shared hit=16"
" -> Index Scan using material_stream_pkey on material_stream mat (cost=0.13..0.56 rows=1 width=9) (actual time=0.004..0.004 rows=1 loops=8)"
" Index Cond: (p.material_stream_id = material_stream_id)"
" Buffers: shared hit=16"
" -> Index Scan using dr_status_pkey on dr_status dr (cost=0.13..0.56 rows=1 width=10) (actual time=0.001..0.001 rows=0 loops=8)"
" Index Cond: (p.dr_status_id = dr_status_id)"
" -> Index Scan using hpc_status_pkey on hpc_status hpc (cost=0.13..0.56 rows=1 width=10) (actual time=0.001..0.001 rows=0 loops=8)"
" Index Cond: (p.hpc_status_id = hpc_status_id)"
" -> Index Scan using module_pkey on module (cost=0.14..0.57 rows=1 width=6) (actual time=0.001..0.001 rows=0 loops=8)"
" Index Cond: (p.module_id = module_id)"
" -> CTE Scan on indexed_comments ic (cost=0.00..4977.59 rows=1106 width=520) (actual time=20.687..71.973 rows=58452 loops=8)"
" Filter: (comment_index = 1)"
" Rows Removed by Filter: 156822"
" Buffers: shared hit=5135, temp read=27667 written=6782"
" -> Index Scan using esn_dates_esn_id_key on esn_dates ed (cost=0.29..8.32 rows=1 width=20) (actual time=0.020..0.021 rows=1 loops=8)"
" Index Cond: (esn_id = 140339)"
" Filter: ((gate_3_stop_actual AND (gate_3_stop >= now())) OR (gate_3_stop IS NULL) OR ((NOT gate_3_stop_actual) AND (gate_3_stop IS NOT NULL) AND (gate_3_stop >= (now() - '730 days'::interval))))"
" Buffers: shared hit=24"
"Planning time: 6.580 ms"
"Execution time: 673.922 ms"
2 个解决方案
解决方案1
0 2021-10-07 03:52:41
It looks so this query returns 200K of rows.看起来这个查询返回 200K 行。 Some clients are very slow when they have to read lot of rows from server.某些客户端在必须从服务器读取大量行时速度非常慢。 Some times can be slow network communication.有时网络通信速度可能会很慢。
解决方案2
0 2021-10-07 06:58:44
If I run the query from postgres client it takes 300ms
So your query is not slow, your client (application code) is.所以你的查询并不慢，你的客户端（应用程序代码）是。
Few things to consider here:这里需要考虑的几件事：
- does the client actually need thousands of rows at once?客户端实际上一次需要数千行吗？ if this is for example user-facing UI, you probably want some pages, and you can limit what you get from DB to one page at the time例如，如果这是面向用户的 UI，您可能需要一些页面，并且您可以将您从 DB 获得的内容限制为一个页面
- does client "buffer" the results (storing them in some kind of list/dictionary), or does your code do any processing in streaming manner (as it gets rows from DB reader).客户端是否“缓冲”结果（将它们存储在某种列表/字典中），或者您的代码是否以流式方式进行任何处理（因为它从 DB 读取器获取行）。 Time to transfer results to client counts into "total query time", and if your client is slow to accept rows from reader, that is your reason.将结果传输到客户端的时间计入“总查询时间”，如果您的客户端从读取器接收行的速度很慢，这就是您的原因。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
为什么这个访问查询需要这么长时间？ - Why is this access query taking so long?
为什么执行此查询需要这么长时间 - Why is this query taking so long to execute
理解为什么SQL查询花了这么长时间 - Understanding why SQL query is taking so long
Postgres - 查询需要很长时间 - Postgres - query taking long time
为什么这个MySQL查询需要这么长时间，我该如何解决？ - Why is this MySQL query taking so long and how can I fix it?
为什么以下 Postgres SQL 查询需要这么长时间？ - Why does the following Postgres SQL query take so long?
为什么此查询在JSONB Gin索引字段上花费这么长时间？ 我可以修复它以便它实际使用索引吗？ - Why is this query taking so long on JSONB Gin index field? Can I fix it so it actually uses the index?
为什么查询花费的时间太长 - why query is taking too long
为什么仅索引扫描需要这么长时间？ - Why Index Only Scan is taking so long?
Postgres SQL 查询嵌套子查询耗时过长 - Postgres SQL Query With Nested Sub-Query Taking Too Long
相关标签