Redshift：在 2 个段之间遇到缓慢的查询性能
[英]Redshift: experiencing slow query performance between 2 segments
问题描述
We're experiencing slow query performance on AWS Redshift.
我们在 AWS Redshift 上遇到查询性能缓慢的问题。 Frequently we see that queries can take ±12 seconds to run, but only very little time (<500ms) is spent actually executing the query (according to the AWS Redshift console for an individual query).我们经常看到查询可能需要 ±12 秒的时间才能运行，但实际执行查询只花费了很少的时间（<500 毫秒）（根据单个查询的 AWS Redshift 控制台）。
Querying from svl_compile we can confirm that the query compilation plan is already compiled.从 svl_compile 查询我们可以确认查询编译计划已经编译。
In svl_query_report we see a long time delay between the start times of 2 segments accounting for the majority of the run time, although the segments themselves all execute very quickly (milliseconds)在 svl_query_report 中，我们看到占运行时间大部分的 2 个段的开始时间之间有很长的延迟，尽管段本身都执行得非常快（毫秒）
1 个解决方案
解决方案1
1 2022-08-22 16:22:19
There are a number of things that could be going on but I suspect network distribution is involved.可能会发生很多事情，但我怀疑涉及到网络分发。 Check STL_DIST.检查 STL_DIST。
Another possibility is that Redshift broke the query up and a subquery is running during that window.另一种可能性是 Redshift 中断了查询，并且在 window 期间运行了一个子查询。 This can happen with very complex queries.这可能发生在非常复杂的查询中。 Review the plan and see if there are any references to computer generated table names (I think they begin with't' but this is just from memory).查看计划并查看是否有任何对计算机生成的表名的引用（我认为它们以't'开头，但这只是来自记忆）。
Spilling to disk could be happening but this seems unlikely given what you have said so far.可能会发生磁盘溢出，但鉴于您目前所说的，这似乎不太可能。 Also queuing delays doesn't seem like a match.排队延迟似乎也不匹配。 Both are possible but not likely.两者都有可能，但不太可能。
If you post more info about how the query is running things will narrow down.如果您发布有关查询如何运行的更多信息，事情将会缩小。 Actual execution report, explain plan, and/or logging table info would help hone in on what is happening during this time window.实际执行报告、解释计划和/或记录表信息将有助于了解在此期间发生的情况 window。
解决方案2
0 2022-08-23 09:31:46
I work with the original poster.我使用原始海报。 I am commenting here to provide more context, with an example query and all details.我在这里发表评论是为了提供更多上下文，包括示例查询和所有详细信息。 Below the query plan, results from svl_compile, stl_dist, svl_query_report, and stl_wlm_query.在查询计划下方，来自 svl_compile、stl_dist、svl_query_report 和 stl_wlm_query 的结果。
We're using a star schema.我们正在使用星型模式。 Our fact table (question_answers), has 25M rows and is using an EVEN Distribution style.我们的事实表 (question_answers) 有 2500 万行，并使用 EVEN 分布样式。 Dimension tables all have an ALL Distribution style.维度表都具有 ALL 分布样式。 We have a d2.large cluster with 4 nodes.我们有一个有 4 个节点的 d2.large 集群。
What we frequently see is that there's a long delay between segments before scanning the fact table.我们经常看到的是，在扫描事实表之前，段之间存在很长的延迟。 The query spends no time in queue.查询不会在队列中花费时间。
Appreciate any help you can offer.感谢您提供的任何帮助。
Here's the query plan:这是查询计划：
-> XN HashAggregate (cost=1143.57..1143.63 rows=1 width=25)
-> XN Hash Join DS_DIST_ALL_NONE (cost=108.15..1143.55 rows=1 width=25)
Hash Cond: ((("outer".questionid)::text = ("inner".questionid)::text) AND ("outer".uniqueformid = "inner".uniqueformid) AND ("outer".companyid = "inner".companyid) AND ("outer".defaultlanguageid = "inner".languageid))
-> XN Hash Left Join DS_DIST_ALL_NONE (cost=103.94..1085.97 rows=593 width=55)
Hash Cond: (("outer".invitationid = "inner".invitationid) AND ("outer".companyid = "inner".companyid))
-> XN Hash Join DS_DIST_ALL_NONE (cost=0.28..982.14 rows=6 width=59)
Hash Cond: (("outer".companyid)::bigint = "inner".companyid)
-> XN Hash Join DS_DIST_ALL_NONE (cost=0.26..982.00 rows=3 width=57)
Hash Cond: (("outer".uniqueformid = "inner".uniqueformid) AND ("outer".companyid = "inner".companyid))
-> XN Seq Scan on question_answers fact_table (cost=0.00..369.48 rows=24489 width=41)
Filter: ((companyid = 28349) AND ((ratingdeleted <> true) OR (ratingdeleted IS NULL)))
-> XN Hash (cost=0.25..0.25 rows=1 width=16)
-> XN Seq Scan on forms (cost=0.00..0.25 rows=1 width=16)
Filter: ((initcap("replace"((journeytype)::text, '_'::text, ' '::text)) ~~* 'Withdrawn'::text) AND (companyid = 28349) AND (CASE WHEN ((formdeleted)::text = 'yes'::text) THEN 'Deleted'::text WHEN ((formdeleted)::text = 'archive'::text) THEN 'Archive'::text ELSE 'Active'::text END = 'Active'::text))
-> XN Hash (cost=0.01..0.01 rows=2 width=14)
-> XN Seq Scan on companies (cost=0.00..0.01 rows=2 width=14)
Filter: (companyid = 28349)
-> XN Hash (cost=74.04..74.04 rows=5924 width=8)
-> XN Seq Scan on tags (cost=0.00..74.04 rows=5924 width=8)
Filter: (companyid = 28349)
-> XN Hash (cost=2.34..2.34 rows=187 width=42)
-> XN Seq Scan on questions (cost=0.00..2.34 rows=187 width=42)
Filter: (companyid = 28349)
svl_query_report (for slice=0) svl_query_report（切片=0）
+---------+---------+-------+------+-------------------------------+-------------------------------+--------------+------+--------+-----------------------------------------+--------------+----------+-----------+-----------------+-----------------+
| query | segment | slice | step | start_time | end_time | elapsed_time | rows | bytes | label | is_diskbased | workmem | is_rrscan | is_delayed_scan | rows_pre_filter |
+---------+---------+-------+------+-------------------------------+-------------------------------+--------------+------+--------+-----------------------------------------+--------------+----------+-----------+-----------------+-----------------+
| 3494144 | 0 | 0 | 0 | 2022-08-22T14:55:23.545+00:00 | 2022-08-22T14:55:23.644+00:00 | 99206 | 163 | 8324 | scan tbl=461589 name=questions | f | 0 | t | f | 179728 |
| 3494144 | 0 | 0 | 1 | 2022-08-22T14:55:23.545+00:00 | 2022-08-22T14:55:23.644+00:00 | 99206 | 163 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 0 | 0 | 2 | 2022-08-22T14:55:23.545+00:00 | 2022-08-22T14:55:23.644+00:00 | 99206 | 163 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 0 | 0 | 3 | 2022-08-22T14:55:23.545+00:00 | 2022-08-22T14:55:23.644+00:00 | 99206 | 163 | 11032 | hash tbl=1219 | f | 66618654 | f | f | 0 |
| 3494144 | 1 | 0 | 0 | 2022-08-22T14:55:23.660+00:00 | 2022-08-22T14:55:23.673+00:00 | 12380 | 5607 | 89712 | scan tbl=461601 name=tags | f | 0 | t | f | 254143 |
| 3494144 | 1 | 0 | 1 | 2022-08-22T14:55:23.660+00:00 | 2022-08-22T14:55:23.673+00:00 | 12380 | 5607 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 1 | 0 | 2 | 2022-08-22T14:55:23.660+00:00 | 2022-08-22T14:55:23.673+00:00 | 12380 | 5607 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 1 | 0 | 3 | 2022-08-22T14:55:23.660+00:00 | 2022-08-22T14:55:23.673+00:00 | 12380 | 5607 | 134568 | hash tbl=1223 | f | 52462878 | f | f | 0 |
| 3494144 | 2 | 0 | 0 | 2022-08-22T14:55:23.726+00:00 | 2022-08-22T14:55:23.727+00:00 | 548 | 1 | 22 | scan tbl=461569 name=companies | f | 0 | t | f | 1248 |
| 3494144 | 2 | 0 | 1 | 2022-08-22T14:55:23.726+00:00 | 2022-08-22T14:55:23.727+00:00 | 548 | 1 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 2 | 0 | 2 | 2022-08-22T14:55:23.726+00:00 | 2022-08-22T14:55:23.727+00:00 | 548 | 1 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 2 | 0 | 3 | 2022-08-22T14:55:23.726+00:00 | 2022-08-22T14:55:23.727+00:00 | 548 | 1 | 32 | hash tbl=1517 | f | 50331648 | f | f | 0 |
| 3494144 | 3 | 0 | 0 | 2022-08-22T14:55:23.790+00:00 | 2022-08-22T14:55:23.798+00:00 | 8769 | 2 | 64 | scan tbl=461581 name=forms | f | 0 | t | f | 9177 |
| 3494144 | 3 | 0 | 1 | 2022-08-22T14:55:23.790+00:00 | 2022-08-22T14:55:23.798+00:00 | 8769 | 2 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 3 | 0 | 2 | 2022-08-22T14:55:23.790+00:00 | 2022-08-22T14:55:23.798+00:00 | 8769 | 2 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 3 | 0 | 3 | 2022-08-22T14:55:23.790+00:00 | 2022-08-22T14:55:23.798+00:00 | 8769 | 2 | 64 | hash tbl=1735 | f | 50331648 | f | f | 0 |
| 3494144 | 4 | 0 | 0 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 62125 | scan tbl=461597 name=question_answers | f | 0 | t | f | 243094 |
| 3494144 | 4 | 0 | 1 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 2 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 3 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | hjoin tbl=1735 | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 4 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 5 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 6 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | hjoin tbl=1517 | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 7 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 8 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 9 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | hjoin tbl=1223 | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 10 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 11 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 12 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | hjoin tbl=1219 | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 13 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 14 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1226 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 4 | 0 | 15 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 1 | 64 | aggr tbl=1795 | f | 4226880 | f | f | 0 |
| 3494144 | 4 | 0 | 16 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 7462 | 0 | 1188 | dist | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 0 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | scan tbl=636400 name=Internal Worktable | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 1 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | aggr tbl=1852 | f | 16907520 | f | f | 0 |
| 3494144 | 5 | 0 | 2 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 3 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | limit | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 4 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | project | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 5 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | return | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 6 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | merge | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 7 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | aggr tbl=1070 | f | 0 | f | f | 0 |
| 3494144 | 5 | 0 | 8 | 2022-08-22T14:55:33.110+00:00 | 2022-08-22T14:55:33.152+00:00 | 41784 | 0 | 0 | project | f | 0 | f | f | 0 |
+---------+---------+-------+------+-------------------------------+-------------------------------+--------------+------+--------+-----------------------------------------+--------------+----------+-----------+-----------------+-----------------+
svl_compile: svl_编译：
+---------+---------+-------+-------------------------------+-------------------------------+---------+
| query | segment | locus | starttime | endtime | compile |
+---------+---------+-------+-------------------------------+-------------------------------+---------+
| 3494144 | 0 | 1 | 2022-08-22T14:55:23.538+00:00 | 2022-08-22T14:55:23.538+00:00 | 0 |
| 3494144 | 1 | 1 | 2022-08-22T14:55:23.577+00:00 | 2022-08-22T14:55:23.577+00:00 | 0 |
| 3494144 | 2 | 1 | 2022-08-22T14:55:23.658+00:00 | 2022-08-22T14:55:23.658+00:00 | 0 |
| 3494144 | 3 | 1 | 2022-08-22T14:55:23.719+00:00 | 2022-08-22T14:55:23.719+00:00 | 0 |
| 3494144 | 4 | 1 | 2022-08-22T14:55:33.100+00:00 | 2022-08-22T14:55:33.100+00:00 | 0 |
| 3494144 | 5 | 1 | 2022-08-22T14:55:33.099+00:00 | 2022-08-22T14:55:33.099+00:00 | 0 |
| 3494144 | 6 | 2 | 2022-08-22T14:55:33.100+00:00 | 2022-08-22T14:55:33.100+00:00 | 0 |
+---------+---------+-------+-------------------------------+-------------------------------+---------+
STL_DIST: STL_DIST：
+---------+-------+---------+------+-------------------------------+-------------------------------+---------+------+-------+---------+---------------+-------------------+
| query | slice | segment | step | starttime | endtime | tasknum | rows | bytes | packets | is_intra_node | distribute_by_lds |
+---------+-------+---------+------+-------------------------------+-------------------------------+---------+------+-------+---------+---------------+-------------------+
| 3494144 | 4 | 4 | 16 | 2022-08-22T14:55:33.120+00:00 | 2022-08-22T14:55:33.139+00:00 | 95 | 0 | 652 | 1 | f | f |
| 3494144 | 1 | 4 | 16 | 2022-08-22T14:55:33.132+00:00 | 2022-08-22T14:55:33.144+00:00 | 92 | 0 | 676 | 1 | f | f |
| 3494144 | 2 | 4 | 16 | 2022-08-22T14:55:33.115+00:00 | 2022-08-22T14:55:33.116+00:00 | 117 | 0 | 0 | 0 | f | f |
| 3494144 | 0 | 4 | 16 | 2022-08-22T14:55:33.111+00:00 | 2022-08-22T14:55:33.119+00:00 | 75 | 0 | 1188 | 1 | f | f |
| 3494144 | 3 | 4 | 16 | 2022-08-22T14:55:33.118+00:00 | 2022-08-22T14:55:33.129+00:00 | 99 | 0 | 792 | 1 | f | f |
| 3494144 | 5 | 4 | 16 | 2022-08-22T14:55:33.118+00:00 | 2022-08-22T14:55:33.143+00:00 | 118 | 0 | 744 | 1 | f | f |
| 3494144 | 6 | 4 | 16 | 2022-08-22T14:55:33.119+00:00 | 2022-08-22T14:55:33.130+00:00 | 93 | 0 | 624 | 1 | f | f |
| 3494144 | 7 | 4 | 16 | 2022-08-22T14:55:33.114+00:00 | 2022-08-22T14:55:33.132+00:00 | 90 | 0 | 620 | 1 | f | f |
+---------+-------+---------+------+-------------------------------+-------------------------------+---------+------+-------+---------+---------------+-------------------+
STL_WLM_QUERY: STL_WLM_QUERY：
+---------+---------------+------------+-------------------------------+-------------------------------+-------------------------------+------------------+-------------------------------+-------------------------------+-----------------+-------------------------------+-------------+--------------+----------------+
| query | service_class | slot_count | service_class_start_time | queue_start_time | queue_end_time | total_queue_time | exec_start_time | exec_end_time | total_exec_time | service_class_end_time | final_state | est_peak_mem | query_priority |
+---------+---------------+------------+-------------------------------+-------------------------------+-------------------------------+------------------+-------------------------------+-------------------------------+-----------------+-------------------------------+-------------+--------------+----------------+
| 3494144 | 6 | 1 | 2022-08-22T14:55:23.462+00:00 | 2022-08-22T14:55:23.463+00:00 | 2022-08-22T14:55:23.463+00:00 | 0 | 2022-08-22T14:55:23.463+00:00 | 2022-08-22T14:55:33.158+00:00 | 9694533 | 2022-08-22T14:55:33.158+00:00 | Completed | 111149056 | n/a |
+---------+---------------+------------+-------------------------------+-------------------------------+-------------------------------+------------------+-------------------------------+-------------------------------+-----------------+-------------------------------+-------------+--------------+----------------+
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
Redshift-查询性能问题 - Redshift — Query Performance Issues
亚马逊红移中的并发查询性能 - concurrent query performance in amazon redshift
Athena Query - “NOT IN” 性能缓慢 - Athena Query - "NOT IN" slow performance
长时间运行的查询-改善Redshift性能的建议 - Long Running Query - Recommendations to improve performance in Redshift
Amazon Redshift - 复制 - 数据加载与查询性能问题 - Amazon Redshift - Replication - Data load Vs Query Performance Issues
Redshift JSON性能 - Redshift JSON Performance
Redshift Spectrum的性能问题 - Performance issues with Redshift Spectrum
在 Amazon Redshift 和 Amazon RDS 之间查询和连接数据 - Query and join data between Amazon Redshift and Amazon RDS
Parquet 数据到 AWS Redshift 速度很慢 - Parquet data to AWS Redshift slow
在 Redshift 上按 random() 排序非常慢 - Order by random() on Redshift is horribly slow
相关标签