Tableau + Redshift 游标缓慢
[英]Tableau + Redshift slowness in cursors
问题描述
I know that Tableau Server uses cursors to refresh extracts, however, for running some simple queries that have large amounts of columns there is a large inconsistency in duration of the execution of some of this cursors.
我知道 Tableau Server 使用游标来刷新数据提取，但是，为了运行一些具有大量列的简单查询，其中一些游标的执行持续时间存在很大的不一致。 For example, when I run:例如，当我运行时：
select * from svl_qlog
where userid = (select usesysid from pg_user where usename='tableau')
order by starttime desc
limit 20;
I get:我得到：
+------+--------+--------+----+--------------------------+--------------------------+---------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------+------------+------------------------------+------------+
|userid|query |xid |pid |starttime |endtime |elapsed |aborted|label |substring |source_query|concurrency_scaling_status_txt|from_sp_call|
+------+--------+--------+----+--------------------------+--------------------------+---------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------+------------+------------------------------+------------+
|108 |14993377|36192048|3270|2021-08-24 03:34:48.862153|2021-08-24 03:38:09.404563|200542410|0 |default |fetch 100000 in "SQL_CUR7"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993370|36192035|3270|2021-08-24 03:34:41.174557|2021-08-24 03:34:41.185152|10595 |0 |default |fetch 100000 in "SQL_CUR6"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993368|36192034|3270|2021-08-24 03:34:40.991779|2021-08-24 03:34:41.021350|29571 |0 |default |Undoing 1 transactions on table 1728726 with current xid 361|NULL |0 - Ran on the main cluster |NULL |
|108 |14993367|36192032|3270|2021-08-24 03:34:40.861741|2021-08-24 03:34:40.907681|45940 |0 |default |fetch 100000 in "SQL_CUR3"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993365|36192025|3262|2021-08-24 03:34:38.135543|2021-08-24 03:34:38.229458|93915 |0 |default |fetch 100000 in "SQL_CUR7"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993363|36192022|3262|2021-08-24 03:34:38.006010|2021-08-24 03:34:38.008911|2901 |0 |default |fetch 100000 in "SQL_CUR6"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993360|36192020|3262|2021-08-24 03:34:37.250081|2021-08-24 03:34:37.885200|635119 |0 |default |Undoing 1 transactions on table 1728724 with current xid 361|NULL |0 - Ran on the main cluster |NULL |
|108 |14993359|36192018|3262|2021-08-24 03:34:35.811267|2021-08-24 03:34:35.865765|54498 |0 |default |fetch 100000 in "SQL_CUR3"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993287|36191920|2934|2021-08-24 03:33:16.921494|2021-08-24 03:33:38.143570|21222076 |0 |default |fetch 100000 in "SQL_CUR7"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993285|36191917|2934|2021-08-24 03:33:16.618563|2021-08-24 03:33:16.623745|5182 |0 |default |fetch 100000 in "SQL_CUR6"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993281|36191916|2934|2021-08-24 03:33:15.619813|2021-08-24 03:33:16.493711|873898 |0 |default |Undoing 1 transactions on table 1728722 with current xid 361|NULL |0 - Ran on the main cluster |NULL |
|108 |14993280|36191914|2934|2021-08-24 03:33:14.720016|2021-08-24 03:33:14.787236|67220 |0 |default |fetch 100000 in "SQL_CUR3"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993071|36191663|2258|2021-08-24 03:30:25.760462|2021-08-24 03:31:05.340131|39579669 |0 |default |fetch 100000 in "SQL_CUR7"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993069|36191660|2258|2021-08-24 03:30:25.359800|2021-08-24 03:30:25.366651|6851 |0 |default |fetch 100000 in "SQL_CUR6"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993064|36191657|2258|2021-08-24 03:30:25.170646|2021-08-24 03:30:25.245196|74550 |0 |default |Undoing 1 transactions on table 1728720 with current xid 361|NULL |0 - Ran on the main cluster |NULL |
|108 |14993063|36191655|2258|2021-08-24 03:30:25.045651|2021-08-24 03:30:25.079935|34284 |0 |default |fetch 100000 in "SQL_CUR3"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993053|36191642|2182|2021-08-24 03:30:18.163032|2021-08-24 03:30:18.381360|218328 |0 |default |fetch 100000 in "SQL_CUR7"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993050|36191638|2182|2021-08-24 03:30:18.029206|2021-08-24 03:30:18.032746|3540 |0 |default |fetch 100000 in "SQL_CUR6"; |NULL |0 - Ran on the main cluster |NULL |
|108 |14993043|36191620|2182|2021-08-24 03:30:15.207471|2021-08-24 03:30:15.853592|646121 |0 |default |Undoing 1 transactions on table 1728718 with current xid 361|NULL |0 - Ran on the main cluster |NULL |
|108 |14993042|36191618|2182|2021-08-24 03:30:14.086680|2021-08-24 03:30:14.131522|44842 |0 |default |fetch 100000 in "SQL_CUR3"; |NULL |0 - Ran on the main cluster |NULL |
+------+--------+--------+----+--------------------------+--------------------------+---------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------+------------+------------------------------+------------+
Just by eyeballing the elapsed column, you can tell the problem.只需观察经过的列，您就可以判断出问题所在。 So, my question is, is there a known reason for some of these cursors to take way longer?所以，我的问题是，是否有已知原因导致其中一些游标运行时间更长？ Also, is Tableau or Redshift the culprit here?另外，Tableau 或 Redshift 是这里的罪魁祸首吗？ And what's with these:这些是什么：
Undoing 1 transactions on table 1728722 with current xid 361
Thanks!谢谢！
1 个解决方案
解决方案1
0 2021-08-24 20:59:13
The first fetch to the cursor runs the query.对 cursor 的第一次提取运行查询。 Subsequent fetches just retrieves the next set of data that is already computed and waiting on the leader node.随后的提取只是检索已经计算并在领导节点上等待的下一组数据。 In your example it looks like your returned data is less than 10K rows as I don't see subsequent fetches for the same transaction.在您的示例中，您返回的数据似乎少于 10K 行，因为我没有看到同一事务的后续提取。 Is this correct or are you missing some fetches?这是正确的还是你错过了一些提取？ Since you say that you are refreshing extracts it seems like more data would be expected but I don't know your situation.既然你说你正在刷新提取物，那么似乎需要更多数据，但我不知道你的情况。
Given this it is likely that the different fetch times are due to differing query complexities and run times.鉴于此，不同的提取时间很可能是由于不同的查询复杂性和运行时间造成的。 If you look at activity (svl_statementtext) for these XIDs you can see what query is being sent to populate the cursor. You can then run these queries yourself and see how complex they are and general run times.如果您查看这些 XID 的活动 (svl_statementtext)，您可以看到正在发送什么查询来填充 cursor。然后您可以自己运行这些查询并查看它们的复杂程度和一般运行时间。
Other possibilities include queuing delays and/or high query load of the database and/or high load on the leader node.其他可能性包括排队延迟和/或数据库的高查询负载和/或领导节点上的高负载。
Do you know the size of the dataset being returned?您知道返回的数据集的大小吗？ If this is very very large then there is likely better ways to get the refresh data to Tableau.如果这非常非常大，那么可能有更好的方法将刷新数据发送到 Tableau。 Pulling data through the leader node with a select query but these methods take effort to set up.使用 select 查询通过领导节点拉取数据，但这些方法需要努力设置。
Questions to answer - What queries go with which fetches?要回答的问题 - 哪些查询 go 与哪些提取？ How long do these take to run standalone?这些需要多长时间才能独立运行？ How much data does each query return?每个查询返回多少数据？ Knowing these answers will help is setting a path for improvement.了解这些答案将有助于设置改进路径。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.