我如何优化这个复杂的 PostgreSQL 查询

Question

我得到了一个复杂的 PostgreSQL 查询，并被要求对其进行优化。 我在确定问题可能出在哪里方面取得了一些进展，但是我对如何优化它的想法已经用完了。 我希望得到一些指示/建议。

此查询是“AI Chatbot”系统的一部分，它返回与聊天机器人进行的唯一对话的数量，并带有一些过滤器。 返回的数据如下所示：

会话ID	用户身份	用户名	渠道	事件计数	操作员	第一个消息日期	最后消息日期	总唯一用户数
对话的ID	用户ID	用户名	对话期间发生的事件数	如果为真，则对话是与“管理员”或内部人员进行的	对话中第一条消息的日期	会话中最后一条消息的日期	唯一用户对话总数

查询的问题似乎与特定的“内部连接”子句有关，连接到一个大表（10540420 行）。

这是注释掉内部连接的查询，这样它将在大约 2 秒内运行。

select 
  "conversation_table"."conversationid" as "conversationId", 
  "du"."userid" as "userId", 
  "du"."username" as "userName", 
  "du"."channel" as "channel", 
  COALESCE(user_events.count, 0):: integer as "eventCount", 
  BOOL_OR(dc.operator_messages > 0) as "operator", 
  DATE_PART(
    'epoch', dc.first_message_timestamp
  )* 1000 as "firstMessageDate", 
  DATE_PART(
    'epoch', dc.last_message_timestamp
  )* 1000 as "lastMessageDate", 
  COUNT(*) OVER():: integer as "totalUniqueUsers" 
from 
  "fact_conversation_data" as "conversation_table" 
  inner join "dim_user" as "du" on "conversation_table"."dim_user_user_id" = "du"."user_id" 
  inner join "dim_time" as "dt" on "conversation_table"."dim_time_time_id" = "dt"."time_id" 
  inner join "dim_conversation" as "dc" on "conversation_table"."conversationid" = "dc"."conversationid" 
--   inner join (
--     select 
--       "conversationid" 
--     from 
--       "fact_milestone_event" as fme
--     where 
--       fme.dim_segment_segment_id in ('20736b82-4515-411f-9bc8-cf4d84ad69ac')
--       group by "conversationid"
--   ) as "fme" on "fme"."conversationid" = "conversation_table"."conversationid" 
  left join (
    select 
      "fme"."conversationid", 
      count("fme"."event_id") 
    from 
      "fact_milestone_event" as "fme" 
    where 
      "fme"."timestamp" >= '2022-12-18 11:00:00.000' 
      and "fme"."timestamp" <= '2023-01-18 10:59:59.999' 
      and "fme"."dim_tenant_tenant_id" = '4621ed8f-d8a4-46e2-a8de-5710751b16b9' 
    group by 
      "fme"."conversationid"
  ) as "user_events" on "conversation_table"."conversationid" = "user_events"."conversationid" 
where 
  "conversation_table"."timestamp" >= '2022-12-18 11:00:00.000' 
  and "conversation_table"."timestamp" <= '2023-01-18 10:59:59.999' 
  and "dt"."bot_zone" = 'Pacific/Auckland' 
  and "du"."is_platform_user" <> true 
  and "conversation_table"."conversationid" in (
    select 
      "notEmptyConservation_table"."conversationid" 
    from 
      (
        select 
          sum("user_messages") as "total_user_message_count", 
          "conversationid" 
        from 
          "fact_conversation_data" 
        where 
          "fact_conversation_data"."dim_tenant_tenant_id" = '4621ed8f-d8a4-46e2-a8de-5710751b16b9' 
          and "fact_conversation_data"."timestamp" >= '2022-12-18 11:00:00.000' 
          and "fact_conversation_data"."timestamp" <= '2023-01-18 10:59:59.999' 
        group by 
          "conversationid"
      ) as "notEmptyConservation_table" 
    where 
      "notEmptyConservation_table"."total_user_message_count" > 1
  ) 
  and "conversation_table"."dim_tenant_tenant_id" = '4621ed8f-d8a4-46e2-a8de-5710751b16b9' 
  and (
    "conversation_table"."user_messages" > 0
  ) 
group by 
  "user_events"."count", 
  "conversation_table"."conversationid", 
  "du"."userid", 
  "du"."username", 
  "du"."channel", 
  "dc"."first_message_timestamp", 
  "dc"."last_message_timestamp" 
order by 
  "lastMessageDate" asc 
limit 
  20

但是，如果我取消注释内部联接，我将获得大约 9 分钟的运行时间。

这是 explain.depesz.com 中慢速查询和执行计划的链接。

这是查询的更快版本的链接，没有有问题的内部联接。

以下是所涉及的两个主要表的索引， fact_milestone_event和fact_conversation_data 。

表名	索引名称	指标定义
事实_对话_数据	fact_conversation_data_conversationid_idx	使用 btree (conversationid) 在 public.fact_conversation_data 上创建索引 fact_conversation_data_conversationid_idx
事实_对话_数据	fact_conversation_data_tenant_id_idx	使用 btree (dim_tenant_tenant_id) 在 public.fact_conversation_data 上创建索引 fact_conversation_data_tenant_id_idx
事实_对话_数据	fact_conversation_data_timestamp_idx	使用 btree（时间戳）在 public.fact_conversation_data 上创建索引 fact_conversation_data_timestamp_idx
事实_对话_数据	conversationid_timeid_unique_idx	使用 btree (conversationid, dim_time_time_id) 在 public.fact_conversation_data 上创建唯一索引 conversationid_timeid_unique_idx
事实_对话_数据	fact_conversation_data_pk	使用 btree (conversation_data_id) 在 public.fact_conversation_data 上创建唯一索引 fact_conversation_data_pk
事实_里程碑_事件	fact_milestone_event_dim_segment_segment_id_idx	使用 btree (dim_segment_segment_id) 在 public.fact_milestone_event 上创建索引 fact_milestone_event_dim_segment_segment_id_idx
事实_里程碑_事件	fact_milestone_event_time_id_idx	使用 btree (dim_time_time_id) 在 public.fact_milestone_event 上创建索引 fact_milestone_event_time_id_idx
事实_里程碑_事件	fact_milestone_event_dim_milestone_milestone_id_idx	使用 btree (dim_milestone_milestone_id) 在 public.fact_milestone_event 上创建索引 fact_milestone_event_dim_milestone_milestone_id_idx
事实_里程碑_事件	fact_milestone_event_conversationid_idx	使用 btree (conversationid) 在 public.fact_milestone_event 上创建索引 fact_milestone_event_conversationid_idx
事实_里程碑_事件	fact_milestone_event_timestamp_idx	使用 btree（时间戳）在 public.fact_milestone_event 上创建索引 fact_milestone_event_timestamp_idx
事实_里程碑_事件	fact_milestone_event_dim_tenant_tenant_id_idx	使用 btree (dim_tenant_tenant_id) 在 public.fact_milestone_event 上创建索引 fact_milestone_event_dim_tenant_tenant_id_idx
事实_里程碑_事件	fact_milestone_event_dim_user_user_id_idx	使用 btree (dim_user_user_id) 在 public.fact_milestone_event 上创建索引 fact_milestone_event_dim_user_user_id_idx
事实_里程碑_事件	fact_milestone_event_pk	使用 btree (event_id) 在 public.fact_milestone_event 上创建唯一索引 fact_milestone_event_pk

我尝试用嵌套 select 的 WHERE 子句替换内部连接。我尝试针对查询运行EXPLAIN ，但无法理解 output。我尝试向内部连接添加更多条件（如下所示）。

inner join (
  select "conversationid" 
  from "fact_milestone_event" as fme
  where 
  fme.dim_segment_segment_id in ('20736b82-4515-411f-9bc8-cf4d84ad69ac') AND
  fme.timestamp >= '2022-12-18 11:00:00.000' AND
  fme.timestamp <= '2023-01-18 10:59:59.999' AND
  fme.dim_tenant_tenant_id = '4621ed8f-d8a4-46e2-a8de-5710751b16b9'
  group by "conversationid"
) as "fme" on "fme"."conversationid" = "conversation_table"."conversationid" 

Answer 1

我认为可以从连接中删除子选择，并将它们的逻辑移到更大的查询中。 与其重复约束，不如检查是否相等。

inner join fact_milestone_event as fme on
  fme.conversationid = conversation_table.conversationid
  and fme.dim_tenant_tenant_id = conversation_table.dim_tenant_tenant_id
  and fme.timestamp = conversation_table.timestamp
where 
  conversation_table.timestamp >= '2022-12-18 11:00:00.000' 
  and conversation_table.timestamp <= '2023-01-18 10:59:59.999' 
  and conversation_table.dim_tenant_tenant_id = '4621ed8f-d8a4-46e2-a8de-5710751b16b9'

notEmptyConservation_table 中的逻辑可以使用现有的 fme 连接。