OFFSET ... FETCH 在高分页值上很慢

Question

这是我的场景：

CREATE TABLE [dbo].[tblSMSSendQueueMain](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [SendMethod] [int] NOT NULL
 CONSTRAINT [PK_tblSMSSendQueueLog] PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

CREATE TABLE [dbo].[tblSMSSendQueueMainSendStatus](
    [ID] [bigint] IDENTITY(1,1) NOT NULL,
    [QueueID] [int] NULL,
    [SendStatus] [int] NULL,
    [StatusDate] [datetime] NULL,
    [UserID] [int] NULL,
 CONSTRAINT [PK_tblSMSSendQueueMainSendStatus] PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO

和一些指标：

CREATE NONCLUSTERED INDEX [IX_tblSMSSendQueueMainSendStatus_SendStatus_Single] ON [dbo].[tblSMSSendQueueMainSendStatus]
(
    [SendStatus] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO

CREATE NONCLUSTERED INDEX [IX_tblSMSSendQueueMain_SendMethod] ON [dbo].[tblSMSSendQueueMain]
(
    [SendMethod] ASC,
    [ID] DESC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO

每个表大约有 13m 行 tblSMSSendQueueMainSendStatus 的 QueueID 列是 tblSMSSendQueueMain 中 ID 列的外键。

服务器有一个 8 核 Xeon CPU 和 8GB RAM。

我为我的分页计划使用 offset 和 fetch，对于 100k 以下的偏移量来说它是完美的并且可以，但是当偏移量上升（超过 100k）时，查询响应很慢，并且需要大约 5 或 6 秒才能运行。

这是我的查询：

SELECT q.ID  
FROM tblSMSSendQueueMain q  
INNER JOIN tblSMSSendQueueMainSendStatus qs  
ON q.ID = qs.QueueID  
WHERE 1 = 1  
AND qs.SendStatus = 5  
AND [SendMethod] = 19  
ORDER BY q.ID desc OFFSET 10 * (1000000 - 1) ROWS  
FETCH NEXT 10 ROWS ONLY 

有谁知道我哪里出错了？

Answer 1

这么慢的原因是服务器获得正确起始行的唯一方法是读取它之前的每一行。

你最好使用Keyset Pagination 。 不是通过起始行号进行分页，而是传入起始键的参数。

为此，您必须返回一个或多个唯一的列，并且为了提高性能，它们应该被很好地索引。

@startingRow作为上一批次的最高ID ，你可以随意获取它。 例如，我使用了ORDER BY ，所以它将是最后一行，或者您的客户端应用程序将能够从变量中检索它。

SELECT TOP (10)
    q.ID  
FROM tblSMSSendQueueMain q  
INNER JOIN tblSMSSendQueueMainSendStatus qs  
ON q.ID = qs.QueueID  
WHERE 1 = 1  
   AND qs.SendStatus = 5  
   AND q.[SendMethod] = 19
   AND qs.ID > @startingRow    -- drop this line for the first query
ORDER BY qs.ID;

---

我必须说，你的查询有点奇怪。 如果外键是q.ID = qs.QueueID ，那么如果你只是查询q.ID ，你会得到多个相同的结果。 我怀疑你实际上只想要q.ID ，在这种情况下这是你的唯一键：

SELECT TOP (10) DISTINCT
    q.ID  
FROM tblSMSSendQueueMain q  
INNER JOIN tblSMSSendQueueMainSendStatus qs  
ON q.ID = qs.QueueID  
WHERE 1 = 1  
   AND qs.SendStatus = 5  
   AND q.[SendMethod] = 19
   AND q.ID > @startingRow    -- drop this line for the first query
ORDER BY q.ID;

或者，我更喜欢EXISTS/IN ，因为它更清楚地说明了要求：

SELECT TOP (10)
    q.ID  
FROM tblSMSSendQueueMain q  
WHERE 1 = 1  
   AND q.[SendMethod] = 19
   AND q.ID IN (
      SELECT qs.QueueID
      FROM tblSMSSendQueueMainSendStatus qs
      WHERE qs.SendStatus = 5
   )
   AND q.ID > @startingRow    -- drop this line for the first query
ORDER BY q.ID;