简体   繁体   English

替换 SQL 服务器中的 3 级深度嵌套游标

[英]Replacing a 3 level deep nested cursors in SQL Server

I have three SQL Server tables that I need to loop trhough and update.我有三个 SQL 服务器表需要循环遍历和更新。 I did it successfully with a cursor but it is so slow that it is pretty pointless sincethe main table with all the data to loop through is over 1,000 rows long.我用 cursor 成功地完成了它,但是它太慢了以至于毫无意义,因为要循环的所有数据的主表超过 1,000 行长。

The tables are (with some sample data):这些表是(带有一些示例数据):

-- The PK is InvoiceId and the IsMajorPart is '0' or '1'.
-- The MajorPartId and SubPartId1 to 4 are "technically" FKs for PartId but aren't hooked up and will not be ever due to some external issues outside of scope.
-- The part Id's can be NULL or empty.
-- This table exists elsewhere and is loaded with Id's being varchars but in transfering they will be going in as int's which is the proper way.
CREATE TABLE dbo.Invoices(
    InvoicdeId varchar(50),
    PartName varchar(255),
    IsMajorPart varchar(1)
    MajorPartId varchar(50),
    SubPartId1 varchar(50),
    SubPartId2 varchar(50),
    SubPartId3 varchar(50),
    SubPartId4 varchar(50));

-- Sampe inserts
INSERT INTO dbo.Invoices VALUES ('1', 'A Part', '0', '', '100', '105', '' ,''):
INSERT INTO dbo.Invoices VALUES ('5', 'E Part', '1', '101', '110', '', '' ,''):
INSERT INTO dbo.Invoices VALUES ('11', 'Z Part', '1', '201', '100', '115', '' ,''):


-- Essentially the old table above is being moved into a normalized, correct tables below.


- The PK is the PartId
CREATE TABLE dbo.Parts
    PartsId int,
    PartName varchar(255)

-- Sampe inserts (that will be updated or inserted by looping through the first table)
INSERT INTO dbo.Parts VALUES (100,'A Part'):
INSERT INTO dbo.Parts VALUES (110,'B Part'):
INSERT INTO dbo.Parts VALUES (201,'C Part'):


-- The PK is the combination of InvoiceId and PartId
CREATE TABLE dbo.InvoiceToParts
    InvoiceId int,
    PartsId int,
    IsMajorPart bit);

-- Sampe inserts (that will be inserted from the first table but conflicts might occur if an InvoiceId from the first table has 2 PartId's that are the same)
INSERT INTO dbo.Parts VALUES (1, 100, 0):
INSERT INTO dbo.Parts VALUES (5, 100, 1):
INSERT INTO dbo.Parts VALUES (17, 201, 0):

The sample INSERTs above are just samples of the data for seeing what is in the tables.上面的示例 INSERT 只是用于查看表中内容的数据示例。

The rules to move Invoices (I don't care what happens to this table), into the correct tables of Parts and InvoiceToParts are below (and these last two tables are the only ones that I care about.Invoices (我不关心这张表发生了什么)移动到正确的PartsInvoiceToParts表中的规则如下(最后两个表是我唯一关心的。

  1. Loop through Invoices and get all the data.遍历Invoices并获取所有数据。
  2. First, find out if IsMajorPart is '1' and then get the MajorPartId .首先,确定IsMajorPart是否为“1”,然后获取MajorPartId
  3. Push the MajorPartId with PartName in Parts table if it DOESN'T already exist.如果MajorPartIdPartName不存在,则将其推送到Parts表中。
  4. Next check InvoiceToParts to see if the PK of InvoiceId and PartId exist.接下来检查InvoiceToParts ,看InvoiceIdPartId的PK是否存在。
  5. If they do, update IsMaorPart to '1'.如果是, IsMaorPart更新为“1”。
  6. If they don't exist, INSERT it.如果它们不存在,请插入它。
  7. Next do the same process for all SubPartId1 to SubPartId4 .接下来对所有SubPartId1SubPartId4执行相同的过程。

I have a nested 3-level cursor which performance-wise ran for over 30min before I stopped it as it wasn't even close to finishing and was sucking up all the resources.我有一个嵌套的 3 级 cursor,它在性能方面运行了超过 30 分钟,然后我停止了它,因为它甚至没有接近完成并且正在吸收所有资源。 I am trying to look for a faster way to do this.我正在尝试寻找一种更快的方法来做到这一点。 The Invoices table can have up to about 5,000 rows in it. Invoices表最多可以包含大约 5,000 行。

You need to unpivot your data and then just do what is called an UPSERT, which has two steps:您需要取消数据透视,然后执行所谓的 UPSERT,它有两个步骤:

  1. If exists, update record(s)如果存在,更新记录
  2. If not exists, insert record(s)如果不存在,则插入记录

Plenty of examples if you search for examples online for UPSERT如果您在线搜索 UPSERT 的示例,则有很多示例

Table Setup表格设置

DROP TABLE IF EXISTS #Invoice
DROP TABLE IF EXISTS #Unpivot
DROP TABLE IF EXISTS #InvoiceToParts
DROP TABLE IF EXISTS #Parts


CREATE TABLE #Parts(
    PartsId int,
    PartName varchar(255)
)

CREATE TABLE #InvoiceToParts(
    InvoiceId int,
    PartsId int,
    IsMajorPart bit
);

CREATE TABLE #Invoice(
    InvoiceId varchar(50),
    PartName varchar(255),
    IsMajorPart varchar(1),
    MajorPartsID varchar(50),
    SubPartsID1 varchar(50),
    SubPartsID2 varchar(50),
    SubPartsID3 varchar(50),
    SubPartsID4 varchar(50)
);

INSERT INTO #Invoice
VALUES ('1', 'A Part', '0', '', '100', '105', '' ,'')
,('5', 'E Part', '1', '101', '110', '', '' ,'')
,('11', 'Z Part', '1', '201', '100', '115', '' ,'')

SQL to Process Data SQL 处理数据

Will first unpivot the data, then load into Parts table first so the ID's can be referenced before inserting into the junction table InvoicetoParts将首先对数据进行逆透视,然后首先加载到 Parts 表中,以便在插入联结表 InvoicetoParts 之前可以引用 ID

SELECT A.InvoiceId
    ,B.*
INTO #Unpivot
FROM #Invoice AS A
CROSS APPLY (
    VALUES 
    (NULLIF(MajorPartsID,''),PartName,IsMajorPart)
    ,(NULLIF(SubPartsID1,''),NULL,0)
    ,(NULLIF(SubPartsID2,''),NULL,0)
    ,(NULLIF(SubPartsID3,''),NULL,0)
    ,(NULLIF(SubPartsID4,''),NULL,0)
) AS B(PartsID,PartName,IsMajorPart)
WHERE B.PartsID IS NOT NULL /*If not data, filter out*/

/*INSERT into table Parts if not exists*/
INSERT INTO #Parts
SELECT PartsID,PartName
FROM #Unpivot AS A
WHERE A.IsMajorPart = 1
AND NOT EXISTS (
    SELECT *
    FROM #Parts AS DTA
    WHERE A.PartsID = DTA.PartsID
    )
GROUP BY PartsID,PartName

/*UPSERT into table dbo.InvoiceParts*/
UPDATE #InvoiceToParts
SET IsMajorPart = B.IsMajorPart
FROM #InvoiceToParts AS A
INNER JOIN #Unpivot AS B
     ON A.InvoiceId = B.InvoiceId
     AND A.PartsId = B.PartsID

INSERT INTO #InvoiceToParts(InvoiceId,PartsId,IsMajorPart)
SELECT InvoiceId
    ,PartsId
    ,IsMajorPart
FROM #Unpivot AS A
WHERE NOT EXISTS (
    SELECT *
    FROM #InvoiceToParts AS DTA
    WHERE A.InvoiceId = DTA.InvoiceID
        AND A.PartsID = DTA.PartsID
    )

SELECT *
FROM #InvoiceToParts

SELECT *
FROM #Parts

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM