如何使用太多参数简化批量SQL插入

Question

So I am trying to insert data that looks like this:

INSERT INTO RELATIONSHIP_CONFIG (USERID, WORKGROUPID, PRIORITY) VALUES 
    (@userId, @WorkgroupId10, @SmartFeedPriority10),
    (@userId, @WorkgroupId11, @SmartFeedPriority11),
    (@userId, @WorkgroupId12, @SmartFeedPriority12),
    (@userId, @WorkgroupId13, @SmartFeedPriority13);

Which generally is very simple and linear as all inserts happen one after the other and performs fine (I think).

The issue is that there is a hard limit with the number of SQL Parameters I am allowed to use- 2100. The upper limit edge case accounts for an insert that is quite a bit above that.

I was thinking about passing the data for WorkgroupId and SmartFeedPriority as a csvs and using a split function to create tables or something like that...

What is the best approach for dealing with data like this?

Maybe creating a stored procedure, passing the @UserId, @WorkgroupId (CSV), and @SmartFeedPriority (CSV) and having linear, one by one inserts done this way, but I am not too sure how the logic for this will look...

Answer 1

Looking at your question it's a bit difficult to suggest a good approach. I'm unable to see how and where the source of the data is.

I see you mentioned a CSV file. You can import data from a CSV file using the below script. Once the data is in a table, you can try one of the below examples.

IF OBJECT_ID('tempdb.dbo.#TempTable', 'U') IS NOT NULL
      DROP TABLE #TempTable ; 

    CREATE TABLE #TempTable 
    (
        [UserID] NVARCHAR(MAX) NULL,
        [WorkGroup] NVARCHAR(MAX) NULL,
        [SmartFeedPriority]  NVARCHAR(MAX) NULL,
    ) 

    BULK INSERT #TempTable
    FROM ' put your csv file path here '
    WITH
    (
    FIELDTERMINATOR = ',', -- comma delimited
    ROWTERMINATOR = '\n',
    CODEPAGE = 65001 --'65001'
    )

If the data you're trying to insert is from a table, you could try selecting the data into the table you need it to be.

Example :

SELECT [UserID], [WorkgroupID], [SmartFeedPriority]
INTO [dbo].[RELATIONSHIP_CONFIG]
FROM [dbo].[SorceTable]

If you would like to take the procedural route you can try the below. The below sample would work if the source of your data is in a table and you would like to individually insert each record.

Example :

procedure for insert

CREATE PROCEDURE [dbo].[CREATE_RELATIONSHIP_CONFIG](@UserId INT, @WorkgroupId INT, @SmartFeedPriority INT)
AS
BEGIN
  INSERT INTO RELATIONSHIP_CONFIG (UserId, WorkgroupId, Priority) VALUES 
(@userId, @WorkgroupId, @SmartFeedPriority)   
END

You can wrap the above procedure in a while loop. I've added a example for it below.

declare @UserId int;
declare @WorkgroupId int;
declare @Priority int;
WHILE EXISTS (SELECT [UserID] FROM #TempTable)
     BEGIN 
        SELECT TOP 1 @UserId=[UserID], @WorkgroupId=[WorkGroup] , @Priority=[SmartFeedPriority] FROM #TempTable

        EXEC [dbo].[CREATE_RELATIONSHIP_CONFIG] @UserId, @WorkgroupId, @Priority

        DELETE TOP (1)FROM #TempTable
     END