Is there a way to reuse variables during executemany?

Question

I'm working on some ETL scripts and I have run into a problem where I'm using a MERGE statement. I am using executemany to run a SQL script that looks like this:

MERGE myTable as target
USING (
  SELECT myID from myTable
  WHERE myID = ?
) AS source (myID) ON (target.myID = source.myID)
WHEN NOT MATCHED THEN
INSERT (myID, myName, myProperty, myOtherProperty)
VALUES(?, ?, ?, ?)
WHEN MATCHED THEN
UPDATE SET myName=?, myProperty=?, etc...;

The issue here is that I need the myID property in multiple places. Both in the SELECT statement, then in both the INSERT and UPDATE statements as well. You cannot reuse properties in the way I have done it because the ? s are filled in with the properties from the dataframe in order and I cannot find a way to reuse the properties.

From my understanding, my last ditch effort should be using a for loop to iterate through the dataframe and do single executes on each row. In my mind, that's the only way I can assign some variables to then use to build my SQL query, however, I'd love to know if there's an easier or better way to go about this.

Answer 1

Yes. You can assign the parameters to local variables and reuse them:

declare @myId int = ?;
declare @myName nvarchar(200) = ?;
declare @myProperty float = ?;
declare @myOtherProperty datetime2(7) = ?;

MERGE myTable as target
USING (
  SELECT myID from myTable
  WHERE myID = @myId
) AS source (myID) ON (target.myID = source.myID)
WHEN NOT MATCHED THEN
INSERT (myID, myName, myProperty, myOtherProperty)
VALUES(@myId, @myName, @myProperty, @myOtherProperty)
WHEN MATCHED THEN
UPDATE SET myName=@myName, myProperty=@myProperty, etc...;

The key enabler here is that you are sending a TSQL batch to SQL Server. It doesn't have to be a single statement.

If you want to send lots of data to SQL Server, though, you can also just send a JSON document using a single NVarchar(max) parameter, and parse it into a table using OPENJSON on the server. That would enable you to send a whole table of data to SQL Server to use in the MERGE.