将巨大的项目列表作为参数传递给 SQL 服务器的最高效方法是什么？

Question

I have a table with millions of entries, and I'm loading data from it using a stored procedure (which applies pagination/sorting and some other things).

My problem is one of the parameters I pass in is a large list of CompanyId and EmployeeId values. These values are structured as CompanyId:EmployeeId and the string would look like this:

@companiesAndEmployees = 'CO5555:EM10001,CO5555:EM10002,CO5555:EM10003,CO7777:EM10004' --can by thousands of items long

CompanyId+EmployeeId is how I can uniquely identify the employees I want to load items from the database for. I use STRING_SPLIT() on this list to "filter" my query for just the employees I want.

SELECT *
FROM MyTable mt
    INNER JOIN STRING_SPLIT(@companiesAndemployees, ',') ce
        ON ce.value = mt.CompanyId + ':' + mt.EmployeeId

My problem is that this list of employees can theoretically be many thousands of items long, and this query begins to choke to the point where I get this SQL timeout exception:

System.Data.Entity.Core.EntityCommandExecutionException: An error occurred while executing the command definition. See the inner exception for details.

System.Data.SqlClient.SqlException: Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

System.ComponentModel.Win32Exception:

What would be the most performant way of retrieving rows only for the users I'm asking for, when there can be many thousands of them?

Answer 1

SqlBulkCopy is an extremely efficient way to get data to the SQL Server.

You can use it with a temporary table, then do a SELECT joining the temp table to MyTable .

---

I'm a bit unclear if you're applying this before or after a stored procedure. If you are doing it after, you can insert the result of the stored proc to a temp table, then join the two.

If you're doing it before, and need to pass this, things get a bit sketchy, but are possible: How to pass a temp table as a parameter into a separate stored procedure