将“计算临时表中的中间结果”SQL模式调整为LINQ？

Question

My team builds a C# web application that can generate ad-hoc reports that recombine data from the same core set of SQL Server 2008 R2 tables in different ways. For example, a single "dashboard" page may combine a list of today's sales in each region, a list of lowest-selling items and their trends over the last week, a list of top-performing salespeople, and 20+ other metrics and charts. Underneath the covers, a typical dashboard page will require at least 30 queries across 20+ different tables.

Unfortunately, it's not practical to "freeze" this data and pre-compute it-- we need to fetch real-time data on-the-fly.

To make these pages fast, our trick has been to identify different queries that pull the same underlying data. Then we compute intermediate results from those underlying tables, cache those results into temp tables, and then join that temp table to other tables to compute the final results. Using this approach we can typically reduce I/O and time required for a particular dashboard by a factor of 10.

Our team would like to apply this same pattern to a similar page that uses LINQ-to-SQL for data access. We like LINQ for programming ease-of-use, for unit testing, etc. But performance stinks for the kind of application described above where we execute multiple queries that may partially depend on the same underlying data.

Of course I can call AsEnumerable() to materialize the intermediate query results, but if the intermediate results are large then getting the results in and out of SQL negates the performance win and creates inefficient parameterized queries with hundreds-item-long IN (@p1, ... ) clauses.

In a perfect world, LINQ-to-SQL would offer an AsServerEnumerable() method which would create a temporary table of intermediate results that I could re-use downstream without leaving the DB.

Does something like this exist?

If not, got any suggestions for how to make our "server-side intermediate materialization" pattern work well on LINQ?

PS - I'm saying "temporary table" and not "table variable" above because temp tables tend to work better with more expensive queries (parallel query plans, non-clustered indexes, etc.). But otherwise all of above would apply to table variables too.

Answer 1

No, that does not exist in raw LINQ and is not pre-canned in any LINQ-style API I'm aware of.

It can exist if you ignore the "LINQ" part of LINQ-to-SQL and just use the db.ExecuteQuery<T>(sql, args) approach, but if you do that you must take care to ensure you are passing an explicit and open connection to the data-context (if you use the connection-string approach, the connection management is handled automatically, and you are not guaranteed to get the same connection between operations - it could be taken from the pool, and as such even if it is the same underlying connection , it will have been reset, dropping any temp tables).

Answer 2

Well, if you have a lot of reads you can consider creating a VIEW instead of a temporary table and add a clustered index to that view. This will materialize the view in the database.

Indexed views can be used by SQL Server two different ways. First, the view can be called directly from a query, as conventional views are currently used. But instead of running the view's underlying SELECT statement and creating the view's result set on the fly, it uses the unique clustered index to display the results of the view almost immediately. Second, any query that is run on SQL Server 2000/2005 is automatically evaluated to see if any existing indexed views exist that would fulfill the query. If so, the Query Optimizer uses the indexed query, even though it has not been specified in the query, greatly speeding the query.

More info: Sql server performance