[英]Is there anything faster than SqlDataReader in .NET?
I need to load one column of strings from table on SqlServer into Array in memory using C#. 我需要使用C#将SqlServer上的表中的一列字符串加载到内存中的Array中。 Is there a faster way than open SqlDataReader and loop through it. 有没有比打开SqlDataReader更快的方法并循环它。 Table is large and time is critical. 表很大,时间很关键。
EDIT I am trying to build .dll and use it on server for some operations on database. 编辑我正在尝试构建.dll并在服务器上使用它来进行数据库上的某些操作。 But it is to slow for now. 但现在要放慢速度。 If this is fastest than I have to redesign the database. 如果这比我重新设计数据库要快。 I tough there may be some solution how to speed thing up. 我很难有可能有一些解决方案如何加快速度。
Data Reader 数据阅读器
About the fastest access you will get to SQL is with the SqlDataReader . 关于最快的访问,你将获得SQL与SqlDataReader 。
Profile it 简介它
It's worth actually profiling where your performance issue is. 值得实际分析您的性能问题。 Usually, where you think the performance issue is, is proven to be totally wrong after you've profiled it. 通常,在您认为性能问题出现之后,在您对其进行分析后证明是完全错误的。
For example it could be: 例如,它可能是:
Profiling each of these in isolation will give you a better idea of where your bottleneck is. 单独分析这些内容可以让您更好地了解瓶颈的位置。 For profiling your code, there is a great article from Microsoft 为了分析您的代码,有一篇来自Microsoft的精彩文章
Cache it 缓存它
The thing to look at to improve performance is to work out if you need to load all that data every time. 看的东西在提高性能的工作, 如果你需要在每次加载所有的数据。 Can the list (or part of it) be cached? 可以缓存列表(或其中的一部分)吗? Take a look at the new System.Runtime.Caching namespace. 看一下新的System.Runtime.Caching命名空间。
Rewrite as T-SQL 重写为T-SQL
If you're doing purely data operations (as your question suggests), you could rewrite your code which is using the data to be T-SQL and run natively on SQL. 如果您正在进行纯粹的数据操作(正如您的问题所示),您可以重写使用数据为T-SQL并在SQL上本机运行的代码。 This has the potential to be much faster, as you will be working with the data directly and not shifting it about. 这有可能更快,因为您将直接处理数据而不是转移它。
If your code has a lot of necessary procedural logic, you can try mixing T-SQL with CLR Integration giving you the benefits of both worlds. 如果您的代码具有许多必要的过程逻辑,您可以尝试将T-SQL与CLR集成混合,为您提供两个世界的好处。
This very much comes down to the complexity (or more procedural nature) of your logic. 这很大程度上归结为逻辑的复杂性(或更多程序性)。
If all else fails 如果一切都失败了
If all areas are optimal (or as near as), and your design is without fault. 如果所有区域都是最佳的(或接近),并且您的设计没有错误。 I wouldn't even get into micro-optimisation, I'd just throw hardware at it . 我甚至不会进入微优化,我只是扔硬件 。
What hardware? 什么硬件? Try the reliability and performance monitor to find out where the bottle neck is. 尝试使用可靠性和性能监视器来找出瓶颈的位置。 Most likely place for the problem you describe HDD or RAM. 最有可能解决您描述HDD或RAM的问题。
如果SqlDataReader
不够快,也许你应该将你的东西存储在其他地方,比如(内存中)缓存。
No. It is actually not only the fastest way - it is the ONLY (!) way. 不,它实际上不仅是最快的方式 - 它是唯一的(!)方式。 All other mechanisms INTERNALLY use a DataReader anyway. 所有其他机制无论如何都在内部使用DataReader。
我怀疑SqlDataReader
与你将获得的一样好。
SqlDataReader is the fastest way. SqlDataReader是最快的方法。 Make sure you use the get by ordinal methods rather than get by column name. 确保使用get by序数方法而不是按列名称获取。 eg GetString(1); 例如GetString(1);
Also worthwhile is experimenting with MinPoolSize in the connection string so that there are always some connections in the pool. 同样值得的是在连接字符串中试验MinPoolSize,以便池中始终存在一些连接。
The SqlDataReader will be the fastest way. SqlDataReader将是最快的方式。 Optimize the use of it, by using the appropriate Getxxx method , which takes an ordinal as parameter. 通过使用适当的Getxxx方法优化其使用,该方法采用序数作为参数。
If it is not fast enough, see if you can tweak your query. 如果速度不够快,请查看是否可以调整查询。 Put a covering index on the column (s) that you want to retrieve. 在要检索的列上放置覆盖索引。 By doing so, Sql Server only has to read the index, and does not have to go to the table directly to retrieve all the info that is required. 通过这样做,Sql Server只需读取索引,而不必直接转到表中以检索所需的所有信息。
What about transforming one column of rows to one row of columns, and having only one row to read? 如何将一列行转换为一行列,并且只读取一行? SqlDataReader
has an optimization for reading a single row ( System.Data.CommandBehavior.SingleRow
argument of ExecuteReader
), so maybe it can improve the speed a bit. SqlDataReader
具有读取单行的优化( ExecuteReader
System.Data.CommandBehavior.SingleRow
参数),因此可能它可以提高一点速度。
I see several advantages: 我看到了几个优点:
reader[0]
), 无需在每次迭代( reader[0]
)上访问数组, reader
) to another one may be faster than looping through elements and adding each one to a new array. 将数组( reader
)克隆到另一个数组可能比循环遍历元素并将每个数组添加到新数组更快。 On the other hand, it has a disadvantage to force SQL database to do more work. 另一方面,强制SQL数据库做更多工作也有一个缺点。
"Provides a way of reading a forward-only stream of rows from a SQL Server database" This is the use of SqlDataReader from MSDN . “提供了一种从SQL Server数据库中读取仅向前行的方法”这是MSDN中SqlDataReader的使用。 The Data structure behind SqlDataReder only allow read forward, it's optimized for reading data in one direction. SqlDataReder背后的数据结构只允许读取,它被优化用于在一个方向上读取数据。 In my opinion, I want to use SqlDataReader than DataSet for simple data reading. 在我看来,我想使用SqlDataReader而不是DataSet来进行简单的数据读取。
You have 4 sets of overheads - Disk Access - .net code (cpu) - SQL server code (cpu) - Time to switch between managed and unmanaged code (cpu) 你有4套开销 - 磁盘访问 - .net代码(cpu) - SQL服务器代码(cpu) - 在托管代码和非托管代码(cpu)之间切换的时间
Firstly is 首先是
select * where column = “junk”
fast enough for you, if not the only solution is to make the disk faster. 对你来说足够快,如果不是唯一的解决方案是让磁盘更快。 (You can get data from SQL Server faster than it can read it) (您可以比SQL Server更快地从SQL Server获取数据)
You may be able to define a Sql Server function in C# then run the function over the column; 您可以在C#中定义Sql Server函数,然后在列上运行该函数; sorry I don't know how to do it. 对不起,我不知道怎么做。 This may be faster than a data reader. 这可能比数据阅读器更快。
If you have more than one CPU, and you know a value the middle of the table, you could try using more than one thread. 如果您有多个CPU,并且您知道表中间的值,则可以尝试使用多个线程。
You may be able to write some TSQL that combines all the strings into a single string using a separator you know is safe. 您可以使用您知道安全的分隔符编写一些TSQL,将所有字符串组合成一个字符串。 Then split the string up again in C#. 然后在C#中再次拆分字符串。 This will reduce the number of round trips between managed and unmanaged code. 这将减少托管代码和非托管代码之间的往返次数。
Some surface-level things to consider that may affect speed (besides a data-reader): 一些需要考虑的表面级别的事情可能会影响速度(除了数据阅读器):
Just random thoughts. 只是随机的想法。 Not sure what might help in your situation. 不确定在你的情况下可能会有什么帮助。
If responsiveness is an issue loading a great deal of data, look at using the asynchronous methods - BeginReader. 如果响应是加载大量数据的问题,请查看使用异步方法 - BeginReader。
I use this all the time for populating large GUI elements in the background while the app continues to be responsive. 我一直使用它来在后台填充大的GUI元素,同时应用程序继续响应。
You haven't said exactly how large this data is, or why you are loading it all into an array. 您还没有确切地说明这些数据有多大,或者为什么要将它们全部加载到数组中。
Often times, for large amounts of data, you may want to leave it in the database or let the database do the heavy lifting. 通常,对于大量数据,您可能希望将其保留在数据库中或让数据库执行繁重的工作。 But we'd need to know what kind of processing you are doing that needs it all in an array at one time. 但是我们需要知道你正在做什么样的处理,一次需要一个数组。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.