简体繁体 English

如何在asp.net中处理大量数据

[英]How do I process large amount of data in asp.net

原文 2013-04-17 09:03:36 2 2 c#/ mysql/ asp.net/ large-data

I have a web project in asp.net/C#/Mysql where the data can be up to some 30,000 rows of data to process. 我在asp.net/C#/Mysql中有一个Web项目，其中数据最多可以处理大约30,000行数据。 Its a reporting tool, and I have to show statistics like counts and sum at several levels. 它是一个报告工具，我必须在几个级别显示计数和总和等统计数据。

I want to know which would be the better way to go around this. 我想知道哪种方法可以解决这个问题。

I can filter my data to limited columns though which I can query. 我可以将我的数据过滤到有限的列，我可以查询。

Now, Is is a good way to get the data (all rows) to my application on load and whenever user queries I can filter that data and do my calculations in the code and show my statistics. 现在，Is是一种在加载时将数据（所有行）提供给我的应用程序的好方法，每当用户查询我可以过滤该数据并在代码中进行计算并显示我的统计信息。
or I can have a stored procedure do all my calculations and every time user queries I can call the stored procedure and get my statistics. 或者我可以有一个存储过程执行所有计算，每次用户查询我都可以调用存储过程并获取我的统计信息。

Thanks 谢谢

2 个解决方案

Databases are optimized to do this kind of data manipulation. 数据库经过优化，可以进行这种数据操作。 And since you reduce network load as well i would vote for the second option. 而且，由于你减少了网络负载，我会投票给第二个选项。

Another possibility is to consider some kind of OLAP solution where transactional data is already consolidated into smaller chunks of data which in turn can be easily queried. 另一种可能性是考虑某种OLAP解决方案，其中事务数据已经合并到较小的数据块中，而这些数据块又可以轻松查询。

I would definetly go with the second option. 我绝对会选择第二种选择。 You are talking about a web application, if You want to get all the data at load, then you must store it somewhere to preserve it during postbacks. 您正在谈论的是Web应用程序，如果您想要在加载时获取所有数据，那么您必须将其存储在某处以便在回发期间保留它。 If you chose to store it in a session state, you will end up consuming the Web server memory, since you have more than one user accessing your site. 如果您选择将其存储在会话状态，则最终会占用Web服务器内存，因为您有多个用户访问您的站点。

If you store it in view state, then you will end up in a very big client response,which will make your page very slow to load on the client side, and will cause network traffic. 如果将它存储在视图状态，那么您将最终得到一个非常大的客户端响应，这将使您的页面在客户端加载非常慢，并将导致网络流量。

Option 2 is the best, because stored procedures are precompiled, which means they are much better in terms of performance. 选项2是最好的，因为存储过程是预编译的，这意味着它们在性能方面要好得多。 You will also reduce network traffic. 您还将减少网络流量。