大量或非规范化记录的存储解决方案

Question

We currently have a MSSQL server with the data stored in a denormalized format in a single table (per record type) where there can be millions of rows containing many columns in some cases. The requirement of the application is to read the data for each record in the application ie each matching row must be sent and read by the application (using a C# DataReader approach), so no aggregations can be made in the SQL server since the actual data is needed.

The querying are in most cases pretty simple on a Date field (integer represented) and on two or three text columns.

Based on the requirement do you guys think that SQL server a good fit or are there other data storages ex: NoSQL solutions thst perhaps would be better/easier to use.

Some of our systems contains hundreds of gigabytes of data in this format and its read only data once imported ie data is never updated.

The bottleneck for us right now seems to be Disk IO performance when returning large results, and because of the size there are no way of having everything cached at all time.

Any tips on different storage solutions that we should look into.

Thanks in advance.

Answer 1

SQL Server is a good fit and come back when you have real amounts of data. Hundreds of GB is a joke - this is 2014, not 1990.

The bottleneck for us right now seems to be Disk IO performance

What discs?

The use of ALL SSD setups in such scenarios is standard for at least 5 years now. I have personally a 3000gb analysis database and am pulling more than 1gb/second from the underlying SSD.

OBVIOUSLY you need hardware appropriate to your requirements.