在SQL Server中快速，大量输入数据

Question

I'm currently in the preparatory phase for a project which will involve (amongst other things) writing lots of data to a database, very fast (ie images (and associated meta-data) from 6 cameras, recording 40+ times a second).

Searching around the web, it seems that 'Big Data' more often applies to a higher rate, but smaller 'bits' (ie market data).

So..

• Is there a more scientific way to proceed than "try it and see what happens"?
• Is "just throw hardware at it" the best approach?
• Is there some technology/white papers/search term that I ought to check out?
• Is there a compelling reason to consider some other database (or just saving to disk)?

Sorry, this is a fairly open-ended question (maybe better for Programmers?)

Answer 1

Is there a more scientific way to proceed than "try it and see what happens"?

No, given your requirements are very unusual.

Is "just throw hardware at it" the best approach?

No, but at some point it is the only approach. You wont get a 400 horse power racing engine just by tuning a fiat panda. You wont get high throughput at any database without appropriate hardware.

Is there some technology/white papers/search term that I ought to check out?

Not a valid question in the context of the question - you ask specifically for sql server.

Is there a compelling reason to consider some other database (or just saving to disk)?

No. As long as you stick relational database the same rules apply pretty much - another may be faster, but not by a wide margin.

Your main problem will be disc IO and network bandwidth, depending on size of the images. Properly size the equipment and you should be fine. At the end this seems less than 300 images per second. Sure you want the images themselves in the database? I normally like that, but this is like storing a movie in pictures and that may be stretching it.

Whatever you do, that is a lot of disc IO and size, so - hardware is the only way to go if you need IOPS etc.