Fetching records from database

Question

In my C# 3.5 application,code performs following steps:

1.Loop through a collection[of length 10]

2.For each item in step 1, fetch records from oracle database by executing a stored proc[here,record count is typically 100]

3.Process items fetched in step 2.

4.Go to next item in step 1.

My question here, with regard to performance, is it a good idea to fetch all items in step #2[ie. 10 * 100=1000 records] in one shot rather than connecting to database in each step and retrieving the 10 records?

Thanks.

Answer 1

Yes it's slightly better because you will lose the overhead of connecting to the DB, but you will still have the overhead of 10 stored procedure calls. If you could find a way to pass all 10 items as parameter to the stored proc and execute just one stored proc call, I think you would get a better performance.

Answer 2

Depending on how intense the connection steps are, it might be better to fetch all the records at once. However, keep in mind that premature optimization is the root of all evil. :-)

Answer 3

Generally it is better to pull all the records from the database in one stored procedure call.

This is countered when the stored procedure call is long running or otherwise extensive enough to cause contention on the table. In your case however with only a 1000 records, I doubt that will be an issue.

Answer 4

Yes, it is an incredibly good idea. The key to database performance is to run as many operations in bulk as possible.

For example, consider just the interaction between PL/SQL and SQL. These two languages run on the same server and are very thoroughly integrated. Yet I routinely see an order of magnitude performance increase when I reduce or eliminate any interaction between the two. I'm sure the same thing applies to interaction between the application and the database.

Even though the number of records may be small, bulking your operations is an excellent habit to get into. It's not premature optimization, it's a best practice that will save you a lot of time and effort later.