How to manage frequent data access in .net application?

Question

I have three tables in my sql Database say Specials , Businesses , Comments . And in my master page i have a prompt area where i need to display alternative data from these 3 tables based on certain conditions during each page refresh (These tables have more than 1000 records). So in that case what will be the best option to retrieve data from these tables?

Accessing data each time from database is not a good idea i know, is there any other good way to do this, like Caching or any other new techniques to effectively manage this. Now it takes too much time to load the page after each page refresh.

Please give your suggestions.

At present what i was planning is to create a SP for data retrieval and to keep the value returned in a Session.

So that we can access the data from this session rather going to DB each time on page refresh. But do not know is there any other effective way to accomplish the same.

Answer 1

Accessing data each time from database is not a good idea

It not always true, it depends on how frequently the data is getting changed. If you choose to cache the data, you will have to revalidate it every time the data is changed. I am assuming you do not want to display a static count or something that once displayed will not change. If that's not the case, you can simply store in cookies and display from there.

Now it takes too much time to load the page after each page refresh.

Do you know what takes too much time? Is it client side code or server side code (use Glimpse to know that)? If server side, is it the code that hits the DB and the query execution time or its server side in memory manipulation.

Generally first step to improve performance is to measure it precisely and in order for you to solve such issues you ought to know where the problem is.

Based on your first statement, If i were you, I would display each count in a separate div which will be refreshed asynchronously. You could choose to update the data periodically using a timer or even better push it from server (use SignalR). The update will happen transparently so no page reload required.

Hope this helps.

Answer 2

I agree that 1000 records doesn't seem like a lot, but if you really aren't concerned about there being a slight delay you may try using HttpContext.Cache object. It's very much like a dictionary with string keys and object values, with the addition that you can set expirations etc...

Excuse typos, on mobile so no compile check:

var tableA = HttpContext.Cache.Get("TableA")
if tableA == null {
    //if its null, there was no copy in the cache so create your
    //object using your database call
    tableA = Array, List, however you store your data
    //add the item to the cache, with an expiration of 1 minute
    HTTPContext.Cache.Insert("TableA", tableA, null, NoAbsoluteExpiration, TimeSpan(0,1,0))
}

Now, no matter how many requests go through, you only hit the database once a minute, or once for however long you think is reasonable considering your needs. You can also trigger a removal of the item from cache, if some particular condition happens.

Answer 3

One suggestion is to think of your database as a mere repository to persist state. Your application tier could cache collections of your business objects, persist them when they change, and immediately return state to your presentation tier (the web page).

This assumes all updates to the data are coming from your page. If the database is being populated from different places, you'll need to either tie everything into a common application tier, or poll the database to update your cache.