How to work with data stored in database?

Question

Good day everyone,

I have some questions, about how to do calculations of data stored in the database. Like, I have a table:

|   ID    | Item name | quantity of items | item price | date |

and for example i have stored 10000 records.

First that I need to do is to pick up items from a date interval, so I wont need the whole database for my calculations. And then I get items from that date interval, I have to add some tables, for example to calculate:

full price = quantity of items *  item price

and store them in new table for each item. So the database for the items picked from the date interval should look like this:

|   ID    | Item name | quantity of items | item price | date |  full price |

The point is that I don't know how to store that items which i picked with date interval. Like, do i have create some temporary table, or something? This will be using an ASP.NET web application, and for calculations in the database I think I will use SQL queries. Maybe there is an easier way to do it? Thank you for your time to help me.

Answer 1

There are few approaches:

• use sql query to calculate that on the fly - this way nothing is stored to the database
• use same or another table to perform calculation
• use calculated field

If you have low database load (few queries per minute, few thousands of rows per fetch) then use first aproach. If calculation on the fly performs poorly (millions of records, x fetches per second...) try second or third aproach. Third one is ok if your db supports calculated and persisted fields, say MSSQL Server.

EDIT:

Typically, as others said, you will perform calculation in your query. That is, as long as your project is simple enough.

First, when the table where you store all the items and their prices becomes attacked with insert/update/deletes from multiple clients, you don't want to block or be blocked by others. You have to understand that eg table X update will possibly block your select from table X until it is finished (look up page/row lock). This means that you are going to love parallel denormalized structure (table with product and the calculated stuff along with it). This is where eg reporting comes into play.

Second, when calculation is simple enough (a*b) and done over not-so-many records, then it's ok. When you have eg 10M records and you have to correlate each row with several other rows and do some aggregation over some groups, there is a chance that calculated/persisted field will save your time - you can gain up to 10-100 times faster result using this approach.

Answer 2

Like other people have said, you can perform these queries on the fly rather than store them.

However, to answer your question, a query like this should do the trick..

I haven't tested this so the syntax might be off a touch, though it will get you on the right track.

Ultimately you have to do an insert with a select

insert into itemFullPrice
select id, itemname, itemqty, itemprice, [date], itemqty*itemprice as fullprice from items where [date] between '2012/10/01' AND '2012/11/01'

again..don't shoot me if i have got the syntax a little off.. it's a busy day today :D

Answer 3

Having 10000 records, it'd not be a good idea to use temporary tables.

You'd better have another table, called ProductsPriceHistory, where you peridodically calculate and store, let's say, monthly reports.

This way, your reports would be faster and you wouldn't have to make calculations everytime you want to get your report.

Be aware this approach is OK if your date intervals are fixed, I mean, monthly, quarterly, yearly, etc. If your date intervals are dynamic, ex. from 10/20/2011 to 10/25/2011, from 05/20/2011 to 08/13/2011, etc, this approach wouldn't work.

Another approach is to make calculations on ASP.Net.

Answer 4

Even with 10000 records, your best bet is to calculate something like this on the fly. This is what structured databases were designed to do.

For instance:

SELECT [quantity of items] * [item price] AS [full price]
  , [MyTable].*
FROM [MyTable]

More complex calculations that involve JOINs to 3 or more tables and thousands of records might lend itself to storing values.

Answer 5

You should separate concerns in your application:

• aspx pages for presentation
• sql server for data persistency
• some kind of intermediate "business" layer for extra logic like fullprice = p * q

Eg if you are using Linq-2-sql for data retrieval, it is very simple to add a the fullprice to your entities. The same for entity framework. Also, if you want, you can already do the computation of p*q in the SQL select. Only if performance really becomes an issue, you can start thinking about temporary tables, views with clustered indexes etc.