Converting C# code to SQL Server stored procedure

Question

I'm writing a small web application for calculating and creating billings for customers. For this, I have the following SQL Server tables:

Each month, there will be added up to 2600 new records in the Billing table with ~3 records in the BillingPriceLine table for each billing and finally, up to 750 records in the BillingPriceLineSpecification table for each record in the BillingPriceLine table. So that means a lot of records in total :-)

So, the initial data that gets insert into the tables each month could look like this (this is just my test data though)

Billing table (starts completely empty):

BillingPriceLine table:

BillingPriceLineSpecification table:

When these records have been added (I'm using SqlBulkCopy for that), I then have to make the following calculations in order to complete a single billing:

1. Calculate BillingPriceLineSpecification.Production based on: BillingPriceLineSpecification.EstimatedProduction - BillingPriceLineSpecification.RealisedProduction

2. Store the sum of each BillingPriceLineSpecification.Production * BillingPriceLineSpecification.Price in a temporary variable for later use

3. Calculate BillingPriceLine.Production based on the sum of all it's BillingPriceLineSpecification.Production

4. Calculate BillingPriceLine.Price based on the sum from step 2 divided by BillingPriceLine.Production

5. Calculate BillingPriceLine.TotalPrice like this:

    ((BillingPriceLine.Production * BillingPriceLine.Price) * BillingPriceLine.Share) / 100) 

6. Calculate Billing.SubTotal based on the sum of all it's BillingPriceLine.TotalPrice

7. Calculate Billing.VAT based on Billing.SubTotal / 4 (Danish VAT is 25%)

8. Calculate Billing.Total which will be Billing.SubTotal + Billing.VAT

I've written some C# code using Entity Framework for doing this, but when I tested it using only 10 records in Billing table instead of 2600, it basically stalled (for a lack of better word)

The C# code I wrote:

using(var ctx = new MyEntities())
{
    foreach (Billing billing in ctx.Billings)
    {
        // Calculate billing price lines from billing price line specifications         
        try
        {
            foreach (BillingPriceLine priceLine in billing.BillingPriceLines)
            {
                // Declare a local variable for holding the specification total sum
                decimal specificationsSum = 0;

                // Loop through billing price line specifications on this price line
                foreach (BillingPriceLineSpecification specification in priceLine.BillingPriceLineSpecifications)
                {
                    // First, check if the estimated production and realised production has a value
                    if (specification.EstimatedProduction.HasValue &&  specification.RealisedProduction.HasValue)
                    {
                        // Calculate production for a price line specification
                        specification.Production = specification.EstimatedProduction.Value - specification.RealisedProduction.Value;

                        // Add to total specification sum
                        specificationsSum += specification.Production*specification.Price;
                    }
                }

                // Set total production on price line
                priceLine.Production = priceLine.BillingPriceLineSpecifications.Sum(x => x.Production);

                // Set price on price line
                priceLine.Price = specificationsSum/priceLine.Production;

                // Set total price on price line
                priceLine.TotalPrice = ((priceLine.Production*priceLine.Price)*priceLine.Share)/100;
            }

            // Set subtotal, VAT and total sum on billing
            billing.Subtotal = billing.BillingPriceLines.Sum(x => x.TotalPrice);
            billing.VAT = billing.Subtotal/4;
            billing.Total = billing.Subtotal + billing.VAT;
        }
        catch
        {
            // Handle error logging here ..
        }
    }

    ctx.SaveChanges();
}

I'm looking to move this calculation process to the SQL Server using a stored procedure, hoping that performance would then be a lot better since it then wouldn't have to run on the web application. However, I'm lacking the skills when it comes to writing T-SQL, to write such a procedure.

Is there anyone who can give me a starting point writing this procedure? :-) And/or enlighten me with even better ways of solving this problem. It'll be greatly appreciated to get some feedback on it.

Thanks in advance.

Answer 1

Try querying ONLY the data required and get all of it at one time (including child collections) and then make the calculations from the data in memory.

---

Here is some sample code of how I have used SQL and SqlBulkCopy to do bulk updating of data from C# without a SP.

First you create a custom object (per table) that holds the data that will be updated.

// custom object
public sealed class CustomObjectWithUpdatedValues
{
    public int BillingID { get; set; }
    public int Field1 { get; set; }
    public int Field2 { get; set; }
    public decimal Field3 { get; set; }
    public decimal Field4 { get; set; }
    public decimal Field5 { get; set; }
}

Then I use the following code to first insert into a new table via SqlBulkCopy. Then I generate an SQL statement to do the bulk update of the live data and then drop the new table.

// do all your calculations and write the data to be updated to a custom object
List<CustomObjectWithUpdatedValues> DataToUpdateList = GetDataToUpdate();

// create a transaction scope incase something fails we can rollback
using (TransactionScope tranScope = new TransactionScope())
{
    using (SqlConnection conn = new SqlConnection("connection_string"))
    {
        // open the connnection
        conn.Open();

        // create a table to temporarily hold the data to update
        string strTableName = "temp_BulkUpdateBillings";
        SqlCommand cmdCreateTable = new SqlCommand
            (
                "CREATE TABLE " + strTableName + " " +
                    "(" +
                        "BillingID int NOT NULL, " +
                        "Field1 int NOT NULL, " +
                        "Field2 int NOT NULL, " +
                        "Field3 decimal(19, 4) NOT NULL, " +
                        "Field4 decimal(19, 4) NOT NULL, " +
                        "Field5 decimal(19, 4) NOT NULL " +
                    ")"
            , conn);
        cmdCreateTable.ExecuteNonQuery();

        // do sql bulk copy to insert data into new table
        using (SqlBulkCopy bcp = new SqlBulkCopy(conn))
        {
            using (var reader = ObjectReader.Create(DataToUpdateList, "BillingID", "Field1", "Field2", "Field3", "Field4", "Field5"))
            {
                bcp.DestinationTableName = strTableName;
                bcp.BatchSize = 1000;
                bcp.BulkCopyTimeout = 300;
                bcp.WriteToServer(reader);
            }
        }

        // update the live records
        SqlCommand cmdBulkUpdate = new SqlCommand
            (
                "UPDATE Billings SET " +
                    "Field1 = temp.Field1, " +
                    "Field2 = temp.Field2, " +
                    "Field3 = temp.Field3, " +
                    "Field4 = temp.Field4, " +
                    "Field5 = temp.Field5 " +
                "FROM " + strTableName + " as temp " +
                "WHERE Billings.ID = temp.BillingID"
            , conn);
        cmdBulkUpdate.ExecuteNonQuery();

        // drop the table
        SqlCommand cmdDropTable = new SqlCommand("DROP TABLE " + strTableName, conn);
        cmdDropTable.ExecuteNonQuery();
    }

    // complete the transaction scope
    tranScope.Complete();
}

The ObjectReader class is from the FastMember library.

Hope that can point you in the right direction.

Answer 2

Hope this could help you

Set Nocount On;

Declare @Billing Table
(
     ID                     Int Identity(514789,1) Primary Key
    ,SubTotal               Numeric(38,10)
    ,Vat                    Numeric(38,10)
    ,Total                  Numeric(38,10)
    ,Status                 Varchar(100)
)

Declare @BillingPriceLine Table
(
     ID                     Int Identity(24527,1) Primary Key
    ,BillingID              Int
    ,Product                Varchar(100) Null
    ,FinancialID            Int
    ,Production             Numeric(38,10)
    ,Unit                   Varchar(100)
    ,Price                  Numeric(38,10)
    ,Share                  Numeric(38,10)
    ,TotalPrice             Numeric(38,10)
)

Declare @BillingPriceLineSpecification Table
(
     ID                     Int Identity(2447820,1) Primary Key
    ,BillingPriceLineID     Int
    ,Production             Numeric(38,10)
    ,EstimatedProduction    Numeric(38,10)
    ,RealisedProduction     Numeric(38,10)
    ,Unit                   Varchar(100)
    ,Price                  Numeric(38,10)
    ,Created                Datetime
    ,Time                   Datetime
)

Insert Into @Billing(SubTotal,Vat,Total) Values
 (Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)
,(Null,Null,Null)

Insert Into @BillingPriceLine(BillingID,FinancialID,Production,Unit,Price,Share,TotalPrice) Values
 (514789,1234,Null,'kWh',Null,50.00,Null)
,(514789,1234,Null,'kWh',Null,50.00,Null)
,(514789,1234,Null,'kWh',Null,50.00,Null)
,(514789,1234,Null,'kWh',Null,50.00,Null)
,(514789,1234,Null,'kWh',Null,50.00,Null)
,(514790,1234,Null,'kWh',Null,50.00,Null)
,(514790,1234,Null,'kWh',Null,50.00,Null)
,(514790,1234,Null,'kWh',Null,50.00,Null)
,(514790,1234,Null,'kWh',Null,50.00,Null)
,(514790,1234,Null,'kWh',Null,50.00,Null)
,(514791,1234,Null,'kWh',Null,50.00,Null)
,(514791,1234,Null,'kWh',Null,50.00,Null)
,(514791,1234,Null,'kWh',Null,50.00,Null)
,(514791,1234,Null,'kWh',Null,50.00,Null)
,(514791,1234,Null,'kWh',Null,50.00,Null)
,(514792,1234,Null,'kWh',Null,50.00,Null)
,(514792,1234,Null,'kWh',Null,50.00,Null)

Insert Into @BillingPriceLineSpecification(BillingPriceLineID,Production,EstimatedProduction,RealisedProduction,Unit,Price,Created,Time) Values
 (24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')
,(24527,0.00,2.00,4.00,'kWh',0.30,'2015-07-30 18:09:34.000','2015-07-30 18:09:34.000')

-------- above one only table preparation -----

Update  bpls
Set     bpls.Production = Abs(bpls.EstimatedProduction - bpls.RealisedProduction)
From    @BillingPriceLineSpecification As bpls

Update  bpl
Set      bpl.Production = bpls.Production
        ,bpl.Price = (bpls.SumProduction / bpls.Production)
        ,bpl.TotalPrice = (bpls.Production * (bpls.SumProduction / bpls.Production) * bpl.Share) / 100
From    @BillingPriceLine As bpl
        Join
        (
            Select   bpls.BillingPriceLineID
                    ,Sum(bpls.Production) As Production
                    ,Sum(bpls.Production * bpls.Price) As SumProduction
            From    @BillingPriceLineSpecification As bpls
            Group By bpls.BillingPriceLineID
        ) As bpls On bpl.ID = bpls.BillingPriceLineID



Update  b
Set      b.SubTotal = bpl.TotalPrice
        ,b.Vat = (bpl.TotalPrice / 4)
        ,b.Total = (bpl.TotalPrice + (bpl.TotalPrice / 4))
From    @Billing As b
        Join
        (
            Select   bpl.BillingID
                    ,Sum(Isnull(bpl.TotalPrice,0)) As TotalPrice
            From    @BillingPriceLine As bpl
            Group By bpl.BillingID  
        ) As bpl On b.ID = bpl.BillingID

---- final result
Select   b.ID
        ,b.SubTotal
        ,b.Vat
        ,b.Total
From    @Billing As b

output :-