How to use transaction in C#.net

Question

I want to insert data into sqlserver with very high speed. So, I Use transaction in c#.net but after use transaction I noticed that the speed of insertion has no difference with not using it. Please check my code and help me to solve it.

code :

void ReadToTable()
    {

        try
        {
            if (dir != "")
            {
                dic.Clear();
                string[] lines = System.IO.File.ReadAllLines(dir);
                foreach (string line in lines)
                {
                    SqlConnection sqlconn = new SqlConnection(DBsetting.Connstring);
                    sqlconn.Open();
                    SqlTransaction transaction = sqlconn.BeginTransaction();
                    if (!string.IsNullOrEmpty(line))
                    {
                        l = line.Split('\t');
                        l[0] = l[0].Trim();
                        l[1] = l[1].Trim();
                        l[2] = l[2].Trim();
                        l[3] = l[3].Trim();
                        if (!string.IsNullOrEmpty(l[0]) && l[3] == "1" || l[3] == "0")
                        {
                            dic.Add(new KeyValuePair<string, string>(l[0], l[1]));
                            PersianCalendar persCal = new PersianCalendar();
                            SqlCommand sqlda = new SqlCommand("insert into readd(IDp,date,day,nobatkari)values(@IDp,@date,@day,@nobatkari)", sqlconn, transaction);
                            sqlda.Transaction = transaction;
                            sqlda.Parameters.AddWithValue("@date", l[1]);
                            sqlda.Parameters.AddWithValue("@IDp", l[0]);
                            sqlda.Parameters.AddWithValue("@day", GetDayOfWeek(GetPerDate(l[1])));
                            sqlda.Parameters.AddWithValue("@nobatkari", "");//ntkari[idx].ToString());
                            sqlda.ExecuteNonQuery();
                            transaction.Commit();
                            sqlconn.Close();
                            sqlconn.Dispose();

                        }
                        else
                        {
                            ReadDataAndInsertIntoTable(line);
                        }
                    }

                }
                RefGrid();
                foreach (var pair in dic)
                {
                    InsertNobatKari(pair.Key, GetPerDate(pair.Value).ToString());
                }
            }

        }
        catch (Exception ex)
        {
            if (ex.Message.Contains("there is no row at position 0"))
            {
                MessageBox.Show("لطفا سطر خالی را انتخاب ننمایید");
            }
            else if (ex.Message.Contains("Cannot insert duplicate key row in object"))
            {
                MessageBox.Show("امکان ورود سطر تکراری وجود ندارد");
            }
            else
            {
                MessageBox.Show(ex.Message);
            }
        }
        finally
        {

        }
    }

Answer 1

I want to insert data into sqlserver with very high speed .so I Use transaction in c#.net but after use transaction I noticed that the speed of insertion has no difference with not using it.

YES you will not see improvement because transactions are not for improving speed. Plus you are not using transaction correctly: you are not committing the transaction.

---

Your Approach

Think about what you are doing. You are looping lines from a file and for each line you are creating a database connection and then going all the way to the database to insert the one record. You do this n times where n is the number of records in your text file.

The other thing is you are using File.ReadAllLines which will read the entire file into memory: What if the file is really large? Can your memory handle it?

---

A Better Approach

A better approach would be to create one connection, and then go to the database and take a batch of records (50, 100 or more) and do them in one trip. Keep doing this until all records are in.

Also, you should use File.ReadLines so it will read each line lazily (it will load the line into memory once it is requested).

To improve speed, use SqlBulkCopy . Make sure you read the parts on column mappings.

Steps:

1. Read one line from the file.

2. Based on this one line create your in-memory table structure.

    string[] strArray = line.Split('\\t'); var dt = new DataTable(); for (int index = 0; index < strArray.Length; index++) dt.Columns.Add(new DataColumn()); 

3. Populate the in-memory datatable from the file:

    do { DataRow row = dt.NewRow(); string[] itemArray = line.Split('\\t'); row.ItemArray = itemArray; dt.Rows.Add(row); i = i + 1; line = sr.ReadLine(); } while (!string.IsNullOrEmpty(line)); 

4. Send the data to the database:

    var bc = new SqlBulkCopy(dbConn, SqlBulkCopyOptions.TableLock, null) { DestinationTableName = "TestData", BatchSize = dt.Rows.Count }; dbConn.Open(); bc.WriteToServer(dt); dbConn.Close(); bc.Close(); 

You can do step 4 in batches if the file is really large. In the above code the sr variable would be an instance of StreamReader but you can use anyway of reading the file.

---

NOTE:

This code is not to be copy pasted but used as a guideline so you will need to tweak it to fit your needs.

Answer 2

You create a transaction for every line inserted, which is pretty much the same as doing everything without a transaction. You probably should do something like this:

    string[] lines = System.IO.File.ReadAllLines(dir);
    SqlConnection sqlconn = new SqlConnection(DBsetting.Connstring);
    sqlconn.Open();
    SqlTransaction transaction = sqlconn.BeginTransaction();
    foreach (string line in lines)
    {
        ...
    }
    transaction.Commit();
    sqlconn.Close();
    sqlconn.Dispose();