从C＃CLR快速插入

Question

I am working on a huge set of data, and using CLR for processing it. The CLR processing is working quick, but I need a quick way to move the processed data to the database(through CLR).

For example, see the following clr code

protected static string Normalize(string s) // space and special character remover
    {
        char[] arr = s.ToCharArray();
        arr = Array.FindAll<char>(arr, (c => char.IsLetterOrDigit(c)));
        return new string(arr).ToLower();
    }

    [Microsoft.SqlServer.Server.SqlProcedure]
    public static void udpNormStr ()
    {
        SqlConnection con = new SqlConnection("context connection = true");
        SqlCommand cmd = new SqlCommand("Select cName from NamesTable", con);
        SqlDataAdapter da = new SqlDataAdapter(cmd);
        DataTable dt = new DataTable();
        DataTable fill = new DataTable();
        fill.Columns.Add("NormName", typeof(string));
        da.Fill(dt);
        cmd.CommandText = "insert into NormTable values (@nName)";
        cmd.Parameters.Add("@nName", SqlDbType.VarChar);
        foreach (DataRow row in dt.Rows)
        {
            fill.Rows.Add(Normalize(row[0].ToString()));
        }
        con.Open();
        foreach (DataRow row in fill.Rows)
        {
            cmd.Parameters["@nName"].Value = row[0];            
            cmd.ExecuteNonQuery();            
        }
        con.Close();
}

It is taking lot of time to execute, and is wasting 90% of that time in the insert operations. Please suggest a better way of moving processed data to database(through CLR).

Answer 1

SqlBulkCopy ; since you have a DataTable already, you can use:

using (var bcp = new SqlBulkCopy(con))
{
    bcp.DestinationTableName = "NormTable";
    bcp.WriteToServer(dt);
}

Note that for streaming data, you can also create a custom IDataReader implementation and feed that to WriteToServer .

Answer 2

Wrap your connection with a SqlTransaction to avoid implicit transactions.

con = new SqlConnection("context connection=true");
con.Open();
using (con)
{
    using (var sqlTrans = con.BeginTransaction()) //one transaction instead of many from each insert implicit
    {
        const string cmdText = @"INSERT INTO [table1] ([a],[b],[c],[d],[e]) VALUES (@a,@b,@c,@d,@e)";
        using (var cmd = new SqlCommand(cmdText, con, sqlTrans))
        {
            var aField = cmd.Parameters.Add("@a", SqlDbType.NVarChar, 255);
            var bField = cmd.Parameters.Add("@b", SqlDbType.Int);
            var cField = cmd.Parameters.Add("@c", SqlDbType.Bit);
            var dField = cmd.Parameters.Add("@d", SqlDbType.NVarChar, 255);
            var eField = cmd.Parameters.Add("@e", SqlDbType.Int);

            //same for all
            cField.Value = false;
            dField.Value = "d";
            eField.Value = 1;

            foreach (var someValue in valueCollection)
            {
                aField.Value = someValue.Key; 
                bField.Value = someValue.Value; 
                cmd.ExecuteNonQuery();
            }
        }
        sqlTrans.Commit();
    }
    con.Close();
}

Answer 3

Try to use SQLBulkCopy Class

this is sample method to Inset DataTable to Database in one Shot

 public static bool SaveDetails(DataTable dbTable)
    {
        try
        {
            SqlConnection conn = new SqlConnection("Data Source=akshay;Initial Catalog=CosmosDB;User Id=sa;Password=Nttdata123");
            conn.Open();
            SqlBulkCopy sbc = new SqlBulkCopy(conn);


            if (dbTable.Rows.Count > 0)
            {
                sbc.DestinationTableName = "Employee";
                sbc.WriteToServer(dbTable);              


            }

            sbc.Close();
            conn.Close();


            return true;



        }
        catch (Exception exp)
        {

            return false;
        }

    }

Answer 4

If this work is SQL/CLR, then that is tricky. One idea might be to make that method only return the data, for example as a CLR Table-Valued Function , and then do the INSERT back in TSQL pulling from the table-valued function.