将数据插入Sql Compact

Question

I have Handheld device that connect to Sql Server database, read the Sql server data and get it on the SQL Compact database that is located on device. This is my code:

public void InsertData() // Function insert data into SQL commapct database
{
    dt = new DataTable();
    dt = SqlServer_GetData_dt("Select id_price, price, id_item  from prices", SqlCeConnection); // get data form sql server
    if (dt.Rows.Count > 0)
    {
        for (int i = 0; i < dt.Rows.Count; i++)
        {
            string sql = "";
            sql = "insert into prices" +
                " ( id_prices, price,id_item) values('"
                    + dt.Rows[i]["id_price"].ToString().Trim() + "', '"
                    + dt.Rows[i]["price"].ToString().Trim() + "', '"
                    + dt.Rows[i]["id_item"].ToString().Trim() + "')";

            obj.SqlCE_WriteData_bit(sql, connection.ConnectionString);//insert into sql compact
        }
    }


}

public DataTable SqlServer_GetData_dt(string query, string conn)
{
    try
    {
        DataTable dt = new DataTable();
        string SqlCeConnection = conn;
        SqlConnection sqlConnection = new SqlConnection(SqlCeConnection);
        sqlConnection.Open();
        {

            SqlDataReader darSQLServer;
            SqlCommand cmdCESQLServer = new SqlCommand();
            cmdCESQLServer.Connection = sqlConnection;
            cmdCESQLServer.CommandType = CommandType.Text;
            cmdCESQLServer.CommandText = query;
            darSQLServer = cmdCESQLServer.ExecuteReader();
            dt.Load(darSQLServer);
            sqlConnection.Close();
        }
        return dt;
    }
    catch (Exception ex)
    {

        MessageBox.Show(ex.ToString());
        DataTable dt = new DataTable();
        return dt;

    }
}
public object SqlCE_WriteData_bit(string query, string conn)
{
    try
    {
        string SqlCeConnection = conn;
        SqlCeConnection sqlConnection = new SqlCeConnection(SqlCeConnection);
        if (sqlConnection.State == ConnectionState.Closed)
        {
            sqlConnection.Open();
        }
        SqlCeCommand cmdCESQLServer = new SqlCeCommand();
        cmdCESQLServer.Connection = sqlConnection;
        cmdCESQLServer.CommandType = CommandType.Text;
        cmdCESQLServer.CommandText = query;
        object i = cmdCESQLServer.ExecuteScalar();
        sqlConnection.Close();
        return i;

    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.ToString());
        return 0;
    }
}

This is all work fine but the problem is that all this work very slow. I have 20 000 row that's need to be inserted into SQL compact database.

Is there any way for faster insert?

Thanks.

Answer 1

Aside from the obvious poor usage of the Connection for every call, you can greatly improve things by also eliminating the query processor altogether. That means don't use SQL. Instead open the destination table with TableDirect and a SqlCeResultset . The iterate through the source data (a DataTable is a bad idea, but that's a completely different thing) and use a series of CreateRecord , SetValues and Insert .

A pretty good example can be found here (though again, I'd use SetValues to set the entire row, not each individual field).

Answer 2

Reuse your connection and don't create a new connection for every INSERT statement.

Instead of passing a connection string to your SqlCE_WriteData_bit method, create the connection once in the InsertData method, and pass the connection object to SqlCE_WriteData_bit .

Answer 3

Put all the data into a DataTable and then use a SqlCeDataAdapter to save all the data with one call to Update . You may have to fiddle with the UpdateBatchSize to get the best performance.

Looking more closely, I see that you already have a DataTable . Looping through it yourself is therefore ludicrous. Just note that, as you have it, the RowState of every DataRow will be Unchanged , so they will not be inserted. I think that you can call DataTable.Load such that all RowState values are left as Added but, if not, then use a SqlDataAdapter instead, set AcceptChangesDuringFill to false and call Fill .

Answer 4

Is there any way for faster insert?

Yes, but it probably won't be "acceptably fast enough" when we're talking about inserting 20k rows.

The problem I can see is that you are opening a connection for every single row you are retrieving from SqlServer_GetData_dt , that is, you open a connection to insert data 20k times...opening a connection is an expensive transaction. You should build the whole query using a StringBuilder object and then execute all the insert statements in one batch.

This will bring some performance gains but don't expect it to solve you're problem, inserting 20k rows will still take some time, specially if indexes need to be re-built. My suggestion is that you should thoroughly analyse your requirements and be a bit smarter about how you approach it. Options are:

• bundle a pre-populated database if possible so your app doesn't have to suffer the population process performance penalties
• if not, run the insert process in the background and access the data only when the pre-population is finished