Help with bulk/batch insert into Sybase from XML with .NET

Question

I want to insert thousands of records into a Sybase database. Is there any good tool available to help me manage bulk inserts with .NET?

The records come in as an XML message to my web service. In the web service I need to load these records into a Sybase database and send load status back.

Answer 1

Think you may need to use BCP

bcp database_name.owner.table_name in datafile -n -U username -P password

Sybase BCP link

However, you may need to write the records to a file first then use the Process class to execute BCP on the file you have just written.

AFAIK there is no Sybase equivalent of SqlBulkCopy

Answer 2

If you have to insert only thousands of records then don't bother about bulk load. Plain old batch insert performs this insert in a few seconds.

A .NET web service with BCP is a too complex solution. Try BCP if you have to insert more than a million records.

Answer 3

I did something like that, and it helped reduce load time significantly,

    public static void InsertModelValueInBulk(DataSet employeData, int clsaId, int batchSize)
    {
        string[] insertStatement = new string[batchSize];
        using (var connection = GetOdbcConnection())
        {
            connection.Open();
            var tran = connection.BeginTransaction();
            try
            {
                int j = 0;
                for (int i = 0; i < employeData.Tables[0].Rows.Count; i++)
                {
                    var row = employeData.Tables[0].Rows[i];
                    var insertItem = string.Format(@"select '{0}',{1}", row["name"], Convert.ToInt32(row["ID"]);
                    insertStatement[j] = insertItem;
                    if (j % (batchSize-1) == 0 && j > 0)
                    {
                        var finalQuery = @" INSERT INTO employee (id, name)
                                         " + String.Join(" union ", insertStatement);
                        using (var cmd = new OdbcCommand(finalQuery, connection, tran))
                        {
                            cmd.ExecuteNonQuery();
                        }
                        j = 0;
                        continue;
                    }
                    else
                    {
                        j = j + 1;
                    }
                }

                if (j > 0)
                {
                    var finalQuery = @"INSERT INTO employee (id, name)
                                     " + String.Join(" union ", insertStatement,0,j-1);
                    using (var cmd = new OdbcCommand(finalQuery, connection, tran))
                    {
                        cmd.ExecuteNonQuery();
                    }
                }
                tran.Commit();
            }
            catch
            {
                tran.Rollback();
                throw;
            }
        }
    }