使用C＃将计算机挂在500mb文件的批量插入中

Question

I have text file of 550mb.When I run this code computer hangs after 10 mins and data insertion fails but this works fine for file of 100mb

 public static void BulkUpdate(string col, System.IO.StreamReader sr, string tableName)
    {
        ConnStr = ConfigurationSettings.AppSettings["ConnStrAffiliate"].Trim();
        SqlConnection myConn = DAO.fetchConnection();
 string[] value = col.Split(new char[] { '\t' },StringSplitOptions.RemoveEmptyEntries);                
        DataTable dt = new DataTable();
        DataRow row;
        foreach (string dc in value)
            {
             dt.Columns.Add(new DataColumn(dc, typeof(string)));
}

        while (!sr.EndOfStream)
        {
            value = sr.ReadLine().Split(new char[] { '\t' });
            List<string> valueList = value.ToList();
            valueList.RemoveAt(valueList.Count - 1);
            value = valueList.ToArray();
            if (value.Length == dt.Columns.Count)
            {
                row = dt.NewRow();
                row.ItemArray = value;
                dt.Rows.Add(row);
            }
        }
        SqlBulkCopy bc = new SqlBulkCopy(ConnStr, SqlBulkCopyOptions.TableLock);
        bc.DestinationTableName = tableName;
        bc.BatchSize = dt.Rows.Count;
        bc.WriteToServer(dt);
        bc.Close();
    }     

Answer 1

I suggest installing Glimpse ( http://getglimpse.com/ ) and then see what the webserver is doing. My first hunch is that you are hitting a timeout value somewhere. You may also want to check the event logs to see if IIS logged an exception.

Answer 2

How much memory do you have available? You could break the file up into smaller parts first and load it into a staging table.

Answer 3

我不知道问题出在哪里，但是我尝试了Sql Server Integration服务来读取文件并将其上传到数据库中。