[英]Insert chunk using linq to sql?
在ac#应用程序中,我需要在sql server 2005中插入很多表或记录。 我在2000记录块中拆分了datatable,使用linq到sql server,但是由于永远不要停止,所以效果不佳! 我有900.000个演奏者,并为此投入了很多。 我做错了什么? 这是我的代码:
int jump = 0;
while (ds.Tables[0].Rows.Count < ds.Tables[0].Rows.Count + 1)
{
String xmlData = ConvertDataTableToXML(ds.Tables[0].AsEnumerable().Skip(jump).Take(2000 + jump).CopyToDataTable());
jump = jump + 2001;
SqlConnection conn = new SqlConnection
("Data Source=SERVER;Initial Catalog=DATABASE;Persist Security Info=True;User ID=USER;Password=PASS;");
conn.Open();
SqlCommand insert = new SqlCommand
("sp_InsertData'" + xmlData + "'", conn);
insert.ExecuteNonQuery();
conn.Close();
}
如果您了解SQL Server中的临时表以及如何使用OPENXML命令,则可以尝试以下操作。 即使您的DBA禁用了BULK INSERTS,它仍然有效
用您自己的字段名替换我的字段名,代码的此部分中的字段映射应与您的表架构/字段定义/数据类型匹配FROM OPENXML (@xmlHandle, '/NewDataSet/XMLDataTable',1) WITH (
ALTER PROCEDURE [dbo].[sp_InsertData]
(@xmlString VARCHAR(MAX))
AS
BEGIN
/* Initialize a handle for the XmlDocument */
DECLARE @xmlHandle INT
/*
Created by @MethodMan you first want to create / declare a TEMP TABLE which
Mimic's the structure of the Target table that you are inserting into
*/
DECLARE @someTargetTable TABLE
(
[EN_INTFC_ID] varchar(25),
[EN_INTFC_LINE_NUM] varchar(5),
[EN_BILL_SOURCE_ID] varchar(10),
[EN_BUSINESS_UNIT] varchar(12),
[EN_ASSET_NAME] varchar(4),
[EN_POSTING_DATE] DateTime,
[EN_FISCAL_YEAR] varchar(4),
[EN_FISCAL_PERIOD] varchar(3),
[EN_CUSTOMER_ID] varchar(50),
[EN_DOC_TYPE] varchar(4),
[EN_TARGET_INVOICE] varchar(16),
[EN_INVOICE_DT] DateTime,
[EN_REVNUE_TYPE] varchar(15),
[EN_QTY] decimal(15,0),
[EN_GROSS_EXT_AMT] decimal(25,2),
[EN_DESCR] varchar(50),
[EN_CONTRACT] varchar(20),
[EN_PRODUCT_TYPE] varchar(15),
[EN_UNIT_OF_MEASURE] varchar(3)
)
/*
Create the XmlDocument using the handle above and the Xml
string as parameters. If your stored procedure has an varchar input
parameter named @xmlString, it would look like this instead:
EXEC sp_xml_preparedocument @xmlHandle output,@xmlString
*/
EXEC sp_xml_preparedocument @xmlHandle output, @xmlString
/*
Use the OPENXML method to query the XmlDocument starting at
/NewDataSet/SampleDataTable node.
*/
INSERT INTO @someTargetTable
SELECT [EN_INTFC_ID],
[EN_INTFC_LINE_NUM],
[EN_BILL_SOURCE_ID],
[EN_BUSINESS_UNIT],
[EN_ASSET_NAME],
[EN_POSTING_DATE],
[EN_FISCAL_YEAR],
[EN_FISCAL_PERIOD],
[EN_CUSTOMER_ID],
[EN_DOC_TYPE],
[EN_TARGET_INVOICE],
[EN_INVOICE_DT],
[EN_REVNUE_TYPE],
[EN_QTY],
[EN_GROSS_EXT_AMT],
[EN_DESCR],
[EN_CONTRACT],
[EN_PRODUCT_TYPE],
[EN_UNIT_OF_MEASURE]
FROM OPENXML (@xmlHandle, '/NewDataSet/XMLDataTable',1)
WITH (
[EN_INTFC_ID] varchar(25) '@EN_INTFC_ID',
[EN_INTFC_LINE_NUM] varchar(5) '@EN_INTFC_LINE_NUM',
[EN_BILL_SOURCE_ID] varchar(10) '@EN_BILL_SOURCE_ID',
[EN_BUSINESS_UNIT] varchar(12) '@EN_BUSINESS_UNIT',
[EN_ASSET_NAME] varchar(4) '@EN_ASSET_NAME',
[EN_POSTING_DATE] DateTime '@EN_POSTING_DATE',
[EN_FISCAL_YEAR] varchar(4) '@EN_FISCAL_YEAR',
[EN_FISCAL_PERIOD] varchar(3) '@EN_FISCAL_PERIOD',
[EN_CUSTOMER_ID] varchar(50) '@EN_CUSTOMER_ID',
[EN_DOC_TYPE] varchar(4) '@EN_DOC_TYPE',
[EN_TARGET_INVOICE] varchar(16) '@EN_TARGET_INVOICE',
[EN_INVOICE_DT] DateTime '@EN_INVOICE_DT',
[EN_REVNUE_TYPE] varchar(15) '@EN_REVNUE_TYPE',
[EN_QTY] decimal(15,0) '@EN_QTY',
[EN_GROSS_EXT_AMT] decimal(25,2) '@EN_GROSS_EXT_AMT',
[EN_DESCR] varchar(50) '@EN_DESCR',
[EN_CONTRACT] varchar(20) '@EN_CONTRACT',
[EN_PRODUCT_TYPE] varchar(15) '@EN_PRODUCT_TYPE',
[EN_UNIT_OF_MEASURE] varchar(3) '@EN_UNIT_OF_MEASURE'
)
/*Insert the records into the table variable */
INSERT INTO Your_Actual_Table_Name (
[EN_INTFC_ID],
[EN_INTFC_LINE_NUM],
[EN_BILL_SOURCE_ID],
[EN_BUSINESS_UNIT],
[EN_ASSET_NAME],
[EN_POSTING_DATE],
[EN_FISCAL_YEAR],
[EN_FISCAL_PERIOD],
[EN_CUSTOMER_ID],
[EN_DOC_TYPE],
[EN_TARGET_INVOICE],
[EN_INVOICE_DT],
[EN_REVNUE_TYPE],
[EN_QTY],
[EN_GROSS_EXT_AMT],
[EN_DESCR],
[EN_CONTRACT],
[EN_PRODUCT_TYPE],
[EN_UNIT_OF_MEASURE] )
(SELECT [EN_INTFC_ID],
[EN_INTFC_LINE_NUM],
[EN_BILL_SOURCE_ID],
[EN_BUSINESS_UNIT],
[EN_ASSET_NAME],
[EN_POSTING_DATE],
[EN_FISCAL_YEAR],
[EN_FISCAL_PERIOD],
[EN_CUSTOMER_ID],
[EN_DOC_TYPE],
[EN_TARGET_INVOICE],
[EN_INVOICE_DT],
[EN_REVNUE_TYPE],
[EN_QTY],
[EN_GROSS_EXT_AMT],
[EN_DESCR],
[EN_CONTRACT],
[EN_PRODUCT_TYPE],
[EN_UNIT_OF_MEASURE]
FROM @someTargetTable)
/* Remove the document from memory */
EXEC sp_xml_removedocument @xmlHandle
END
// your sql command below.
SqlCommand insert = new SqlCommand ("sp_InsertData '" + xmlData + "'", conn);
insert.CommandTimeout = 5000;
insert.ExecuteNonQuery();
conn.Close()
//我的代码以及如何利用using(){} statement
以及代码将DataTable转换为XML,然后将该xml传递给我上面已描述的存储过程
private bool ProcessSomeDataTableToXML(DataTable dataTable)
{
String xmlData = ConvertDataTableToXML(dataTable);
var ConnString = System.Configuration.ConfigurationManager.ConnectionStrings["yourdatabase"].ConnectionString;
using (SqlConnection connection = new SqlConnection(ConnString))
{
using (SqlCommand command = new SqlCommand("sp_InsertData '" + xmlData + "'", connection))
{
connection.Open();
try
{
command.ExecuteNonQuery();
fileInserted = true;
}
catch (SqlException sqlEx)
{
fileInserted = false;
Console.WriteLine(sqlEx.Message);
}
}
}
return fileInserted;
}
private static string ConvertDataTableToXML(DataTable dtData)
{
DataSet dsData = new DataSet();
StringBuilder sbSQL;
StringWriter swSQL;
string XMLformat;
try
{
sbSQL = new StringBuilder();
swSQL = new StringWriter(sbSQL);
dsData.Merge(dtData, true, MissingSchemaAction.AddWithKey);
dsData.Tables[0].TableName = "XMLDataTable";
foreach (DataColumn col in dsData.Tables[0].Columns)
{
col.ColumnMapping = MappingType.Attribute;
}
dsData.WriteXml(swSQL, XmlWriteMode.WriteSchema);
XMLformat = sbSQL.ToString();
sbSQL = null;
swSQL = null;
return XMLformat;
}
catch (Exception sysException)
{
throw sysException;
}
}
一个明显的问题是你的while
条件:
ds.Tables[0].Rows.Count < ds.Tables[0].Rows.Count + 1
即count < count + 1
即true
您的循环旨在永不停止。 您可以将其更改为
while (jump < ds.Tables[0].Rows.Count)
1)您有一个无限循环(ds.Tables [0] .Rows.Count <ds.Tables [0] .Rows.Count +1)
2)连接管理:您无需在每次迭代中打开关闭连接。 完成连接后,请使用using块来处理连接。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.