[英]Fast insert relational(normalized) data tables into SQL Server 2008 database
I'm trying to find a better and faster way to insert pretty massive amount of data(~50K rows) than the Linq that I'm using now. 我正在尝试找到一种比我现在使用的Linq更好,更快的方法来插入大量数据(约5万行)。 The data I'm trying to write to a local db is in a list of ORM mapped data serialized and received from WCF. 我要写入本地数据库的数据在从WCF序列化并接收的ORM映射数据的列表中。 I'm keen on using SqlBulkCopy, but the problem is that the tables are normalized and are actually a sequence or interconnected tables with one-to-many relationships. 我热衷于使用SqlBulkCopy,但问题是这些表已规范化,实际上是具有一对多关系的序列表或互连表。
Here's some code that illustrates my point: 这是一些说明我观点的代码:
foreach (var meeting in meetingsList)
{
int meetingId = dbAccess.InsertM(value1, value2...);
foreach (var competition in meeting.COMPETITIONs)
{
int competitionId = dbAccess.InsertC(meetingid, value1, value2...);
foreach(var competitor in competition.COMPETITORs)
{
int competitorId = dbAccess.InsertCO(comeetitionId, value1,....)
// and so on
}
}
}
where dbAccess.InsertMeeting looks something like this: dbAccess.InsertMeeting看起来像这样:
// check if meeting exists
int meetingId = GetMeeting(meeting, date);
if (meetingId == 0)
{
// if meeting doesn't exist insert new
var m = new MEETING
{
NAME = name,
DATE = date
}
_db.InsertOnSubmit(m);
_db.SubmitChanges();
}
Thanks in advance for any answers. 预先感谢您的任何答案。 Bojan 博扬
I would still use SqlBulkCopy
to quickly copy your data from the external file into a staging table that has the same (flat) structure as the file (you'll need to create that table ahead of time) 我仍将使用SqlBulkCopy
将数据从外部文件快速复制到具有与文件相同(扁平)结构的登台表中(您需要提前创建该表)
Once it's loaded, you can split up the data across multiple tables using eg a stored procedure or something - should be pretty fast since everything's on the server already. 加载后,您可以使用存储过程等方法将数据拆分到多个表中,因为所有内容都已经在服务器上,所以应该很快。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.