[英]Entity Framework update/insert multiple entities
Just a bit of an outline of what i am trying to accomplish. 只是我想要完成的一些概述。 We keep a local copy of a remote database (3rd party) within our application. 我们在应用程序中保留远程数据库(第三方)的本地副本。 To download the information we use an api. 要下载信息,我们使用api。 We currently download the information on a schedule which then either inserts new records into the local database or updates the existing records. 我们目前按计划下载信息,然后将新记录插入本地数据库或更新现有记录。 here is how it currently works 这是它目前的工作方式
public void ProcessApiData(List<Account> apiData)
{
// get the existing accounts from the local database
List<Account> existingAccounts = _accountRepository.GetAllList();
foreach(account in apiData)
{
// check if it already exists in the local database
var existingAccount = existingAccounts.SingleOrDefault(a => a.AccountId == account.AccountId);
// if its null then its a new record
if(existingAccount == null)
{
_accountRepository.Insert(account);
continue;
}
// else its a new record so it needs updating
existingAccount.AccountName = account.AccountName;
// ... continue updating the rest of the properties
}
CurrentUnitOfWork.SaveChanges();
}
This works fine, however it just feels like this could be improved. 这工作正常,但只是感觉这可以改善。
Any suggestions on how i can improve this would be brilliant. 关于如何改善这一点的任何建议都会很棒。 I'm still fairly new to c# so i'm still searching for the best way to do things. 我仍然是c#的新手,所以我仍然在寻找最好的办法。
I'm using .net 4.5.2 and Entity Framework 6.1.3 with MSSQL 2014 as the backend database 我使用.net 4.5.2和Entity Framework 6.1.3与MSSQL 2014作为后端数据库
Attach(newAccount, originalAccount)
to update an existing entity. 假设apiData中的类与您的实体相同,您应该能够使用Attach(newAccount, originalAccount)
来更新现有实体。 For bulk inserts I use AddRange(listOfNewEntitities)
. 对于批量插入,我使用AddRange(listOfNewEntitities)
。 If you have a lot of entities to insert it is advisable to batch them. 如果您要插入大量实体,建议批量处理它们。 Also you may want to dispose and recreate the DbContext
on each batch so that it's not using too much memory. 此外,您可能希望在每个批处理中处置并重新创建DbContext
,以便它不会占用太多内存。
var accounts = new List<Account>(); var context = new YourDbContext(); context.Configuration.AutoDetectChangesEnabled = false; foreach (var account in apiData) { accounts.Add(account); if (accounts.Count % 1000 == 0) // Play with this number to see what works best { context.Set<Account>().AddRange(accounts); accounts = new List<Account>(); context.ChangeTracker.DetectChanges(); context.SaveChanges(); context?.Dispose(); context = new YourDbContext(); } } context.Set<Account>().AddRange(accounts); context.ChangeTracker.DetectChanges(); context.SaveChanges(); context?.Dispose();
For bulk updates, there's not anything built in in LINQ to SQL. 对于批量更新,LINQ to SQL中没有内置任何内容。 There are however libraries and solutions to address this. 然而,有一些库和解决方案来解决这个问题。 See eg Here for a solution using expression trees. 有关使用表达式树的解决方案,请参阅此处 。
For EFCore you can use this library: 对于EFCore,您可以使用此库:
https://github.com/borisdj/EFCore.BulkExtensions https://github.com/borisdj/EFCore.BulkExtensions
And for EF 6 this one: 对于EF 6这个:
https://github.com/TomaszMierzejowski/EntityFramework.BulkExtensions https://github.com/TomaszMierzejowski/EntityFramework.BulkExtensions
Both are extending DbContext
with Bulk operations and have the same syntax call: 两者都使用Bulk操作扩展DbContext
并具有相同的语法调用:
context.BulkInsert(entitiesList);
context.BulkUpdate(entitiesList);
context.BulkDelete(entitiesList);
EFCore version have additionally BulkInsertOrUpdate
method. EFCore版本还有BulkInsertOrUpdate
方法。
List vs. Dictionary 列表与字典
You check in a list every time if the entity exists which is bad. 如果实体存在不良,则每次都检入一个列表。 You should create a dictionary instead to improve performance. 您应该创建一个字典来提高性能。
var existingAccounts = _accountRepository.GetAllList().ToDictionary(x => x.AccountID);
Account existingAccount;
if(existingAccounts.TryGetValue(account.AccountId, out existingAccount))
{
// ...code....
}
Add vs. AddRange 添加vs. AddRange
You should be aware of Add vs. AddRange performance when you add multiple records. 添加多个记录时,您应该了解Add vs. AddRange性能。
So at 10,000 entities, Add method have taken 875x more time to add entities in the context simply. 因此,在10,000个实体中,Add方法仅需要875倍的时间在上下文中添加实体。
To fix it: 要解决这个问题:
In your case, you will need to create an InsertRange method to your repository. 在您的情况下,您需要为存储库创建一个InsertRange方法。
EF Extended EF扩展
You are right. 你是对的。 This library updates all data with the same value. 该库使用相同的值更新所有数据。 That is not what you are looking for. 那不是你想要的。
Disclaimer : I'm the owner of the project Entity Framework Extensions 免责声明 :我是项目实体框架扩展的所有者
This library may perfectly fit for your enterprise if you want to improve your performance dramatically. 如果您想要显着提高性能,此库可能非常适合您的企业。
You can easily perform: 您可以轻松执行:
Example: 例:
public void ProcessApiData(List<Account> apiData)
{
// Insert or Update using the primary key (AccountID)
CurrentUnitOfWork.BulkMerge(apiData);
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.