Best way to update a collection of SQL data

Question

I am writing a .net/entity framework code snippet that's supposed to update/delete a bunch of MS SQL rows with the latest data passed from UI. Say the table originally has 20 rows and the latest collection contains 15 records. Out of the 15, 9 have changes and 6 remain the same. So 9 rows will be updated, and the 5 rows that are not in the latest collection, will be deleted from the table. So my question is, what's the best way of doing this: If I iterate over all 20 rows and try to find each of them, it would be O(mn). Deleting all table rows and re-insert them may be faster but I am not sure. All help appreciated!

Answer 1

So you have a user interface element filled with items of some class. You also have a database with a table filled with items of the same class.

IEnumerable<MyClass> userInterfaceElements = ...
IQueryable<MyClass> databaseElements = ...

Note: the query is not executed yet!

You want to update the database such, that after your update your database contains the items from your user interface elements.

• User interface elements that are not in the database yet will be added
• Database elements that are not in the user interface need to be removed
• User interface elements that are also in the database need to be updated.

You didn't write how you decide whether a user interface element is also in the database. Let's assume you don't invent primary keys. This means that elements with a default value (zero) for your primary key are elements that are not in the database.

var itemsToAdd = userInterfaceElements.Where(row => row.Id == 0);
var itemsToUpdate = userInterfaceElements.Where(row => row.Id != 0);

var idsItemsToKeep = itemsToUpdate.Select(row => row.Id);
var itemsToRemove = databaseElements.Where(row => !idsItemsToKeep.Contains(row.Id))

The last one: remove all items that have an Id that is not in your user interface elements anymore.

Note: we still have not executed any query!

Adding the items to your database will change databaseElements, so before you make any changes you need to materialize the items

var addList = itemsToAdd.ToList();
var updateList = itemsToUpdate.ToList();
var removeList = itemsToRemove.ToList();

By now you've queried your database exactly once: you fetched all items to remove. You can't order entity framework to remove items without fetching them first.

dbContext.MyClasses.RemoveRange(removeList);
dbContext.MyClasses.AddRange(addList);

To update in entity framework, the proper method would be to fetch the data and then change the properties.

Some people prefer to attach the items to the dbContext's change tracker and tell that it is changed. This might be dangerous however, if someone else has changed some properties of these items, especially if you don't show these values in your user interface elements. So only do this if you really have a long list of items to update.

Proper way:

foreach(var itemToUpdate in updateList)
{
     var fetchedItem = dbContext.MyClasses.Find(itemToUpdate.Id);
     // TODO: update changed properties of the fetchedItem with values from itemToUpdate
}

Dangerous method:

foreach(var itemToUpdate in updateList)
{
    dbContext.Entry(itemToUpdate).State = entityState.Modified;
}

Finally:

dbContext.SaveChanges();

Improved delete method

You've got a problem when you filled your user interface element with database values, and some other process removed one of these values from your database.

When your code looks at the primary key, it will think it is in the database, however it is not there anymore. What to do with this element? Add it again? Act as if the user also wanted it to be deleted?

To solve this kind of problems, quite often people don't delete items from their database, but declare them obsolete instead. They add a Boolean column to the table that indicates whether the item is to be deleted in the near future. This solves the problem that people want to update items while others want them to be removed.

Regularly, every month or so, a process is started to remove all obsolete objects. The chance that you want to update an obsolete object are much lower.

If this needs to be full save: don't remember a Boolean obsolete, but the obsolete date. Periodically remove all items that are obsolete for a longer time.

The nice thing about the obsolete, is that if someone declared an item obsolete by accident, there is still some time to repair this.