如何在超过 1,000,000,000 条记录中更改 substring？

Question

I have a database table with columns dbValues and dbKeys . The dbKeys value is a string containing keys separated with ; .

dbKeys 1: "name;age;color;...price"
dbKeys 2: "city;street;age;...favourite_meal"
...

dbValues has same formatting as dbKeys but contains values for this keys.

dbValues 1: "Peter;18;blue;...64"
dbValues 2: "London;Main;40;...applepie"
...

There are more than 1 000 000 000 rows. I need to delete some keys and its values from all of the records where the key is, for example the age key, so the result would be:

dbKeys 1: "name;color;...price"
dbKeys 2: "city;street;...favourite_meal"
...

dbValues 1: "Peter;blue;...64"
dbValues 2: "London;Main;...applepie"
...

Do you have any recommendation how to modify all the records with the specific key value in the dbKeys column in the most effective way? I prefer some SQL, or something in .NET / C#.

My solution is to create for-cycle over all the records (or over every 1000 records) and change every record separately. But it doesn't look very effective to me. So I'm opened for any other suggestion.

Answer 1

This will be really difficult in SQL but pretty easy in C#. Just:

• Load the record
• Split the values (for both columns) into lists with String.Split()
• Loop through the keys. If there is a match, delete the index from both lists.
• Join the lists back into strings with String.Join()
• Save the record

You might want to save in a batch rather than individually.

Of course storing the data in this form is not ideal in the first place. But you probably already know that.