best algorithm to delete records from database?

Question

Folks,

There are 2500 records in master table city which has references in approx. 22 another tables as city ID or city name.

Now i will have to delete 2000 records from city master table and cross check in another 22 tables where city ID exists and if references exist then modify the ID.

I made a simple script in PHP but it took 15-16 hours and i cannot do same on production server. All 22 tables have on average 700,000 to 800,000 rows.

Please provide best possible solution.

Thanks

Answer 1

Okay, this is a shot in the dark, but here goes:

Remove the cities one at a time. So for each city, update all the other 22 massive tables. This will be slower overall, but will execute in smaller chunks (1/2500th of 16 hours ~= 20-25 seconds). Once the tables are updated, remove that row from the master city table. Rinse and repeat ~2500 times.

Answer 2

I made a simple script in PHP but it took 15-16 hours and i cannot do same on production server

If it's simple then why not post it here? Without knowing your algorithm and data schema we can't guess what the best algorithm would be.

Certainly you should already have ensured that you're using appropriate indexes for your queries and that the mysql instance is tuned (you should probably be configuring more sort_buffer, repair_threads, thread_cache and max_sort_file_size than the default).

In general bulk operations will be much faster done in SQL rather than PHP. And its probably a lot simpler to change the foreign key references before you delete the master record (not to monetion you're not breaking your data integrity during the process).