mysql update thousands of rows with one statement

Question

I have a table that has literally over 200,000 rows and is growing that need to be updated at given time intervals.

I used to do it with a for each loop and it worked until recently. Now I'm getting php timeout issues.

The update is quite simple. It uses a unique id for the row and updates the quantity.

I looked at using CASE but it would seem there is a limit on how many rows could be updated.

Any advise would be appreciated.

Answer 1

Well, here's a stab in the dark. You mention "update the quantity". There are a couple angles you could take.

In your code, determine how many unique "quantities" you have to update.
Let's say you have 200K products to update, but stock levels tend to stay in the 0 to 50 range.
Then you'd have only about 50 SQL statements to make. They might, however, have rather large concatenated IN statements attached. The guts of your FOR loop would look something like this:
```
 $s = "update myTable set quantity = 10 where id in (124,1215,1232,4444,4532,46789);"; $r = $db->query( $s ); 
```
Another approach would be to have a local copy of the data, and submit a delta for only those products whose quantities have actually changed since the update. That is, for all products that have increased by 1, concatenate the product IDs, and do:
```
 $s="UPDATE myTable SET quantity = (quantity + 1) WHERE id IN (124,1215,1232,4444,4532,46789)"; $r = $db->query( $s ); 
```

It's not terribly difficult to make these concatenated IN statements. Say we have $array_of_ids_that_went_up_by_5:

$in_statement = "( ".implode(",",$array_of_ids_that_went_up_by_5)." );

You might need to rtrim() a comma from the end.

Should be quite a bit more efficient than 200K SQL statements.