MySQL: Optimizing UPDATE of many rows in a large table

Question

The table is organized using a nested set model . When I'm inserting something I need to move everything with left/right values > destination to the left.

UPDATE projects SET rgt = rgt + 2 WHERE rgt >= @superRgt;

This query can take a couple of seconds to complete, which is not acceptable. My question is; how can this query be optimized? is it possible to ..

• Normalize/defragment/reorder the physical layout of the table?
• Create better indexes?
• Just be smarter and avoid the problem?

We're using Innodb table, and already have indexes on left, right and left/right. The table has ~100k rows.

Answer 1

No magic bullets, but a few thoughts...

I know that it's proposed quite a bit, I've never seen the physical layout of a table change MySQL performance (with any significance) for real workloads.

Although indexes can speed up queries, having too many indexes can slow updates down due to the fact that the indexes need to reflect the updates in data. Note that your left index is effectively irrelevant since it's the lead field on the left/right index. Having said that, since you will be likely typically be using ranging queries, the indexes on left and right are likely sufficient (that is, I might be inclined to drop the left/right index unless you know it's being used). Loosely speaking, MySQL can only use the later part of compound indexes if all the preceding columns are used in equality references.

Regardless, to speed up the execution of your query, is there any chance that your left/right fields can take on negative values? If that's the case, then you could "move" the data on the smaller side of the pivot point left or right --- whichever will cause a smaller number of rows to be updated.

Note that if you're updating too many rows, MySQL won't use an index at all. There is a heuristic threshold of the percentage of table rows (often reported to be ~30%) after which MySQL will refuse to use an index. That is, at a certain point, it's better to do one disk seek and scan the whole table rather than do disk-seeks for >30% of the rows in the table.

Stepping back to fundamentals, have you deviated from the (poor) default innodb configurion settings? See this article for some pointers. At the very least, make sure your dataset can fit in the innodb_buffer_pool_size (if you have the RAM), and change the innodb_flush_log_at_trx_commit to 0 or 2 if your application allows for it.

Answer 2

所有其他方法相同，使用ENGINE = MEMORY创建重复的临时表应加快更新速度。