Which index will speed up the query?

Question

In the table of 350 million records, the structure is:

CREATE TABLE `table` (
    `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `job_id` int(10) unsigned NOT NULL,
    `lock` mediumint(6) unsigned DEFAULT '0',
    `time` timestamp NULL DEFAULT NULL,
    PRIMARY KEY (`id`),
    KEY `job_id` (`job_id`),
    KEY `lock` (`lock`),
    KEY `time` (`time`)
) ENGINE=MyISAM;

What index should I create to speed up the query:

UPDATE `table` SET `lock` = 1 WHERE `lock` = 0 ORDER BY `time` ASC LIMIT 500;

Answer 1

lock is declared to be NULLable . Does this mean that the value is often NULL ? If so, then there is a nasty problem in MyISAM (not InnoDB) that may lead to 500 additional fragmentation hits.

When a MyISAM row is updated and it becomes longer, then the row will not longer fit where it is. (Now my detailed knowledge gets fuzzy.) The new row will be put somewhere else and/or it will be broken into two parts, with a link between the parts. That implies writes in two places.

As Gordon pointed out, any change to any indexed column, lock in your case, involved a costly index update -- remove a 'row' from one place in the index's BTree and add a row in another place.

Does lock have only values 0 or 1? Then use TINYINT (1 byte), not MEDIUMINT (3 bytes).

You should check MAX(id) . If it is clean, id's max will be about 350M (not too close to the limit of 4B). But if there has been any churn, it may be much closer to the limit.

I, too, advocate switching to InnoDB. However your 10GB (data+indexes) will grow to 20-30GB in the conversion.

Are you "locking the oldest unlocked" thingies? Will you then do a select to see what got locked?

If this is too slow, don't do 500 at once, pick a lower number.

With InnoDB, can you avoid locking? Perhaps transactional locking would suffice?

I think we need to see the rest of the environment -- other tables, job "flow", etc. There may be other things we can suggest.

And I second the motion for INDEX(lock, time) . But when doing so, DROP the index on just lock as being redundant.

And when converting to InnoDB, do all the index changes in the same ALTER . This will run faster than separate passes.

Answer 2

For this query:

UPDATE `table`
    SET `lock` = 1
    WHERE `lock` = 0
    ORDER BY `time` ASC
    LIMIT 500;

The best index is table(lock, time) . Do note, however, that the update also needs to update the index, so you should test how well this works in practice. Do not make this a clustered index. That will just slow down the process.