Why isn't MySQL using an index for this query when it uses one for 99% of other queries of an identical form?

Question

I have a database which I make a lot of queries to like this:

mysql> explain     SELECT  time, user, x, y, z, type, action
        FROM  co_block
        WHERE  time >= 1642497664
          AND  wid = 4
          AND  x <= -7650
          AND  x >= -7651;

+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
| id | select_type | table    | partitions | type  | possible_keys | key  | key_len | ref  | rows    | filtered | Extra                 |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
|  1 | SIMPLE      | co_block | NULL       | range | wid           | wid  | 10      | NULL | 3940334 |    33.33 | Using index condition |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+

This query returns 101 rows in 0.7 seconds.

Almost all of them return very quickly, but occasionally I run into one that takes an eternity, like this almost-identical query which differs only in the X range that is searched.

mysql> explain SELECT time, user, x, y, z, type, action FROM co_block WHERE time >= 1642497664 AND wid = 4 AND x <= -7651 AND x >= -7652;
+----+-------------+----------+------------+------+---------------+------+---------+------+-----------+----------+-------------+
| id | select_type | table    | partitions | type | possible_keys | key  | key_len | ref  | rows      | filtered | Extra       |
+----+-------------+----------+------------+------+---------------+------+---------+------+-----------+----------+-------------+
|  1 | SIMPLE      | co_block | NULL       | ALL  | wid           | NULL | NULL    | NULL | 116322801 |     3.59 | Using where |
+----+-------------+----------+------------+------+---------------+------+---------+------+-----------+----------+-------------+

This query returns 85 rows after a painful 68 seconds.

If I use FORCE INDEX to force the use of the index, the query returns in only 1.2 seconds, so it obviously can be used and to great benefit, and indeed MySQL seems to actually use it in 99.99% of the queries of this sort that I make. So why is it choosing not to use the index in this case?

The "explain" of the FORCE INDEX queries, in case they're insightful:

mysql> explain SELECT time, user, x, y, z, type, action FROM co_block FORCE INDEX (`wid`) WHERE time >= 1642497664 AND wid = 4 AND x <= -7650 AND x >= -7651;
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
| id | select_type | table    | partitions | type  | possible_keys | key  | key_len | ref  | rows    | filtered | Extra                 |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
|  1 | SIMPLE      | co_block | NULL       | range | wid           | wid  | 10      | NULL | 3940334 |    33.33 | Using index condition |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

mysql> explain SELECT time, user, x, y, z, type, action FROM co_block FORCE INDEX (`wid`) WHERE time >= 1642497664 AND wid = 4 AND x <= -7651 AND x >= -7652;
+----+-------------+----------+------------+-------+---------------+------+---------+------+----------+----------+-----------------------+
| id | select_type | table    | partitions | type  | possible_keys | key  | key_len | ref  | rows     | filtered | Extra                 |
+----+-------------+----------+------------+-------+---------------+------+---------+------+----------+----------+-----------------------+
|  1 | SIMPLE      | co_block | NULL       | range | wid           | wid  | 10      | NULL | 12543806 |    33.33 | Using index condition |
+----+-------------+----------+------------+-------+---------------+------+---------+------+----------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

Table definition:

CREATE TABLE `co_block` (
  `rowid` bigint NOT NULL AUTO_INCREMENT,
  `time` int DEFAULT NULL,
  `user` int DEFAULT NULL,
  `wid` int DEFAULT NULL,
  `x` int DEFAULT NULL,
  `y` int DEFAULT NULL,
  `z` int DEFAULT NULL,
  `type` int DEFAULT NULL,
  `data` int DEFAULT NULL,
  `meta` blob,
  `blockdata` blob,
  `action` int DEFAULT NULL,
  `rolled_back` tinyint(1) DEFAULT NULL,
  PRIMARY KEY (`rowid`),
  KEY `wid` (`wid`,`x`,`z`,`time`),
  KEY `user` (`user`,`time`),
  KEY `type` (`type`,`time`)
) ENGINE=InnoDB AUTO_INCREMENT=252511481 DEFAULT CHARSET=utf8mb3

Index info:

mysql> show index from co_block;
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| Table    | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| co_block |          0 | PRIMARY  |            1 | rowid       | A         |   116288328 |     NULL |   NULL |      | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | wid      |            1 | wid         | A         |         886 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | wid      |            2 | x           | A         |      135141 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | wid      |            3 | z           | A         |    22393332 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | wid      |            4 | time        | A         |    71268960 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | user     |            1 | user        | A         |       49756 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | user     |            2 | time        | A         |    13558248 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | type     |            1 | type        | A         |       33126 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
| co_block |          1 | type     |            2 | time        | A         |    28455380 |     NULL |   NULL | YES  | BTREE      |         |               | YES     | NULL       |
+----------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+

Likely irrelevant notes:

The actual queries I'm making are grabbing 16x16 chunks by specifying both an X range and a Z range, but in trying to debug the problem myself, the Z range specified doesn't seem to be relevant to the problem, so I removed it from the queries.

In one of the many "why isn't MySQL using my index" questions I read, the problem was that the condition specified by the WHERE clause included almost all of the records. This made me wonder if the problem might be the order of the "x >= -7652" and "x <= -7651" conditions, as the first would include almost all of the records in the database, and only the second actually eliminates most of them. So I reversed the order of these two conditions, but it made no difference. I also tried using BETWEEN to specify the range but that too had no effect.

Though perhaps only due to insufficient effort to find an example case, it seems to be required that there is both an "x >=" and an "x <=" condition.

The time constrain is not required but I included it above because otherwise there is much more data returned by the queries. Here are the "explain" queries with the time constraint removed:

mysql> explain SELECT time, user, x, y, z, type, action FROM co_block WHERE wid = 4 AND x <= -7651 AND x >= -7652;
+----+-------------+----------+------------+------+---------------+------+---------+------+-----------+----------+-------------+
| id | select_type | table    | partitions | type | possible_keys | key  | key_len | ref  | rows      | filtered | Extra       |
+----+-------------+----------+------------+------+---------------+------+---------+------+-----------+----------+-------------+
|  1 | SIMPLE      | co_block | NULL       | ALL  | wid           | NULL | NULL    | NULL | 116325588 |    10.78 | Using where |
+----+-------------+----------+------------+------+---------------+------+---------+------+-----------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

mysql> explain SELECT time, user, x, y, z, type, action FROM co_block WHERE wid = 4 AND x <= -7650 AND x >= -7651;
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
| id | select_type | table    | partitions | type  | possible_keys | key  | key_len | ref  | rows    | filtered | Extra                 |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
|  1 | SIMPLE      | co_block | NULL       | range | wid           | wid  | 10      | NULL | 3940334 |   100.00 | Using index condition |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)

Query results requested by nnichols:

mysql> SELECT COUNT(*) FROM co_block FORCE INDEX (wid) WHERE wid = 4 AND x <= -7650 AND x >= -7651;
+----------+
| COUNT(*) |
+----------+
|  2958623 |
+----------+
1 row in set (0.86 sec)

mysql> explain SELECT COUNT(*) FROM co_block FORCE INDEX (wid) WHERE wid = 4 AND x <= -7650 AND x >= -7651;
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------+
| id | select_type | table    | partitions | type  | possible_keys | key  | key_len | ref  | rows    | filtered | Extra                    |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------+
|  1 | SIMPLE      | co_block | NULL       | range | wid           | wid  | 10      | NULL | 3940334 |   100.00 | Using where; Using index |
+----+-------------+----------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------+
1 row in set, 1 warning (0.00 sec)

mysql> SELECT COUNT(*) FROM co_block FORCE INDEX (wid) WHERE wid = 4 AND x <= -7651 AND x >= -7652;
+----------+
| COUNT(*) |
+----------+
|  8982424 |
+----------+
1 row in set (2.38 sec)

mysql> explain SELECT COUNT(*) FROM co_block FORCE INDEX (wid) WHERE wid = 4 AND x <= -7651 AND x >= -7652;
+----+-------------+----------+------------+-------+---------------+------+---------+------+----------+----------+--------------------------+
| id | select_type | table    | partitions | type  | possible_keys | key  | key_len | ref  | rows     | filtered | Extra                    |
+----+-------------+----------+------------+-------+---------------+------+---------+------+----------+----------+--------------------------+
|  1 | SIMPLE      | co_block | NULL       | range | wid           | wid  | 10      | NULL | 12543806 |   100.00 | Using where; Using index |
+----+-------------+----------+------------+-------+---------------+------+---------+------+----------+----------+--------------------------+
1 row in set, 1 warning (0.00 sec)

Answer 1

Add these two indexes and see if things improve:

INDEX(wid, x, time),
INDEX(wid, time, x)

Also, run this once:

ANALYZE TABLE co_block;

How big is the table compared to innodb_buffer_pool_size ? If quite big, then think about whether some of the INTs (4 bytes each) can be shrunk to smaller datatypes.

This might help, too:

ORDER BY x, time

Answer 2

As already discussed in the comments and mentioned by @RickJames, you really should update the datatypes to better fit the data. I would suggest starting with -

ALTER TABLE `co_block` 
    CHANGE COLUMN `time` `time` INT UNSIGNED NOT NULL, /* 5 => 4 bytes per index entry */
    CHANGE COLUMN `wid` `wid` TINYINT NOT NULL,        /* 5 => 1 byte */
    CHANGE COLUMN `x` `x` SMALLINT NOT NULL,           /* 5 => 2 bytes */
    CHANGE COLUMN `z` `z` SMALLINT NOT NULL;           /* 5 => 2 bytes */

This will reduce your wid index from 20 bytes => 9 bytes per row; less than half the size. Assuming similar changes can be made to all the integer fields, your space saving on both the data and index will be considerable. That reduction in size will also lead to improved performance.

If you are concerned about possible impact on the plugin then do some digging online or speak to the developer. Do you need to retain all the data? Should you implement a purge routine to remove data older than x months?