why is mysql not using the right index?

Question

I have a problem where the correct index is no being used.

I have the following indexes on an innodb table (about 500,000 rows) :

+-------------+------------+------------------------+--------------+------------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table       | Non_unique | Key_name               | Seq_in_index | Column_name            | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------------+------------+------------------------+--------------+------------------------+-----------+-------------+----------+--------+------+------------+---------+
| osdate_user |          0 | PRIMARY                |            1 | id                     | A         |      419700 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          0 | email                  |            1 | email                  | A         |      419700 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          0 | username               |            1 | username               | A         |      419700 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | lastvisit              |            1 | lastvisit              | A         |      419700 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | active                 |            1 | active                 | A         |           8 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | gender                 |            1 | gender                 | A         |          88 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | regdate                |            1 | regdate                | A         |      419700 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | lastupdate             |            1 | lastupdate             | A         |      419700 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | password               |            1 | password               | A         |      419700 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | age                    |            1 | age                    | A         |         190 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | is_new                 |            1 | is_new                 | A         |           8 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | private_photos         |            1 | private_photos         | A         |           8 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | pictures_cnt           |            1 | pictures_cnt           | A         |          10 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | pictures_cnt           |            2 | private_photos         | A         |          10 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | status                 |            1 | status                 | A         |          19 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | status                 |            2 | active                 | A         |          19 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | status                 |            3 | gender                 | A         |          19 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | status                 |            4 | age                    | A         |          19 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | status                 |            5 | country                | A         |        7630 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | status                 |            6 | city                   | A         |       46633 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | status                 |            7 | pictures_cnt           | A         |       83940 |     NULL | NULL   | YES  | BTREE      |         |
| osdate_user |          1 | status                 |            8 | private_photos         | A         |      139900 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | status                 |            9 | lang                   | A         |      209850 |     NULL | NULL   |      | BTREE      |         |
| osdate_user |          1 | status                 |           10 | is_new                 | A         |      209850 |     NULL | NULL   | YES  | BTREE      |         |
+-------------+------------+------------------------+--------------+------------------------+-----------+-------------+----------+--------+------+------------+---------+

This query :

EXPLAIN EXTENDED SELECT user.id, user.active
FROM osdate_user user
WHERE user.active =1
AND user.status =  'active'
AND user.gender =  'M'
AND user.age
BETWEEN 19 
AND 35 
AND user.pictures_cnt >0
AND user.private_photos =  '0'
ORDER BY user.lastvisit DESC 
LIMIT 0 , 24

Displays the following plan :

+----+-------------+---------+-------------+--------------------------------------------------------------+------------------------------+---------+---------------------------+-------+----------+----------------------------------------------------------------------------+
| id | select_type | table   | type        | possible_keys                                                | key                          | key_len | ref                       | rows  | filtered | Extra                                                                      |
+----+-------------+---------+-------------+--------------------------------------------------------------+------------------------------+---------+---------------------------+-------+----------+----------------------------------------------------------------------------+
|  1 | SIMPLE      | user    | index_merge | PRIMARY,active,gender,age,private_photos,pictures_cnt,status | gender,private_photos,active | 4,1,2   | NULL                      | 28204 |   100.00 | Using intersect(gender,private_photos,active); Using where; Using filesort |
|  1 | SIMPLE      | userext | eq_ref      | userid                                                       | userid                       | 4       | db_name.user.id |     1 |   100.00 | Using index                                                                |
+----+-------------+---------+-------------+--------------------------------------------------------------+------------------------------+---------+---------------------------+-------+----------+----------------------------------------------------------------------------+

My question is why isn't the query using the index "status"

When I force it to use the index "status" - it takes 0.2s, if I don't it takes 2.5s

Any help much appreciated. regards.

Answer 1

u have several indexes that overlap the fields they index. For example pictures_cnt and status
The optimizer "sees" your query as closer to use the fields of the pictures_cnt rather than the status index. If u will have more fields in the where that belongs to the status index, it might decide to use that index instead of picture_cnt

Do notice, it can't use both.

If u think the optimizer is wrong (as is the case on uncommon occasions) u can force it to use the index u want it to use. do benchmark it though.

Answer 2

After searching a bit, I found some recommendations regarding not using columns that allow null in indexes. I changed the columns in the status index to all be non-null, and now that index is being used.

As is usually the case, the simplest and cleanest solutions are the best.