MySQL latency caused by using index; Using temporary; Using filesort

Question

I have a query that joins two tables and orders the data on the primary key. This is resulting in the very popular problem of MySQL "Using index; Using temporary; Using filesort."

The issue is causing a severe latency problem in my production tables with about 400k records.

Here's more info:

I have two tables: Doctor and Area. The Doctor table has a foreign key pointing to Area.

Doctor:

+-----------------------------+---------------+------+-----+---------+----------------+
| Field                       | Type          | Null | Key | Default | Extra          |
+-----------------------------+---------------+------+-----+---------+----------------+
| id                          | int(11)       | NO   | PRI | NULL    | auto_increment |
| area_id                     | int(11)       | NO   | MUL | NULL    |                |
+-----------------------------+---------------+------+-----+---------+----------------+

Doctor indexes:

+---------------+------------+------------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table         | Non_unique | Key_name               | Seq_in_index | Column_name      | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+------------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| doctor        |          0 | PRIMARY                |            1 | id               | A         |        5546 |     NULL | NULL   |      | BTREE      |         |               |
| doctor        |          1 | doctor_dfd0e917        |            1 | area_id          | A         |          29 |     NULL | NULL   |      | BTREE      |         |               |
+---------------+------------+------------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Area:

+------------------------+-------------+------+-----+---------+----------------+
| Field                  | Type        | Null | Key | Default | Extra          |
+------------------------+-------------+------+-----+---------+----------------+
| id                     | int(11)     | NO   | PRI | NULL    | auto_increment |
+------------------------+-------------+------+-----+---------+----------------+

And the Area indexes:

+---------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table         | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| area          |          0 | PRIMARY  |            1 | id          | A         |          24 |     NULL | NULL   |      | BTREE      |         |               |
+---------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

I'm trying to run the following query:

SELECT `doctor`.`id`, 
       `area`.`id` 
FROM 
       `doctor` 
INNER JOIN 
       `area` ON (`doctor`.`area_id` = `area`.`id`) 
ORDER BY 
      `doctor`.`id` DESC LIMIT 100;

The EXPLAIN returns the following (with the problematic Using index; Using temporary; Using filesort):

+----+-------------+---------------+-------+------------------------+------------------------+---------+--------------+------+----------------------------------------------+
| id | select_type | table         | type  | possible_keys          | key                    | key_len | ref          | rows | Extra                                        |
+----+-------------+---------------+-------+------------------------+------------------------+---------+--------------+------+----------------------------------------------+
|  1 | SIMPLE      | area          | index | PRIMARY                | PRIMARY                | 4       | NULL         |   24 | Using index; Using temporary; Using filesort |
|  1 | SIMPLE      | doctor        | ref   | doctor_dfd0e917        | doctor_dfd0e917        | 4       | area.id      |  191 | Using index                                  |
+----+-------------+---------------+-------+------------------------+------------------------+---------+--------------+------+----------------------------------------------+

If I remove the ORDER BY clause, I get the desired effect:

+----+-------------+---------------+-------+------------------------+------------------------+---------+--------------+------+----------------------------------------------+
| id | select_type | table         | type  | possible_keys          | key                    | key_len | ref          | rows | Extra                                        |
+----+-------------+---------------+-------+------------------------+------------------------+---------+--------------+------+----------------------------------------------+
|  1 | SIMPLE      | area          | index | PRIMARY                | PRIMARY                | 4       | NULL         |   24 | Using index                                  |
|  1 | SIMPLE      | doctor        | ref   | doctor_dfd0e917        | doctor_dfd0e917        | 4       | area.id      |  191 | Using index                                  |
+----+-------------+---------------+-------+------------------------+------------------------+---------+--------------+------+----------------------------------------------+

Why is the ORDER BY clause causing problems here even though I'm using the primary key?

Thank you in advance.

Answer 1

It seems that you only have one area per doctor. See how this query works:

SELECT d.id,
       (SELECT a.id FROM area a ON a.id = d.area_id) as area_id
FROM doctor d 
ORDER BY d.id DESC
LIMIT 100;

If you are using inner join to test for the presence of a doctor in the table, then add:

SELECT d.id,
       (SELECT a.id FROM area a ON a.id = d.area_id) as area_id
FROM doctor d 
WHERE EXISTS (SELECT 1 FROM area a ON a.id = d.area_id)
ORDER BY d.id DESC
LIMIT 100;

There is a good chance that both of these will scan the doctors table in order, picking up the information from area as needed.