使用ORDERBY时MySQL慢速JOIN查询

Question

I have a problem with this query:

SELECT a.*
FROM smartressort AS s
JOIN smartressort_to_ressort AS str
    ON s.id = str.smartressort_id
JOIN article_to_ressort AS atr
    ON str.ressort_id = atr.ressort_id
JOIN article AS a FORCE INDEX (source_created)
    ON atr.article_id = a.id    
WHERE
    s.id = 1
ORDER BY
    a.created_at DESC
LIMIT 25;

This one is realy slow, it some times takes 14 sec.

EXPLAIN show this:

1   SIMPLE  s   const   PRIMARY PRIMARY 4   const   1   Using index; Using temporary; Using filesort
1   SIMPLE  str ref PRIMARY,ressort_id  PRIMARY 4   const   1   Using index
1   SIMPLE  atr ref PRIMARY,article_id  PRIMARY 4   com.nps.lvz-prod.str.ressort_id 1262    Using index
1   SIMPLE  a   ALL NULL    NULL    NULL    NULL    146677  Using where; Using join buffer (flat, BNL join)

so the last "all" type is realy bad. But i already tried to force using the index, with no luck.

The Article Table looks like this:

CREATE TABLE `article` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`node_id` varchar(255) NOT NULL DEFAULT '',
`object_id` varchar(255) DEFAULT NULL,
`headline_1` varchar(255) NOT NULL DEFAULT '',
`created_at` datetime(3) NOT NULL,
`updated_at` datetime(3) NOT NULL,
`teaser_text` longtext NOT NULL,
`content_text` longtext NOT NULL,
PRIMARY KEY (`id`),
KEY `article_nodeid` (`node_id`),
KEY `article_objectid` (`object_id`),
KEY `source_created` (`created_at`)
) ENGINE=InnoDB AUTO_INCREMENT=161116 DEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC;

When i remove the FORCE INDEX, the Explain gets better, but the query is still slow.

Explain Without force index:

1   SIMPLE  s   const   PRIMARY PRIMARY 4   const   1   Using index; Using temporary; Using filesort
1   SIMPLE  str ref PRIMARY,ressort_id  PRIMARY 4   const   1   Using index
1   SIMPLE  atr ref PRIMARY,article_id  PRIMARY 4   com.nps.lvz-prod.str.ressort_id 1262    Using index
1   SIMPLE  a   eq_ref  PRIMARY PRIMARY 4   com.nps.lvz-prod.atr.article_id 1   

And for another smartressort id(3) it looks like this:

1   SIMPLE  s   const   PRIMARY PRIMARY 4   const   1   Using index; Using temporary; Using filesort
1   SIMPLE  str ref PRIMARY,ressort_id  PRIMARY 4   const   13  Using index
1   SIMPLE  atr ref PRIMARY,article_id  PRIMARY 4   com.nps.lvz-prod.str.ressort_id 1262    Using index
1   SIMPLE  a   eq_ref  PRIMARY PRIMARY 4   com.nps.lvz-prod.atr.article_id 1   

Here we have 13 Ressorts for one Smartressort. Rows: 1x1x13x1262x1 = 16.406

1) What can i do to make this request faster?

2) What's wrong with the source_created index?

Answer 1

The SELECT * you have in your query is ugly, and this can often be an index killer. It can preclude the use of an index, because most indices you would define would not cover every column demanded by the SELECT * . The approach of this answer is to index all other tables in your query, which would therefore incentivize MySQL to just do a single scan over the article table.

CREATE INDEX idx1 ON article_to_ressort (article_id, ressort_id);
CREATE INDEX idx2 ON smartressort_to_ressort (ressort_id, smartressort_id);

These two indices should speed up the joining process. Note that I did not define an index for the smartressort table, assuming that its id column is already a primary key. I would probably write your query starting with the article table, and joining outwards, but it should not really matter.

Also, forcing an index is mostly either a bad idea or not necessary. The optimizer can usually figure out when it is best to use an index.

Answer 2

SELECT many columns FROM tables ORDER BY something LIMIT few is a notorious performance antipattern; it has to retrieve and order a whole mess of rows and columns, just to discard all but a few rows of the result set.

The trick is to figure out which values of article.id you need in your result set, then retrieve just those values. It's called a deferred join .

This should get you that set of id values. There's probably no need to join the smartressort table because smartressort_to_ressort contains the id values you need.

                 SELECT a.id
                   FROM article a
                   JOIN article_to_ressort atr ON a.id = atr.article_id
                   JOIN smartressort_to_ressort str ON atr.ressort_id = str.ressort_id
                  WHERE str.smartressort_id = 1
                  ORDER BY a.created_at DESC
                  LIMIT 25

Then you can use this as a subquery to get the rows you need.

SELECT a.*
  FROM article a
 WHERE a.id IN (
                 SELECT a.id
                   FROM article a
                   JOIN article_to_ressort atr ON a.id = atr.article_id
                   JOIN smartressort_to_ressort str ON atr.ressort_id = str.ressort_id
                  WHERE str.smartressort_id = 1
                  ORDER BY a.created_at DESC
                  LIMIT 25
               )
 ORDER BY a.created_at DESC

The second ORDER BY makes sure the rows from article are in a predictable order. Your index optimization work, then, need only apply to the subquery.

Answer 3

In addition to @TimBiegelsen 's great answer, I would recommend to modify your source_created index:

...
KEY `source_created` (`id`, `created_at`)

The gain would be that MySQL could use it for sorting, and wouldn't need to fetch all 16406 rows. It may or may not help, but worth to try (perhaps with explicite declaration to use it)

Answer 4

To start with: You can remove the smartressort table from your query, as it doesn't add anything to it.

The following is your query rewritten. We want all ressorts for smart ressort #1 and then all articles for these ressorts. Of these we show the newest 25.

SELECT *
FROM article
WHERE id IN
(
  SELECT article_id
  FROM article_to_ressort 
  WHERE ressort_id IN
  (
    SELECT ressort_id
    FROM smartressort_to_ressort
    WHERE smartressort_id = 1
  )
)
ORDER BY created_at DESC
LIMIT 25;

Now which indexes would be needed to help the DBMS with this? Start with the inner table ( smartressort_to_ressort ). We access all records with a given smartressort_id and we want to get the associated ressort_id . So the index should contain these two columns in this order. Same for article_to_ressort and its ressort_id and article_id . At last we want to select the articles by the found article IDs and order by their created_at .

CREATE INDEX idx1 ON smartressort_to_ressort (smartressort_id, ressort_id);
CREATE INDEX idx2 ON article_to_ressort (ressort_id, article_id);
CREATE INDEX idx3 ON article (id, created_at);

Anyway, these indexes are just an offer to the DBMS. It may decide against them. This is especially true for the index on the article table. How many rows does the DBMS expect to access for one smartressort_id , ie how many rows may be in the IN clause? If the DBMS thinks that this might well be about 10% of all article IDs, it may already decide to rather read the table sequentially than muddle it's way through the index for so many rows.

Answer 5

So for me the solution was this:

SELECT a.*
FROM article as a  USE INDEX (source_created)
where a.id in (
             SELECT atr.article_id
               from smartressort_to_ressort str 
               JOIN article_to_ressort atr  ON atr.ressort_id = str.ressort_id
              WHERE str.smartressort_id = 1
) 
ORDER BY a.created_at DESC
LIMIT 25;

This only needs ~35ms. Explain looks like this:

1   PRIMARY a   index   NULL    source_created  7   NULL    1   
1   PRIMARY <subquery2> eq_ref  distinct_key    distinct_key    4   func    1
2   MATERIALIZED    str ref PRIMARY,ressort_id,idx1 PRIMARY 4   const   1   Using index
2   MATERIALIZED    atr ref PRIMARY,article_id,idx2 PRIMARY 4   com.nps.lvz-prod.str.ressort_id 1262    Using index

Even so, this query Explain looks better for me, but i don't know why exactly:

explain SELECT a.*, NOW()
FROM article as a  USE INDEX (source_created)
where a.id in (SELECT atr.article_id
    FROM smartressort AS s
    JOIN smartressort_to_ressort AS str
    ON s.id = str.smartressort_id
    JOIN article_to_ressort AS atr
    ON str.ressort_id = atr.ressort_id
    WHERE s.id = 1
) 
ORDER BY a.created_at DESC
LIMIT 25;

Output:

1   PRIMARY s   const   PRIMARY PRIMARY 4   const   1   Using index
1   PRIMARY a   index   NULL    source_created  7   NULL    25  
1   PRIMARY str ref PRIMARY,ressort_id,idx1 PRIMARY 4   const   1   Using index
1   PRIMARY atr eq_ref  PRIMARY,article_id,idx2 PRIMARY 8   com.nps.lvz-prod.str.ressort_id,com.nps.lvz-prod.a.id   1   Using index; FirstMatch(a)