Simple SELECT mysql query very slow (using intersect)

Question

A query that used to work just fine on a production server has started becoming extremely slow (in a matter of hours).

This is it:

SELECT * FROM news_articles WHERE published = '1' AND news_category_id = '4' ORDER BY date_edited DESC LIMIT 1;

This takes up to 20-30 seconds to execute (the table has ~200.000 rows)

This is the output of EXPLAIN :

+----+-------------+---------------+-------------+----------------------------+----------------------------+---------+------+------+--------------------------------------------------------------------------+
| id | select_type | table         | type        | possible_keys              | key                        | key_len | ref  | rows | Extra                                                                    |
+----+-------------+---------------+-------------+----------------------------+----------------------------+---------+------+------+--------------------------------------------------------------------------+
|  1 | SIMPLE      | news_articles | index_merge | news_category_id,published | news_category_id,published | 5,5     | NULL | 8409 | Using intersect(news_category_id,published); Using where; Using filesort |
+----+-------------+---------------+-------------+----------------------------+----------------------------+---------+------+------+--------------------------------------------------------------------------+

Playing around with it, I found that hinting a specific index ( date_edited ) makes it much faster:

SELECT * FROM news_articles USE INDEX (date_edited) WHERE published = '1' AND news_category_id = '4' ORDER BY date_edited DESC LIMIT 1;

This one takes milliseconds to execute .

EXPLAIN output for this one is:

+----+-------------+---------------+-------+---------------+-------------+---------+------+------+-------------+
| id | select_type | table         | type  | possible_keys | key         | key_len | ref  | rows | Extra       |
+----+-------------+---------------+-------+---------------+-------------+---------+------+------+-------------+
|  1 | SIMPLE      | news_articles | index | NULL          | date_edited | 8       | NULL |    1 | Using where |
+----+-------------+---------------+-------+---------------+-------------+---------+------+------+-------------+

Columns news_category_id , published and date_edited are all indexed.

The storage engine is InnoDB.

This is the table structure:

CREATE TABLE `news_articles` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `title` text NOT NULL,
  `subtitle` text NOT NULL,
  `summary` text NOT NULL,
  `keywords` varchar(500) DEFAULT NULL,
  `body` mediumtext NOT NULL,
  `source` varchar(255) DEFAULT NULL,
  `source_visible` int(11) DEFAULT NULL,
  `author_information` enum('none','name','signature') NOT NULL     DEFAULT 'name',
  `date_added` datetime NOT NULL,
  `date_edited` datetime NOT NULL,
  `views` int(11) DEFAULT '0',
  `news_category_id` int(11) DEFAULT NULL,
  `user_id` int(11) DEFAULT NULL,
  `c_forwarded` int(11) DEFAULT '0',
  `published` int(11) DEFAULT '0',
  `deleted` int(11) DEFAULT '0',
  `permalink` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `user_id` (`user_id`),
  KEY `news_category_id` (`news_category_id`),
  KEY `published` (`published`),
  KEY `deleted` (`deleted`),
  KEY `date_edited` (`date_edited`),
  CONSTRAINT `news_articles_ibfk_3` FOREIGN KEY (`news_category_id`) REFERENCES `news_categories` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
  CONSTRAINT `news_articles_ibfk_4` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=192588 DEFAULT CHARSET=utf8

I could possibly change all queries my web application does to hint using that index. but this is considerable work.

Is there some way to tune MySQL so that the first query is made more efficient without actually rewriting all queries?

Answer 1

just a few tips..

1 - It seems to me the fields published and news_category_id are INTEGER. If so, please remove the single quotes from your query. It can make a huge difference when comes to performance;

2 - Also, I'd say that your field published has no many different values (it is probably 1 - yes and 0 - no, or something like that). If I'm right, this is not a good field to index at all. The parse in this case still has to go through all the records to find what it is looking for; In this case move the news_category_id to be the first field in your WHERE clause.

3 - "Don't forget about the most left index". This affirmation is valid for your SELECT, JOINS, WHERE, ORDER BY. Even the position of the columns on the table are imporant, keep the indexed ones on the top. Indexes are your friend as long as you know how to play with them.

Hope it can help you in somehow..

SELECT * FROM news_articles WHERE published = '1' AND news_category_id = '4' ORDER BY date_edited DESC LIMIT 1;

Answer 2

Original: SELECT * FROM news_articles WHERE published = 1 AND news_category_id = 4 ORDER BY date_edited DESC LIMIT 1;

Since you have LIMIT 1 , you're only selecting the latest row. ORDER BY date_edited tells MySQL to sort then take 1 row off the top. This is really slow, and why USE INDEX would help.

Try to match MAX(date_edited) in the WHERE clause instead. That should get the query planner to use its index automatically.

Choose MAX(date_entered): SELECT * FROM news_articles WHERE published = 1 AND news_category_id = 4 AND date_edited = (select max(date_edited) from news_articles);

Answer 3

Please change your query to :

SELECT * FROM news_articles WHERE published = 1 AND news_category_id = 4 ORDER BY date_edited DESC LIMIT 1;

Please note that i have removed quotes from '1' and '4' data provided in query

The difference in the datatype passed and the column structure does not allow mysql to be able to use the index on these 2 columns.