Sometimes MySQL stops using right index

Question

Several times I had such situation: query that has worked quickly started work in 1000-10_000 times slower in one moment when there was no changes. MySQL stops using proper index and I have to use FORCE INDEX(..) . It happens with queries to big tables with 10-300M records.

MySQL: 5.6.23 (AWS RDS, db.r3.xlarge)

There is the last issue:

table1 (175M records)

CREATE TABLE `table1` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `site_id` int(11) NOT NULL,
  `created_at` datetime DEFAULT NULL,
  `type` varchar(25) DEFAULT NULL,
  ...
  PRIMARY KEY (`id`),
  UNIQUE KEY `index_table1_on_site_id_and_..._and_type_and_...` (`site_id`,`...`,`type`,`...`),
  KEY `index_table1_on_created_at_and_site_id` (`created_at`,`site_id`),
  KEY `index_table1_on_site_id_and_type_and_created_at_and_...` (`site_id`,`type`,`created_at`,`...`) USING BTREE,
  KEY `index_table1_on_site_and_type_and_..._and_created` (`site_id`,`type`,`..._id`,`created_at`),
) ENGINE=InnoDB AUTO_INCREMENT=... DEFAULT CHARSET=utf8

table2 (2M records)

CREATE TABLE `table2` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `table1_id` int(11) NOT NULL,
  ...
  PRIMARY KEY (`id`),
  ...
) ENGINE=InnoDB AUTO_INCREMENT=... DEFAULT CHARSET=utf8

request:

SELECT  `table1`.* FROM `table1` 
INNER JOIN `table2` ON `table2`.`table1_id` = `table1`.`id` 
WHERE `table1`.`type` IN ('...', '...') 
  AND `table1`.`site_id` = ... 
  AND (table1.created_at >= '...') 
  AND (table1.created_at <= '...')  
ORDER BY `table1`.`id` DESC LIMIT 30 offset 0;

was ~10-80ms now > 420 sec

request with FORCE INDEX :

SELECT  `table1`.* FROM `table1` USE INDEX (`index_table1_on_site_id_and_type_and_created_at_and_...`) 
INNER JOIN `table2` ON `table2`.`table1_id` = `table1`.`id` 
WHERE `table1`.`type` IN ('...', '...') 
  AND `table1`.`site_id` = ... 
  AND (table1.created_at >= '...') 
  AND (table1.created_at <= '...')  
ORDER BY `table1`.`id` DESC LIMIT 30 offset 0;

~85 ms

EXPLAINE: without FORCE

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: table1
         type: index
possible_keys: PRIMARY,index_table1_on_site_id_and_..._and_type_and_...,index_table1_on_created_at_and_site_id,index_table1_on_type,index_table1_on_site_id_and_type_and_created_at_and_...,index_table1_on_site_and_type_and_..._and_created
          key: PRIMARY
      key_len: 4
          ref: NULL
         rows: 9257179
        Extra: Using where
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: table2
         type: eq_ref
possible_keys: ...
          key: ...
      key_len: 4
          ref: db.table1.id
         rows: 1
        Extra: Using index

with FORCE

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: table1
         type: range
possible_keys: index_table1_on_site_id_and_type_and_created_at_and_...
          key: index_table1_on_site_id_and_type_and_created_at_and_...
      key_len: 88
          ref: NULL
         rows: 499
        Extra: Using index condition; Using filesort
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: table2
         type: eq_ref
possible_keys: ...
          key: ...
      key_len: 4
          ref: db.table1.id
         rows: 1
        Extra: Using index

Is there any solutions to avoid such unpredictable MySQL behaviour? I can't add FORCE INDEX to all requests, what to do?

PS:

SELECT * FROM `table1` 
INNER JOIN `table2` ON `table2`.`table1_id` = `table1`.`id` 
WHERE `table1`.`site_id` = ... ;

returns just 122 records

PSS: Crazy, but request works faster for wider time period

AND (table1.created_at >= '2016-07-01') AND (table1.created_at <= '2016-07-07)  

420 sec

AND (table1.created_at >= '2016-06-01') AND (table1.created_at <= '2016-07-07)      

85ms

Answer 1

If the tables have changed, you can try running ANALYZE TABLE ( http://dev.mysql.com/doc/refman/5.7/en/analyze-table.html ) to update the stats synchronously. InnoDB persists optimizer stats which has some limitations .

Based on the date ranges, I also wonder if it would be just as fast if you did

AND (table1.created_at >= '2016-06-01') AND (table1.created_at <= '2016-06-07)'

assuming that the older data has more stable stats, and it's not the size that makes the difference.