Optimize SQL query in MySQL 5.6

Question

System is Amazon RDS (MySql 5.6.x), Moodle 2.8 based software Here is the table that I'm currently working on:

CREATE TABLE `mdl_course_categories` (
  `id` bigint(10) NOT NULL AUTO_INCREMENT,
  `name` varchar(255) NOT NULL DEFAULT '',
  `idnumber` varchar(100) DEFAULT NULL,
  `description` longtext,
  `descriptionformat` tinyint(2) NOT NULL DEFAULT '0',
  `parent` bigint(10) NOT NULL DEFAULT '0',
  `sortorder` bigint(10) NOT NULL DEFAULT '0',
  `coursecount` bigint(10) NOT NULL DEFAULT '0',
  `visible` tinyint(1) NOT NULL DEFAULT '1',
  `visibleold` tinyint(1) NOT NULL DEFAULT '1',
  `timemodified` bigint(10) NOT NULL DEFAULT '0',
  `depth` bigint(10) NOT NULL DEFAULT '0',
  `path` varchar(255) NOT NULL DEFAULT '',
  `theme` varchar(50) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `mdl_courcate_par_ix` (`parent`),
  KEY `mdl_carcoute_tmid` (`timemodified`,`id`),
  KEY `mdl_tm_field` (`timemodified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

The original query I had looks like this:

SELECT 
    id,
    name,
    description,
    FROM_UNIXTIME(timemodified) AS timemodified,
    timemodified AS traw
FROM
    mdl_course_categories
WHERE
    timemodified BETWEEN 1360602072 AND 1446736233
        OR 
    id > 0
ORDER BY id ASC
LIMIT 0 , 50000

Explain for this query is:

+----+-------------+-----------------------+-------+----------------------------------------+---------+---------+------+------+----------+-------------+
| id | select_type | table                 | type  | possible_keys                          | key     | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-----------------------+-------+----------------------------------------+---------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | mdl_course_categories | index | PRIMARY,mdl_carcoute_tmid,mdl_tm_field | PRIMARY | 8       | NULL |   68 |   100.00 | Using where |
+----+-------------+-----------------------+-------+----------------------------------------+---------+---------+------+------+----------+-------------+

I was suggested by database specialist to avoid OR and transform query into this:

SELECT * FROM
(
    (
        SELECT 
            id,
            name,
            description,
            FROM_UNIXTIME(timemodified) AS timemodified,
            timemodified AS traw
        FROM
            mdl_course_categories
        WHERE
            timemodified BETWEEN 1360602072 AND 1446736233
    )
    UNION ALL
    (
        SELECT 
            id,
            name,
            description,
            FROM_UNIXTIME(timemodified) AS timemodified,
            timemodified AS traw
        FROM
            mdl_course_categories
        WHERE
            id > 0
    )
) t
ORDER BY id ASC
LIMIT 0 , 50000

Explain looks like this:

+----+--------------+-----------------------+-------+--------------------------------+-------------------+---------+------+------+----------+-----------------------+
| id | select_type  | table                 | type  | possible_keys                  | key               | key_len | ref  | rows | filtered | Extra                 |
+----+--------------+-----------------------+-------+--------------------------------+-------------------+---------+------+------+----------+-----------------------+
|  1 | PRIMARY      | <derived2>            | ALL   | NULL                           | NULL              | NULL    | NULL |   80 |   100.00 | Using filesort        |
|  2 | DERIVED      | mdl_course_categories | range | mdl_carcoute_tmid,mdl_tm_field | mdl_carcoute_tmid | 8       | NULL |   12 |   100.00 | Using index condition |
|  3 | UNION        | mdl_course_categories | range | PRIMARY                        | PRIMARY           | 8       | NULL |   68 |   100.00 | Using where           |
| NULL | UNION RESULT | <union2,3>            | ALL   | NULL                           | NULL              | NULL    | NULL | NULL |     NULL | Using temporary       |
+----+--------------+-----------------------+-------+--------------------------------+-------------------+---------+------+------+----------+-----------------------+

What do you think? Can this be made any better?

Answer 1

The optimal query would include the limit and order by in the subqueries and use the right indexes:

SELECT *
FROM ((SELECT id, name, description,
              FROM_UNIXTIME(timemodified) AS timemodified, timemodified AS traw
       FROM mdl_course_categories
       WHERE timemodified BETWEEN 1360602072 AND 1446736233
       ORDER BY id
       LIMIT 50000
      ) UNION ALL
      (SELECT id, name, description,
              FROM_UNIXTIME(timemodified) AS timemodified, timemodified AS traw
       FROM mdl_course_categories
       WHERE id > 0 AND
             NOT timemodified BETWEEN 1360602072 AND 1446736233
       ORDER BY id
       LIMIT 50000
      )
     ) t
ORDER BY id ASC
LIMIT 0 , 50000;

The indexes you want are mdl_course_categories(id, timemodified) and mdl_course_categories(timemodified) . Unfortunately, you cannot necessarily out of the sorting, because the where clause has inequalities. However, sorting 100,000 records should be better than all the records.

Note that the second WHERE clause is changed to exclude records from the first one.