create index on tables for mysql query that involves join,group/order-by,union and like

Question

Below is the explain output for the slow query that has 10's of "copying to tmp table" state in mysql processlist.

explain SELECT distinct
    (radgroupreply.groupname),
    count(distinct (radusergroup.username)) AS users
FROM
    radgroupreply
        LEFT JOIN
    radusergroup ON radgroupreply.groupname = radusergroup.groupname
WHERE
    (radgroupreply.groupname NOT LIKE 'FB-%' AND radgroupreply.groupname NOT LIKE '%Dropped%')
GROUP BY radgroupreply.groupname 
UNION SELECT distinct
    (radgroupcheck.groupname),
    count(distinct (radusergroup.username))
FROM
    radgroupcheck
        LEFT JOIN
    radusergroup ON radgroupcheck.groupname = radusergroup.groupname
WHERE
    (radgroupcheck.groupname NOT LIKE 'FB-%' AND radgroupcheck.groupname NOT LIKE '%Dropped%')
GROUP BY radgroupcheck.groupname
ORDER BY groupname asc;

+----+--------------+---------------+------+---------------+------+---------+------+-------+----------------------------------------------+
| id | select_type  | table         | type | possible_keys | key  | key_len | ref  | rows  | Extra                                        |
+----+--------------+---------------+------+---------------+------+---------+------+-------+----------------------------------------------+
|  1 | PRIMARY      | radgroupreply | ALL  | NULL          | NULL | NULL    | NULL |   456 | Using where; Using temporary; Using filesort |
|  1 | PRIMARY      | radusergroup  | ALL  | NULL          | NULL | NULL    | NULL | 10261 |                                              |
|  2 | UNION        | radgroupcheck | ALL  | NULL          | NULL | NULL    | NULL |   167 | Using where; Using temporary; Using filesort |
|  2 | UNION        | radusergroup  | ALL  | NULL          | NULL | NULL    | NULL | 10261 |                                              |
|NULL| UNION RESULT | <union1,2>    | ALL  | NULL          | NULL | NULL    | NULL |  NULL | Using filesort                               |
+----+--------------+---------------+------+---------------+------+---------+------+-------+----------------------------------------------+
5 rows in set (0.00 sec)

I cant get my head around this query to create compound/single index and optimize since it has multiple joins, group by and like operations.

Answer 1

Here are three observations to get started.

1. The select distinct is unnecessary (the group by takes care of that).
2. The left join s are unnecessary (the where clauses turn them into inner joins).
3. The UNION should probably be UNION ALL . I doubt you really want to incur the overhead of removing duplicates.

So, you can write the query as:

SELECT rr.groupname, count(distinct rg.username) AS users  
FROM radgroupreply rr JOIN
     radusergroup rg
     ON rr.groupname = rg.groupname                                                  
WHERE rr.groupname NOT LIKE 'FB-%' AND rr.groupname NOT LIKE '%Dropped%'
GROUP BY rr.groupname 
UNION ALL
SELECT rc.groupname, count(rg.username)
FROM radgroupcheck rc JOIN
     radusergroup rg
     ON rc.groupname = rg.groupname                                 
WHERE rc.groupname NOT LIKE 'FB-%' AND rc.groupname NOT LIKE '%Dropped%'
GROUP BY rc.groupname
ORDER BY groupname asc;

This query can take advantage of indexes on radusergroup(groupname) . I am guessing an index on rc(radusergroup) would be used.

I would also advice you to remove the DISTINCT in COUNT(DISTINCT) if it is not necessary.