MySQL Query optimization - speed up|index use

Question

I have the following MySQL query that takes more than a day to execute:

SELECT SN,NUMBER FROM a

WHERE SN IN 
    (SELECT LOWER_SN FROM b
    WHERE HIGHER_ED  LIKE "%c1" AND LOWER_ED LIKE "%16")
AND ED LIKE "%16"

The subquery takes 21 seconds to run and returns 11035 rows. I have indices on a:

SHOW INDEX FROM a

Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment

0 a 1 wob1 1 ED A 756095 None None BTREE
1 a 1 wob2 1 SN A 2268287 None None BTREE
2 a 1 wob3 1 ED A 756095 None None BTREE
3 a 1 wob3 2 SN A 9073150 None None BTREE
4 a 1 wob4 1 NUMBER A 18146301 None None YES BTREE
5 a 1 wob5 1 SN A 2268287 None None BTREE
6 a 1 wob5 2 NUMBER A 18146301 None None YES BTREE

EXPLAIN gives:

# id, select_type, table, type, possible_keys, key, key_len, ref, rows, Extra

'1', 'PRIMARY', 'a', 'ALL', NULL, NULL, NULL, NULL, '18146301', 'Using where'

'2', 'DEPENDENT SUBQUERY', 'b', 'index_subquery', 'cfg2', 'cfg2', '47', 'func', '6', 'Using where'

Why doesn't it use the indices? How can i speedup the query?

Answer 1

SELECT a.SN, a.NUMBER 
FROM a
WHERE EXISTS (
   SELECT * 
   FROM b
   WHERE b.LOWER_SN = a.SN
      AND b.HIGHER_ED  LIKE "%c1" 
      AND b.LOWER_ED LIKE "%16"
)
AND a.ED LIKE "%16"

I am not usually a proponent of correlated subqueries for a number of reasons; but since you cannot benefit from indexes, this might work out.

It will effectively run the subquery in the background for each and every a record (or if the query optimizer is helping, maybe only the a records matching the a.ED LIKE condition). Normally, this will more costly; but the EXISTS subquery can run much much faster than the previous subquery you had since it only has to evaluate b records with specific LOWER_SN values, and EXISTS gives it an "early out" when finds a single match instead of finding every match. You should index b.LOWER_SN to make sure the b records for each a record are identified as quickly as possible.

If the above version is still slow, it might be worth trying this:

SELECT a2.SN, a2.NUMBER 
FROM (SELECT a.SN, a.NUMBER FROM a WHERE AND a.ED LIKE "%16") AS a2
WHERE EXISTS (
   SELECT * 
   FROM b
   WHERE b.LOWER_SN = a2.SN
      AND b.HIGHER_ED  LIKE "%c1" 
      AND b.LOWER_ED LIKE "%16"
)

It basically just forces the a table to be filtered on the a.ED LIKE condition first if the optimizer doesn't do it.