Slow Group_concat query

Question

I tried to create a normalized database. Because of this, I am attempting to join 21 tables. I am using an index on almost all of the tables in the join as seen in the below screenshot of the EXPLAIN statement.

http://i.imgur.com/V5hQu.png

All lookup_xxxxx tables have one index on 2 columns (content_id and xxxxxx_id) All xxxxxx tables have one index on 1 column (xxxxx_id)

I only have about 10 or so rows in each table . The problem I am seeing is that the two goup_concat prolong the query an extra 200ms. I allow the submission fields, peripheral and programming language to have multiple values. Without them, the query is less than 80ms. My question is whether or not I should do away with the group_concats and do individual query's for them or reconstruct my database.

The lookup_xxxxx tables store each allowed value and then the other tables such as peripheral link the submission to the allowed value via the content_id. Everything is referenced to the submission content_id . The content table holds essential info such as member id, name, etc.

I apologize if my post is not clear enough.

mysql> describe peripheral;
+------------------+----------+------+-----+---------+-------+
| Field            | Type     | Null | Key | Default | Extra |
+------------------+----------+------+-----+---------+-------+
| peripheral_id    | int(2)   | NO   | PRI | NULL    |       |
| peripheral       | char(30) | NO   |     | NULL    |       |
| peripheral_total | int(5)   | NO   |     | NULL    |       |
+------------------+----------+------+-----+---------+-------+

mysql> select * from peripheral;
+---------------+-----------------+------------------+
| peripheral_id | peripheral      | peripheral_total |
+---------------+-----------------+------------------+
|             1 | periph 1        |                0 |
|             2 | periph 2        |                1 |
|             3 | periph 3        |                3 |
+---------------+-----------------+------------------+

:

mysql> describe lookup_peripheral;
+---------------+---------+------+-----+---------+-------+
| Field         | Type    | Null | Key | Default | Extra |
+---------------+---------+------+-----+---------+-------+
| content_id    | int(10) | NO   | MUL | NULL    |       |
| peripheral_id | int(2)  | NO   |     | NULL    |       |
+---------------+---------+------+-----+---------+-------+  


mysql> mysql> select * from lookup_peripheral;
+------------+---------------+
| content_id | peripheral_id |
+------------+---------------+
|         74 |             2 |
|         74 |             5 |
|         75 |             2 |
|         75 |             5 |
|         76 |             3 |
|         76 |             4 |
+------------+---------------+

SELECT group_concat(DISTINCT peripheral.peripheral_id) as peripheral_id, group_concat(DISTINCT programming_language.programming_language_id) as programming_language_id, c.member_name, c.member_id, c.added_date_time, c.title, c.raw_summary, c.raw_all_content, c.meta_tags, c.main_pic_thumb, application.application_id, architecture.architecture_id, compiler.compiler_id, device_family.device_family_id, difficulty.difficulty_id, ide.ide_id, programmer.programmer_id, table_name.table_name_id, device_name.device_name
FROM (content as c)
INNER JOIN lookup_peripheral ON 76 = lookup_peripheral.content_id
INNER JOIN peripheral ON peripheral.peripheral_id = lookup_peripheral.peripheral_id
INNER JOIN lookup_programming_language ON 76 = lookup_programming_language.content_id
INNER JOIN programming_language ON programming_language.programming_language_id = lookup_programming_language.programming_language_id
.......
LEFT OUTER JOIN device_name ON device_name.content_id = c.content_id
INNER JOIN table_name ON table_name.table_name_id = lookup_table_name.table_name_id
WHERE `c`.`content_id` = '76'  

Answer 1

I think you should also INDEX the lookup_peripheral.peripheral_id field . Indexing foreign keys makes INNER JOINs faster. Also, do you really need the DISTINCT clause since you are concatenating ID fields? On second thought, maybe you could omit the GROUP_CONCATs and INNER JOINs (some of your JOIN conditions overlap I believe, correct me if I'm mistaken)

... (SELECT GROUP_CONCAT(l.peripheral_id) from lookup_peripheral lp where lp.content_id = 76) as peripheral_id, ...

Hope this helps.