MySQL查詢優化，大表，使用臨時文件排序

Question

我需要您的幫助來優化查詢。 一個表是一個具有數百萬個條目的日志表，我嘗試將查詢破壞為<1s。 我的查詢應該給出總體概述，因此應該很快。 我相信我可以借助腳本在列表上進行多個簡單查詢。 但是腳本可以做什么，mysql我可以嗎？我至少希望如此。 也許並不是所有部分都得到最好的利用，但是我陷入了一個查詢，該查詢使一個臨時表和文件排序（我發現這確實很糟糕）。 在閱讀時，我發現使用了一些整潔且位置良好的索引，但是現在我陷入了一個特定的問題。

讓我向您展示我的最終查詢及其結果：

SELECT
    ps.SERVER_ID,
    ps.FULLNAME,
    SUM(CASE WHEN pml.ID_TYPE = 3 THEN 1 ELSE 0 END) 'amount_warning',
    SUM(CASE WHEN pml.ID_TYPE = 4 THEN 1 ELSE 0 END) 'amount_error',
    SUM(CASE WHEN pml.ID_TYPE = 5 THEN 1 ELSE 0 END) 'amount_alert',
    SUM(CASE WHEN pml.ID_TYPE = 7 THEN 1 ELSE 0 END) 'amount_critical'
FROM
    PAR_SERVER ps
INNER JOIN
    PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID
WHERE
    pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY )
GROUP BY
    ps.SERVER_ID;

這是我得到的：

mysql> [thequeryabove]
[...]
59 rows in set (11.69 sec)

mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type   | possible_keys               | key     | key_len | ref                       | rows    | Extra                                        |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
|  1 | SIMPLE      | pml   | ALL    | SERVER_ID,SERVER_ID-ID_TYPE | NULL    | NULL    | NULL                      | 4014447 | Using where; Using temporary; Using filesort |
|  1 | SIMPLE      | ps    | eq_ref | PRIMARY                     | PRIMARY | 4       | database.pml.SERVER_ID |       1 |                                              |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)

這是我當前的表格設置：

mysql> describe PAR_SERVER;
+----------------+--------------+------+-----+---------+----------------+
| Field          | Type         | Null | Key | Default | Extra          |
+----------------+--------------+------+-----+---------+----------------+
| SERVER_ID      | int(255)     | NO   | PRI | NULL    | auto_increment |
| FULLNAME       | varchar(255) | YES  |     | NULL    |                |
| SHORTNAME      | varchar(255) | YES  | MUL | NULL    |                |
+----------------+--------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

mysql> show indexes from PAR_SERVER;
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table      | Non_unique | Key_name  | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_SERVER |          0 | PRIMARY   |            1 | SERVER_ID   | A         |         142 |     NULL | NULL   |      | BTREE      |         |
| PAR_SERVER |          1 | shortname |            1 | SHORTNAME   | A         |         142 |     NULL | NULL   | YES  | BTREE      |         |
+------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
2 rows in set (0.00 sec)

mysql> select count(*) from PAR_SERVER;
+----------+
| count(*) |
+----------+
|      142 |
+----------+
1 row in set (0.00 sec)

mysql> describe PAR_MONITORINGv2_LOG;
+--------------+----------+------+-----+---------+----------------+
| Field        | Type     | Null | Key | Default | Extra          |
+--------------+----------+------+-----+---------+----------------+
| ID           | int(11)  | NO   | PRI | NULL    | auto_increment |
| ID_TYPE      | int(11)  | NO   | MUL | NULL    |                |
| ID_SERVICE   | int(11)  | NO   | MUL | NULL    |                |
| SERVER_ID    | int(11)  | NO   | MUL | NULL    |                |
| MESSAGE      | tinytext | NO   |     | NULL    |                |
| CREATED_DATE | datetime | NO   |     | NULL    |                |
+--------------+----------+------+-----+---------+----------------+
6 rows in set (0.00 sec)

mysql> show indexes from PAR_MONITORINGv2_LOG;
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table                | Non_unique | Key_name          | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| PAR_MONITORINGv2_LOG |          0 | PRIMARY           |            1 | ID          | A         |     3998188 |     NULL | NULL   |      | BTREE      |         |
| PAR_MONITORINGv2_LOG |          1 | ID_TYPE           |            1 | ID_TYPE     | A         |           7 |     NULL | NULL   |      | BTREE      |         |
| PAR_MONITORINGv2_LOG |          1 | ID_SERVICE        |            1 | ID_SERVICE  | A         |           5 |     NULL | NULL   |      | BTREE      |         |
| PAR_MONITORINGv2_LOG |          1 | SERVER_ID         |            1 | SERVER_ID   | A         |          66 |     NULL | NULL   |      | BTREE      |         |
| PAR_MONITORINGv2_LOG |          1 | SERVER_ID-ID_TYPE |            1 | SERVER_ID   | A         |          66 |     NULL | NULL   |      | BTREE      |         |
| PAR_MONITORINGv2_LOG |          1 | SERVER_ID-ID_TYPE |            2 | ID_TYPE     | A         |         258 |     NULL | NULL   |      | BTREE      |         |
+----------------------+------------+-------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
6 rows in set (0.00 sec)

mysql> select count(*) from PAR_MONITORINGv2_LOG;
+----------+
| count(*) |
+----------+
|  3998386 |
+----------+
1 row in set (0.00 sec)

這是逐步打破我的查詢的時間結果。 修復每個零件花費了這么長時間后，我可能會逐步進行。 但是目前，對於該問題，當前僅運行時間為2.30秒的查詢才有意義。

mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID WHERE pml.CREATED_DATE > date_sub( NOW( ) , INTERVAL 7 DAY ) GROUP BY ps.SERVER_ID;
[...]
59 rows in set (6.41 sec)

mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type   | possible_keys               | key     | key_len | ref                       | rows    | Extra                                        |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
|  1 | SIMPLE      | pml   | ALL    | SERVER_ID,SERVER_ID-ID_TYPE | NULL    | NULL    | NULL                      | 4014788 | Using where; Using temporary; Using filesort |
|  1 | SIMPLE      | ps    | eq_ref | PRIMARY                     | PRIMARY | 4       | database.pml.SERVER_ID |       1 |                                              |
+----+-------------+-------+--------+-----------------------------+---------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)

mysql> SELECT ps.SERVER_ID, ps.FULLNAME FROM PAR_SERVER ps INNER JOIN PAR_MONITORINGv2_LOG pml ON ps.SERVER_ID = pml.SERVER_ID GROUP BY ps.SERVER_ID;
[...]
59 rows in set (2.30 sec)

mysql> explain [thequeryabove]
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
| id | select_type | table | type   | possible_keys               | key       | key_len | ref                       | rows    | Extra                                        |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
|  1 | SIMPLE      | pml   | index  | SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID | 4       | NULL                      | 4015694 | Using index; Using temporary; Using filesort |
|  1 | SIMPLE      | ps    | eq_ref | PRIMARY                     | PRIMARY   | 4       | database.pml.SERVER_ID |       1 |                                              |
+----+-------------+-------+--------+-----------------------------+-----------+---------+---------------------------+---------+----------------------------------------------+
2 rows in set (0.00 sec)

mysql> SELECT pml.SERVER_ID FROM PAR_MONITORINGv2_LOG pml GROUP BY pml.SERVER_ID;
[...]
65 rows in set (0.00 sec)

mysql> explain [thequeryabove]
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
| id | select_type | table | type  | possible_keys | key       | key_len | ref  | rows | Extra                    |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
|  1 | SIMPLE      | pml   | range | NULL          | SERVER_ID | 4       | NULL |   67 | Using index for group-by |
+----+-------------+-------+-------+---------------+-----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)

我可以通過定義（SERVER_ID，ID_TYPE）的索引來大大改善查詢，如下面的示例查詢所確認：

mysql> SELECT count(*) 'count_warnings' FROM PAR_MONITORINGv2_LOG pml WHERE pml.SERVER_ID = 191 AND pml.ID_TYPE = 3 GROUP BY pml.SERVER_ID;
[...]
1 row in set (0.01 sec)

mysql> explain [thequeryabove]
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
| id | select_type | table | type | possible_keys                       | key               | key_len | ref         | rows  | Extra       |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
|  1 | SIMPLE      | pml   | ref  | ID_TYPE,SERVER_ID,SERVER_ID-ID_TYPE | SERVER_ID-ID_TYPE | 8       | const,const | 10254 | Using index |
+----+-------------+-------+------+-------------------------------------+-------------------+---------+-------------+-------+-------------+
1 row in set (0.00 sec)

我現在陷入了最細分的查詢中，執行時間僅為2.30秒。 我不知道如何在沒有任何where子句的查詢中使用索引。

Answer 1

通過在PAR_MONITORINGv2_LOG（CREATED_DATE，SERVER_ID，ID_TYPE）上添加復合索引，您的查詢無疑將獲得最大的收益。 但是，我建議即使在CREATED_DATE簡單的索引也可以CREATED_DATE提高性能。