我需要优化以下SQL查询

Question

我需要优化以下sql查询，因为我一直在等待过去45分钟才能执行。

    create database comparelogs;
    use comparelogs;
    create table compare(field varchar(1080));

    load data local infile 'c:/path/text.txt'
    into table compare
    lines terminated by '\n';

以下查询是耗时的查询。

    SELECT
    field,
    (SELECT COUNT(*) FROM compare T2 WHERE T2.field = T1.field) AS Count,
    @row_num := if(@prev_value=field,@row_num+1,1) as Occurrence,
    @prev_value := field as previous_value
    FROM compare T1
    order by field;

    input data:(from text.txt file)
    /Jols/AAP/AAP.36/Ads\AdS_7.75x10_1.29.75_k
    /Jols/AAP/AAP.36/Ads\PhyJobPl4x1034_BW_1.5.pdf
    /Jls/AAP/AAP.36Ads\Cusins BW_rint_FPbw_1.3.pdf
    /Jouals/AAP/AAP.36Ads\GeneryjWebdBW_1.6.pdf AAR356BF
    /Jals/AAP/AAP.36Ads\GeneralWyjyjyAW_1.6.pdf AAR356BF
    /Jls/AAP/AAP.36Ads\XtraCredit filler ad - PROPRIETARY ONLY_4C_1.0.pdf
    /Jos/AAP/AAP.36Ads\AAP_May-Jun_10_Havel's_EchoBlock_1.0.pdf
    /Jls/AAP/AAP.36//As\Roc76SolidAdbw_1.0.pdf
    /Jls/AAP/AAP.36//A\RAPM_fi7ul ad_BW_1.0.pdf
    /Jls/AAP/AAP.36/As\AAP_1_11_Bjhkytaun_Dyjidance_1.0.pdf
    /Jls/AAP/AAP.36/s\AAPyjs36_yj.2.pdf
     ..........................................upto 200000 records.

已经超过45分钟了，它仍在执行，我如何优化它以快速获得结果。 我正在使用mysql56。

Answer 1

这是您正在加载的数据的好选择。 这将需要一些时间。 以下相同的问题可能对您有帮助

http://www.mysqlperformanceblog.com/2008/07/03/how-to-load-large-files-safely-into-innodb-with-load-data-infile/

关于性能的另一个链接

http://www.mysqlperformanceblog.com/2008/07/03/how-to-load-large-files-safely-into-innodb-with-load-data-infile/

Answer 2

我认为MySQL正在执行SELECT语句。 您为什么不运行SHOW PROCESSLIST （我认为使用LOAD DATA INFILE加载200k行不会花费很长时间。）

以我的经验，关联子查询在MySQL中非常慢。 你可以试试这个吗？

您可以在这里进行测试。 http://www.sqlfiddle.com/#!2/ebcfe/7

SET @row_num := 0;
SET @prev_value := '';

SELECT field,
    @row_num := if(@prev_value=field,@row_num+1,1) as Occurrence,
    @prev_value := field as previous_value
FROM
(
    SELECT
      T1.field, cnt
    FROM compare T1 INNER JOIN
        (
            SELECT field, COUNT(*) AS cnt
            FROM compare
            GROUP BY field
        ) AS T2 ON T1.field = T2.field
    ORDER BY T1.field
) x;
+--------+------------+----------------+
| field  | Occurrence | previous_value |
+--------+------------+----------------+
| field1 |          1 | field1         |
| field2 |          1 | field2         |
| field2 |          2 | field2         |
| field3 |          1 | field3         |
| field3 |          2 | field3         |
| field3 |          3 | field3         |
| field4 |          1 | field4         |
| field4 |          2 | field4         |
| field4 |          3 | field4         |
| field4 |          4 | field4         |
+--------+------------+----------------+