堆栈内存溢出
  繁体   English   中英

MySQL - 查询获取速度非常慢

[英]MySQL - query fetching very slow

 原文  2021-04-12 14:06:40  1   1    mysql

问题描述

CREATE TABLE `offers` (
  `id` mediumint(8) unsigned NOT NULL auto_increment,
  `cap_id` varchar(255),
  `deposit_value` varchar(255) default NULL,
  `term` varchar(255) default NULL,
  `annual_mileage` varchar(255) default NULL,
  `finance_type` varchar(255) default NULL,
  `monthly_payment` mediumint default NULL,
  `stock` varchar(255) default NULL,
  PRIMARY KEY (`id`)
) AUTO_INCREMENT=1;

INSERT INTO `offers` (`cap_id`,`deposit_value`,`term`,`annual_mileage`,`finance_type`,`monthly_payment`,`stock`) VALUES ("93897","6","24","12000","B",363,"0"),("90626","1","24","12000","P",810,"0"),("93607","6","36","25000","B",172,"1"),("35877","1","48","10000","B",678,"1"),("77134","3","24","10000","P",454,"0"),("47080","6","36","10000","B",737,"0"),("46392","3","24","30000","P",261,"0"),("21418","3","36","20000","B",390,"1"),("80175","9","48","10000","B",713,"1"),("91487","6","48","12000","P",778,"1");
INSERT INTO `offers` (`cap_id`,`deposit_value`,`term`,`annual_mileage`,`finance_type`,`monthly_payment`,`stock`) VALUES ("18311","9","24","20000","B",384,"0"),("41740","9","24","12000","P",674,"1"),("69030","9","24","10000","P",518,"0"),("53342","3","36","8000","B",746,"1"),("65566","6","36","30000","P",145,"1"),("63172","6","48","5000","P",698,"1"),("79712","1","48","30000","B",330,"1"),("90505","1","36","8000","B",458,"0"),("42393","1","36","20000","B",363,"1"),("70454","9","24","5000","B",673,"1");
INSERT INTO `offers` (`cap_id`,`deposit_value`,`term`,`annual_mileage`,`finance_type`,`monthly_payment`,`stock`) VALUES ("81215","1","48","8000","B",472,"0"),("76538","3","24","15000","B",226,"0"),("05094","9","36","12000","P",721,"1"),("57363","9","48","5000","B",777,"1"),("23233","1","48","12000","B",381,"0"),("40542","3","48","12000","P",610,"0"),("63824","3","24","12000","B",761,"1"),("17686","3","24","5000","P",893,"1"),("57669","9","48","30000","P",805,"0"),("21864","1","24","25000","P",530,"0");
INSERT INTO `offers` (`cap_id`,`deposit_value`,`term`,`annual_mileage`,`finance_type`,`monthly_payment`,`stock`) VALUES ("48360","3","48","12000","P",159,"0"),("88614","3","48","20000","B",730,"0"),("17693","1","24","10000","B",298,"0"),("34049","6","48","20000","B",728,"0"),("15038","9","24","10000","P",720,"1"),("31809","1","36","20000","P",237,"0"),("49277","9","48","25000","P",235,"1"),("54607","1","24","12000","P",661,"1"),("65098","1","48","20000","P",548,"1"),("76440","9","48","10000","P",495,"1");

CREATE TABLE `offers_lowest` (
  `id` mediumint(8) unsigned NOT NULL auto_increment,
  `date` DATE,
  `cap_id` varchar(255),
  `deposit` varchar(255) default NULL,
  `term` varchar(255) default NULL,
  `mileage` varchar(255) default NULL,
  `finance_type` varchar(255) default NULL,
  `lowest_price` mediumint default NULL,
  `stock` varchar(255) default NULL,
  PRIMARY KEY (`id`)
) AUTO_INCREMENT=1;

INSERT INTO `offers_lowest` (`date`,`cap_id`,`deposit`,`term`,`mileage`,`finance_type`,`lowest_price`,`stock`) VALUES ("2021-04-09","93897","6","24","12000","B",363,"0"),("2021-04-010","90626","1","24","12000","P",810,"0"),("2021-04-010","93607","6","36","25000","B",172,"1"),("2021-04-10","35877","1","48","10000","B",678,"1"),("2021-04-11","77134","3","24","10000","P",454,"0"),("2021-04-11","47080","6","36","10000","B",737,"0"),("2021-04-11","46392","3","24","30000","P",261,"0"),("2021-04-11","21418","3","36","20000","B",390,"1"),("2021-04-12","80175","9","48","10000","B",713,"1"),("2021-04-12","91487","6","48","12000","P",778,"1");
INSERT INTO `offers_lowest` (`date`,`cap_id`,`deposit`,`term`,`mileage`,`finance_type`,`lowest_price`,`stock`) VALUES ("2021-04-09","18311","9","24","20000","B",384,"0"),("2021-04-010","41740","9","24","12000","P",674,"1"),("2021-04-010","69030","9","24","10000","P",518,"0"),("2021-04-10","53342","3","36","8000","B",746,"1"),("2021-04-11","65566","6","36","30000","P",145,"1"),("2021-04-11","63172","6","48","5000","P",698,"1"),("2021-04-11","79712","1","48","30000","B",330,"1"),("2021-04-11","90505","1","36","8000","B",458,"0"),("2021-04-12","42393","1","36","20000","B",363,"1"),("2021-04-12","70454","9","24","5000","B",673,"1");
INSERT INTO `offers_lowest` (`date`,`cap_id`,`deposit`,`term`,`mileage`,`finance_type`,`lowest_price`,`stock`) VALUES ("2021-04-09","81215","1","48","8000","B",472,"0"),("2021-04-09","76538","3","24","15000","B",226,"0"),("2021-04-010","05094","9","36","12000","P",721,"1"),("2021-04-10","57363","9","48","5000","B",777,"1"),("2021-04-11","23233","1","48","12000","B",381,"0"),("2021-04-11","40542","3","48","12000","P",610,"0"),("2021-04-11","63824","3","24","12000","B",761,"1"),("2021-04-11","17686","3","24","5000","P",893,"1"),("2021-04-12","57669","9","48","30000","P",805,"0"),("2021-04-12","21864","1","24","25000","P",530,"0");
INSERT INTO `offers_lowest` (`date`,`cap_id`,`deposit`,`term`,`mileage`,`finance_type`,`lowest_price`,`stock`) VALUES ("2021-04-09","48360","3","48","12000","P",159,"0"),("2021-04-09","88614","3","48","20000","B",730,"0"),("2021-04-010","17693","1","24","10000","B",298,"0"),("2021-04-10","34049","6","48","20000","B",728,"0"),("2021-04-11","15038","9","24","10000","P",720,"1"),("2021-04-11","31809","1","36","20000","P",237,"0"),("2021-04-11","49277","9","48","25000","P",235,"1"),("2021-04-11","54607","1","24","12000","P",661,"1"),("2021-04-12","65098","1","48","20000","P",548,"1"),("2021-04-12","76440","9","48","10000","P",495,"1");

CREATE INDEX idx_profile_grouping ON offers (cap_id, deposit_value, term, annual_mileage);

CREATE INDEX idx_specials_query ON offers_lowest (cap_id, deposit, term, mileage);

SQLFiddle

所以，我正在尝试加入两个表 - 一个是我们平台上的活动报价列表（我们是一个汽车租赁比较网站），另一个是自定义表，它每天记录每辆车的最佳价格（cap_id），以及每个可能的财务状况，即 9 个月的存款价值、24 个月的合同和每年 8,000 英里等。

这是为了确定什么是“好”价格的最终目标，output 包含MIN(offers.monthly_payment)和MIN(offers_lowest.lowest_price)以及计算的百分比差异，历史价格数据取自 7 -day 日期范围不包括当天。

offers_lowest是从每天的offers中汇总的，找到按以下条件分组的最低monthly_payment付款： cap_id, deposit_value, term, annual_mileage, finance_type

到目前为止，这是我想出的：

SELECT 
    s.cap_id,
    s.deposit,
    s.term,
    s.mileage,
    s.best_price,
    f.previous_best,
    (( s.best_price - f.previous_best ) / f.previous_best ) * 100 AS difference,
    s.stock
FROM
    (SELECT
        o.cap_id,
        o.deposit_value as deposit,
        o.term,
        o.annual_mileage as mileage,
        o.finance_type,
        MIN(o.monthly_payment) AS best_price,
        o.stock,
        o.brand_id
    FROM
        offers o USE INDEX(idx_profile_grouping)
    WHERE o.finance_type = 'P'
    GROUP BY o.cap_id, o.deposit_value, o.term, o.annual_mileage
    ) s 
    
INNER JOIN

    (SELECT
        ol.cap_id,
        ol.deposit,
        ol.term,
        ol.mileage,
        MIN(lowest_price) as previous_best
    FROM
        offers_lowest ol USE INDEX(idx_specials_query)
    WHERE finance_type = 'P'
        AND ol.date > CURDATE() - INTERVAL 7 DAY
        AND ol.date <= CURDATE() - INTERVAL 1 DAY
    GROUP BY ol.cap_id, ol.deposit, ol.term, ol.mileage
    ) f ON s.cap_id = f.cap_id AND s.deposit = f.deposit AND s.term = f.term AND s.mileage = f.mileage
GROUP BY s.cap_id, s.deposit, s.term, s.mileage

问题在于速度。 当我单独运行任一子查询时，运行时间 <0.1 秒，然后获取大约 75 秒。 当我运行整个过程时，它会一直运行直到我停止它（>10 分钟）。

这些索引是我在 cap_id、deposit、term 和 mileage 字段（两个表）上创建的，以试图加快速度。

解释返回这个：

＃ ID 选择类型 桌子 分区 类型 可能的键 钥匙 key_len 参考 行 过滤 额外的
1 基本的 全部 965741 100.00 使用哪里； 使用临时的； 使用文件排序
1 基本的 参考 <auto_key0> <auto_key0> 10 f.cap_id f.存款 f.术语 f.里程
3 衍生的 奥尔 指数 idx_specials_query idx_specials_query 13 17386818 5.55 使用哪里
2 衍生的 ○ 指数 idx_profile_grouping idx_profile_grouping 10 4800964 50.00 使用哪里

并且 SHOW PROFILE 返回这个：

地位 期间
开始 0.000017
检查查询缓存以进行查询 0.000171
检查权限 0.000006
检查权限 0.000005
打开表 0.000020
在里面 0.000109
系统锁 0.000009
优化 0.000005
优化 0.000011
统计数据 0.000033
准备 0.000037
排序结果 0.000007
优化 0.000015
统计数据 0.000017
准备 0.000017
排序结果 0.000012
统计数据 0.000046
准备 0.000021
创建 tmp 表 0.000020
排序结果 0.000007
执行 0.000013
发送数据 0.000012
执行 0.000004
发送数据 68.205032
将 HEAP 转换为磁盘 0.542756
发送数据 3.132642
执行 0.000019
发送数据 7.968966
将 HEAP 转换为磁盘 0.746200
发送数据 22.506716
将 HEAP 转换为磁盘 0.567816
发送数据 4.886870
创建排序索引 2.151519
结尾 0.000019
查询结束 0.000010
删除 tmp 表 0.002051
查询结束 0.000019
收盘 0.000005
删除 tmp 表 0.001385
收盘 0.000009
删除 tmp 表 0.002780
收盘 0.000017
释放物品 0.000069
清理干净 0.000007
打扫干净 0.000021

由此看来，我还没有写出最有效的查询..

我们使用 AWS RDS 实例，运行 MySQL 5.7.12。

如果我可以提供任何其他内容来添加背景，请告诉我。

1 个解决方案

解决方案1
 1  2021-04-12 16:40:09

  • FROM ( SELECT... ) JOIN ( SELECT... )效率低下。 尝试将其中至少一个移到外部SELECT中。 （如果您为JOIN添加合适的索引，这可能会消除“创建排序索引”。它也可能会消除大多数神秘的“发送数据”。）
  • 该日期范围仅包括 6 个 DATE。
  • 如果一行有 MIN(offers.monthly_payment) 而另一行有 MIN(offers_lowest.lowest_price) 怎么办？
  • 不要将 VARCHAR(255) 用于数值。 例如，MEDIUMINT 表示里程。 （这可能会消除“将 HEAP 转换为磁盘”。）
  • 注意 PROFILE 是多么无用； 大多数时候它喜欢说“发送数据”（占总数的 95%）。 我可以确定 4% 并建议修复，见上文。 另外 1% 是不可避免的噪音。
  • 第一个GROUP BY是不合适的，因为它没有说明为 stock 或 brand_id 赋予哪个值。 如有必要，在找到信息的 rest 后，通过额外的JOIN获取它们。
  • offers_lowest需求INDEX(finance_type, date)
  • 摆脱“索引提示”（ USE INDEX... ）

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 MySQL查询速度很慢 MySQL中查询非常慢 MYSQL（NOT IN）查询非常慢 MySql查询速度很慢 mysql查询很慢 MySQL-查询速度非常慢 MySQL查询非常慢 MySQL查询非常慢 非常慢的mysql SELECT查询 Mysql查询（order by）很慢
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM