简体   繁体   中英

MySQL query with dependent subqueries - how can I speed it up?

I have a query that I am trying to speed up. I am sure that there is a way to make this more efficient but I have a lack of knowledge.

orders - this shows one row per order received (currently 800,000 rows)

orders_financials - this shows one row per financial transaction associated with the order (currently 2,000,000 rows)

The orders_financials table will contain multiple rows per order, the category of the transaction is shown in financialType

So, for example, for one order there might be 5 rows in orders_financials

financialType = 14: value = 10.00
financialType = 12: value = 7.00
financialType = 18: value = 2.50
financialType = 15: value = 0.30
financialType = 17: value = 7.50

For each order there will be a varying amount of data. Over time more data is added to the system and new rows are added to orders_financials.

What I am trying to do is to run a query that allows me to get one row per order and for this row to calculate the totals of several categories of financial transactions.

The query below should give me all orders placed by customerID 279:

select 
orders.orderID, 
customers.username, 
(select group_concat(financialRef) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (17) ) ref,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (17) ) costs_cost,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (12,13) ) costs_exp,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (14) ) costs_sell,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (15) ) costs_sell_out,                                       
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (1,2,3,9,11) ) extras_in,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (4,5) ) extras_out,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (20) ) extras_back,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (18) ) insurance_in,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (10,6) ) insurance_out,
(select sum(amount) from orders_financials where orders_financials.orderID=orders.orderID and financialType IN (19) ) insurance_back                                            
from orders 
left join customers on orders.customerID=customers.customerID 
where customers.customerID='279'
order by orderID

(FYI, there are 18,263 records in the orders table where customerID=279)

You'll see that financialType's of 12 or 13 can be totalled to come up with costs_exp. Likewise 1,2 3,9,11 are all extras_in.

The end result is a nice query with a lot of data but it gives a detailed overview for every order. In the where clause I can search by customer, by service, by date range, etc.

My problem is that this query takes ages to run. I must be doing it in a very inefficient way as each element of the query individually is fast.

From slow log I can see it's iterating through 422,000,000 rows

Query_time: 528.107584  Lock_time: 0.000652 Rows_sent: 1388  Rows_examined: 422442417

There must be a more efficient way but I am struggling to see.

EDIT - Here is the EXPLAIN of the above query:

+----+--------------------+-------------------+-------+-----------------------+---------------+---------+---------------------------+--------+--------------------------+
| id | select_type        | table             | type  | possible_keys         | key           | key_len | ref                       | rows   | Extra                    |
+----+--------------------+-------------------+-------+-----------------------+---------------+---------+---------------------------+--------+--------------------------+
|  1 | PRIMARY            | customers         | const | PRIMARY               | PRIMARY       | 4       | const                     |      1 |                          |
|  1 | PRIMARY            | orders            | ref   | customerID            | customerID    | 5       | const                     |  24802 | Using where; Using index |
| 12 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | financialType | 4       | const                     |      1 | Using where              |
| 11 | DEPENDENT SUBQUERY | orders_financials | range | orderID,financialType | financialType | 4       | NULL                      |      2 | Using where              |
| 10 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | financialType | 4       | const                     |  49717 | Using where              |
|  9 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | financialType | 4       | const                     |      1 | Using where              |
|  8 | DEPENDENT SUBQUERY | orders_financials | range | orderID,financialType | financialType | 4       | NULL                      |      2 | Using where              |
|  7 | DEPENDENT SUBQUERY | orders_financials | range | orderID,financialType | financialType | 4       | NULL                      |      5 | Using where              |
|  6 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | financialType | 4       | const                     |    139 | Using where              |
|  5 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | orderID       | 4       | p4db_admin.orders.orderID | 236338 | Using where              |
|  4 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | orderID       | 4       | p4db_admin.orders.orderID | 236338 | Using where              |
|  3 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | financialType | 4       | const                     |  99966 | Using where              |
|  2 | DEPENDENT SUBQUERY | orders_financials | ref   | orderID,financialType | financialType | 4       | const                     |  99966 | Using where              |
+----+--------------------+-------------------+-------+-----------------------+---------------+---------+---------------------------+--------+--------------------------+
13 rows in set (0.00 sec)
SELECT  a.orderID, 
        b.username,
        GROUP_CONCAT(CASE WHEN c.financialType = 17 THEN c.financialRef END) ref,
        SUM(CASE WHEN c.financialType IN (12,13) THEN c.amount END) costs_cost,
        SUM(CASE WHEN c.financialType = 14 THEN c.amount END) costs_exp,
        SUM(CASE WHEN c.financialType = 15 THEN c.amount END) costs_sell,
        SUM(CASE WHEN c.financialType IN (1,2,3,9,11) THEN c.amount END) extras_in,
        SUM(CASE WHEN c.financialType IN (4,5) THEN c.amount END) extras_out,
        SUM(CASE WHEN c.financialType = 20 THEN c.amount END) extras_back,
        SUM(CASE WHEN c.financialType = 18 THEN c.amount END) insurance_in,
        SUM(CASE WHEN c.financialType IN (10,6) THEN c.amount END) insurance_out,
        SUM(CASE WHEN c.financialType = 19 THEN c.amount END) insurance_back
FROM    orders a
        LEFT JOIN customers b
            ON a.customerID = b.customerID 
        LEFT JOIN orders_financials c
            ON a.orderID = c.orderID
GROUP   BY a.orderID, b.username

add also index on orders_financials.financialType

ALTER TABLE orders_financials ADD INDEX (financialType)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM