MySQL 左外连接查询太慢

Question

I have 3 tables:

orders (5429850 rows), order_lines (PK id, FK order_id, 18530647 rows), order_line_allocations (PK id, FK order_line_id, 112594 rows)

order_lines and order_line_allocations stores the accounts related info. For single account case we store account data in order_lines table and for multiple accounts use-case we store data in allocations table.

I am trying to write a query to fetch orders satisfying user account restrictions.

SELECT `orders`.`id` FROM `orders` 
LEFT JOIN `order_lines` ON `order_lines`.`order_id` = `orders`.`id` 
LEFT JOIN order_line_allocations ON order_line_allocations.order_line_id = order_lines.id
LEFT JOIN (select sec_accounts.id from accounts sec_accounts
  WHERE (sec_accounts.account_type_id = 344 AND ((`sec_accounts`.`segment_1` = 'MS')))
) as non_alloc_accounts ON non_alloc_accounts.id = order_lines.account_id AND order_lines.allocation_count = 0
LEFT JOIN (select sec_accounts.id from accounts sec_accounts
  WHERE (sec_accounts.account_type_id = 344 AND ((`sec_accounts`.`segment_1` = 'MS')))
) as alloc_accounts ON alloc_accounts.id = order_line_allocations.account_id 
WHERE (`orders`.line_count = 0 OR
       `orders`.account_type_id IS NULL OR
       `orders`.account_type_id IN (NULL) OR 
       (non_alloc_accounts.id is not null) OR
       (alloc_accounts.id is not null)
) ORDER BY `orders`.`id` ASC LIMIT 90 \G;

This query is taking 7 seconds to complete which is too long

However, for other user:

SELECT `orders`.`id` FROM `orders` 
LEFT JOIN `order_lines` ON `order_lines`.`order_id` = `orders`.`id` 
LEFT JOIN order_line_allocations ON order_line_allocations.order_line_id = order_lines.id
LEFT JOIN (select sec_accounts.id from accounts sec_accounts
  WHERE (sec_accounts.account_type_id = 8 AND ((`sec_accounts`.`segment_1` = '2D')))
) as non_alloc_accounts ON non_alloc_accounts.id = order_lines.account_id AND order_lines.allocation_count = 0
LEFT JOIN (select sec_accounts.id from accounts sec_accounts
  WHERE (sec_accounts.account_type_id = 8 AND ((`sec_accounts`.`segment_1` = '2D')))
) as alloc_accounts ON alloc_accounts.id = order_line_allocations.account_id 
WHERE (`orders`.line_count = 0 OR
       `orders`.account_type_id IS NULL OR
       `orders`.account_type_id IN (NULL) OR 
       (non_alloc_accounts.id is not null) OR
       (alloc_accounts.id is not null)
) ORDER BY `orders`.`id` ASC LIMIT 90 \G;

^ This just takes 0.03 sec. Notice the different is just in where clause of LEFT JOINS. Specifically "=8" vs "=344" in 2 places.

No. of accounts matching 8 and '2D' is 175667, for 344 and 'MS' its just 5648 .

I have the required indexes on all the tables

Table indexes:

orders

PRIMARY KEY (`id`),
UNIQUE KEY `index_orders_on_po_number` (`po_number`),
KEY `index_orders_on_account_type_id` (`account_type_id`),
KEY `index_orders_on_created_by` (`created_by_id`),
KEY `index_orders_on_last_exported_at` (`last_exported_at`),
KEY `index_orders_on_status` (`status`),
KEY `index_orders_on_updated_at` (`updated_at`),
KEY `index_orders_on_updated_by` (`updated_by_id`),
KEY `index_orders_on_created_at_and_status` (`created_at`,`status`),
KEY `index_status_supplier_id` (`status`,`supplier_id`),
KEY `index_orders_on_line_count` (`line_count`)

order_lines

PRIMARY KEY (`id`),
UNIQUE KEY `index_order_lines_on_bulk_price_id` (`bulk_price_id`),
KEY `index_order_lines_on_account_type_id` (`account_type_id`),
KEY `index_order_lines_on_created_at` (`created_at`),
KEY `index_order_lines_on_created_by` (`created_by_id`),
KEY `index_order_lines_on_order_id` (`order_id`),
KEY `index_order_lines_on_status` (`status`),
KEY `index_order_lines_on_updated_at` (`updated_at`),
KEY `index_order_lines_on_updated_by` (`updated_by_id`),
KEY `index_order_lines_on_order_id_and_position` (`order_id`,`position`),
KEY `index_order_lines_on_order_id_and_line_num` (`order_id`,`line_num`),
KEY `index_ol_on_header_id_reporting_total_savings_pct_created_at` (`order_id`,`reporting_total`,`savings_pct`,`created_at`),
KEY `index_ol_on_order_id_supplier_id_reporting_total` (`order_id`,`supplier_id`,`reporting_total`),
KEY `index_ol_on_order_id_commodity_id_reporting_total_ela_id` (`order_id`,`commodity_id`,`reporting_total`,`extra_line_attribute_id`),
KEY `index_order_lines_on_allocation_count_and_account_id` (`allocation_count`,`account_id`),
KEY `index_order_lines_on_account_id` (`account_id`),
KEY `index_order_lines_on_allocation_count` (`allocation_count`),
KEY `index_ol_on_acc_id_alloc_count_order_id` (`account_id`,`allocation_count`,`order_id`),
KEY `index_ol_on_order_id_account_id_alloc_count` (`order_id`,`account_id`,`allocation_count`)

order_line_allocations

PRIMARY KEY (`id`),
KEY `index_order_line_allocations_on_account_id` (`account_id`),
KEY `index_order_line_allocations_on_account_type_id` (`account_type_id`),
KEY `index_order_line_allocations_on_order_id` (`order_id`),
KEY `index_order_line_allocations_on_order_line_id` (`order_line_id`)

accounts

PRIMARY KEY (`id`),
KEY `index_accounts_on_account_type_id_and_active` (`account_type_id`,`active`),
KEY `index_accounts_on_segment_1` (`segment_1`),
KEY `index_accounts_on_segment_10` (`segment_10`),
KEY `index_accounts_on_segment_11` (`segment_11`),
KEY `index_accounts_on_segment_12` (`segment_12`),
KEY `index_accounts_on_segment_13` (`segment_13`),
KEY `index_accounts_on_segment_14` (`segment_14`),
KEY `index_accounts_on_segment_15` (`segment_15`),
KEY `index_accounts_on_segment_16` (`segment_16`),
KEY `index_accounts_on_segment_17` (`segment_17`),
KEY `index_accounts_on_segment_18` (`segment_18`),
KEY `index_accounts_on_segment_19` (`segment_19`),
KEY `index_accounts_on_segment_2` (`segment_2`),
KEY `index_accounts_on_segment_20` (`segment_20`),
KEY `index_accounts_on_segment_3` (`segment_3`),
KEY `index_accounts_on_segment_4` (`segment_4`),
KEY `index_accounts_on_segment_5` (`segment_5`),
KEY `index_accounts_on_segment_6` (`segment_6`),
KEY `index_accounts_on_segment_7` (`segment_7`),
KEY `index_accounts_on_segment_8` (`segment_8`),
KEY `index_accounts_on_segment_9` (`segment_9`),
KEY `index_accounts_on_account_type_id` (`account_type_id`)

EXPLAIN PLAN for slow query

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: orders
   partitions: NULL
         type: index
possible_keys: index_orders_on_account_type_id,index_orders_on_line_count
          key: PRIMARY
      key_len: 4
          ref: NULL
         rows: 10
     filtered: 100.00
        Extra: NULL
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: order_lines
   partitions: NULL
         type: ref
possible_keys: index_order_lines_on_order_id,index_order_lines_on_order_id_and_position,index_order_lines_on_order_id_and_line_num,index_ol_on_header_id_reporting_total_savings_pct_created_at,index_ol_on_order_id_supplier_id_reporting_total,index_ol_on_order_id_commodity_id_reporting_total_ela_id,index_ol_on_order_id_account_id_alloc_count
          key: index_ol_on_order_id_commodity_id_reporting_total_ela_id
      key_len: 4
          ref: perf_amazon_qas1481_utf8mb4.orders.id
         rows: 2
     filtered: 100.00
        Extra: NULL
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: order_line_allocations
   partitions: NULL
         type: ref
possible_keys: index_order_line_allocations_on_order_line_id
          key: index_order_line_allocations_on_order_line_id
      key_len: 5
          ref: perf_amazon_qas1481_utf8mb4.order_lines.id
         rows: 3
     filtered: 100.00
        Extra: NULL
*************************** 4. row ***************************
           id: 1
  select_type: SIMPLE
        table: sec_accounts
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,index_accounts_on_account_type_id_and_active,index_accounts_on_segment_1,index_accounts_on_account_type_id
          key: PRIMARY
      key_len: 4
          ref: perf_amazon_qas1481_utf8mb4.order_lines.account_id
         rows: 1
     filtered: 100.00
        Extra: Using where
*************************** 5. row ***************************
           id: 1
  select_type: SIMPLE
        table: sec_accounts
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,index_accounts_on_account_type_id_and_active,index_accounts_on_segment_1,index_accounts_on_account_type_id
          key: PRIMARY
      key_len: 4
          ref: perf_amazon_qas1481_utf8mb4.order_line_allocations.account_id
         rows: 1
     filtered: 100.00
        Extra: Using where

EXPLAIN PLAN for fast query:

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: orders
   partitions: NULL
         type: index
possible_keys: index_orders_on_account_type_id,index_orders_on_line_count
          key: PRIMARY
      key_len: 4
          ref: NULL
         rows: 10
     filtered: 100.00
        Extra: NULL
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: order_lines
   partitions: NULL
         type: ref
possible_keys: index_order_lines_on_order_id,index_order_lines_on_order_id_and_position,index_order_lines_on_order_id_and_line_num,index_ol_on_header_id_reporting_total_savings_pct_created_at,index_ol_on_order_id_supplier_id_reporting_total,index_ol_on_order_id_commodity_id_reporting_total_ela_id,index_ol_on_order_id_account_id_alloc_count
          key: index_ol_on_order_id_commodity_id_reporting_total_ela_id
      key_len: 4
          ref: perf_amazon_qas1481_utf8mb4.orders.id
         rows: 2
     filtered: 100.00
        Extra: NULL
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: order_line_allocations
   partitions: NULL
         type: ref
possible_keys: index_order_line_allocations_on_order_line_id
          key: index_order_line_allocations_on_order_line_id
      key_len: 5
          ref: perf_amazon_qas1481_utf8mb4.order_lines.id
         rows: 3
     filtered: 100.00
        Extra: NULL
*************************** 4. row ***************************
           id: 1
  select_type: SIMPLE
        table: sec_accounts
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,index_accounts_on_account_type_id_and_active,index_accounts_on_segment_1,index_accounts_on_account_type_id
          key: PRIMARY
      key_len: 4
          ref: perf_amazon_qas1481_utf8mb4.order_lines.account_id
         rows: 1
     filtered: 100.00
        Extra: Using where
*************************** 5. row ***************************
           id: 1
  select_type: SIMPLE
        table: sec_accounts
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,index_accounts_on_account_type_id_and_active,index_accounts_on_segment_1,index_accounts_on_account_type_id
          key: PRIMARY
      key_len: 4
          ref: perf_amazon_qas1481_utf8mb4.order_line_allocations.account_id
         rows: 1
     filtered: 100.00
        Extra: Using where

How to speed up the query?

Answer 1

These composite indexes may help:

order_lines:  INDEX(order_id, account_id, allocation_count)
order_line_allocations:  INDEX(order_line_id,  account_id)
accounts:  INDEX(account_type_id, segment_1)

When adding a composite index, DROP index(es) with the same leading columns. That is, when you have both INDEX(a) and INDEX(a,b) , toss the former.

Can the two LEFT JOINs into accounts be combined?

For further discussion, please provide SHOW CREATE TABLE for each table. And EXPLAIN for both 8 and 344.

OR makes it impractical to use an INDEX for optimization.