How do I optimize this mysql query with nested joins?

Question

Below is my query which I need to optimize.

SELECT
        UPPER(IFNULL(op.supplier_payment_method,s.default_payment_method)) AS Payment_Method,
        op.supplier_payment_date AS Payment_Date,
        Date_format(IFNULL(op.supplier_payment_date,op.ship_date),'%M %Y') AS Payment_Month,
        s.supplier_name AS Farm,
        op.sub_order_id AS Order_num,
        Date_format(op.ship_date,'%b-%d-%Y') AS Ship_Date,
        op.farm_credit AS Farm_Credit,
        op.credit_memo AS Credit_Memo,
        op.credit_description AS Credit_Description,
        opb.boxes AS Box_Type,
        CONCAT('$',FORMAT(op.box_charge,2)) AS Box_Charge,
        op.invoice_num AS Invoice_num,
        CONCAT('$',FORMAT(op.invoice_amt,2)) AS Invoice_Amt,
        CONCAT('$',FORMAT(op.total_invoice_amt,2)) AS Total_Invoice_Amt,
        CONCAT(op.UM_qty,' ',op.UM_type) AS St_Bu_Qty,
        op.PO_Product_Name AS Invoice_desc,
        CONCAT('$',FORMAT((op.price_um*op.um_qty),2)) AS Cost_product_cms,
        op.supplier_invoice_note AS Supplier_Invoice_Notes,
        CONCAT('$',FORMAT(op.cms_invoice_cost,2)) AS CMS_Invoice_diff,
        CONCAT('$',FORMAT(op.total_farm_cost,2)) AS Farm_Cost
    FROM 
        orders_products op
        INNER JOIN 
            suppliers s ON s.supplier_id = op.supplier_name
        LEFT JOIN
            (
                SELECT 
                    sub_order_id, 
                    GROUP_CONCAT(CONCAT(box_type_qty,' ',bo.box_option_name) SEPARATOR ', ') AS boxes
                FROM
                    order_products_boxes opb
                    INNER JOIN box_options bo ON bo.id=opb.box_type_id
                GROUP BY
                    opb.sub_order_id
            ) opb ON opb.sub_order_id = op.sub_order_id
    WHERE
        op.order_active=0
        AND op.ship_date>='2013-03-01'
        AND op.ship_date<='2013-04_01'
    ORDER BY op.ship_date DESC

As you can see, the query comprises of 4 tables, each containing around 20k-30k rows each. So as soon as I add in the sub-query, the query becomes exceptionally slow. It takes approx 1.5mins to fetch just 500 rows of record. Is there a way to speed things up within a single query?

Answer 1

SELECT
    UPPER(IFNULL(op.supplier_payment_method,s.default_payment_method)) AS Payment_Method,
    op.supplier_payment_date AS Payment_Date,
    Date_format(IFNULL(op.supplier_payment_date,op.ship_date),'%M %Y') AS Payment_Month,
    s.supplier_name AS Farm,
    op.sub_order_id AS Order_num,
    Date_format(op.ship_date,'%b-%d-%Y') AS Ship_Date,
    op.farm_credit AS Farm_Credit,
    op.credit_memo AS Credit_Memo,
    op.credit_description AS Credit_Description,
     (
            SELECT 
                GROUP_CONCAT(CONCAT(box_type_qty,' ',bo.box_option_name) SEPARATOR ', ') 
            FROM
                order_products_boxes opb
                INNER JOIN box_options bo ON bo.id=opb.box_type_id
            GROUP BY
                opb.sub_order_id
        ) AS Box_Type,
    CONCAT('$',FORMAT(op.box_charge,2)) AS Box_Charge,
    op.invoice_num AS Invoice_num,
    CONCAT('$',FORMAT(op.invoice_amt,2)) AS Invoice_Amt,
    CONCAT('$',FORMAT(op.total_invoice_amt,2)) AS Total_Invoice_Amt,
    CONCAT(op.UM_qty,' ',op.UM_type) AS St_Bu_Qty,
    op.PO_Product_Name AS Invoice_desc,
    CONCAT('$',FORMAT((op.price_um*op.um_qty),2)) AS Cost_product_cms,
    op.supplier_invoice_note AS Supplier_Invoice_Notes,
    CONCAT('$',FORMAT(op.cms_invoice_cost,2)) AS CMS_Invoice_diff,
    CONCAT('$',FORMAT(op.total_farm_cost,2)) AS Farm_Cost
FROM 
    orders_products op
    INNER JOIN 
        suppliers s ON s.supplier_id = op.supplier_name
    LEFT JOIN

                order_products_boxes opb (
                INNER JOIN box_options bo ON bo.id=opb.box_type_id
        ) opb ON opb.sub_order_id = op.sub_order_id
WHERE
    op.order_active=0
    AND op.ship_date>='2013-03-01'
    AND op.ship_date<='2013-04_01'
ORDER BY op.ship_date DESC

Answer 2

Your left-join query that pre-concatenates the sub-box options is querying the ENTIRE DATABASE FIRST, then only joins to those that are within the criteria you are limiting for the outer WHERE clause. It may be a little more effort, but pull the same outer criteria into your inner query should significantly help. As for doing a field-level sql-select on a per-every-row basis could be a performance killer.

You should only need to change at the "FROM" clause...

from
   orders_products op
      INNER JOIN suppliers s 
         ON op.supplier_name = s.supplier_id 
      LEFT JOIN
         ( SELECT 
                 opb2.sub_order_id, 
                 GROUP_CONCAT(CONCAT(box_type_qty,' ',bo.box_option_name) SEPARATOR ', ') AS boxes
              FROM
                 orders_products op2
                    JOIN order_products_boxes opb2
                       on op2.sub_order_id = opb2.sub_order_id
                       INNER JOIN box_options bo 
                          ON opb2.box_type_id = bo.id
              WHERE
                     op2.order_active = 0
                 AND op2.ship_date >= '2013-03-01'
                 AND op2.ship_date <= '2013-04_01'
              GROUP BY
                 opb2.sub_order_id  ) opb 
         ON op.sub_order_id = opb.sub_order_id
WHERE
       op.order_active = 0
   AND op.ship_date >= '2013-03-01'
   AND op.ship_date <= '2013-04_01'