using a local java program I have access to a database on a vps whose tables involved have an average of about 1,500 records.
Unfortunately, despite the records are not many, I have performance issues around 10sec for the extraction.
The query is as follows:
SELECT DISTINCT P.advanced_stock_management,
PA.id_product, PA.reference, PL.name,
round(P.wholesale_price,2),
round((P.price + P.price * 22 / 100), 2),
round(SP.reduction,2),
SA.quantity, PQ.is_true
FROM db.ps_product AS P
INNER JOIN db.ps_product_attribute AS PA
ON P.id_product=PA.id_product
INNER JOIN db.ps_product_lang AS PL
ON PA.id_product=PL.id_product
INNER JOIN db.ps_stock_available AS SA
ON SA.id_product_attribute=PA.id_product_attribute
LEFT OUTER JOIN db.ps_product_quantity_real AS PQ
ON PA.id_product=PQ.id_product
AND PA.reference=PQ.reference
LEFT OUTER JOIN db.ps_specific_price AS SP
ON PA.id_product=SP.id_product
WHERE P.active = 1;
How can I improve structure of the query and increase performance?
Thanks in advance.
You should have proper indexes on the following columns:
table_name (column_name)
ps_product (id_product)
ps_product_attribute (id_product)
ps_product_attribute (reference)
ps_product_attribute (id_product_attribute)
ps_product_lang (id_product)
ps_stock_available (id_product_attribute)
ps_product_quantity_real (id_product)
ps_product_quantity_real (reference)
ps_specific_price (id_product)
By rewrite the query with a proper indentation:
SELECT DISTINCT
product.advanced_stock_management,
attribute.id_product,
attribute.reference,
lang.name,
round(product.wholesale_price,2),
round((product.price + product.price * 22 / 100),2),
round(price.reduction,2),
availablity.quantity,
quantity.is_true
FROM db.ps_product AS product
INNER JOIN db.ps_product_attribute AS attribute
ON product.id_product = attribute.id_product
INNER JOIN db.ps_product_lang AS lang
ON lang.id_product = attribute.id_product
INNER JOIN db.ps_stock_available AS availablity
ON availablity.id_product_attribute = attribute.id_product_attribute
LEFT OUTER JOIN db.ps_product_quantity_real AS quantity
ON attribute.id_product = quantity.id_product
AND attribute.reference = quantity.reference
LEFT OUTER JOIN db.ps_specific_price AS price
ON price.id_product = attribute.id_product
WHERE product.active = 1;
We can see that the central table is in fact ps_product_attribute
. Let's make it the start of the query:
SELECT DISTINCT
product.advanced_stock_management,
attribute.id_product,
attribute.reference,
lang.name,
round(product.wholesale_price,2),
round((product.price + product.price * 22 / 100),2),
round(price.reduction,2),
availablity.quantity,
quantity.is_true
FROM db.ps_product_attribute AS attribute
INNER JOIN db.ps_product AS product
ON attribute.id_product = product.id_product
INNER JOIN db.ps_product_lang AS lang
ON attribute.id_product = lang.id_product
INNER JOIN db.ps_stock_available AS availablity
ON attribute.id_product_attribute = availablity.id_product_attribute
LEFT OUTER JOIN db.ps_product_quantity_real AS quantity
ON attribute.id_product = quantity.id_product
AND attribute.reference = quantity.reference
LEFT OUTER JOIN db.ps_specific_price AS price
ON attribute.id_product = price.id_product
WHERE product.active = 1;
The query looks very good now.
Do you have indexes on every field in this query? If not, you should!
ALTER TABLE `ps_product_attribute` ADD INDEX `id_product` (`id_product`)
ALTER TABLE `ps_product_attribute` ADD INDEX `reference` (`reference`)
ALTER TABLE `ps_product` ADD INDEX `id_product` (`id_product`)
ALTER TABLE `ps_product_lang` ADD INDEX `id_product` (`id_product`)
ALTER TABLE `ps_stock_available` ADD INDEX `id_product_attribute` (`id_product_attribute`)
ALTER TABLE `ps_product_quantity_real` ADD INDEX `id_product` (`id_product`)
ALTER TABLE `ps_product_quantity_real` ADD INDEX `reference` (`reference`)
ALTER TABLE `ps_specific_price` ADD INDEX `id_product` (`id_product`)
With this database size, the query should runs under 1 second.
You've two options
Pre-fetch 1500 records in memory and then join using record matching ( no database ). Pre-fetch provides speed and removed network latency in accessing records from Database but ya record matching will be slow and tiresome.
Use apache spark with local cluster ( bigdata joins ). I've personally executed 1million * 2 join under 9ms. This must be implemented using in-memory RDD
only.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.