大表的 MySQL 查询和 LEFT JOIN

Question

I am trying to optimize query (below) which takes 80 minutes to execute. :/ I have very large table prodaja with 21m rows and actual_stock with 960k rows.

SELECT
    p.NazivMat,
    sum(p.Kolicina) AS ProdajaKol,
    sum(p.Iznos) AS ProdajaIznos,
    s.Kolicina AS TrenutnaZaliha,
    s.Iznos AS TrenZalIznos 
FROM
    prodaja p
LEFT JOIN actual_stock s ON s.BrojSklad = p.BrojSklad 
    AND s.SifraMat = p.SifraMat 
WHERE
    p.Dobavljac = 1664 
    AND p.DatumOtprem BETWEEN '2020-12-10' 
    AND '2020-12-11'

I have set Indexes on fields BrojSklad and SifraMat but it does not change much at all as I have dates and range changing and query can take (run) forever if 10 days range is selected (with this query).

Is there any other way(s) to get same result with different query or two of them like "prefetch" and store in temp table and run another one?

Table with 20m rows is pain in the but. :/

UPDATE: 30. Dec

Thanks for all responds below. For sake of simplicity, I've shorten the query, the long version is below. I did add GROUP BY and the end of it, that's sorted.

EXPLAIN SELECT
    cm_prodaja.NazivGrupe,
    cm_prodaja.Grupa,
    cm_prodaja.DatumOtprem,
    cm_prodaja.SifraMat,
    cm_prodaja.BarCode,
    cm_prodaja.SifArtOdDob,
    cm_prodaja.NazivMat, 
    sum(cm_prodaja.Kolicina) AS Kolicina,
    sum(cm_prodaja.Iznos) AS Iznos,
    IFNULL (zaliha_artikala_radnje.Kolicina, 0) AS TrenutnaZaliha,
    IFNULL (zaliha_artikala_radnje.Iznos, 0) AS TrenZalIznos
FROM
    cm_prodaja
    LEFT JOIN zaliha_artikala_radnje ON zaliha_artikala_radnje.BrojSklad = cm_prodaja.BrojSklad 
    AND zaliha_artikala_radnje.SifraMat = cm_prodaja.SifraMat 
WHERE
    cm_prodaja.Dobavljac = 1664 
    AND cm_prodaja.DatumOtprem BETWEEN '2020-08-10' 
    AND '2020-08-11'
    
    GROUP BY cm_prodaja.BrojSklad, cm_prodaja.NazivRadnje, cm_prodaja.SifraMat,
cm_prodaja.BarCode, cm_prodaja.SifArtOdDob, cm_prodaja.NazivMat,
cm_prodaja.Kolicina, cm_prodaja.Iznos, cm_prodaja.Dobavljac,
cm_prodaja.NazivDobavljaca, cm_prodaja.Proizvodjac,
cm_prodaja.NazivProizvodjaca, cm_prodaja.Grupa, cm_prodaja.NazivGrupe   

I made it a bit faster by adding missing Index on zaliha_artikala_radnje.BrojSklad and zaliha_artikala_radnje.SifraMat .

Another thing I did is enabling partitioning and I've set to "split" table by year (months) on 4 sections/year and that helped a lot.

I've added image with EXPLAIN result.

Answer 1

Add these composite indexes, with the columns in the order given:

p:  (Dobavljac, DatumOtprem)
s:  (SifraMat, BrojSklad, Iznos, Kolicina)

If you need further assistance, please provide SHOW CREATE TABLE and fix the syntax error: ... LEFT The JOIN...

What is the datatype of DatumOtprem ? I am worried about the endpoints of the BETWEEN .

Another problem... The query has SUM() , but no GROUP BY ; what is the intent?

Answer 2

thanks for suggestions. I ended up with using PHP way to JOIN data described here: https://www.koolreport.com/docs/processes/join/

My original query took 3-40 minutes to give results for 1-7 days selected in filter. Now it takes 2-5 seconds, where final result has 1k - 8k rows.

I did try different methods and anything what has JOIN inside query dropped performance drastically. As said, KoolReport JOIN function solved my problem. I created two queries, both of them are getting their sets of data ordered by SifraMat and match same field Dobavljac.