Improve query speed suggestions

Question

For self education I am developing an invoicing system for an electricity company. I have multiple time series tables, with different intervals. One table represents consumption, two others represent prices. A third price table should be still incorporated. Now I am running calculation queries, but the queries are slow. I would like to improve the query speed, especially since this is only the beginning calculations and the queries will only become more complicated. Also please note that this is my first database i created and exercises I have done. A simplified explanation is preferred. Thanks for any help provided.

I have indexed: DATE, PERIOD_FROM, PERIOD_UNTIL in each table. This speed up the process from 60 seconds to 5 seconds.

The structure of the tables is the following:

CREATE TABLE `apxprice` (
 `APX_id` int(11) NOT NULL AUTO_INCREMENT,
 `DATE` date DEFAULT NULL,
 `PERIOD_FROM` time DEFAULT NULL,
 `PERIOD_UNTIL` time DEFAULT NULL,
 `PRICE` decimal(10,2) DEFAULT NULL,
 PRIMARY KEY (`APX_id`)
) ENGINE=MyISAM AUTO_INCREMENT=28728 DEFAULT CHARSET=latin1

CREATE TABLE `imbalanceprice` (
 `imbalanceprice_id` int(11) NOT NULL AUTO_INCREMENT,
 `DATE` date DEFAULT NULL,
 `PTU` tinyint(3) DEFAULT NULL,
 `PERIOD_FROM` time DEFAULT NULL,
 `PERIOD_UNTIL` time DEFAULT NULL,
 `UPWARD_INCIDENT_RESERVE` tinyint(1) DEFAULT NULL,
 `DOWNWARD_INCIDENT_RESERVE` tinyint(1) DEFAULT NULL,
 `UPWARD_DISPATCH` decimal(10,2) DEFAULT NULL,
 `DOWNWARD_DISPATCH` decimal(10,2) DEFAULT NULL,
 `INCENTIVE_COMPONENT` decimal(10,2) DEFAULT NULL,
 `TAKE_FROM_SYSTEM` decimal(10,2) DEFAULT NULL,
 `FEED_INTO_SYSTEM` decimal(10,2) DEFAULT NULL,
 `REGULATION_STATE` tinyint(1) DEFAULT NULL,
 `HOUR` int(2) DEFAULT NULL,
 PRIMARY KEY (`imbalanceprice_id`),
 KEY `DATE` (`DATE`,`PERIOD_FROM`,`PERIOD_UNTIL`)
) ENGINE=MyISAM AUTO_INCREMENT=117427 DEFAULT CHARSET=latin

CREATE TABLE `powerload` (
 `powerload_id` int(11) NOT NULL AUTO_INCREMENT,
 `EAN` varchar(18) DEFAULT NULL,
 `DATE` date DEFAULT NULL,
 `PERIOD_FROM` time DEFAULT NULL,
 `PERIOD_UNTIL` time DEFAULT NULL,
 `POWERLOAD` int(11) DEFAULT NULL,
 PRIMARY KEY (`powerload_id`)
) ENGINE=MyISAM AUTO_INCREMENT=61039 DEFAULT CHARSET=latin

Now when running this query:

SELECT  i.DATE, i.PERIOD_FROM, i.TAKE_FROM_SYSTEM, i.FEED_INTO_SYSTEM,
        a.PRICE, p.POWERLOAD, sum(a.PRICE * p.POWERLOAD)
    FROM  imbalanceprice i, apxprice a, powerload p
    WHERE  i.DATE = a.DATE
      and  i.DATE = p.DATE
      AND  i.PERIOD_FROM >= a.PERIOD_FROM
      and  i.PERIOD_FROM = p.PERIOD_FROM
      AND  i.PERIOD_FROM < a.PERIOD_UNTIL
      AND  i.DATE >= '2018-01-01'
      AND  i.DATE <= '2018-01-31'
    group by  i.DATE

I have run the query with explain and get the following result: Select_type, all simple partitions all null possible keys a,p = null i = DATE Key a,p = null i = DATE key_len a,p = null i = 8 ref a,p = null i = timeseries.a.DATE,timeseries.p.PERIOD_FROM rows a = 28727 p = 61038 i = 1 filtered a = 100 p = 10 i = 100 a extra: using where using temporary using filesort b extra: using where using join buffer (block nested loop) c extra: null

Preferably I run a more complicated query for a whole year and group by month for example with all price tables incorporated. However, this would be too slow. I have indexed: DATE, PERIOD_FROM, PERIOD_UNTIL in each table. The calculation result may not be changed, in this case quarter hourly consumption of two meters multiplied by hourly prices.

Answer 1

"Categorically speaking," the first thing you should look at is indexes.

Your clauses such as WHERE i.DATE = a.DATE ... are categorically known as INNER JOINs, and the SQL engine needs to have the ability to locate the matching rows "instantly." (That is to say, without looking through the entire table!)

• FYI: Just like any index in real-life – here I would be talking about "library card catalogs" if we still had such a thing – indexes will assist both "equal to" and "less/greater than" queries. The index takes the computer directly to a particular point in the data, whether that's a "hit" or a "near miss."

Finally, the EXPLAIN verb is very useful: put that word in front of your query, and the SQL engine should "explain to you" exactly how it intends to carry out your query. (The SQL engine looks at the structure of the database to make that decision.) Although the EXPLAIN output is ... (heh) ... "not exactly standardized," it will help you to see if the computer thinks that it needs to do something very time-wasting in order to deliver your answer.