MySql Query Optimization. Too slow
Question
I am trying to get some data, but the query takes more then 5 minutes (it should be as fast as few milliseconds).
This is the table
CREATE TABLE IF NOT EXISTS `bookings` (
`bkid`
int(11) NOT NULL AUTO_INCREMENT,
`idBuilding`
int(11) NOT NULL,
`checkinDate`
date NOT NULL,
`baggage_checkin`
int(11) DEFAULT NULL,
`baggage_checkin_check`
tinyint(1) DEFAULT NULL,
`arrivalAtAirportTime`
time DEFAULT NULL,
`returnFlightTime`
time DEFAULT NULL,
`arrivalAtAirportTime_check`
tinyint(1) NOT NULL DEFAULT '0',
`flightNumber`
varchar(50) DEFAULT NULL,
`checkinTime`
time NOT NULL DEFAULT '15:00:00',
`checkoutDate`
date NOT NULL,
`baggage_checkout`
int(11) DEFAULT NULL,
`baggage_checkout_check`
tinyint(1) DEFAULT NULL,
`checkoutTime`
time NOT NULL DEFAULT '12:00:00',
`agency`
varchar(255) NOT NULL,
`distributor`
varchar(255) NOT NULL,
`number`
varchar(255) NOT NULL,
`arrivalby`
varchar(50) DEFAULT NULL,
`adults`
int(3) NOT NULL,
`actualAdults`
int(10) DEFAULT NULL,
`doublebeds`
int(3) DEFAULT NULL,
`children`
int(3) NOT NULL,
`singlebeds`
int(3) DEFAULT NULL,
`price`
decimal(8, 2) NOT NULL,
`received`
decimal(8, 2) NOT NULL DEFAULT '0.00',
`createdBy`
int(11) NOT NULL,
`createdTime`
int(11) NOT NULL,
`modifiedBy`
int(11) NOT NULL,
`modifiedTime`
int(11) NOT NULL,
`notes`
longblob NOT NULL,
`checkinBy`
int(11) NOT NULL,
`checkoutBy`
int(11) NOT NULL,
`keys_number`
int(11) DEFAULT NULL,
`keys_number_kept`
int(11) DEFAULT NULL,
`keys_number_back`
int(11) DEFAULT NULL,
`mobile_wifi`
int(11) DEFAULT NULL,
`mobile_wifi_back`
tinyint(1) NOT NULL DEFAULT '0',
`usb_sticks`
int(11) DEFAULT '0',
`bkstatus`
tinyint(4) NOT NULL,
`status`
int(11) NOT NULL,
`booking_fee`
decimal(8, 2) NOT NULL,
`online_guest`
decimal(10, 2) NOT NULL,
`sales`
decimal(10, 2) NOT NULL,
`sales_vat`
decimal(10, 2) DEFAULT NULL,
`distribution`
decimal(10, 2) NOT NULL,
`host_margin`
decimal(10, 2) NOT NULL,
`cleaning_fee`
decimal(8, 2) NOT NULL,
`taxi_to_office`
int(11) DEFAULT NULL,
`taxi_to_office_check`
varchar(50) DEFAULT NULL,
`taxi_apartment_to_office`
int(11) DEFAULT NULL,
`taxi_apartment_to_office_check`
varchar(50) DEFAULT NULL,
`taxi_to_apartment`
int(11) NOT NULL,
`taxi_to_apartment_check`
varchar(50) DEFAULT NULL,
`taxi_to_airport`
int(11) NOT NULL,
`taxi_to_airport_check`
varchar(50) DEFAULT NULL,
`idOrigBuilding`
int(11) NOT NULL,
`guest_satisfaction`
enum('-1', '0', '1') NOT NULL DEFAULT '0',
`guest_informed_status`
enum('1', '2', '3', '4') NOT NULL DEFAULT '1',
`channel_informed_status`
enum('1', '2', '3', '4') NOT NULL DEFAULT '1',
`mail_sent`
tinyint(1) DEFAULT '0',
`mail_sent_date`
timestamp NULL DEFAULT NULL,
`mail_sent_reminder`
tinyint(1) DEFAULT '0',
`mail_sent_reminder_date`
datetime DEFAULT NULL,
`mail_sent_lastreminder`
tinyint(1) DEFAULT '0',
`mail_sent_lastreminder_date`
datetime DEFAULT NULL,
`DELbirthDate`
date DEFAULT NULL,
`DELcity`
varchar(50) DEFAULT NULL,
`cleaning_percentage`
varchar(50) NOT NULL DEFAULT '0',
`rent_percentage`
varchar(50) NOT NULL DEFAULT '0',
`checkout_percentage`
int(3) NOT NULL DEFAULT '0',
`groupType`
varchar(50) NOT NULL,
`deposit`
double NOT NULL,
`idrefundstatustype`
int(11) NOT NULL,
`prepayment`
decimal(10, 2) NOT NULL,
`distribution_changed`
int(11) NOT NULL,
`checkinDT`
datetime NOT NULL,
`checkoutDT`
datetime NOT NULL,
PRIMARY KEY(`bkid`),
KEY `number` (`number`),
KEY `createdBy` (`createdBy`, `modifiedBy`),
KEY `idBuilding` (`idBuilding`),
KEY `checkinDate` (`checkinDate`),
KEY `checkoutDate` (`checkoutDate`),
KEY `checkinTime` (`checkinTime`),
KEY `checkoutTime` (`checkoutTime`),
KEY `checkinDT` (`checkinDT`, `checkoutDT`)
) ENGINE = MyISAM DEFAULT CHARSET = utf8 AUTO_INCREMENT = 15934;
Here is the query
select
concat(bu.streetName,' ',bu.houseNumber) as address,
concat(co.firstName,' ',co.middleName,' ',co.lastName) as ownerName,
a.city as city,
a.area as area,
bu.minGuest as minGuest,
bu.maxGuest as maxGuest,
bu.bedrooms as bedrooms,
bu.bathrooms as bathrooms,
bo.checkinDate as checkinDate,
bo.checkinTime as checkinTime,
bo.checkoutDate as checkoutDate,
bo.checkoutTime as checkoutTime,
concat(ifnull(cg.firstName,''),' ',ifnull(cg.middleName,''),' ',ifnull(cg.lastName,'')) as guestName,
bo.adults as guestsCount,
bo.bkid as bkid,
bo.idBuilding as bid,
bu.price as price,
bu.percentage as isPercent,
bu.isActive as isActive,
ifnull(f.hashname,-1) as iconName,
f.ext as iconExt
from
bookings bo
left join
buildings bu
on
bu.id = bo.idBuilding
left join
items i
on
i.guid = bu.guid
left join
images im
on
im.guid = i.icon
left join
files f
on
f.hashname = im.96_hash
left join
areas a
on
a.id = bu.idArea
left join
booking_participant bp
on
bp.bookings_id = bo.bkid
and
bp.role_id = 2
left join
contacts co
on
co.id = bu.idContactOwner
left join
contacts cg
on
cg.id = bp.contacts_id
left join
bookings nextBo
on
nextBo.idBuilding = bo.idBuilding
and
nextBo.checkinDT >= bo.checkoutDT
left join
jobs nextInspection
on
nextInspection.jobtype = 2
and
nextInspection.endDT < nextBo.checkinDT
and
nextInspection.endDT > bo.checkoutDT
left join
jobs nextCleaning
on
nextCleaning.jobtype = 1
and
nextCleaning.endDT < nextBo.checkinDT
and
nextCleaning.endDT > bo.checkoutDT
where
bo.checkinDT>='2014-07-18 00:00:00'
and
bo.checkinDT<='2014-07-18 23:59:59'
group by
bo.bkid
order by
bo.checkinDate, bo.checkinTIme, bu.streetName, bu.houseNumber, bo.createdTime asc;
This is the explain result:
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: bo
type: range
possible_keys: checkinDT
key: checkinDT
key_len: 8
ref: NULL
rows: 40
Extra: Using where; Using temporary; Using filesort
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: bu
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bo.idBuilding
rows: 1
Extra:
*************************** 3. row ***************************
id: 1
select_type: SIMPLE
table: i
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: company.bu.guid
rows: 1
Extra:
*************************** 4. row ***************************
id: 1
select_type: SIMPLE
table: im
type: eq_ref
possible_keys: guid
key: guid
key_len: 8
ref: company.i.icon
rows: 1
Extra:
*************************** 5. row ***************************
id: 1
select_type: SIMPLE
table: f
type: ref
possible_keys: hash
key: hash
key_len: 42
ref: company.im.96_hash
rows: 1
Extra:
*************************** 6. row ***************************
id: 1
select_type: SIMPLE
table: a
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bu.idArea
rows: 1
Extra:
*************************** 7. row ***************************
id: 1
select_type: SIMPLE
table: bp
type: ref
possible_keys: PRIMARY,bookings_id,bookings_id_2
key: bookings_id
key_len: 4
ref: company.bo.bkid
rows: 2
Extra: Using index
*************************** 8. row ***************************
id: 1
select_type: SIMPLE
table: co
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bu.idContactOwner
rows: 1
Extra:
*************************** 9. row ***************************
id: 1
select_type: SIMPLE
table: cg
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bp.contacts_id
rows: 1
Extra:
*************************** 10. row ***************************
id: 1
select_type: SIMPLE
table: nextBo
type: ref
possible_keys: idBuilding,checkinDT
key: idBuilding
key_len: 4
ref: company.bo.idBuilding
rows: 71
Extra:
*************************** 11. row ***************************
id: 1
select_type: SIMPLE
table: nextInspection
type: ref
possible_keys: endDT,jobtype
key: jobtype
key_len: 4
ref: const
rows: 2749
Extra:
*************************** 12. row ***************************
id: 1
select_type: SIMPLE
table: nextCleaning
type: ref
possible_keys: endDT,jobtype
key: jobtype
key_len: 4
ref: const
rows: 16040
Extra:
12 rows in set (0.00 sec)
Do you maybe have an idea how can I speed up this query?
It will be used for selecting bookings from a server, so it should be much faster. For bookings that happened in a period of 1 day the query takes more then 5 minutes, so for multiple days it is even worse..
This one executed in 0.01 seconds.
Done this, but still slow.
Here is output of optimize.
mysql> OPTIMIZE TABLE bookings;
+--------------------------+----------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------------------+----------+----------+----------+
| company.bookings | optimize | status | OK |
+--------------------------+----------+----------+----------+
1 row in set (0.02 sec)
Output of myisamchk
Variables (--variable-name=value)
and boolean options {FALSE|TRUE} Value (after reading options)
--------------------------------- ----------------------------------------
character-sets-dir (No default value)
data-file-length 0
keys-used 18446744073709551615
max-record-length 9223372036854775807
set-auto-increment 0
set-collation (No default value)
sort-records 9
tmpdir (No default value)
key-buffer-size 520192
key-cache-block-size 1024
myisam-block-size 1024
read-buffer-size 262136
write-buffer-size 262136
sort-buffer-size 2097144
myisam-sort-buffer-size 2097144
sort-key-blocks 16
decode-bits 9
ft-min-word-len 4
ft-max-word-len 84
ft-stopword-file (No default value)
stats-method nulls_unequal
You are correct, the query starts slowing down after adding the jobs join. Before joining nextBo, execution time is 0.01 After nextBo join, it goes up to 0.1 After the first jobs join it goes up to few minutes
I made the change you suggested, but it is still not faster. Here is the new query
select
concat(bu.streetName,' ',bu.houseNumber) as address,
concat(co.firstName,' ',co.middleName,' ',co.lastName) as ownerName,
a.city as city,
a.area as area,
bu.minGuest as minGuest,
bu.maxGuest as maxGuest,
bu.bedrooms as bedrooms,
bu.bathrooms as bathrooms,
bo.checkinDate as checkinDate,
bo.checkinTime as checkinTime,
bo.checkoutDate as checkoutDate,
bo.checkoutTime as checkoutTime,
concat(ifnull(cg.firstName,''),' ',ifnull(cg.middleName,''),' ',ifnull(cg.lastName,'')) as guestName,
bo.adults as guestsCount,
bo.bkid as bkid,
bo.idBuilding as bid,
bu.price as price,
bu.percentage as isPercent,
bu.isActive as isActive,
ifnull(f.hashname,-1) as iconName,
f.ext as iconExt,
(
select
clnid
from
jobs nextInspection
where
nextInspection.jobtype = 2
and
nextInspection.endDT < nextBo.checkinDT
and
nextInspection.endDT > bo.checkoutDT
order by
nextInspection.endDT asc
limit 1
) as nextInspect,
(
select
clnid
from
jobs nextInspection
where
nextInspection.jobtype = 2
and
nextInspection.endDT < nextBo.checkinDT
and
nextInspection.endDT > bo.checkoutDT
order by
nextInspection.endDT asc
limit 1
) as nextClean
from
bookings bo
left join
buildings bu
on
bu.id = bo.idBuilding
left join
items i
on
i.guid = bu.guid
left join
images im
on
im.guid = i.icon
left join
files f
on
f.hashname = im.96_hash
left join
areas a
on
a.id = bu.idArea
left join
booking_participant bp
on
bp.bookings_id = bo.bkid
and
bp.role_id = 2
left join
contacts co
on
co.id = bu.idContactOwner
left join
contacts cg
on
cg.id = bp.contacts_id
left join
bookings nextBo
on
nextBo.idBuilding = bo.idBuilding
and
nextBo.checkinDT >= bo.checkoutDT
where
bo.checkinDT>='2014-07-18 00:00:00'
and
bo.checkinDT<='2014-07-18 23:59:59'
group by
bo.bkid
order by
bo.checkinDate, bo.checkinTIme, bu.streetName, bu.houseNumber, bo.createdTime asc;
The explain now says
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: bo
type: range
possible_keys: checkinDT
key: checkinDT
key_len: 8
ref: NULL
rows: 74
Extra: Using where; Using temporary; Using filesort
*************************** 2. row ***************************
id: 1
select_type: PRIMARY
table: bu
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bo.idBuilding
rows: 1
Extra:
*************************** 3. row ***************************
id: 1
select_type: PRIMARY
table: i
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: company.bu.guid
rows: 1
Extra:
*************************** 4. row ***************************
id: 1
select_type: PRIMARY
table: im
type: eq_ref
possible_keys: guid
key: guid
key_len: 8
ref: company.i.icon
rows: 1
Extra:
*************************** 5. row ***************************
id: 1
select_type: PRIMARY
table: f
type: ref
possible_keys: hash
key: hash
key_len: 42
ref: company.im.96_hash
rows: 1
Extra:
*************************** 6. row ***************************
id: 1
select_type: PRIMARY
table: a
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bu.idArea
rows: 1
Extra:
*************************** 7. row ***************************
id: 1
select_type: PRIMARY
table: bp
type: ref
possible_keys: PRIMARY,bookings_id,bookings_id_2
key: bookings_id
key_len: 4
ref: company.bo.bkid
rows: 2
Extra: Using index
*************************** 8. row ***************************
id: 1
select_type: PRIMARY
table: co
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bu.idContactOwner
rows: 1
Extra:
*************************** 9. row ***************************
id: 1
select_type: PRIMARY
table: cg
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bp.contacts_id
rows: 1
Extra:
*************************** 10. row ***************************
id: 1
select_type: PRIMARY
table: nextBo
type: ref
possible_keys: idBuilding,checkinDT
key: idBuilding
key_len: 4
ref: company.bo.idBuilding
rows: 70
Extra:
*************************** 11. row ***************************
id: 3
select_type: DEPENDENT SUBQUERY
table: nextInspection
type: index
possible_keys: endDT,jobtype
key: endDT
key_len: 8
ref: NULL
rows: 10
Extra: Using where
*************************** 12. row ***************************
id: 2
select_type: DEPENDENT SUBQUERY
table: nextInspection
type: index
possible_keys: endDT,jobtype
key: endDT
key_len: 8
ref: NULL
rows: 10
Extra: Using where
12 rows in set (0.00 sec)
Actually I just found out that I forgot to join the jobs table on idBuilding, so it was getting jobs of all buildings instead for the current one only. Anyways, the execution is now 1 seconds instead of 5 minutes, but the application should be much more responsive than that (around 0.01 seconds per day is acceptable, so if the search is for whole year then it would be around 3 seconds in total instead of 5 minutes like it is now..)
Here is the latest query
select
concat(bu.streetName,' ',bu.houseNumber) as address,
concat(co.firstName,' ',co.middleName,' ',co.lastName) as ownerName,
a.city as city,
a.area as area,
bu.minGuest as minGuest,
bu.maxGuest as maxGuest,
bu.bedrooms as bedrooms,
bu.bathrooms as bathrooms,
bo.checkinDate as checkinDate,
bo.checkinTime as checkinTime,
bo.checkoutDate as checkoutDate,
bo.checkoutTime as checkoutTime,
concat(ifnull(cg.firstName,''),' ',ifnull(cg.middleName,''),' ',ifnull(cg.lastName,'')) as guestName,
bo.adults as guestsCount,
bo.bkid as bkid,
bo.idBuilding as bid,
bu.price as price,
bu.percentage as isPercent,
bu.isActive as isActive,
ifnull(f.hashname,-1) as iconName,
f.ext as iconExt,
( select
clnid
from
jobs nextInspection
where
nextInspection.idBuilding = bo.idBuilding
and
nextInspection.jobtype = 2
and
nextInspection.endDT < nextBo.checkinDT
and
nextInspection.endDT > bo.checkoutDT
) as inspectionClnid,
( select
clnid
from
jobs nextCleaning
where
nextCleaning.idBuilding = bo.idBuilding
and
nextCleaning.jobtype = 1
and
nextCleaning.endDT < nextBo.checkinDT
and
nextCleaning.endDT > bo.checkoutDT
) as cleaningClnid
from
bookings bo
left join
buildings bu
on
bu.id = bo.idBuilding
left join
items i
on
i.guid = bu.guid
left join
images im
on
im.guid = i.icon
left join
files f
on
f.hashname = im.96_hash
left join
areas a
on
a.id = bu.idArea
left join
booking_participant bp
on
bp.bookings_id = bo.bkid
and
bp.role_id = 2
left join
contacts co
on
co.id = bu.idContactOwner
left join
contacts cg
on
cg.id = bp.contacts_id
left join
bookings nextBo
on
nextBo.idBuilding = bo.idBuilding
and
nextBo.checkinDT >= bo.checkoutDT
where
bo.checkinDT>='2014-07-18 00:00:00'
and
bo.checkinDT<='2014-07-18 23:59:59'
group by
bo.bkid
order by
bo.checkinDate, bo.checkinTIme, bu.streetName, bu.houseNumber, bo.createdTime asc;
And this is the explain result
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: bo
type: range
possible_keys: checkinDT
key: checkinDT
key_len: 8
ref: NULL
rows: 73
Extra: Using where; Using temporary; Using filesort
*************************** 2. row ***************************
id: 1
select_type: PRIMARY
table: bu
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bo.idBuilding
rows: 1
Extra:
*************************** 3. row ***************************
id: 1
select_type: PRIMARY
table: i
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 8
ref: company.bu.guid
rows: 1
Extra:
*************************** 4. row ***************************
id: 1
select_type: PRIMARY
table: im
type: eq_ref
possible_keys: guid
key: guid
key_len: 8
ref: company.i.icon
rows: 1
Extra:
*************************** 5. row ***************************
id: 1
select_type: PRIMARY
table: f
type: ref
possible_keys: hash
key: hash
key_len: 42
ref: company.im.96_hash
rows: 1
Extra:
*************************** 6. row ***************************
id: 1
select_type: PRIMARY
table: a
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bu.idArea
rows: 1
Extra:
*************************** 7. row ***************************
id: 1
select_type: PRIMARY
table: bp
type: ref
possible_keys: PRIMARY,bookings_id,bookings_id_2
key: bookings_id
key_len: 4
ref: company.bo.bkid
rows: 2
Extra: Using index
*************************** 8. row ***************************
id: 1
select_type: PRIMARY
table: co
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bu.idContactOwner
rows: 1
Extra:
*************************** 9. row ***************************
id: 1
select_type: PRIMARY
table: cg
type: eq_ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: company.bp.contacts_id
rows: 1
Extra:
*************************** 10. row ***************************
id: 1
select_type: PRIMARY
table: nextBo
type: ref
possible_keys: idBuilding,checkinDT
key: idBuilding
key_len: 4
ref: company.bo.idBuilding
rows: 70
Extra:
*************************** 11. row ***************************
id: 3
select_type: DEPENDENT SUBQUERY
table: nextCleaning
type: ref
possible_keys: idBuilding,endDT,jobtype
key: idBuilding
key_len: 4
ref: func
rows: 128
Extra: Using where
*************************** 12. row ***************************
id: 2
select_type: DEPENDENT SUBQUERY
table: nextInspection
type: ref
possible_keys: idBuilding,endDT,jobtype
key: idBuilding
key_len: 4
ref: func
rows: 128
Extra: Using where
12 rows in set (0.00 sec)
1 answers
solution1
1 2014-07-23 07:04:57
Interesting. Actually, your query looks pretty good , the indexes look fine, your only WHERE clause variable is bo.checkinDT, which is indexed (although just part of a bigger index, but it' sa left part...).
What is the purpose of the GROUP BY ? You are not using any grouping functions. Are you shorthanding DISTINCT?
Can you try:
- Running the same query without the joins (
SELECT * FROM bookings bo where bo.checkinDT>='2014-07-18 00:00:00' and bo.checkinDT<='2014-07-18 23:59:59'). Is that slow too? If not, it might be the joins/group by playing. - Creating a seperate index on
bookings.checkinDT? - Running an
OPTIMIZE TABLE bookingsonce? - Running
myisamchk --sort-index --sort-records=9(if 9 is the checkinDT index)?
(I'll update my answer as you provide details).
Ok, since the base query (only on bookings) is fast, the joins are slowing it down. Can you do a SELECT * for the base table, and one by one add a LEFT JOIN?
I'm guessing the jobs table (2749 rows) and (16040 rows) are causing the problem.
Am I seeing things wrong, or are you (currently) not even using the table?
If all you need is the next date for both, a subquery might be faster, you could try putting this in your select statement:
(SELECT * FROM jobs nextInspection
WHERE nextInspection.jobtype = 2
and nextInspection.endDT < nextBo.checkinDT
and nextInspection.endDT > bo.checkoutDT
ORDER BY nextInspection.endDT ASC
LIMIT 1)
Gets the first "inspection" after the current checkout but before the next checkin.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
MySQL query with JOIN and GROUP BY optimization. Is it possible?
MySQL query optimization. Avoiding temporary & filesort
MySQL query optimization. Many SELECT + INNER JOIN
Query optimization. Query without NOT IN
Mysql Slow Query Optimization
MySQL Optimization: DISTINCT too slow
MySQL query is too slow
mysql query too slow
MySQL query too slow?
MySQL query optimization, VERY SLOW
Related Tags