MySQL查询在大表上运行非常慢

Question

I am trying to run the following query on a very large table with over 90 million of rows increasing

SELECT COUNT(DISTINCT device_uid) AS cnt,  DATE_FORMAT(time_start, '%Y-%m-%d') AS period 
FROM game_session 
WHERE account_id = -2 AND DATE_FORMAT(time_start '%Y-%m-%d') BETWEEN CURDATE() - INTERVAL 90 DAY AND CURDATE()
GROUP BY period 
ORDER BY period DESC

I have the following table structure:

CREATE TABLE `game_session` (
  `session_id` bigint(20) NOT NULL,
  `account_id` bigint(20) NOT NULL,
  `authentification_type` char(2) NOT NULL,
  `source_ip` char(40) NOT NULL,
  `device` char(50) DEFAULT NULL COMMENT 'Added 0.9',
  `device_uid` char(50) NOT NULL,
  `os` char(50) DEFAULT NULL COMMENT 'Added 0.9',
  `carrier` char(50) DEFAULT NULL COMMENT 'Added 0.9',
  `protocol_version` char(20) DEFAULT NULL COMMENT 'Added 0.9',
  `lang_key` char(2) NOT NULL DEFAULT 'en',
  `instance_id` char(100) NOT NULL,
  `time_start` datetime NOT NULL,
  `time_end` datetime DEFAULT NULL,
  PRIMARY KEY (`session_id`),
  KEY `game_account_session_fk` (`account_id`),
  KEY `lang_key_fk` (`lang_key`),
  KEY `lookup_active_session_idx` (`account_id`,`time_start`),
  KEY `lookup_finished_session_idx` (`account_id`,`time_end`),
  KEY `start_time_idx` (`time_start`),
  KEY `lookup_guest_session_idx` (`device_uid`,`time_start`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

How can I optimize this?

Thank for your answer

Answer 1

DATE_FORMAT(time_start '%Y-%m-%d') sounds expensive.
Every calculation on a column reduces the use of indexes. You probably run in to a full index scan + calculation of DATE_FORMAT for each value instead of a index lookup / range scan.

Try to store the computed value in the column (or create a computed index if mysql supports it). Or even better rewrite your conditions to compare directly to the value stored in the column.

Answer 2

好吧，90mlns太多了，但是我怀疑它不会使用start_time_idx因为可以避免这种操作（您可以操作与之比较的值，如果mysql是，则每个查询也只能执行一次）足够聪明），您是否检查了EXPLAIN ？

Answer 3

You may want to group and sort by time_start instead of the period value you create when the query is run. Sorting by period requires all of those values to be generated before any sorting can be done.

Answer 4

Try swapping out your WHERE clause with the following: WHERE account_id = -2 AND time_start BETWEEN CURDATE() - INTERVAL 90 DAY AND CURDATE()

MySQL will still catch the dates between, the only ones you'll need to worry about are the ones from today, which might get truncated due to technically being greater than midnight.

You can fix that by incrementing the second CURDATE( ) with CURDATE( ) + INTERVAL 1 DAY

Answer 5

I'd change

BETWEEN CURDATE() - INTERVAL 90 DAY AND CURDATE()

to

> (CURDATE() - INTERVAL 90 DAY)

You don't have records from future, do you?

Answer 6

Change the query to:

SELECT COUNT(DISTINCT device_uid) AS cnt
     , DATE_FORMAT(time_start, '%Y-%m-%d') AS period 
FROM game_session 
WHERE account_id = -2 
  AND time_start >= CURDATE() - INTERVAL 90 DAY 
  AND time_start <  CURDATE() + INTERVAL 1 DAY
GROUP BY DATE(time_start) DESC

so the index of (account_id, time_start) can be used for the WHERE part of the query.

---

If it's still slow - the DATE(time_start) does not look very good for performance - add a date_start column and store the date part of time_start .

Then add an index on (account_id, date_start, device_uid) which will further improve performance as all necessary info - for the GROUP BY date_start and the COUNT(DISTINCT device_uid) parts - will be on the index:

SELECT COUNT(DISTINCT device_uid) AS cnt
     , date_start                 AS period 
FROM game_session 
WHERE account_id = -2 
  AND date_start BETWEEN CURDATE() - INTERVAL 90 DAY 
                     AND CURDATE()
GROUP BY date_start DESC