Mysql optimization

Question

I'm currently trying to optimize a MYSQL statement that is taking quite some time. The table this is running on is 600k+ and the query is taking over 10 seconds.

SELECT DATE_FORMAT( timestamp, '%Y-%m-%d' ) AS date, COUNT( DISTINCT (
email
) ) AS count
FROM log
WHERE timestamp > '2009-02-23'
AND timestamp < '2020-01-01'
AND TYPE = 'play'
GROUP BY date
ORDER BY date DESC

I've just indexes on timestamp and type and also one on timestamp_type (type_2).

Here is the explain results, the problem seems to be a file sort but I don't know how to get around this...

id: 1
select_type: SIMPLE
table: log
type: ref
possible_keys: type,timestamp,type_2
key: type_2
key_len: 1
ref: const
rows: 226403
Extra: Using where; Using filesort

Thanks

Answer 1

Things to try:

• Have a separate date column (indexed) and use that instead of your timestamp column
• Add an index across type and date
• Use BETWEEN (don't think it will affect the speed but it's easier to read)

So ideally you would

1. Create a date column and fill it using UPDATE table SET date = DATE(timestamp)
2. Index across type and date
3. Change your select to ... type = ? AND date BETWEEN ? AND ?

Answer 2

Try rewriting to filter on TYPE alone first. Then apply your date range and aggregates. Basically create an inline view that filters type down. I know it's likely that the optimizer is doing this already, but when trying to improve performance I find it's helpful to be very certain of what things are happening first.

Answer 3

1. DATE_FORMAT will not utilizing the indexes.

   2. You can still use the below query to utilize the index on timestamp column

      SELECT timestamp AS date, COUNT( DISTINCT ( email ) ) AS count FROM log WHERE timestamp > '2009-02-23 00:00:00' AND timestamp < '2020-01-01 23:59:59' AND TYPE = 'play' GROUP BY date ORDER BY date DESC

   3. Format the datetime value to date while printing/using