mysql日期查询总是做全盘扫描

Question

I am trying to do a supposedly very simple query. I have a table with datetime column with timestamps.

I need to find all parent table rows which do not have a timestamp of the last 5 minutes. This may change row to row as described below. I read a number of articles, trying changing my query a lot, but still my query is not using index properly.

1) the access table shown below may have more than one rows of mon.id. 2) I need to find all mon.id's which do not have a row in access table, with a lastaccess_date datetime within the last mon.duration minutes. 3) the access table may have more than 1 rows, so the row with latest timestamp needs to be checked for the duration logic.

Tables are as below:

mon (parent)
-----------
id,payload,duration

access (child)
---
id,mon_id,lastaccess_date

Current query is

select id,payload,elapsed,duration from 
(SELECT mon.id,payload,TIMESTAMPDIFF(MINUTE, lastaccess_date, NOW()) as elapsed,duration
    FROM mon
    inner JOIN access_log log on mon.id=log.monitor_id
order by lastaccess_date desc
 ) as t1
GROUP BY id
having elapsed>duration

I also made number of other queries, but these do not seem to be efficient. If I have 100 rows, then these queries are not using index and doing full table scan.

Please suggest an efficient query which can uses indexes. If required, I can tweak the table design a bit if it helps for this case.

mysql EXPLAIN of this query is something like below:

EDIT : As per comment, and what I had already tried before, I even changed the query to a drastic:

select monitor_id
  from access_log
 WHERE access_dt not between date_sub(now(),INTERVAL 5 MINUTE) and now()

now I am not touching the access_dt DATETIME column in the where clause, but still its doing a full table scan. The query returns 40 rows out of 100 rows in this test scenario.

Here is the EXPLAIN now:

id, select_type, table, type, possible_keys, key, key_len, ref, rows, filtered, Extra
'1', 'SIMPLE', 'access_log', 'ALL', 'access_dt', NULL, NULL, NULL, '100', '100.00', 'Using where'

Answer 1

There are several possibilities for your second query's EXPLAIN not being what you expect.

First of all, don't waste time worrying about the EXPLAIN results for small tables. This is aa tiny table right now, and your query is returning more than half of it. The MySQL query planner may have not chosen the index simply because it did not seem to be selective enough to be worth the trouble of paging into RAM and using. If that's the case, the situation may change as your tables grow.

Second, you have this clause:

WHERE access_dt not between date_sub(now(),INTERVAL 5 MINUTE) 
                        and now()

The not may prove to be unhelpful, because it gets performed as if it were

WHERE (    access_dt < date_sub(now(),INTERVAL 5 MINUTE)
        OR access_dt > now() )

OR clauses are no fun for MySQL to evaluate. If you happen to know that access_dt values cannot be in the future, you can do.

WHERE access_dt < date_sub(now(), INTERVAL 5 MINUTE)

and that's eligible for an index range scan.

Thirdly, you appear to be misusing GROUP BY in your first query. Do you mean ORDER BY ? It's hard to figure out what you need. Read this: http://dev.mysql.com/doc/refman/5.6/en/group-by-handling.html

Finally, let's take a look at your inner query in your first query, and try to optimize it. You started with this, which I have edited to show the tables from which each column comes.

SELECT mon.id, mon.payload,
      TIMESTAMPDIFF(MINUTE, log.lastaccess_date, NOW()) as elapsed,
      mon.duration
 FROM mon
inner JOIN access_log log ON mon.id=log.monitor_id
order by log lastaccess_date desc

Let's adjust this by adding the timestamp selection criterion to your ON clause.

  ...
  FROM mon
 INNER JOIN access_log LOG 
       ON mon.id = log.monitor_id
     AND log.lastaccess_date < DATE_SUB(NOW(),INTERVAL mon.duration MINUTE)

That will select the rows you want. When you get relatively large tables (at least 10K rows in access_log ) you should experiment with the following two compound indexes to see whether one or the other gives you better results.

 (monitor_id, lastaccess_date)
 (lastaccess_date, monitor_id)