MySQL find date range with count

Question

In MySQL, it is fairly easy to find the number of records that exist within some time interval.

SELECT COUNT(*) FROM records WHERE create_date > '2018-01-01 01:15:00' AND create_date < '2018-01-01 02:15:00'

But I want to do the opposite, sort of. Rather than providing a time interval and getting a count of records, I want to provide a count of records and check if a X minute time interval exists where more than Y many records were created. Getting the exact time interval is not essential, only if one exists or not. At a higher level, I am attempting to identify if there was any X minute "surge" when more than Y records where created during the course of a day.

For example, in the past 24 hours was there any 1 hour interval where a "surge" of more than 50 new records occurred?

I have already ruled out dividing the 24 hours into blocks of 1 hour intervals and checking each block. This does not work because the "surge" could span two sequential 1 hour blocks, such as 25 records at the end of the 01:00:00 block and 25 records at the beginning of the 02:00:00 block.

Answer 1

This should do it:

SELECT COUNT(*)
FROM records r1 
WHERE
    (SELECT COUNT(*) FROM records r2 
    WHERE ABS(UNIX_TIMESTAMP(r1.create_date) - UNIX_TIMESTAMP(r2.create_date)) < X) > Y

What this does is count how many records have more than Y records that have been created within X seconds after or before each record.

So basically it will return >=1 if there are any, 0 if not.

Answer 2

So if you wanted to sort by hours you would want to group the records. Here I'm using the built-in functions that return parts of a timestamp, year() , month() , dayofmonth() , hour() . Since you can't use an aggregate function in the where clause I had to use having to limit by the count requirement.

select date(create_date),
hour(create_date),
count(*) as surge from records
where create_date > curdate() - interval 1 day
group by year(create_date), month(create_date),
dayofmonth(create_date), hour(create_date)
having count(*) > 50;

Another method to accomplish your goal might be to select the count of records and group by the interval in question. In this case I'm adding an hour to the create_date to get your 1 hour suggested interval. Anytime the count is greater than 50 it returns a row. Notice I'm also grouping by the hour . This is to prevent multiple starts for a "surge" within the same hour:

select create_date,count(*) as surge from records
group by year(create_date), month(create_date),
dayofmonth(create_date),hour(create_date),
(create_date + interval 1 hour - create_date) having count(*) > 50;

The problem with this however is that some surges may last longer than 1 hour, but it should give you the moment the "surge" started.