get latest records from mysql result

Question

I have my mysql query, which i use in php script:

select distinct dhcp_logs_public.service,
dhcp_logs_public.ip,
dhcp_logs_public.date as date,
dhcp_logs_public.until as until
from dhcp_logs_public
where dhcp_logs_public.ip in('79.109.1.200','71.109.160.123','21.110.151.110')
AND dhcp_logs_public.date >= DATE_ADD(NOW(), INTERVAL -48 Hour)

which gives me this kind of result:

service         ip                date            until
PONS1   79.109.1.200    11.10.2014 17:17    11.10.2014 19:17
PONS2   71.109.160.123  11.10.2014 19:09    12.10.2014 3:09
PONS2   71.109.160.123  11.10.2014 23:09    12.10.2014 7:09
PONS2   71.109.160.123  12.10.2014 3:09     12.10.2014 11:09
PONS3   71.109.160.123  12.10.2014 7:09     12.10.2014 15:09
PONS4   71.109.160.123  12.10.2014 16:19    12.10.2014 17:19
PONS4   71.109.160.123  12.10.2014 16:49    13.10.2014 0:49
PONS5   21.110.151.110  13.10.2014 9:22     13.10.2014 11:22
PONS5   21.110.151.110  13.10.2014 10:34    13.10.2014 12:34
PONS5   21.110.151.110  13.10.2014 11:46    13.10.2014 13:46
PONS5   21.110.151.110  13.10.2014 11:46    13.10.2014 13:46

I need to modify my query to achieve this: (latest record for every ip address)

service         ip                date            until
PONS1   79.109.1.200    11.10.2014 17:17    11.10.2014 19:17
PONS4   71.109.160.123  12.10.2014 16:49    13.10.2014 0:49
PONS5   21.110.151.110  13.10.2014 11:46    13.10.2014 13:46

I tried things like sub-query, but my table is too large (~ 1 billion rows) to process it quite quickly. Also I tried to add max(date) with group by ip, but no luck..

Any ideas?

Answer 1

You could use not exists to select all rows where another row with the same ip and a more recent date does not exist (which means that the selected rows are the most recent for their ip addresses)

select d.service,
d.ip,
d.date,
d.until
from dhcp_logs_public d
where d.ip in('79.109.1.200','71.109.160.123','21.110.151.110')
and d.date >= DATE_ADD(NOW(), INTERVAL -48 Hour)
and not exists (
    select 1 from dhcp_logs_public d2
    where d2.ip = d.ip
    and d2.date > d.date
)

This query can take advantage of a composite index on (ip,date)

Edit

If you can rely on the id column to determine how recent a row is then the following might be faster

select d.service,
d.ip,
d.date,
d.until
from dhcp_logs_public d
where d.ip in in('79.109.1.200','71.109.160.123','21.110.151.110')
and d.date >= DATE_ADD(NOW(), INTERVAL -48 Hour)
and d.id = (select max(id) from dhcp_logs_public d2 where d2.ip = d.ip)

or with a derived table instead of a subquery

select d.service,
d.ip,
d.date,
d.until
from dhcp_logs_public d
join (
    select max(id) max_id
    from dhcp_logs_public
    where d.ip in in('79.109.1.200','71.109.160.123','21.110.151.110')
    and d.date >= DATE_ADD(NOW(), INTERVAL -48 Hour)
    group by ip 
) t1 on t1.max_id = d.id

Answer 2

Are you sure GROUP BY doesn't work ?

Else try this :

SELECT DISTINCT 
    MAX(dhcp_logs_public.service) as service,
    dhcp_logs_public.ip,
    MAX(dhcp_logs_public.date) as date,
    MAX(dhcp_logs_public.until) as until

    FROM dhcp_logs_public
    WHERE dhcp_logs_public.ip in('79.109.1.200','71.109.160.123','21.110.151.110')
    AND dhcp_logs_public.date >= DATE_ADD(NOW(), INTERVAL -96 Hour)
    GROUP BY dhcp_logs_public.ip