Get the max date based on multiple columns of an R dplyr / tidyverse dataframe

Question

From a csv file that looks like this:

Date	Timestamp	Units	Name	Condition	Obj	Param	Attrib1	Atrrib2	Result
2019-07-31	2019-08-01 01:16:09	m3	n01	a1	o1	Nap	TP	IN	34937
2019-07-31	2019-08-01 01:16:10	m3	n01	a2	o2	Nap	TP	OUT	36673.09
2019-11-06	2019-11-18 20:21:06	mg/l	n01	a3	o3	NO3	TP	OUT	1
2019-11-06	2019-11-18 20:21:06	mg/l	n01	z5	o4	BOD	IO	IN	220
2019-11-06	2019-11-18 20:21:06	mg/l	n01	z5	o4	BOD	TP	IN	220
2019-11-06	2019-11-18 20:21:06	mg/l	n01	z6	o1	NO2	TP	OUT	0.31
2019-11-06	2019-11-18 20:21:13	mg/l	n01	a11	o4	Ntot	IO	IN	47
2019-11-06	2019-11-18 20:21:13	mg/l	n01	a11	o4	Ntot	TP	IN	47
2021-01-06	2021-01-07 02:15:06	m3	n01	a1	o1	Nap	TP	IN	17909
2021-01-06	2021-01-07 02:15:07	m3	n01	a2	o2	Nap	TP	OUT	19216.19

I want to remove the rows with the last (or max) Timestamp per value in column Date and column Condition .
The resulting table should not have the duplicated timestamps "2019-11-18 20:21:06" and "2019-11-18 20:21:13" (Which Condition and Result values were [z5, a11] and [220, 47] respectively).

Date	Timestamp	Units	Name	Condition	Obj	Param	Attrib1	Atrrib2	Result
2019-07-31	2019-08-01 01:16:09	m3	n01	a1	o1	Nap	TP	IN	34937
2019-07-31	2019-08-01 01:16:10	m3	n01	a2	o2	Nap	TP	OUT	36673.09
2019-11-06	2019-11-18 20:21:06	mg/l	n01	a3	o3	NO3	TP	OUT	1
2019-11-06	2019-11-18 20:21:06	mg/l	n01	z5	o4	BOD	IO	IN	220
2019-11-06	2019-11-18 20:21:06	mg/l	n01	z6	o1	NO2	TP	OUT	0.31
2019-11-06	2019-11-18 20:21:13	mg/l	n01	a11	o4	Ntot	IO	IN	47
2021-01-06	2021-01-07 02:15:06	m3	n01	a1	o1	Nap	TP	IN	17909
2021-01-06	2021-01-07 02:15:07	m3	n01	a2	o2	Nap	TP	OUT	19216.19

I found two links ( 1 and 2 ) to come up with the following R script

library(tidyverse)
# Group per Date and Condition and filter max Timestamp
df <- read.csv("./Example.csv") %>%
    mutate(Date = as.POSIXct(Date, format = "%Y-%m-%d")) %>%
    mutate(Timestamp = as.POSIXct(Timestamp, format = "%Y-%m-%d %H:%M:%S")) %>%
    group_by(Date, Condition) %>%
    filter(Timestamp == max(Timestamp)) %>%
    distinct()
write_csv(df, file = "./ExampleResult.csv")

But I cannot get the wished results.
What is wrong with the approach? Is there any other easier way?
Thank you!

Answer 1

You have multiple values at max(Timestamp) . To resolve this I'd suggest to use dplyr::slice_max and setting with_ties = FALSE .

Here's some code to get what you're after.

df %>% 
  mutate(Date = as.POSIXct(Date, format = "%Y-%m-%d")) %>%
  mutate(Timestamp = as.POSIXct(Timestamp, format = "%Y-%m-%d %H:%M:%S")) %>%
  group_by(Date, Condition) %>%
  slice_max(order_by = Timestamp, n = 1, with_ties = FALSE)

But depending on your application, you may want to be explicit about how to resolve those ties by supplying additional variables to the order_by argument.

Answer 2

Try using the following:

library(dplyr)

read.csv("./Example.csv") %>%
#df %>%
  mutate(Date = as.Date(Date), 
        Timestamp = as.POSIXct(Timestamp, format = "%Y-%m-%d %H:%M:%S")) %>%
  distinct(Date, Condition, Result, .keep_all = TRUE) -> result

result

#        Date           Timestamp Units Name Condition Obj Param Attrib1 Atrrib2   Result
#1 2019-07-31 2019-08-01 01:16:09    m3  n01        a1  o1   Nap      TP      IN 34937.00
#2 2019-07-31 2019-08-01 01:16:10    m3  n01        a2  o2   Nap      TP     OUT 36673.09
#3 2019-11-06 2019-11-18 20:21:06  mg/l  n01        a3  o3   NO3      TP     OUT     1.00
#4 2019-11-06 2019-11-18 20:21:06  mg/l  n01        z5  o4   BOD      IO      IN   220.00
#5 2019-11-06 2019-11-18 20:21:06  mg/l  n01        z6  o1   NO2      TP     OUT     0.31
#6 2019-11-06 2019-11-18 20:21:13  mg/l  n01       a11  o4  Ntot      IO      IN    47.00
#7 2021-01-06 2021-01-07 02:15:06    m3  n01        a1  o1   Nap      TP      IN 17909.00
#8 2021-01-06 2021-01-07 02:15:07    m3  n01        a2  o2   Nap      TP     OUT 19216.19

data

df <- structure(list(Date = c("2019-07-31", "2019-07-31", "2019-11-06", 
"2019-11-06", "2019-11-06", "2019-11-06", "2019-11-06", "2019-11-06", 
"2021-01-06", "2021-01-06"), Timestamp = c("2019-08-01 01:16:09", 
"2019-08-01 01:16:10", "2019-11-18 20:21:06", "2019-11-18 20:21:06", 
"2019-11-18 20:21:06", "2019-11-18 20:21:06", "2019-11-18 20:21:13", 
"2019-11-18 20:21:13", "2021-01-07 02:15:06", "2021-01-07 02:15:07"
), Units = c("m3", "m3", "mg/l", "mg/l", "mg/l", "mg/l", "mg/l", 
"mg/l", "m3", "m3"), Name = c("n01", "n01", "n01", "n01", "n01", 
"n01", "n01", "n01", "n01", "n01"), Condition = c("a1", "a2", 
"a3", "z5", "z5", "z6", "a11", "a11", "a1", "a2"), Obj = c("o1", 
"o2", "o3", "o4", "o4", "o1", "o4", "o4", "o1", "o2"), Param = c("Nap", 
"Nap", "NO3", "BOD", "BOD", "NO2", "Ntot", "Ntot", "Nap", "Nap"
), Attrib1 = c("TP", "TP", "TP", "IO", "TP", "TP", "IO", "TP", 
"TP", "TP"), Atrrib2 = c("IN", "OUT", "OUT", "IN", "IN", "OUT", 
"IN", "IN", "IN", "OUT"), Result = c(34937, 36673.09, 1, 220, 
220, 0.31, 47, 47, 17909, 19216.19)),class = "data.frame",row.names = c(NA,-10L))