根据下一行中另一列中的值从一列中提取值，然后找到持续时间直到再次达到该值

Question

I have a dataframe where I need to calculate the duration of a flooding event.

In order to do this, I need the 'From' Datetime:

Datetime of the row above when 'Flooded == TRUE'

and the 'To' Datetime:

Datetime of when the Temp > or = (Temp of the row above when 'Flooded == TRUE')

   > head(FloodingDuration)
    # A tibble: 6 x 8
    # Groups:   NestID, Nest, Year [1]
      Beach  Nest  Year Datetime             Temp NestID   TempDrop Flooded
      <chr> <dbl> <dbl> <dttm>              <dbl> <fct>       <dbl> <lgl>  
    1 LB        1  2014 2014-01-12 09:00:00  27.2 LB1_2014  0       FALSE  
    2 LB        1  2014 2014-01-12 10:00:00  27.2 LB1_2014 -0.0110  FALSE  
    3 LB        1  2014 2014-01-12 11:00:00  27.2 LB1_2014 -0.0190  FALSE  
    4 LB        1  2014 2014-01-12 12:00:00  27.2 LB1_2014 -0.00300 FALSE  
    5 LB        1  2014 2014-01-12 13:00:00  27.2 LB1_2014 -0.0290  FALSE  
    6 LB        1  2014 2014-01-12 14:00:00  27.1 LB1_2014 -0.00400 FALSE

I have some of the code, see below, I need the 'From' and 'To' to work.

FloodingDuration = group_by (TempData, NestID, Nest, Year) %>%
      filter(minute(Datetime) == 0) %>%
      mutate(TempDrop = Temp - lag(Temp, n=1, default = first(Temp))) %>%
      mutate(Flooded = TempDrop < -0.45) %>%
      group_by(NestID) %>%

      mutate(From = Datetime of Temp at (row above "Flooded == TRUE")) %>%
      mutate(To = Datetime of Temp >= Temp at (row above "Flooded == TRUE"))

      mutate(Duration = as.numeric(difftime(From, To, unit = "days" ))) %>% 
      mutate(MaxDuration = max(Duration) %>% 
      distinct(NestID, MaxDuration)

For example:

Row 8083 where Flooded==TRUE

From = Datetime from row 8082

To = row not seen, Datetime where Temp >= 28.920

FYI:

There are 112 different NestIDs, each with around 1200 rows of data.

24 of the NestIDs will have at least 1 Flooded==TRUE, sometimes consecutively, sometimes after the temperature has recovered from the first event.

After each Flooding event, the temperature will gradually rise again.

If it is only possible to find the Duration for the first Flooded==TRUE event for each Flooded NestID, this would still be great.

Answer 1

@Bex .. to get you started, the following might help.
As a rule of thumb, when you have multiple conditions, checks, etc. this may not always be able to perform in a "vectorised" manner, ie with a (single) pipe.

I take you step by step through it. You trim the code to make it a bit more elegant.

(I) reproducible data example and first steps
To allow others to chip in and help working out an elegant solution ... tadaaa ... a reproducible data snapshot . I adapted the Temp column to get the conditions you are referring to:

df <- data.frame(
    Datetime = seq(from = ymd_hms("2014-04-02 05:00:00"), to = ymd_hms("2014-04-02 15:00:00"), by = "1 hour")
    ,Temp = c(29.083, 29.088, 29.091, 28.920, 27.934, 27.359, 27.491, 27.99, 28.92, 28.93, 28.28)
    ,Flooded = c(rep(FALSE, 4), rep(TRUE, 2), rep(FALSE, 5)))

If you are unsure about how your algorithm develops, I recommend to code it step by step introducing interim results in new columns - this is where dplyr and friends prevail!

To get the Datetime for one of your instances, just use the lag() construction you applied earlier in your pipe.
Let's do the same for the Temp, as you want to check on this condition later.
We want to check for when conditions hold. That is a classical "if-then". Using if_else() requires type consistency. Thus, I wrap the NA in the respective datatype.

df <- df %>% 
  mutate(From = if_else(Flooded == TRUE, lag(Datetime, default = first(Datetime)),as.POSIXct(NA))
        ,Temp_before = if_else(Flooded == TRUE, lag(Temp, default = first(Temp)),as.double(NA)))

You now have:

              Datetime   Temp Flooded                From Temp_before
1  2014-04-02 05:00:00 29.083   FALSE                <NA>          NA
2  2014-04-02 06:00:00 29.088   FALSE                <NA>          NA
3  2014-04-02 07:00:00 29.091   FALSE                <NA>          NA
4  2014-04-02 08:00:00 28.920   FALSE                <NA>          NA
5  2014-04-02 09:00:00 27.934    TRUE 2014-04-02 08:00:00      28.920
6  2014-04-02 10:00:00 27.359    TRUE 2014-04-02 09:00:00      27.934
7  2014-04-02 11:00:00 27.491   FALSE                <NA>          NA
8  2014-04-02 12:00:00 27.990   FALSE                <NA>          NA
9  2014-04-02 13:00:00 28.920   FALSE                <NA>          NA
10 2014-04-02 14:00:00 28.930   FALSE                <NA>          NA
11 2014-04-02 15:00:00 28.280   FALSE                <NA>          NA

(II) check for the next temp condition

For each Flooded == TRUE we would need to check the remainder of the dataframe. This is better done in a separate function that you call for each instance of Flooded == TRUE .
The idea is to take the dataframe, trim it to the start datetime, and then check when the next time the temp condition is met.

check_next_temp_condition <- function(.start_date, .df = df){
    check_df <- .df %>% 
        filter(Datetime >= .start_date) %>%             # trim to start position
        mutate(Temp_cond = Temp >= first(Temp_before))  # check temp condition
}
#-------- let's quickly test what we have
start <- lubridate::ymd_hms("2014-04-02 09:00:00")
(   check_next_temp_condition(start)   )      # brackets around call will print result

This gives

             Datetime   Temp Flooded                From Temp_before Temp_cond
1 2014-04-02 09:00:00 27.934    TRUE 2014-04-02 08:00:00      28.920     FALSE
2 2014-04-02 10:00:00 27.359    TRUE 2014-04-02 09:00:00      27.934     FALSE
3 2014-04-02 11:00:00 27.491   FALSE                <NA>          NA     FALSE
4 2014-04-02 12:00:00 27.990   FALSE                <NA>          NA     FALSE
5 2014-04-02 13:00:00 28.920   FALSE                <NA>          NA      TRUE
6 2014-04-02 14:00:00 28.930   FALSE                <NA>          NA      TRUE
7 2014-04-02 15:00:00 28.280   FALSE                <NA>          NA     FALSE

What follows is standard. We subset for the first time the condition is met and extract the timestamp.

check_next_temp_condition <- function(.start_date, .df = df){
    check_df <- .df %>% filter(Datetime >= .start_date) %>%
        mutate(Temp_cond = Temp >= first(Temp_before)) %>%
        filter(Temp_cond == TRUE)   # filter for condition
    #---------- extract Datetime
    if(nrow(check_df) >= 1){ To_date = check_df$Datetime[1]   # check for result
    }else{                   To_date = as.POSIXct(NA)}        # ensure if no result
}

# Test again, if you like
(   check_next_temp_condition(start)   ) 

(III) running the check function over the dataframe

The final stretch is to apply the function based on the condition to the rows where the condition is met. ((This is where the vectorised approach for the pipe would break)).
dplyr offers you a function rowwise() that allows to iterate over each row. You need to remove the rowwise grouping before walking further in a pipe, etc. Alternatively, you can use other loop approaches.

Below, I just iterate over all rows, eg also rows for which the conditions does not apply. To save processing time, you may want to make this conditional again.

df <- df %>% 
    rowwise() %>% 
    mutate(To = check_next_temp_condition(Datetime)) %>% 
    ungroup()     # undo the rowwise()!

You now have a dataframe that allows you to wrap up your other calculations.

df
# A tibble: 11 x 6
   Datetime             Temp Flooded From                Temp_before To                 
   <dttm>              <dbl> <lgl>   <dttm>                    <dbl> <dttm>             
 1 2014-04-02 05:00:00  29.1 FALSE   NA                         NA   NA                 
 2 2014-04-02 06:00:00  29.1 FALSE   NA                         NA   NA                 
 3 2014-04-02 07:00:00  29.1 FALSE   NA                         NA   NA                 
 4 2014-04-02 08:00:00  28.9 FALSE   NA                         NA   NA                 
 5 2014-04-02 09:00:00  27.9 TRUE    2014-04-02 08:00:00        28.9 2014-04-02 13:00:00
 6 2014-04-02 10:00:00  27.4 TRUE    2014-04-02 09:00:00        27.9 2014-04-02 12:00:00
 7 2014-04-02 11:00:00  27.5 FALSE   NA                         NA   NA                 
 8 2014-04-02 12:00:00  28.0 FALSE   NA                         NA   NA                 
 9 2014-04-02 13:00:00  28.9 FALSE   NA                         NA   NA                 
10 2014-04-02 14:00:00  28.9 FALSE   NA                         NA   NA                 
11 2014-04-02 15:00:00  28.3 FALSE   NA                         NA   NA