What is the most efficient way to add a column that is a binary indicator of a recurring number in time series dataframe?

Question

I have a dataframe that is similar to this example dataframe:

example <- data.frame(id = c("1","1","1", "1", "2", "2", "2"),
                      amount = c(2300, 1765, 2300, 1500, 35, 180, 180),
                      date = c("2010-11-01", "2010-11-02", "2010-11-03", "2010-11-04", "2010-11-01", "2010-11-02", "2010-11-03"))

I want to add a column that will have a 1 that indicates if an amount is a recurring amount. A recurring amount can only be considered recurring if the amount repeats within the same id. So it would look like this:

desiredResult <- data.frame(id = c("1","1","1", "1", "2", "2", "2"),
                      amount = c(2300, 1765, 2300, 1500, 2300, 180, 180),
                      date = c("2010-11-01", "2010-11-02", "2010-11-03", "2010-11-04", "2010-11-01", "2010-11-02", "2010-11-03"),
                      probableRecurringAmount = c(1,0,1,0,0,1,1)) 

The dataset is very large and I am having a hard time coming up with an efficient solution. I was considering adding keys to a column based on combinations of these other columns, but I want to only have a binary flag.

Answer 1

You can do it like this:

library(dplyr)    
example %>%
  group_by(id, amount) %>%
  mutate(probableRecurringAmount  = ifelse(n() > 1, 1, 0))

# A tibble: 7 x 4
# Groups:   id, amount [5]
# id      amount date       probableRecurringAmount
#<fct>  <dbl> <fct>                        <dbl>
#1 1       2300 2010-11-01                       1
#2 1       1765 2010-11-02                       0
#3 1       2300 2010-11-03                       1
#4 1       1500 2010-11-04                       0
#5 2         35 2010-11-01                       0
#6 2        180 2010-11-02                       1
#7 2        180 2010-11-03                       1

Answer 2

You can use duplicated to find duplicated rows, then join with the original data to flag both the original and the duplicate.

library(tidyverse)
example <- data.frame(id = c("1","1","1", "1", "2", "2", "2"),
                      amount = c(2300, 1765, 2300, 1500, 35, 180, 180),
                      date = c("2010-11-01", "2010-11-02", "2010-11-03", "2010-11-04", "2010-11-01", "2010-11-02", "2010-11-03"))

# Find duplicated rows
dups = example %>% 
  select(id, amount) %>% 
  mutate(recurring=as.numeric(duplicated(.))) %>% 
  filter(recurring==1)

# Flag both the original and duplicated rows as recurring
example %>% left_join(dups, ) %>% 
  replace_na(list(recurring=0))
#> Joining, by = c("id", "amount")
#>   id amount       date recurring
#> 1  1   2300 2010-11-01         1
#> 2  1   1765 2010-11-02         0
#> 3  1   2300 2010-11-03         1
#> 4  1   1500 2010-11-04         0
#> 5  2     35 2010-11-01         0
#> 6  2    180 2010-11-02         1
#> 7  2    180 2010-11-03         1

^{Created on 2020-01-14 by the reprex package (v0.3.0)}

Answer 3

We can use duplicated from base R

example$recurring <-  +(duplicated(example[c('id', 'amount')])|
         duplicated(example[c('id', 'amount')], fromLast = TRUE))
example$recurring
#[1] 1 0 1 0 0 1 1