R中如何根据账户和时间间隔创建新特征，比如最后一天、上周、上个月等？

Question

PS: My data has a million rows so I am looking for a solution that wouldn't take so long to run on a large data.

This is my transaction data

id          from    to          date        amount  
<int>       <fctr>  <fctr>      <date>      <dbl>
19521       6644    6934        2005-01-01  700.0
19524       6753    8456        2005-01-01  600.0
19523       9242    9333        2005-01-01  1000.0
19522       9843    9115        2005-01-01  900.0
19525       7075    6510        2005-02-01  400.0
19526       8685    7207        2005-02-01  1100.0
…           …       …           …           …
1052337     9866    5992        2010-12-31  139.1
1052768     9866    5797        2010-12-31  72.1
1054271     9866    6697        2010-12-31  95.8
1055597     9866    9736        2010-12-31  278.9
1053519     9868    8644        2010-12-31  242.8
1052790     9869    8399        2010-12-31  372.2

I want to create new features, based on the accounts in the "from" column, and time intervals such as last day, last week, last month, etc

For example, I want to calculate max amounts the accounts in the "from" column sent in last day/last week/last month and save this information to the data as new columns

The following data is what I expect the output to be:

id      from    to      date        amount  max_amount_in_last_day  max_amount_in_last_week max_amount_in_last_month    max_amount_in_last_3month
<int>   <fctr>  <fctr>  <date>      <dbl>   <dbl>                   <dbl>                  <dbl>                       <dbl>
19521   6644    6934    2005-01-01  700.0   700.0                   700.0                   700.0                       700.0
16214   6644    6874    2005-06-30  288.8   288.8                   288.8                   288.8                       288.8
16806   6644    9746    2005-06-30  83.7    288.8                   288.8                   288.8                       288.8
15220   6644    6671    2005-07-23  21716.0 21716.0                 21716.0                 21716.0                     21716.0
11975   6644    5384    2005-08-31  216.7   216.7                   216.7                   216.7                       21716.0
26579   6644    6041    2005-12-31  159.2   159.2                   159.2                   159.2                       159.2
35893   6644    8567    2006-01-31  169.6   169.6                   169.6                   169.6                       169.6
39425   6644    5599    2006-02-12  16230.0 16230.0                 16230.0                 16230.0                     16230.0
43606   6644    6454    2006-03-06  5859.0  5859.0                  5859.0                  16230.0                     16230.0
52052   6644    8836    2006-04-13  7269.0  7269.0                  7269.0                  7269.0                      16230.0
...

How can I achieve this?

To calculate max_amount_in_last_day , I tried the following code, but it works very poorly on a large dataset. Is there a better approach to this?

y  <- NULL

for(i in data$id){
  
  date <- data[data$id==i,]$date
  
  acc <- data[data$id==i,]$from
  
  df <- data[data$from==acc & data$date==date,]
        
        df <- df %>% mutate(max_trx_amount_in_last_day=max(df$amount)) 

        tmp <- df
        y <- rbind(y, tmp) %>% distinct
        

}

y

Answer 1

First, define a function to compute the statistic you want within a certain number of previous days.

previousDays <- function(X, from, date, days, FUN){
  FUN <- match.fun(FUN)
  Y <- X[X[['from']] == from, ]
  i <- Y[['date']] >= date - days + 1 & Y[['date']] <= date
  if(sum(i) > 0) FUN(Y[i, 'amount']) else NA_real_
}

Now, apply the function to each row, by a specified number of days. In the case below, 7 days, the last week.

applyPrev <- function(X, days, FUN){
  FUN <- match.fun(FUN)
  sapply(seq_along(X[['id']]), function(i){
    f <- X[i, 'from']
    d <- X[i, 'date']
    previousDays(df1, f, d, days, FUN)
  })
}

applyPrev(df1, 7, max)

Data

df1 <- read.table(text = "
id          from    to          date        amount  
19521       6644    6934        2005-01-01  700.0
19524       6753    8456        2005-01-01  600.0
19523       9242    9333        2005-01-01  1000.0
19522       9843    9115        2005-01-01  900.0
19525       7075    6510        2005-02-01  400.0
19526       8685    7207        2005-02-01  1100.0
1052337     9866    5992        2010-12-31  139.1
1052768     9866    5797        2010-12-31  72.1
1054271     9866    6697        2010-12-31  95.8
1055597     9866    9736        2010-12-31  278.9
1053519     9868    8644        2010-12-31  242.8
1052790     9869    8399        2010-12-31  372.2
", header = TRUE)