R - 如何在 data.table 中最有效地滞后/领先多个列

Question

Having a large data.table that stores one date column (monthly) and then a bunch of different variables of interest measured at the respective dates for various subjects/IDs. Now I want to add for a subset of those variables (only some columns) newly computed columns that lead AND lag those columns by multiple periods all at once. Is that doable? See below for the illustration of some example data that represents the high-level structure of my table and for what I tried so far

Date        ID   Var_A   Var_B   Var_C
2000-01-31  1    100     500     1000
2000-02-28  1    200     600     1100
2000-03-31  1    300     700     1200
2000-04-30  1    400     800     1300 
2000-01-31  2    100     500     1000
2000-02-28  2    200     600     1100
2000-03-31  2    300     700     1200
2000-04-30  2    400     800     1300

dt[, `:=`(Var_A_Lag_1 = shift(Var_A_Lag_1, 1),
          Var_A_Lead_1 = shift(Var_A_Lead_1, 1, type = 'lead'),
          Var_A_Lag_2 = shift(Var_A_Lag_1, 2),
          Var_A_Lead_2 = shift(Var_A_Lead_1, 2, type = 'lead'),
          Var_B_Lag_1 = shift(Var_B_Lag_1, 1),
          Var_B_Lead_1 = shift(Var_B_Lead_1, 1, type = 'lead'),
          Var_B_Lag_2 = shift(Var_B_Lag_1, 2),
          Var_B_Lead_2 = shift(Var_B_Lead_1, 2, type = 'lead')),
   by = ID]

But that cannot be efficient can it? I tried sth that I thought was very intuitive and would work but no luck.

cols_to_edit <- which(sapply(dt, is.numeric))
cols_to_edit <- colnames(dt[, cols_to_edit, with = FALSE])

# col names od shifted variables
col_names_lag_1 <- paste(cols_to_edit, "lag_1", sep = "_")
col_names_lag_2 <- paste(cols_to_edit, "lag_2", sep = "_")
col_names_lead_1 <- paste(cols_to_edit, "lead_1", sep = "_")
col_names_lead_2 <- paste(cols_to_edit, "lead_2", sep = "_")

# colnames for differences 
col_names_lag_1_d <- paste("d", cols_to_edit, "lag_1", sep = "_")
col_names_lag_2_d <- paste("d", cols_to_edit, "lag_2", sep = "_")
col_names_lead_1_d <- paste("d", cols_to_edit, "lead_1", sep = "_")
col_names_lead_2_d <- paste("d", cols_to_edit, "lead_2", sep = "_")

# Execute the shift command
dt_2[, (col_names_lag_1) := shift(cols_to_edit, 1), by = ID] 
# would have repeated for all new columns as defined above but it is not working. 

I basically want all numeric variables in this table shifted, say by 1 and 2 respectively in either directio. The newly computed values should then be assigned to the columns named by the name vector declared above. Didn't find any other question that was similiar to my case here. Do you have any idea or know a best practice for doing this?

The context: The variables are selected metrics as input for a regression model requiring the input to be available in that format.

Answer 1

How's this in a nice little for loop:

cols <- grep("Var", names(dt), value = TRUE)
for ( i in 1:2 ) { # update for the number of shifts
  
  lag_names <- paste(cols, "Lag", i, sep = "_")
  dt[, (lag_names) := lapply(.SD, shift, i, type = "lag"), .SDcols = cols]
  
  lead_names <- paste(cols, "Lead", i, sep = "_")
  dt[, (lead_names) := lapply(.SD, shift, i, type = "lead"), .SDcols = cols]
  
}