R中data.table列表中的唯一值

Question

I am working with a rather large data set (8M obs) and I am trying to obtain unique values/balances for a given date AND not in the previous date/date range. My date range covers 156 monthly observations. Here is an example of what and how I am proceeding but I'm sure there is a more efficient way.

library(data.table)
df = data.frame(ID = c("1234", "5678", "1234", "1112"))
set.seed(1234)
df$Bal = cbind(rnorm(4))
df$Date = as.Date(c("2017-12-31",rep("2018-01-31",3)))
setDT(df)
df[,.(count = uniqueN(ID)), by = Date]
tmp = split(df[,.SD,.SDcols = 1:3], by = "Date")
table(tmp[[2]][,ID] %in% tmp[[1]][,ID])
# FALSE  TRUE 
#   2     1

Essentially, 2 FALSE would represent new IDs and 1 TRUE would represent existing. Additionally, I would like the sum of the balance information. For example

Sum of old balance: -0.1226248
Sum of new balance: -2.068269

In turn, my new data frame would be

            New_Balance Old_Balance New_Accts Exisiting_Accts
2018-01-31  -2.068269   -0.1226248      2           1

Any help would be greatly appreciated!

Answer 1

The code below reproduces the expected result for the given sample dataset of two months:

library(data.table)
# make sure data is ordered by Date
setorder(df, Date)
# mark first appearance of each ID
df[, Acct := "Old"][rowid(ID) == 1L, Acct := "New"][]
# cumulative balances for each ID
df[, Balance := cumsum(Bal), by = ID][]
# reshape from long to wide format
dcast(df, Date ~ Acct, fun = list(sum, length), value.var = "Balance")

  Date Balance_sum_New Balance_sum_Old Balance_length_New Balance_length_Old 1: 2017-12-31 -1.207066 0.0000000 1 0 2: 2018-01-31 -2.068268 -0.1226246 2 1 

Note the code needs to be verified against a more elaborate sample dataset containing more months.