Generically aggregate columns using data.table

Question

I am trying generically aggregate data using data.table package. I have multiple columns that I want to aggregate. I create a initial data table using the following script:

library(data.table)
dt <- data.table(x.1 = rnorm(10, 20, 3), x.2 = rnorm(10, 20, 3), x.3 = rnorm(10, 20, 3),
                 y.1 = rnorm(10, 20, 3), y.2 = rnorm(10, 20, 3), y.3 = rnorm(10, 20, 3),
                 z.1 = rnorm(10, 20, 3), z.2 = rnorm(10, 20, 3), z.3 = rnorm(10, 20, 3))

What I am trying to achieve is to aggregate columns {x1, x2, x3, y1, y2, y3, z1, z2, z3} => {x.total, y.total, z.total} by applying sum to each group of columns.

I can do this using for loops like this:

prefixes <- c('x', 'y', 'z')
initial.colnames <- c(names(dt))

for (i in 1:nrow(dt)){
  for (pref in prefixes){
    dt[,eval(paste0(pref, '.total')) := sum(dt[i, eval(grep(pref, initial.colnames))]), with = TRUE]
  }
}

However, I want to apply using in-line data table construct, something like this:

dt[, eval(paste0(prefixes, '.total')) := sum(dt[,eval(grep(prefixes, initial.colnames))]), with = F]

But this does not give me the required results.

Maybe there are some ideas how would I do that in the right way?

Answer 1

Here's a way to aggregate with melt :

mDT = melt(dt[, r := .I], measure.vars = patterns(prefixes), value.name=prefixes)

mDT[, lapply(.SD, sum), by=r, .SDcols=prefixes]


     r        x        y        z
 1:  1 63.65898 65.41892 56.40470
 2:  2 60.58634 62.71055 48.69771
 3:  3 50.12036 60.06289 66.38637
 4:  4 55.42629 63.38670 56.98914
 5:  5 59.94042 54.28727 49.20218
 6:  6 59.51313 67.53499 59.24097
 7:  7 63.26874 62.23262 60.70875
 8:  8 54.90082 76.09135 58.79787
 9:  9 56.35402 52.11372 60.37903
10: 10 52.77926 55.06044 53.75093

Answer 2

We can use Map with Reduce

dt[,paste0(prefixes, '.total'):= Map(function(i)  Reduce('+',as.list(.SD[,i, with=FALSE])), 
                                split(names(dt), sub('\\..*', '', names(dt))))]