how can i melt a data.table with concatenated column names

Question

I'm using dcast.data.table to convert a long data.table to a wide data.table

library(data.table)
library(reshape2)

set.seed(1234)
dt.base <- data.table(A = rep(c(1:3),2), B = rep(c(1:2),3), C=c(1:4,1,2),thevalue=rnorm(6))


#from long to wide using dcast.data.table()
dt.cast <- dcast.data.table(dt.base, A ~ B + C, value.var = "thevalue", fun = sum)

#now some stuff happens e.g., please do not bother what happens between dcast and melt
setkey(dt.cast, A)
dt.cast[2, c(2,3,4):=1,with = FALSE]

now i want to melt the data.table back again to the original column layout and here i'm stuck, how do I separate the concatenated columnames from the casted data.table, this is my problem

dt.melt <- melt(dt.cast,id.vars = c("A"), value.name = "thevalue")

I need two columns instead of one

the result that i'm looking for can be produced with this code

#update
dt.base[A==2 & B == 1 & C == 1, thevalue :=1]
dt.base[A==2 & B == 2 & C == 2, thevalue :=1]

#insert (2,1,3 was not there in the base data.table)
dt.newrow <- data.table(A=2, B=1, C=3, thevalue = 1)
dt.base <-rbindlist(list(dt.base, dt.newrow))
dt.base

As always any help is appreciated

Answer 1

Would that work for you?

colnames <- c("B", "C")
dt.melt[, (colnames) := (colsplit(variable, "_", colnames))][, variable := NULL]
subset(dt.melt, thevalue != 0)
# or dt.melt[thevalue != 0, ]

#   A   thevalue B C
#1: 1 -1.2070657 1 1
#2: 2  1.0000000 1 1
#3: 2  1.0000000 1 3
#4: 3  1.0844412 1 3
#5: 2  1.0000000 2 2
#6: 3  0.5060559 2 2
#7: 1 -2.3456977 2 4

If your data set isn't representable and there could be zeros in valid rows, here's alternative approach

colnames <- c("B", "C")
setkey(dt.melt[, (colnames) := (colsplit(variable, "_",colnames))][, variable := NULL], A, B, C)
setkey(dt.base, A, B, C)
dt.base <- dt.melt[rbind(dt.base, data.table(A = 2, B = 1, C = 3), fill = T)]
dt.base[, thevalue.1 := NULL]

##    A B C   thevalue
## 1: 1 1 1 -1.2070657
## 2: 1 2 4 -2.3456977
## 3: 2 1 1  1.0000000
## 4: 2 2 2  1.0000000
## 5: 3 1 3  1.0844412
## 6: 3 2 2  0.5060559
## 7: 2 1 3  1.0000000

Edit

As. suggested by @Arun, the most efficient way would be to use @AnandaMahto cSplit function, as it is using data.table too, ie,

cSplit(dt.melt, "variable", "_")

Second Edit

In order to save the manual merges, you can set fill = NA (for example) while dcasting and then do everything in one go with csplit , eg

dt.cast <- dcast.data.table(dt.base, A ~ B + C, value.var = "thevalue", fun = sum, fill = NA)
setkey(dt.cast, A)
dt.cast[2, c(2,3,4):=1,with = FALSE]
dt.melt <- melt(dt.cast,id.vars = c("A"), value.name = "thevalue")
dt.cast <- cSplit(dt.melt, "variable", "_")[!is.na(thevalue)]
setnames(dt.cast, 3:4, c("B","C"))

#    A   thevalue B C
# 1: 1 -1.2070657 1 1
# 2: 2  1.0000000 1 1
# 3: 2  1.0000000 1 3
# 4: 3  1.0844412 1 3
# 5: 2  1.0000000 2 2
# 6: 3  0.5060559 2 2
# 7: 1 -2.3456977 2 4