矩阵列表到数据框列表

Question

I have a list containing lists of matrices, as below:

set.seed(123)

mat1 <- matrix(rnorm(9,1,2), ncol=3, nrow=3)
mat2 <- matrix(rnorm(9,1,3), ncol=3, nrow=3)

mynames <- c("a","b","c")

colnames(mat1) <- mynames
colnames(mat2) <- mynames

rownames(mat1) <- mynames
rownames(mat2) <- mynames

finallist <- list(val1 = list(subval1 = mat1), val2 = list(subval1 = mat2))

I was looking to get an output as:

goal <- data.frame(val1 = rnorm(9,1,2), val2 = rnorm(9,1,3), subval = rep("subval1",9), origrownames = rep(mynames, 3), origcolumnnames = rep(mynames,each=3))

I know there might be an intermediary dataframe which I can use reshape on, but I can't seem to get anything close. I have tried do.call("rbind", finallist) , but this does not seem to preserve the names of the top level list and the child list. Additionally, the sublists contain 2000 matrices each, with each matrix 20x20 in dimension, and I plan on using this function 20+ times, so I'm looking for something that isn't too slow.

Answer 1

The particular structure of this data can be accessed by a fairly rare method called recursive indexing. Here is three lines that will produce the result.

# build row and column names variables
mydf <- data.frame(origrownames = rep(mynames, 3), origcolumnnames = rep(mynames, each=3))
# use matrix subsetting to extract val1 and val2 variables
mydf[c("val1", "val2")] <- list(finallist[[c(1,1)]][as.matrix(mydf)],
                                finallist[[2:1]][as.matrix(mydf)])
# extract subval1 from list
mydf$subval <- names(finallist$val1)

The point of interest here is the second line, which first uses recursive indexing (the [[c(1, 1)]] and [[2:1]] ) to pull out elements in the nested lists and then uses matrix subsetting on the row and column names of the matrix to pull out the values in the desired order (see ?"[" for details on both of these methods).

The output from these extractions are wrapped in a list and then fed to mydf[c("va1", "val2")] which adds them to the data.frame with the desired names.

This returns

mydf
  origrownames origcolumnnames       val1       val2  subval
1            a               a -0.1209513 -0.3369859 subval1
2            b               a  0.5396450  4.6722454 subval1
3            c               a  4.1174166  2.0794415 subval1
4            a               b  1.1410168  2.2023144 subval1
5            b               b  1.2585755  1.3320481 subval1
6            c               b  4.4301300 -0.6675234 subval1
7            a               c  1.9218324  6.3607394 subval1
8            b               c -1.5301225  2.4935514 subval1
9            c               c -0.3737057 -4.8998515 subval1

You can reorder the columns using

mydf <- mydf[c("val1", "val2", "subval", "origrownames", "origcolumnnames")]

Answer 2

You could do

tmp <- simplify2array(unlist(finallist, FALSE))
setNames(cbind(expand.grid(dimnames(tmp)[-3]), apply(tmp, 3, c), 'subval1'),
         c('origrownames', 'origcolumnames', names(finallist), 'subval'))
#  origrownames origcolumnames       val1       val2  subval
#1            a              a -0.1209513 -0.3369859 subval1
#2            b              a  0.5396450  4.6722454 subval1
#3            c              a  4.1174166  2.0794415 subval1
#4            a              b  1.1410168  2.2023144 subval1
#5            b              b  1.2585755  1.3320481 subval1
#6            c              b  4.4301300 -0.6675234 subval1
#7            a              c  1.9218324  6.3607394 subval1
#8            b              c -1.5301225  2.4935514 subval1
#9            c              c -0.3737057 -4.8998515 subval1

Although the 'subval' variable seems redundant (it can only take one value). In my opinion this makes more sense

setNames(as.data.frame.table(simplify2array(lapply(finallist, '[[', 1))),
         c('origrownames', 'origcolumnames', 'variable', 'value'))
#   origrownames origcolumnames variable      value
#1             a              a     val1 -0.1209513
#2             b              a     val1  0.5396450
#3             c              a     val1  4.1174166
#4             a              b     val1  1.1410168
#5             b              b     val1  1.2585755
#6             c              b     val1  4.4301300
#7             a              c     val1  1.9218324
#8             b              c     val1 -1.5301225
#9             c              c     val1 -0.3737057
#10            a              a     val2 -0.3369859
#11            b              a     val2  4.6722454
#12            c              a     val2  2.0794415
#13            a              b     val2  2.2023144
#14            b              b     val2  1.3320481
#15            c              b     val2 -0.6675234
#16            a              c     val2  6.3607394
#17            b              c     val2  2.4935514
#18            c              c     val2 -4.8998515