R中数据框的列表列表

Question

I have to cope with an ugly list called ul that looks like this:

[[1]]
[[1]]$param
     name     value 
"Section"       "1" 

[[1]]$param
   name   value 
"field"     "1" 

[[1]]$param
          name          value 
"final answer"            "1" 

[[1]]$param
    name    value 
"points"   "-0.0" 


[[2]]
[[2]]$param
     name     value 
"Section"       "1" 

[[2]]$param
   name   value 
"field"     "2" 

[[2]]$param
          name          value 
"final answer"            "1" 

[[2]]$param
    name    value 
"points"    "1.0" 


[[3]]
[[3]]$param
     name     value 
"Section"       "1" 

[[3]]$param
   name   value 
"field"     "3" 

[[3]]$param
          name          value 
"final answer"        "0.611" 

[[3]]$param
    name    value 
"points"    "1.0" 

I would like to convert the list to a simple data frame, ie

Section    field    final answer    points
      1        1               1      -0.0
      1        2               1       1.0
      1        3           0.611       1.0

Is there any straightforward way to achieve that? or do I have to make a function accessing each list individually and binding it to a dataframe?

The data is imported from an uglier xml file, so if someone wants to play with it there is a link to the RData file . Sorry for not having reproducible code. Thank you very much.

Answer 1

There is probably a better solution, but this should get you started. First, we load some libraries

R> library(plyr)
R> library(reshape2)

Then handle your lists in two parts.

##lapply applies ldply to each list element in turn
ul1 = lapply(ul, ldply)

##We then do the same again
dd = ldply(ul1)[,2:3]

Next we label output according to their list order

R> dd$num = rep(1:3, each=4)

Then we convert from long to wide format

R> dcast(dd, num ~ name)

  num field final answer points Section
1   1     1            1   -0.0       1
2   2     2            1    1.0       1
3   3     3        0.611    1.0       1

Answer 2

An answer to a similar problem was given by Marc Schwartz at this link : https://stat.ethz.ch/pipermail/r-help/2006-August/111368.html

I'm copying it in case the link is deleted.

 as.data.frame(sapply(a, rbind))

   V1 V2 V3
1  a  b  c
2  1  3  5
3  2  4  6

or:

as.data.frame(t(sapply(a, rbind)))
   V1 V2 V3
1  a  1  2
2  b  3  4
3  c  5  6

Answer 3

As the structure of the ul is consistent, you can simply get each column individually (using only base R):

section <- vapply(ul, function(x) as.numeric(x[[1]][2]), 0)
field <- vapply(ul, function(x) as.numeric(x[[2]][2]), 0)
final_answer <- vapply(ul, function(x) as.numeric(x[[3]][2]), 0)
points <- vapply(ul, function(x) as.numeric(x[[4]][2]), 0)

(Note, I use vapply instead of sapply as it is faster and reliably returns a vector, which is needed here).
Then you can simply put it all together:

> data.frame(section, field, final_answer, points)
  section field final_answer points
1       1     1        1.000      0
2       1     2        1.000      1
3       1     3        0.611      1

Note that I transformed everything into numeric . If you want to retain everything as characters, delete the as.numeric and exchange 0 with "" in each call to vapply .

---

Late update:

There is actually a nice oneliner that extracts the complete data:

do.call("rbind", lapply(ul, function(x) as.numeric(vapply(x, "[", i = 2, ""))))

which gives:

     [,1] [,2]  [,3] [,4]
[1,]    1    1 1.000    0
[2,]    1    2 1.000    1
[3,]    1    3 0.611    1

to get the colnames use:

> vapply(ul[[1]], "[", i = 1, "")
         param          param          param          param 
     "Section"        "field" "final answer"       "points" 

Answer 4

I'm not sure what you mean by "a function accessing each list individually", but this is pretty straightforward using "lapply" and "do.call('rbind',...)":

I couldn't load your .RData file, so this code works for the list:

ul <- list(param = list(
             c(name = "Section", value = "1"),
             c(name = "field", value = "1"),
             c(name = "final answer", value = "1"),
             c(name = "points", value = "-0.0")),
           param = list(
             c(name = "Section", value = "1"),
             c(name = "field", value = "2"),
             c(name = "final answer", value = "1"),
             c(name = "points", value = "1.0")))

You may have to tweak the details if your list is different; the general principal will remain the same. Just to keep the code clean, let's define the 'extractitem' function that's going to pull out all of the names or values for ul[[1]], ul[[2]], etc. This function is a little more general than you need.

extractitem <- function(listelement, item)
  unname(lapply(listelement, function(itemblock) itemblock[item]))

Now we'll just use lapply to walk through ul element by element; for each element, we extract the values into a data frame, then name the columns according to the 'names'.

rowlist <- lapply(ul, function(listelement) {
  d <- data.frame(extractitem(listelement, "value"), stringsAsFactors = FALSE)
  names(d) <- unlist(extractitem(listelement, "name"))
  d
})

rowlist is now a list of data frames; we can consolidate them into a single data frame with 'rbind'. The nice thing about using data frames in the previous step (as opposed to vectors or something with lower overhead) is that rbind will reorder the columns if necessary, so if the order of the fields changes from element to element, we're still all right.

finaldf <- do.call("rbind", rowlist)

We still need to change the elements fo finaldf from "character" to whatever's appropriate for your application through, eg

finaldf$points <- as.numeric(finaldf$points)

and so on. The last step cleans up the data frame by stripping the automatically-generated row names:

rownames(finaldf) <- NULL

In case you need to tweak things, the general idea is to write a function that will format each ul[[i]] as a data frame with the correct column names; then invoke that function on each element of ul with lapply; and finally collapse the resulting list with do.call("rbind",...).