如何在R中的reshape包中循环dcast函数

Question

Being a relatively new R user, I have trouble with any looping functions. I have looked at many tutorials but the examples in them are usually very basic and therefore easy to execute. However I need create slightly more complex loops and am having a lot of trouble figuring out how to do so. There are a few related looping questions on here and other forums but none match exactly what I need and though I have tried to adapt other answers for my current problem, I keep running into errors.

I have 2000 .csv files with data tabulated in long-format data (simplified example):

solution1    
> sol1     sol2     Istat
> s1       s2       0.435
> s1       s3       0.456
> s1       s4       0.845
> s1       s5       0.234

It is basically a summary of pairwise comparisons of 2000 individual solutions that I have, with the similarity between solutions summarised in an 'Istat' value.

I am trying to dcast each of these 2000 .csv files into wide-format table (using the reshape package in R) so they look like (following example above):

     s1     s2     s3     s4     s5
s1   NA     0.435  0.456  0.845  0.234

I know how to do this just once with a single .csv file:

stat.cast <- dcast(solution1, sol2 ~ sol1, value.var="Istat")

But I can't seem to work it into into a for loop function or even with lapply , which seems like it could be a possible solution here too.

The closest I was able to get with a for function:

 # Get files from directory
loopout = "/Users/jc219806/Documents/Chapter 1/ANALYSES/R work/Istat/last_LoopOut/"
# List of file names inside folder
solutions <- list.files(loopout)
# Read all 2000 files inside
all.data <- lapply(solutions, read.csv, header=TRUE)
# Loop for performing reshape cast function to each listed dataframe
for (i in 1:length(all.data))
  {
  all.cast <- dcast(all.data, sol2 ~ sol1, value.var="Istat")
  }

But it keeps giving me the error that it is unable to recognise the "Istat" value from the input - even though it is there in the list of dataframes I have ("solutions" object in code above).

And with the lapply function:

lapply(solutions, dcast(all.data, sol2 ~ sol1, value.var="Istat"))

I get the same type of error:

Error: value.var (Istat) not found in input

I don't understand why because it is listed in the list of dataframes, as one of the variables in each of the 2000 dataframes. It seems like I am not getting it to loop through each of my 2000 .csv files properly, but I don't know how to fix that. I was also wondering if it were also possible to write the code so that it loops through binding all the 2000 outputs together according to column names? It's looping crazy.

I hope this is not as complicated a problem as it seems to me to be. Any help (along with some detailed explanations) or useful direction would be massively and sincerely appreciated. Thanks

Answer 1

You wrote:

for (i in 1:length(all.data))
  {
  all.cast <- dcast(all.data, sol2 ~ sol1, value.var="Istat")
  }

What you should have written:

all.cast <- list()
for (i in 1:length(all.data)) {
  all.cast[[i]] <- dcast(all.data[[i]], sol2 ~ sol1, value.var = "Istat")
}

But a more "R-esque" solution would be:

all.cast <- lapply(all.data, dcast, sol2 ~ sol1, value.var = "Istat")

Hopefully this makes it clear what you did wrong.

Answer 2

"all.data" is a list of dataframes. To loop over the list, you can use lapply and an anonymous function call (just to be clear) and apply dcast on that.

library(reshape2)
lapply(all.data, function(x) dcast(x, sol1 ~ sol2, value.var="Istat"))

Or instead of doing individual dcast , the list can be rbind to a dataframe with a grouping variable for each list element and then either do dcast or spread from library(tidyr)

library(dplyr)
library(tidyr)
unnest(all.data, group) %>% 
                  spread(sol2, Istat)

Or using data.table

library(data.table)
dcast(rbindlist(Map(cbind, all.data, group=seq_along(all.data))),
                 group + sol1 ~sol2, value.var='Istat')

data

all.data <- structure(list(solution1 = structure(list(sol1 = c("s1", 
"s1", 
"s1", "s1"), sol2 = c("s2", "s3", "s4", "s5"), Istat = c(0.435, 
0.456, 0.845, 0.234)), .Names = c("sol1", "sol2", "Istat"), 
class =     "data.frame", row.names = c(NA, 
-4L)), solution2 = structure(list(sol1 = c("s1", "s1", "s1", 
"s1"), sol2 = c("s2", "s3", "s4", "s5"), Istat = c(0.42, 0.536, 
0.945, 0.324)), .Names = c("sol1", "sol2", "Istat"), 
class =    "data.frame", row.names = c(NA, 
-4L))), .Names = c("solution1", "solution2"))

Answer 3

I would melt your "all.data" list and then dcast it to a wide form. Something like:

## Sample data
set1 <- set2 <- data.frame(sol1 = c("s1", "s1", "s1", "s1"), 
                   sol2 = c("s2", "s3", "s4", "s5"), 
                   Istat = c(0.435, 0.456, 0.845, 0.234))
set2$Istat <- set2$Istat + 1 ## Just to see some different data

all.data <- mget(ls(pattern = "set\\d+")) ## use your actual object

## The reshaping
library(reshape2)
dcast(melt(all.data, id.vars = c("sol1", "sol2")), 
      L1 + sol1 ~ sol2, value.var = "value")
#     L1 sol1    s2    s3    s4    s5
# 1 set1   s1 0.435 0.456 0.845 0.234
# 2 set2   s1 1.435 1.456 1.845 1.234

If your "all.data" object has names, "L1" will reflect those names, which can be quite convenient in the long run.