Problem Iterating through dataframes contained in a list in R

Question

Context I read a csv and separated the content into 11 different dataframes. I then put the dataframes into a list like so.

SourceCSV= read.csv("dt1Summary.csv",header=TRUE, sep = ';')

df1=SourceCSV[SourceCSV$Number == 122]
df2=SourceCSV[SourceCSV$Number == 430]
...
df11=SourceCSV[SourceCSV$Number == 1830]

dfList = list(df1, df2, ..., df11)

Then I cleaned the dataframes of rows with NA-Values like so

for (i in 1:length(dfList)) {
 dfList[[i]]=dfList[[i]][complete.cases(dfList[[i]]),]
}

Now when I try to run any code that needs to modify the cells in the dataframes in the same manner for every element in the list, I cant seem to figure out how to call the contents correctly.

I want to add a column with the value 1 in every cell to every dataframe.

But when I run:

for (i in 1:length(dfList)){
    dfList[[i]]$extraCol = 1
}

The dataframes just get replaced by 11 values of 1.

Can you explain how I can properly call the cells in the dataframes in the list? And how can I properly loop through the list without using the length method?

Answer 1

I've tried your code with a custom dataset and I have no problem with your code. Here is my code to generate the dataset and run the code you posted:

dfList <- list("df1" = data.frame("x1" = rnorm(1000), 
                                  "x2" = rnorm(1000), 
                                  "x3" = rnorm(1000)), 
               "df2" = data.frame("x1" = rnorm(1000), 
                                  "x2" = rnorm(1000), 
                                  "x3" = rnorm(1000)), 
               "df3" = data.frame("x1" = rnorm(1000), 
                                  "x2" = rnorm(1000), 
                                  "x3" = rnorm(1000)))
dfList[[1]][c(15,108,201,405,673,702),] <- NA
dfList[[2]][c(105,18,207,504,67,802),] <- NA
dfList[[3]][c(150,408,102,566,773,902),] <- NA

for (i in 1:length(dfList)) {
  dfList[[i]]=dfList[[i]][complete.cases(dfList[[i]]),]
}
for (i in 1:length(dfList)){
  dfList[[i]]$extraCol = 1
}
dfList
$df1
          x1         x2         x3 extraCol
1  0.6898781 -0.1514055  1.2448713        1
2 -0.5443420 -0.8995352 -0.4034141        1
3  0.7767239  0.5620077  0.1774943        1
 [ reached 'max' / getOption("max.print") -- omitted 991 rows ]

$df2
          x1        x2         x3 extraCol
1 -0.7885374 0.9506792  0.6453008        1
2 -1.5811700 0.1110525 -0.3572549        1
3 -1.4067713 0.1001205 -0.8444532        1
 [ reached 'max' / getOption("max.print") -- omitted 991 rows ]

$df3
         x1         x2          x3 extraCol
1 1.5309524 -0.9326038 -0.04479456        1
2 0.5882008  0.9105592 -0.82232054        1
3 1.2161384 -0.4759543 -0.64703306        1
 [ reached 'max' / getOption("max.print") -- omitted 991 rows ]

However, I think your problem is the lines df1=SourceCSV[SourceCSV$Number == 122] and similar. I suppose that your SourceCSV is a data.frame , so you should index it as df1=SourceCSV[SourceCSV$Number == 122,] to index all columns of the dataframe. Check that dfX objects are data.frames .

Also, if you want to avoid using the length() function, you can use lapply to apply a function to every data.frame in dfList .

dfList <- lapply(dfList, function(x) {x[complete.cases(x),]})
dfList <- lapply(dfList, function(x) {x$extraCol = 1; return(x)})