R: Import data from multiple file paths and store in a list

Question

I have some data which I would like to read in from seperate .txt files based on individual paths. A sample folder structure with txt files can be downloaded here .

The (sample) data.frame I have looks like this

data <-   structure(list(Name = structure(c(1L, 3L, 4L, 2L), .Label = c("Test1", "Test10", "Test2", "Test6"), class = "factor"), Metadata = structure(c(3L, 4L, 1L, 2L), .Label = c("asdajl7", "asfhas", "sgash", "uashas8"), class = "factor"), Filepath = structure(c(2L, 3L, 4L, 1L), .Label = c("", "Folder1/File8.txt", "Folder7/file2.txt", "Folder9/File19.txt"), class = "factor")), .Names = c("Name", "Metadata", "Filepath"), class = "data.frame", row.names = c(NA, -4L))

data
    Name Metadata           Filepath
1  Test1    sgash  Folder1/File8.txt
2  Test2  uashas8  Folder7/file2.txt
3  Test6  asdajl7 Folder9/File19.txt
4 Test10   asfhas

In order to make a reproducible example I tried to implement the following function to adjust the filepath to the place where you saved the Folder structure from the Download above.

# Choose path to unzipped Data directory
choose.dir <- function() {
  system("osascript -e 'tell app \"R\" to POSIX path of (choose folder with prompt \"Choose Data Folder:\")' > /tmp/R_folder",
         intern = FALSE, ignore.stderr = TRUE)
  p <- system("cat /tmp/R_folder && rm -f /tmp/R_folder", intern = TRUE)
  return(ifelse(length(p), p, NA))
}
a <- choose.dir()
if(is.na(a)) stop("No folder", call. = F)
# paste ready to use path together
data$completepath <- paste0(a,"/",data$Filepath)
data$completepath <- gsub("//", "/", data$completepath)

The data.frame now looks like this (I unzipped the folder structure to my Desktop):

data
    Name Metadata           Filepath                               completepath
1  Test1    sgash  Folder1/File8.txt  /Users/XYZ/Desktop/Data/Folder1/File8.txt
2  Test2  uashas8  Folder7/file2.txt  /Users/XYZ/Desktop/Data/Folder7/file2.txt
3  Test6  asdajl7 Folder9/File19.txt /Users/XYZ/Desktop/Data/Folder9/File19.txt
4 Test10   asfhas                                      /Users/XYZ/Desktop/Data/

How could I read in the data from the different .txt files using a loop so that I get the following list structure?

   List with 3 elements
        1.1 (5 observations and 5 Variables)
        $Name chr[1:5] Test1 Test1 Test1 Test1 Test1
        $Year num[1:5] 1783 1784 1785 1786 1787
        $data1 num[1:5] 12 53 13.1 12.9 16
        $data2 num[1:5] 56 5 532 27 9
        $data3 num[1:5] 0.1 9 42 2 13
        1.2 (4 observations and 3 variables)
        $Name chr[1:4] Test2 Test2 Test2 Test2
        $Year num[1:4] 1387 1388 1389 1390
        $data num[1:4] 78.9 27 12.3 0.9
        1.3 (3 observations and 3 variables)
        $Name chr[1:3] Test6 Test6 Test6
        $Test1 chr[1:3] hajshf asfhah ashsa
        $Year num[1:3] 2001 2002 2003

What I tried is the following, but this doesn't work as the empty filepath of Test10 is causing problems. Can someone help me?

# read in the data
f <- file.path(data$completepath)
d <- lapply(f, read.table)

Answer 1

You don't have to do it this way, you could write the path out instead

setwd("/home/christie/Downloads/Data/")

This will give all of the paths to every file in the working directory

files<-list.files(getwd(),recursive=TRUE)

This reads them all into the list

d<-lapply(files,function(x) read.table(x, header=T))

str(d)
List of 3
 $ :'data.frame':   5 obs. of  4 variables:
  ..$ data1: num [1:5] 12 53 13.1 12.9 16
  ..$ data2: int [1:5] 56 5 532 27 9
  ..$ data3: num [1:5] 0.1 9 42 2 13
  ..$ year : int [1:5] 1783 1784 1785 1786 1787
 $ :'data.frame':   4 obs. of  2 variables:
  ..$ data: num [1:4] 78.9 27 12.3 0.9
  ..$ year: int [1:4] 1387 1388 1389 1390
 $ :'data.frame':   3 obs. of  3 variables:
  ..$ data1: int [1:3] 18 39 371
  ..$ Test1: Factor w/ 3 levels "asfhah","ashsa",..: 3 1 2
  ..$ Year : int [1:3] 2001 2002 2003