R for loop works only for one file

Question

I have been trying to run a for loop for multiple csv files in R. But the loop only runs for the first file.

I want to import csv files and then create a directory for each csv file where the analysis of its data will be stored. After creating directories, I'm having problem with setting that as a working directory each time I run my code. My code works fine when it is just one file but it fails when I use the for loops.

Code:

## Setting the working directory and path
setwd("path")
path <- "path"

##to extract the filename from each path
files <- list.files(path=path, pattern="*.csv")
for(file in files)
{
  temp <- which(strsplit(file, "")[[1]]==".")
  assign(
    gsub(" ","",substr(file, 1, temp-1)), 
    read.csv(paste(path,file,sep="")))
}

##To create a new directory for each file and set that as the new working directory.

    for(i in seq(1, length(files), by = 1)){
      fileName <- read.csv(files[i])
      base <- strsplit(files[i], ".csv")[[1]]
      dir <- dir.create(paste(path,base, sep = "/"))
      setwd(getwd(dir))

Further analysis with results stored in the newly set working directory.

Creating variables

Date_Time <- strptime(fileName$Date...Time, format = "%d/%m/%Y %H:%M")
  fileName$month <- months(Date_Time,abbreviate = TRUE)  #creates month column (char)
  fileName$julian <- Date_Time$yday  #creates julian day column
  fileName$year <- Date_Time$year + 1900  #creates year column
  fileName$hour <- Date_Time$hour  #creates hour column
  fileName$weeknum <- round(Date_Time$yday/7,0)
  fileName$numericdate <- fileName$year+fileName$julian/366  #numeric value of date

  #Identify and remove empty columns
  fileName <- as.data.table(fileName)
  fileName <- fileName[,which(unlist(lapply(fileName, function(x)!all(is.na(x))))),with=F]
  dim(fileName) # to check if empty columns have been eliminated
  head(fileName) #to find appropriate column name for PM10 data
  PM10 <- fileName$PM10_BAM #substitue in a common variable for further calculations
  fileName$PM10_BAM <- as.numeric(as.character(PM10))

  ##to view basic seasonal pattern through the data
  df_eve <- subset(fileName, hour>=18)
  jpeg(file = "seasonal pattern observed in the evenings.jpg")
  with(df_eve, boxplot(PM10_BAM ~ weeknum, main = "seasonal pattern observed in the evenings", xlab = "weeknum", ylab = "PM10", outline = FALSE, na.rm = T))
  dev.off()
}


Errors:

Error in file(file, "rt") : cannot open the connection In addition: Warning messages:
1: In dir.create(paste(path, base, sep = "/")) :
  '/Users/ayushikachhara/Desktop/Work/CSV//EW_Matamata' already exists
2: NAs introduced by coercion 
3: In file(file, "rt") :
  cannot open file 'EW_Ngaruawahia.csv': No such file or directory

EW_Matamata and EW_Ngaruawahia are the files in the initially set working directory. But since I import them and then change the directory, I don't understand why I keep getting the 3rd error message.

Any help is appreciated since I'm at a learning stage :)

Answer 1

Check this line of your code .

      dir <- dir.create(paste(path,base, sep = "/"))
      setwd(getwd(dir))

Now when it gets the first file it creates a New Dir and sets the working dir as the Directory of the newly created Dir .So when it looks for the second file in the current Dir ,its Obviously is not present there