merge many csv files with different columns and headers in R

Question

I have a bit of a complex issue and appreciate any advice. I have about 100 csv files corresponding to different years of data collection that I have brought into R as a list. I need to merge them, but the files do not all have the same number of columns, the columns are not in the same order, and some column headings are uppercase while others are lowercase. Here is a shortened example of the datasets within the list:

ID_code = c(1,2,3,4,5)
stuff = sample(x = c("A", "B", "C"), size = 5, replace = TRUE)
THINGS = sample(x = c(5, 10, 27), size = 5, replace = TRUE)
extras = rnorm(5, mean=0)
dat1 = as.data.frame(cbind(ID_code, stuff, THINGS, extras))

ID_code = c(6,7,8,9,10)
THINGS = sample(x = c(5, 10, 27), size = 5, replace = TRUE)
stuff = sample(x = c("A", "B", "C"), size = 5, replace = TRUE)
dat2 = as.data.frame(cbind(ID_code, THINGS, stuff))

ID_code = c(11,12,13,14,15)
EXTRAS = rnorm(5, mean=0)
stuff = sample(x = c("A", "B", "C"), size = 5, replace = TRUE)
things = sample(x = c(5, 10, 27), size = 5, replace = TRUE)
dat3 = as.data.frame(cbind(ID_code, EXTRAS, stuff, things))

The ID_code is unique for each row because it includes the year and other factors. So for this example I want one file with 15 rows for each ID_code and all columns (stuff, things, and extras). How would I merge all the files in the list and set all the column headings to lowercase so R doesn't think "things" and "THINGS" are different columns?

Answer 1

We can get all the datasets starts with dat and ends with digits ( \\d+ ) into a list ( mget ), change the column names to lower case ( tolower ), merge them within Reduce

Reduce(function(...) merge(..., all = TRUE),
        lapply(mget(ls(pattern = '^dat\\d+$')), 
     function(x) setNames(x, tolower(names(x)))))

---

Or it could be bind_rows

library(dplyr)
library(purrr)
mget(ls(pattern = '^dat\\d+$')) %>%
      map_dfr(~ .x %>%
                rename_all(tolower))