从R中的数据框列表中匹配日期

Question

I have a list of 100+ time series dataframes my.list with daily observations for each product in its own data frame. Some values are NA without any record of the date. I would like to update each data frame in this list to show the date and NA if it does not have a record on this date.

Dates:

start = as.Date('2016/04/08')
full <- seq(start, by='1 days', length=10)

Sample Time Series Data:

d1 <- data.frame(Date = seq(start, by ='2 days',length=5), Sales = c(5,10,15,20,25))
d2 <- data.frame(Date = seq(start, by= '1 day', length=10),Sales = c(1, 2, 3,4,5,6,7,8,9,10))
my.list <- list(d1, d2)

I want to merge all full date values into each data frame, and if no match exists then sales is NA:

   my.list

[[d1]]
Date    Sales
2016-04-08    5
2016-04-09    NA
2016-04-10    10
2016-04-11    NA
2016-04-12    15
2016-04-13    NA
2016-04-14    20
2016-04-15    NA
2016-04-16    25
2016-04-17    NA


[[d2]]
Date    Sales
2016-04-08    1
2016-04-09    2
2016-04-10    3
2016-04-11    4
2016-04-12    5
2016-04-13    6
2016-04-14    7
2016-04-15    8
2016-04-16    9
2016-04-17    10

Answer 1

If I understand correctly, the OP wants to update each of the dataframes in my.list to contain one row for each date given in the vector of dates full

Base R

In base R, merge() can be used as already mentioned by Hack-R . However, th answer below expands this to work on all dataframes in the list:

# creat dataframe from vector of full dates
full.df <- data.frame(Date = full)
# apply merge on each dataframe in the list
lapply(my.list, merge, y = full.df, all.y = TRUE)

 [[1]] Date Sales 1 2016-04-08 5 2 2016-04-09 NA 3 2016-04-10 10 4 2016-04-11 NA 5 2016-04-12 15 6 2016-04-13 NA 7 2016-04-14 20 8 2016-04-15 NA 9 2016-04-16 25 10 2016-04-17 NA [[2]] Date Sales 1 2016-04-08 1 2 2016-04-09 2 3 2016-04-10 3 4 2016-04-11 4 5 2016-04-12 5 6 2016-04-13 6 7 2016-04-14 7 8 2016-04-15 8 9 2016-04-16 9 10 2016-04-17 10 

Caveat

The answer assumes that full covers the overall range of Date of all dataframes in the list.

In order to avoid any mishaps, the overall range of Date can be retrieved from the available data in my.list :

overall_date_range <- Reduce(range, lapply(my.list, function(x) range(x$Date)))
full <- seq(overall_date_range[1], overall_date_range[2], by = "1 days")

---

Using rbindlist()

Alternatively, the list of dataframes which are identical in structure can be stored in a large dataframe. An additional attribute indicates to which product each row belongs to. The homogeneous structure simplifies subsequent operations.

The code below uses the rbindlist() function from the data.table package to create a large data.table . CJ() ( cross join ) creates all combinations of dates and product id which is then merged / joined to fill in the missing dates:

library(data.table)
all_products <- rbindlist(my.list, idcol = "product.id")[
  CJ(product.id = unique(product.id), Date = seq(min(Date), max(Date), by = "1 day")), 
  on = .(Date, product.id)]
all_products

  product.id Date Sales 1: 1 2016-04-08 5 2: 1 2016-04-09 NA 3: 1 2016-04-10 10 4: 1 2016-04-11 NA 5: 1 2016-04-12 15 6: 1 2016-04-13 NA 7: 1 2016-04-14 20 8: 1 2016-04-15 NA 9: 1 2016-04-16 25 10: 1 2016-04-17 NA 11: 2 2016-04-08 1 12: 2 2016-04-09 2 13: 2 2016-04-10 3 14: 2 2016-04-11 4 15: 2 2016-04-12 5 16: 2 2016-04-13 6 17: 2 2016-04-14 7 18: 2 2016-04-15 8 19: 2 2016-04-16 9 20: 2 2016-04-17 10 

Subsequent operations can be grouped by product.id , eg, to determine the number of valid sales data for each product:

all_products[!is.na(Sales), .(valid.sales.data = .N), by = product.id]

  product.id valid.sales.data 1: 1 5 2: 2 10 

Or, the totals sales per product:

all_products[, .(total.sales = sum(Sales, na.rm = TRUE)), by = product.id]

  product.id total.sales 1: 1 75 2: 2 55 

If required for some reason the result can be converted back to a list by

split(all_products, by = "product.id")