如果Date超出给定间隔，则在数据框中将值设置为NA

Question

I have two dataframes, df1 and df2 .

df1 contains values for different products X1 , X2 ,and so on at different times. df2 contains the true start and end date for some of the products. I want to replace the values outside of the given date intervals in df2 by NA , as shown in the final table df3 .

Create df1 and df2 :

df1=data.frame(matrix(NA,10,6))
df1[,1]=(c(seq(as.Date("2012-01-01"),as.Date("2012-10-01"),by="1 month")))
df1[,2]=c(1:10); df1[,3]=c(12:21); df1[,4]=c(0.5:10); df1[,5]=c(5:14); df1[,6]=c(10:19)
colnames(df1)=c("Date","X1","X2","X3","X4","X5")
df2=data.frame(matrix(data=c("X1","X2","X4","2012-02-01","2012-04-01","2012-06-01","2012-09-01","2012-06-01","2012-10-01"),3,3))
colnames(df2)=c("Name","Start","End")

Output:

   > df1
         Date X1 X2  X3 X4 X5
1  2012-01-01  1 12 0.5  5 10
2  2012-02-01  2 13 1.5  6 11
3  2012-03-01  3 14 2.5  7 12
4  2012-04-01  4 15 3.5  8 13
5  2012-05-01  5 16 4.5  9 14
6  2012-06-01  6 17 5.5 10 15
7  2012-07-01  7 18 6.5 11 16
8  2012-08-01  8 19 7.5 12 17
9  2012-09-01  9 20 8.5 13 18
10 2012-10-01 10 21 9.5 14 19
> df2
  Name      Start        End
1   X1 2012-02-01 2012-09-01
2   X2 2012-04-01 2012-06-01
3   X4 2012-06-01 2012-10-01

Final output should look like this:

 df3
       Date  X1  X2  X3 X4 X5
1  2012-01-01 NA NA 0.5 NA 10
2  2012-02-01  2 NA 1.5 NA 11
3  2012-03-01  3 NA 2.5 NA 12
4  2012-04-01  4 15 3.5 NA 13
5  2012-05-01  5 16 4.5 NA 14
6  2012-06-01  6 17 5.5 10 15
7  2012-07-01  7 NA 6.5 11 16
8  2012-08-01  8 NA 7.5 12 17
9  2012-09-01  9 NA 8.5 13 18
10 2012-10-01 NA NA 9.5 14 19

Answer 1

I am sure there is a more elegant way, but you could create a matrix of the indices that meet your criterion, where you set the elements to 1 if it is within your interval for that product and NA if it isn't. Assuming you are dealing with numerical values you can then multiply your data frame with that index matrix:

Example:

library(dplyr)
## Convert your dates to Date-objects:
df2 <- df2 %>% dplyr::mutate(Start = as.Date(Start), End = as.Date(End))

## Create a matrix of indices (TRUE/FALSE):
indMx <- lapply(names(df1)[-1], function(product){
            (df1$Date >= df2$Start[df2$Name == product]) & 
                    (df1$Date <= df2$End[df2$Name == product]) 
        }) %>% do.call('cbind',.)

## Multiply with NA^indMx, which gives you NA in place of FALSE and 
## 1 in place of TRUE:
df1[,-1] <- df1[,-1]*NA^indMx

df1
#          Date X1 X2  X3
# 1  2012-01-01  1 12 0.5
# 2  2012-02-01 NA 13 1.5
# 3  2012-03-01 NA 14 2.5
# 4  2012-04-01 NA NA 3.5
# 5  2012-05-01 NA NA 4.5
# 6  2012-06-01 NA NA  NA
# 7  2012-07-01 NA 18  NA
# 8  2012-08-01 NA 19  NA
# 9  2012-09-01 NA 20  NA
# 10 2012-10-01 10 21  NA

Answer 2

Here is one solution with data.table . There might be a more elegant method using non-equi joins.

for(i in seq_len(nrow(df2))) df1[!(Date %between% df2[i,.(Start, End)]), df2[i, Name] := NA]

Here, you run through each row of df2, subset df1 based on dates outside of the start and end dates in the current row of df2, and then assign NA to the variable given in df2.

This returns

df1
          Date X1 X2  X3
 1: 2012-01-01 NA NA  NA
 2: 2012-02-01  2 NA  NA
 3: 2012-03-01  3 NA  NA
 4: 2012-04-01  4 15  NA
 5: 2012-05-01  5 16  NA
 6: 2012-06-01  6 17 5.5
 7: 2012-07-01  7 NA 6.5
 8: 2012-08-01  8 NA 7.5
 9: 2012-09-01  9 NA 8.5
10: 2012-10-01 NA NA 9.5

---

update

If the data is constructed as was updated in the original post, then run this line first to convert the Names variable in df2 to a character vector (starts out as a factor). Then the above code will work for the new dataset.

# convert data.frames to data.tables
setDT(df1)
setDT(df2)

# convert factor to character
df2[, Name := as.character(Name)]

---

data

library(data.table)
# read in data
df1 <- fread("Date X1 X2  X3
2012-01-01  1 12 0.5
2012-02-01  2 13 1.5
2012-03-01  3 14 2.5
2012-04-01  4 15 3.5
2012-05-01  5 16 4.5
2012-06-01  6 17 5.5
2012-07-01  7 18 6.5
2012-08-01  8 19 7.5
2012-09-01  9 20 8.5
2012-10-01 10 21 9.5")

df2 <- fread("  Name      Start        End
X1 2012-02-01 2012-09-01
X2 2012-04-01 2012-06-01
X3 2012-06-01 2012-10-01")

# convert to date type
df1[, Date := as.Date(Date)]
df2[, c("Start", "End")  := .(as.Date(Start), as.Date(End))]

Answer 3

Using dplyr and tidyr ...

library(tidyr)
library(dplyr)

df3 <- df1 %>% gather(key=Name,value=value,-Date) %>% #convert to long form
  left_join(df2) %>% #merge in date limits
  mutate(ind=(as.Date(Date)>=as.Date(Start) & as.Date(Date)<=as.Date(End))) %>% #check valid 
  mutate(value=replace(value,!ind,NA)) %>% #replace invalid with NA
  select(Date,Name,value) %>% #remove unnecessary variables
  spread(key=Name,value=value) #convert back to rectangular form

df3
         Date X1 X2  X3 X4 X5
1  2012-01-01 NA NA 0.5 NA 10
2  2012-02-01  2 NA 1.5 NA 11
3  2012-03-01  3 NA 2.5 NA 12
4  2012-04-01  4 15 3.5 NA 13
5  2012-05-01  5 16 4.5 NA 14
6  2012-06-01  6 17 5.5 10 15
7  2012-07-01  7 NA 6.5 11 16
8  2012-08-01  8 NA 7.5 12 17
9  2012-09-01  9 NA 8.5 13 18
10 2012-10-01 NA NA 9.5 14 19