滚动平均值与不同数量的观察值

Question

I'm trying to construct a rolling mean for a dataset over the past 6 months. The data is on a daily basis and has more than 100.000 rows from which I provided a sample below.

# A tibble: 100 × 5
       ID    MONTH       DATE VALUE   R_MEAN
   <fctr>    <dbl>     <date> <dbl>    <dbl>
1     634 20160200 2016-02-03     2 0.000000
2    1700 20150300 2015-03-02     3 0.000000
3    1700 20150400 2015-04-01     7 3.000000
4    1700 20150400 2015-04-09     1 5.000000
5    1700 20150700 2015-07-02    26 3.666667
6    1700 20150800 2015-08-03     1 9.250000
7    1700 20150900 2015-09-01     2 7.600000
8    1700 20151000 2015-10-01     5 7.400000
9    1700 20151000 2015-10-07    10 7.833333
10   1700 20151100 2015-11-02     8 8.800000
# ... with 90 more rows

My goal is to create a moving average over the past 6 months, so for example for an ID: X and DATE value of 20160101 I want to get the average VALUE of all rows which have the same ID and where the DATE value is between 20150601 and 20160101. When no previous values are available I assume an average value of zero.

I thought of using some sort of expanding grid approach, but as I have a lot of ID's (close to 30.000), expanding the grid on aa daily basis over a period of 2 years would result in an enormous grid.

Answer 1

Here I use dplyr . I inner_join the table on itself, then filter the relevant previous rows, per row in the source data, and calculate the mean value.

Finally I left_join the original data on the processed data and replace NA using coalesce .

The 6 months window is calculated by substracting 182 days from the DATE . You could also use lubridate to make it a period in months. Personally I prefer to work with a fixed window of days, that does not depend on the different amount of days each month has.

str <- '
row ID  MONTH DATE  VALUE R_MEAN
1 634 20160200 2016-02-03     2 0.000000
2 1700 20150300 2015-03-02     3 0.000000
3 1700 20150400 2015-04-01     7 3.000000
4 1700 20150400 2015-04-09     1 5.000000
5 1700 20150700 2015-07-02    26 3.666667
6 1700 20150800 2015-08-03     1 9.250000
7 1700 20150900 2015-09-01     2 7.600000
8 1700 20151000 2015-10-01     5 7.400000
9 1700 20151000 2015-10-07    10 7.833333
10  1700 20151100 2015-11-02     8 8.800000
'

file <- textConnection(str)

raw <- read.table(file, header = T)

library(dplyr)

df <- raw %>% mutate(DATE = as.Date(DATE,'%Y-%m-%d'))

prev <- df %>% inner_join(df, by = 'ID') %>%
  filter(DATE.y > DATE.x-182, DATE.y < DATE.x) %>%
  group_by(row.x) %>% summarise(meanVALUE = mean(VALUE.y)) %>%
  rename(row = row.x)

df %>% left_join(prev, by='row') %>% mutate(meanVALUE = coalesce(meanVALUE,0))

result:

   row   ID    MONTH       DATE VALUE   R_MEAN meanVALUE
1    1  634 20160200 2016-02-03     2 0.000000  0.000000
2    2 1700 20150300 2015-03-02     3 0.000000  0.000000
3    3 1700 20150400 2015-04-01     7 3.000000  3.000000
4    4 1700 20150400 2015-04-09     1 5.000000  5.000000
5    5 1700 20150700 2015-07-02    26 3.666667  3.666667
6    6 1700 20150800 2015-08-03     1 9.250000  9.250000
7    7 1700 20150900 2015-09-01     2 7.600000  8.750000
8    8 1700 20151000 2015-10-01     5 7.400000  7.500000
9    9 1700 20151000 2015-10-07    10 7.833333  7.000000
10  10 1700 20151100 2015-11-02     8 8.800000  8.800000

Answer 2

Maybe this helps:

   for (i in 1:levels(df$ID))
     mean(df$value[df$DATE>(Sys.date()-182) & 
                   df$ID==levels(df$ID)[i]],
           na.rm=T)