在过去x分钟间隔内滚动时间序列的最大/最小/总和

Question

I have a financial time series data.frame with microsecond precision:

timestamp                    price  volume
2017-08-29 08:00:00.345678   99.1   10
2017-08-29 08:00:00.674566   98.2   5
....
2017-08-29 16:00:00.111234   97.0   3
2017-08-29 16:00:01.445678   96.5   5

In total: around 100k records per day.

I saw a couple of functions where I can specify the width of the rolling windows, eg k = 10. But the k is expressed as a number of observations and not minutes.

I need to calculate runing/rolling Max, Min of Price series and a runing/rolling sum of Volume series like that:

1. starting with a timestamp exactly 5 minutes after the begin of the time series
2. for every following timestamp: look back for 5 minutes interval and
3. calculate the rolling statistics.

How to calculate this effectivly?

Answer 1

Your data

I wasn't able to capture milliseconds (but the solution should still work)

library(lubridate)
df <- data.frame(timestamp = ymd_hms("2017-08-29 08:00:00.345678", "2017-08-29 08:00:00.674566", "2017-08-29 16:00:00.111234", "2017-08-29 16:00:01.445678"),
                 price=c(99.1, 98.2, 97.0, 96.5),
                 volume=c(10,5,3,5))

purrr and dplyr solution

library(purrr)
library(dplyr)
timeinterval <- 5*60   # 5 minute

Filter df for observations within time interval, save as list

mdf <- map(1:nrow(df), ~df[df$timestamp >= df[.x,]$timestamp & df$timestamp < df[.x,]$timestamp+timeinterval,])

Summarise for each data.frame in list

statdf <- map_df(mdf, ~.x %>% 
                          summarise(timestamp = head(timestamp,1),
                                    max.price = max(price), 
                                    max.volume = max(volume),
                                    sum.price = sum(price),
                                    sum.volume = sum(volume),
                                    min.price = min(price), 
                                    min.volume = min(volume)))

Output

                timestamp max.price max.volume sum.price sum.volume
1 2017-08-29 08:00:00      99.1         10     197.3         15
2 2017-08-29 08:00:00      98.2          5      98.2          5
3 2017-08-29 16:00:00      97.0          5     193.5          8
4 2017-08-29 16:00:01      96.5          5      96.5          5
  min.price min.volume
1      98.2          5
2      98.2          5
3      96.5          3
4      96.5          5

Answer 2

As I was looking for a backward calculation (start with a timestamp and look 5 minutes back) I slightly modified the great solution by #CPak as follows:

mdf <- map(1:nrow(df), ~df[df$timestamp <= df[.x,]$timestamp & df$timestamp > df[.x,]$timestamp - timeinterval,])

statdf <- map_df(mdf, ~.x %>% 
                      summarise(timestamp_to = tail(timestamp,1),
                                timestamp_from = head(timestamp,1),
                                max.price = max(price), 
                                min.price = min(price),
                                sum.volume = sum(volume),
                                records = n()))

In addition, I added records = n() to see how many records have been used in the intervals.

One caveat: the code takes 10 mins on mdf and another 6 mins for statdf on a dataset with 100K+ records.

Any ideas how to optimize it? Thank you!