R时间加上开始/停止时间

Question

I have a set of time series data that has a start and stop time. Each event can last from few seconds to few days, I need to calculate the sum, in this example the total memory used, every hour of the jobs active at the time. Here is a sample of the data:

mem_used    start_time            stop_time 
16          2015-10-24 17:24:41   2015-10-25 04:19:44   
80          2015-10-24 17:24:51   2015-10-25 03:14:59   
44          2015-10-24 17:25:27   2015-10-25 01:16:10   
28          2015-10-24 17:25:43   2015-10-25 00:00:31   
72          2015-10-24 17:30:23   2015-10-24 23:58:31   

In this case it should give something like:

time                  total_mem
2015-10-24 17:00:00   240
2015-10-24 18:00:00   240
...
2015-10-25 00:00:00   168
2015-10-25 01:00:00   140
2015-10-25 02:00:00   96
2015-10-25 03:00:00   96
2015-10-25 04:00:00   16

I'm trying to do something with the aggregate function but I can not figure it out. Any ideas? Thanks.

Answer 1

Here's how I would do it, using lubridate .

First, make sure that your dates are in POSIXct format:

dat$start_time = as.POSIXct(dat$start_time, format = "%Y-%m-%d %H:%M:%S")
dat$stop_time = as.POSIXct(dat$stop_time, format = "%Y-%m-%d %H:%M:%S")

Then make an interval object with lubridate:

library(lubridate)
dat$interval <- interval(dat$start_time, dat$stop_time)

Now we can make a vector of times, replace these with your desired times:

z <- seq(start = dat$start_time[1], stop = dat$stop_time[5], by = "hours")

And sum those where we have an overlap:

out <- data.frame(times = z,
                  mem_used = sapply(z, function(x) sum(dat$mem_used[x %within% dat$interval]))) 

                times mem_used
1 2015-10-24 17:24:41       16
2 2015-10-24 18:24:41      240
3 2015-10-24 19:24:41      240
4 2015-10-24 20:24:41      240
5 2015-10-24 21:24:41      240
6 2015-10-24 22:24:41      240
7 2015-10-24 23:24:41      240

Here's the data used:

structure(list(mem_used = c(16L, 80L, 44L, 28L, 72L), start_time = structure(c(1445721881, 
1445721891, 1445721927, 1445721943, 1445722223), class = c("POSIXct", 
"POSIXt"), tzone = ""), stop_time = structure(c(1445761184, 1445757299, 
1445750170, 1445745631, 1445745511), class = c("POSIXct", "POSIXt"
), tzone = "")), .Names = c("mem_used", "start_time", "stop_time"
), row.names = c(NA, -5L), class = "data.frame")

Answer 2

Here is another solution based on dplyr and lubridate . Make sure first to have the data in the right format (eg date in POSIXct )

library(dplyr)
library(lubridate)

glimpse(df)
## Observations: 5
## Variables: 3
## $ mem_used   (int) 16, 80, 44, 28, 72
## $ start_time (time) 2015-10-24 17:24:41, 2015-10-24 17:24:51...
## $ end_time   (time) 2015-10-25 04:19:44, 2015-10-25 03:14:59...

Then we will just keep the hour (removing minutes and seconds) since we want to aggregate per hour.

### Remove minutes and seconds
minute(df$start_time) <- 0
second(df$start_time) <- 0
minute(df$end_time) <- 0
second(df$end_time) <- 0

The most important step now, is to create a new data.frame with one row for each hour between start_time and end_time . For example, if on the first line of the original data.frame we have 5 hours between start_time and end_time , we will end with 5 rows and the value mem_used duplicated 5 times.

###
n <- nrow(df)
l <- lapply(1:n, function(i) {
  date <- seq.POSIXt(df$start_time[i], df$end_time[i], by = "hour")
  mem_used <- rep(df$mem_used[i], length(date))
  data.frame(time = date, mem_used = mem_used)
})


df <- Reduce(rbind, l)
glimpse(df)
## Observations: 47
## Variables: 2
## $ time     (time) 2015-10-24 17:00:00, 2015-10-24 18:00:00, ...
## $ mem_used (int) 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,...

Finally, we can now aggregate using dplyr or aggregate (or other similar functions)

df %>%
  group_by(time) %>%
  summarise(tot = sum(mem_used))
##                   time   tot
##                 (time) (int)
## 1  2015-10-24 17:00:00   240
## 2  2015-10-24 18:00:00   240
## 3  2015-10-24 19:00:00   240
## 4  2015-10-24 20:00:00   240
## 5  2015-10-24 21:00:00   240
## 6  2015-10-24 22:00:00   240
## 7  2015-10-24 23:00:00   240
## 8  2015-10-25 00:00:00   168
## 9  2015-10-25 01:00:00   140
## 10 2015-10-25 02:00:00    96
## 11 2015-10-25 03:00:00    96
## 12 2015-10-25 04:00:00    16

## Or aggregate
aggregate(mem_used ~ time, FUN = sum, data = df)