To extract date, hour and weekday from a UTC timestamp -R

Question

I have a database with a column for created time in UTC, eg

created_utc
1 1430438400
2 1430438410
3 1430438430
4 1430438455
5 1430438470
6 1430438480

I want to extract date, hour and is.weekend into separate columns. I have tried,

db_subset %>% mutate(hour = as.POSIXlt(created_utc, origin ='1970-01-01')$hour)

but it fails to recognise the created_utc object. I tried coercing it into a data frame and then,

df_comments <- db_subset %>% 
                select(created_utc) %>%
                        collect() %>%
                            data.frame() %>%
                               mutate(hour = as.POSIXlt(created_utc, origin ='1970-01-01')$hour)

but it fails with error: invalid subscript type 'closure'

Can somebody help where am I going wring and how can I extract the hours,date etc?

Answer 1

One option if we are using dplyr would be to convert to POSIXct (as the POSIXlt class is not supported) and extract the hour using lubridate .

library(lubridate)
library(dplyr)
db_subset %>%
    mutate(hour=hour(as.POSIXct(created_utc, origin='1970-01-01')))
#   created_utc hour
#1  1430438400   20
#2  1430438410   20
#3  1430438430   20
#4  1430438455   20
#5  1430438470   20
#6  1430438480   20

data

db_subset <- structure(list(created_utc = c(1430438400L, 1430438410L, 
1430438430L, 
1430438455L, 1430438470L, 1430438480L)), .Names = "created_utc", 
class = "data.frame", row.names = c("1", "2", "3", "4", "5", "6"))

Answer 2

I wold first suggest to convert your created_utc to a POSIXct class (instead of POSIXlt and then extract all the data you need. Here's a simple illustration using the data.table package

library(data.table)
setDT(df)[, created_utc := as.POSIXct(created_utc, origin = '1970-01-01')]
df[, `:=`(Date = as.Date(created_utc),
          Hour = hour(created_utc),
          isWeekend = wday(created_utc) %in% c(7L, 1L))]
df
#            created_utc       Date Hour isWeekend
# 1: 2015-05-01 03:00:00 2015-05-01    3     FALSE
# 2: 2015-05-01 03:00:10 2015-05-01    3     FALSE
# 3: 2015-05-01 03:00:30 2015-05-01    3     FALSE
# 4: 2015-05-01 03:00:55 2015-05-01    3     FALSE
# 5: 2015-05-01 03:01:10 2015-05-01    3     FALSE
# 6: 2015-05-01 03:01:20 2015-05-01    3     FALSE

Answer 3

All of this can be done with base R:

R> df <- data.frame(created_utc=c(1430438400, 1430438410, 1430438430,
+                                 1430438455, 1430438470, 1430438480))
R> df
  created_utc
1  1430438400
2  1430438410
3  1430438430
4  1430438455
5  1430438470
6  1430438480
R> 
R> # so far so good -- we just have the data
R> # so let's make it a date time object 
R> 
R> df[,1] <- as.POSIXct(df[,1], origin="1970-01-01")
R> df
          created_utc
1 2015-04-30 19:00:00
2 2015-04-30 19:00:10
3 2015-04-30 19:00:30
4 2015-04-30 19:00:55
5 2015-04-30 19:01:10
6 2015-04-30 19:01:20
R> 
R> ## we can use this to extract Date, Hour and Weekend computations
R> 
R> df[,"date"] <- as.Date(df[,1])
R> df[,"hour"] <- as.POSIXlt(df[,1])$hour
R> df[,"isWeekend"] <- as.POSIXlt(df[,1])$wday < 1 || as.POSIXlt(df[,1])$wday > 5
R> df
          created_utc       date hour isWeekend
1 2015-04-30 19:00:00 2015-05-01   19     FALSE
2 2015-04-30 19:00:10 2015-05-01   19     FALSE
3 2015-04-30 19:00:30 2015-05-01   19     FALSE
4 2015-04-30 19:00:55 2015-05-01   19     FALSE
5 2015-04-30 19:01:10 2015-05-01   19     FALSE
6 2015-04-30 19:01:20 2015-05-01   19     FALSE
R>