Why does xts shift a date one day back when creating an xts object from a data frame when TZ is not specified?

Question

Let me start by saying that I took a look at ?xts , realised that this is a timezone related problem and seem to have resolved it, but I don't understand why it was happening. So: I have a simple data frame of price data. When I convert it to an xts object the first date of the xts object is a day earlier than the first date in the data frame. If I specify the time zone the dates match problem disappears. I thought at first it might be because xts() assumes that an order.by date without TZ specified is UMT, and Sys.timezone() gives "JST" for me but I don't see why that would lead to a date that is a full day earlier...?

Q. Why is this happening?

require(xts)
aa <- structure(list(Date = structure(c(6822, 6823, 6824, 6825, 6826,
6829), class = "Date"), Open = c(2145, 2126, 2130, 2148, 2144,
2137), High = c(2148, 2131, 2141, 2152, 2146, 2151), Low = c(2124,
2111, 2128, 2140, 2135, 2136), Close = c(2124, 2120, 2141, 2140,
2140, 2149), Volume = c(0L, 0L, 0L, 0L, 0L, 0L)), .Names = c("Date",
"Open", "High", "Low", "Close", "Volume"), row.names = c(NA,
6L), class = "data.frame")

str(aa)
aa

bb <- xts(aa[5], order.by = aa$Date)
str(bb)
bb ## first date is a day earlier than the first day of the data frame

bb <- xts(aa[5], order.by = aa$Date, tzone = Sys.getenv("TZ"))
str(bb)
bb ## first dates in xts object and data frame match...

This is on:

sessionInfo():
R version 2.15.1 (2012-06-22)

Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United Kingdom.1252
[2] LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] grid      stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
 [1] gridExtra_0.9.1 scales_0.2.2    plyr_1.7.1      ggplot2_0.9.2.1
 [5] lubridate_1.2.0 quantmod_0.3-17 TTR_0.21-1      xts_0.8-8      
 [9] zoo_1.7-9       Defaults_1.1-1 

loaded via a namespace (and not attached):
 [1] colorspace_1.2-0   dichromat_1.2-4    digest_0.5.2       gtable_0.1.1      
 [5] labeling_0.1       lattice_0.20-10    MASS_7.3-22        memoise_0.1       
 [9] munsell_0.4        proto_0.3-9.2      RColorBrewer_1.0-5 reshape2_1.2.1    
[13] stringr_0.6.1     

Answer 1

I do not know, and I cannot exactly reproduce your problem, but I believe it has to do with the Date s being coerced to POSIXct and back.

This line is in the code for the xts function:

if (inherits(order.by, "Date") && !missing(tzone)) 
    order.by <- .POSIXct(unclass(order.by) * 86400, tz = tzone)

In your first call, tzone is missing, so this code is not executed. In your second call, tzone is not missing , so it is executed.

If you step through the code in xts.R, you can see that (if tzone is missing ) the Dates in aa$Date are coerced to POSIXct .

index <- as.numeric(as.POSIXct(order.by))

---

I think the issue is that as.Date.POSIXct has a default of tz="UTC" , so that is used unless you specify a different one.

x <- structure(1290125760, 
               tzone = structure("America/Chicago", .Names = "TZ"), 
               tclass = c("POSIXt", "POSIXct"), 
               class = c("POSIXct", "POSIXt"))
x
#[1] "2010-11-18 18:16:00 CST"
str(x)
#POSIXct[1:1], format: "2010-11-18 18:16:00"
as.Date(x)
#[1] "2010-11-19"
as.Date(x, origin=as.Date("1970-01-01"), tz="America/Chicago")
#[1] "2010-11-18"