[datatable-help] Automagic timezone convertion on data.table rbind

[Nicolas Chapados]

Hi Tom,

Thanks for your reply, and for the trick of forcibly establishing the
timezone to GMT.

I'm surprised of the behavior with c(), and can understand that data.table
inherits it.  However, for the record, when rbinding data.frames (instead of
data.tables), there is no loss of timezone info (this is what I'm currently
doing in my code, building an intermediate data.frame, and then converting
it wholesale to data.table when it's done -- timezones stay what they should
be, even though this is not optimal).  So apparently, there are multiple
instances of core R code doing date vector concatenation, and they disagree
as to timezone handling...

I will impatiently await the integer date and time classes.  (I currently
have to manually set the storage.mode of a POSIXct vector to "integer" to be
able to use them as keys in my data.tables.)  Moreover, I looked into chron
and it's not suitable for a few things I need from POSIXct.

    Best,
    + Nicolas


On Tue, Aug 31, 2010 at 10:05 PM, Short, Tom <TShort at epri.com> wrote:

>  Nicolas,
>
> That's not really a data.table issue. It's generally a problem with
> POSIXct. See here:
>
> > x = seq(as.POSIXct("2010-01-01", "GMT"), as.POSIXct("2010-01-10", "GMT"),
> length.out=2)
> >
> > c(x,x)
> [1] "2009-12-31 19:00:00 EST" "2010-01-09 19:00:00 EST"
> [3] "2009-12-31 19:00:00 EST" "2010-01-09 19:00:00 EST"
> To get around this, I generally fix the timezone for the system as follows:
>
> > Sys.setenv(TZ="GMT") # time zones drive me crazy!
> >
> > c(x,x)
> [1] "2010-01-01 GMT" "2010-01-10 GMT" "2010-01-01 GMT" "2010-01-10 GMT"
> Also, if you just have dates, it's better to stick with the Date class (or
> chron). In the development version of data.table, we also have integer date
> and time classes that work well with data.table (faster sorting and
> grouping).
>
> - Tom
>
>
>  ------------------------------
> *From:* datatable-help-bounces at lists.r-forge.r-project.org [mailto:
> datatable-help-bounces at lists.r-forge.r-project.org] *On Behalf Of *Nicolas
> Chapados
> *Sent:* Tuesday, August 31, 2010 15:40
> *To:* datatable-help at lists.r-forge.r-project.org
> *Subject:* [datatable-help] Automagic timezone convertion on data.table
> rbind
>
> Dear list,
>
> I'm using a data.table that contains date columns (in POSIXct format).
>  There appears to be an issue with automatic timezone conversion when
> rbinding from NULL (or an empty data.table), as would occur when one is
> progressively building up the table in a loop.
>
> For example:
>
>  > require(data.table)
> Loading required package: data.table
>
> ## Currently located in the North-America Eastern Standard Time zone.
> > Sys.timezone()
> [1] "America/Montreal"
> > a <- NULL
>
> ## Create a table with a single date column (GMT timezone)
> > b <- data.table(x = seq(as.POSIXct("2010-01-01", "GMT"),
> as.POSIXct("2010-01-10", "GMT"), length.out=10))
> > b
>                x
>  [1,] 2010-01-01
>  [2,] 2010-01-02
>  [3,] 2010-01-03
>  [4,] 2010-01-04
>  [5,] 2010-01-05
>  [6,] 2010-01-06
>  [7,] 2010-01-07
>  [8,] 2010-01-08
>  [9,] 2010-01-09
> [10,] 2010-01-10
>
> ## Bind it from NULL: Oops!  The timezone changes to EST!
> > rbind(a,b)
>                         x
>  [1,] 2009-12-31 19:00:00
>  [2,] 2010-01-01 19:00:00
>  [3,] 2010-01-02 19:00:00
>  [4,] 2010-01-03 19:00:00
>  [5,] 2010-01-04 19:00:00
>  [6,] 2010-01-05 19:00:00
>  [7,] 2010-01-06 19:00:00
>  [8,] 2010-01-07 19:00:00
>  [9,] 2010-01-08 19:00:00
> [10,] 2010-01-09 19:00:00
>
> ## Same behavior from an empty data table.
> > rbind(data.table(), b)
>                         x
>  [1,] 2009-12-31 19:00:00
>  [2,] 2010-01-01 19:00:00
>  [3,] 2010-01-02 19:00:00
>  [4,] 2010-01-03 19:00:00
>  [5,] 2010-01-04 19:00:00
>  [6,] 2010-01-05 19:00:00
>  [7,] 2010-01-06 19:00:00
>  [8,] 2010-01-07 19:00:00
>  [9,] 2010-01-08 19:00:00
> [10,] 2010-01-09 19:00:00
>
> This behavior is somewhat puzzling.  Any pointers as to how to preserve
> timezone information would be greatly appreciated!  For the record, I'm
> with R version 2.9.2 and time.date version 1.4.1 (the latest on CRAN).
>
> Best regards,
> + Nicolas Chapados
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20100831/b51838b1/attachment.htm>