[datatable-help] IDateTime and plotting thereof

Matthew Dowle mdowle at mdowle.plus.com
Sat Dec 17 00:28:47 CET 2011


Farrel,
Please raise a bug.report(package="data.table") for this one. The
content can just be a link back to this thread. That way it won't be
forgotten. Please include the result of sessionInfo() if not already in
the thread, since the version numbers of R and the packages are very
important, especially at the moment.
Matthew

On Fri, 2011-12-16 at 12:02 -0500, Farrel Buchinsky wrote:
> I discovered that if I wanted control of the dates on the x axis I had
> to convert the IDate to as.POSIXct as well otherwise strange and
> undesirable things started happening - for instance all the dates were
> the same if I tried to format the dates in a particular way. So here
> is what I finished with my original data set.
> 
> 
> qplot(data=ape.ph,x=as.POSIXct(date,tz="EST"),
> y=as.POSIXct(timei,tz="EST")) + 
>    scale_y_datetime(name="time",format="%H:%M") +
>    scale_x_datetime(name="November",major="1 day", format = "%a %d")
> 
> Farrel Buchinsky
> Google Voice Tel: (412) 567-7870
> 
> 
> 
> On Fri, Dec 16, 2011 at 10:35, Farrel Buchinsky <fjbuch at gmail.com>
> wrote:
>         Dear Matthew
>         
>         
>         Thank you very much for your help.
>         I found something fascinating. I do not know the details of
>         POSIXct and iTime but look at this.
>         
>         
>         library(data.table)
>         library(ggplot2)
>         
>         
>         datetimes <- c("2011 NOV18 09:29:16", "2011 NOV18 10:42:40",
>         "2011 NOV18 10:47:12", 
>         "2011 NOV18 11:06:01", "2011 NOV18 11:35:34", "2011 NOV18
>         11:51:09", 
>         "2011 NOV18 12:23:16", "2011 NOV18 12:30:01", "2011 NOV18
>         12:34:28", 
>         "2011 NOV18 12:51:45", "2011 NOV18 12:56:37", "2011 NOV18
>         13:28:17", 
>         "2011 NOV18 13:37:41", "2011 NOV18 13:49:07", "2011 NOV18
>         14:47:04", 
>         "2011 NOV18 15:16:39", "2011 NOV18 15:17:10", "2011 NOV18
>         15:19:19", 
>         "2011 NOV19 08:28:00", "2011 NOV19 09:20:55", "2011 NOV19
>         10:41:25", 
>         "2011 NOV19 10:41:41", "2011 NOV19 12:53:15", "2011 NOV20
>         11:22:59", 
>         "2011 NOV20 19:53:25", "2011 NOV21 07:16:35", "2011 NOV21
>         08:50:45", 
>         "2011 NOV21 09:32:59", "2011 NOV21 09:43:01", "2011 NOV21
>         10:29:52", 
>         "2011 NOV21 10:34:23", "2011 NOV21 10:37:09", "2011 NOV21
>         10:55:18", 
>         "2011 NOV21 11:43:46", "2011 NOV21 11:46:50", "2011 NOV21
>         12:10:16", 
>         "2011 NOV21 12:16:44", "2011 NOV21 13:53:42", "2011 NOV21
>         13:53:43", 
>         "2011 NOV21 14:06:06", "2011 NOV21 14:23:39", "2011 NOV21
>         15:14:21", 
>         "2011 NOV21 15:35:53", "2011 NOV21 15:46:29", "2011 NOV21
>         15:49:37", 
>         "2011 NOV21 16:45:32", "2011 NOV21 20:36:31", "2011 NOV22
>         08:56:03", 
>         "2011 NOV22 09:33:12", "2011 NOV22 09:39:52", "2011 NOV22
>         10:40:46", 
>         "2011 NOV22 11:19:48", "2011 NOV22 11:23:32", "2011 NOV22
>         12:04:16", 
>         "2011 NOV22 12:07:04", "2011 NOV22 12:23:31", "2011 NOV22
>         12:28:54", 
>         "2011 NOV22 12:31:14", "2011 NOV22 12:53:31", "2011 NOV22
>         12:54:11", 
>         "2011 NOV22 13:41:54", "2011 NOV22 15:34:11", "2011 NOV23
>         07:03:44", 
>         "2011 NOV23 07:04:57", "2011 NOV23 08:01:42", "2011 NOV23
>         08:37:14", 
>         "2011 NOV23 08:57:55", "2011 NOV23 09:00:34", "2011 NOV23
>         09:10:38", 
>         "2011 NOV23 09:21:19", "2011 NOV23 09:23:58", "2011 NOV23
>         10:48:10", 
>         "2011 NOV23 11:23:58", "2011 NOV23 11:49:14", "2011 NOV23
>         12:19:44", 
>         "2011 NOV23 12:31:34", "2011 NOV23 12:48:00", "2011 NOV23
>         13:22:08", 
>         "2011 NOV23 13:30:36", "2011 NOV23 14:01:56", "2011 NOV23
>         14:38:15", 
>         "2011 NOV23 14:54:02", "2011 NOV23 15:33:31", "2011 NOV23
>         15:49:12", 
>         "2011 NOV25 09:21:42", "2011 NOV25 09:22:25", "2011 NOV25
>         09:24:22", 
>         "2011 NOV25 10:07:42", "2011 NOV25 15:03:15", "2011 NOV28
>         08:12:32", 
>         "2011 NOV28 09:20:39", "2011 NOV28 09:26:09", "2011 NOV28
>         10:25:47", 
>         "2011 NOV28 11:19:40", "2011 NOV28 11:54:54", "2011 NOV28
>         14:33:53", 
>         "2011 NOV28 15:08:25", "2011 NOV28 15:09:25", "2011 NOV28
>         15:23:07", 
>         "2011 NOV28 15:23:30", "2011 NOV28 15:50:35", "2011 NOV29
>         08:22:39", 
>         "2011 NOV29 09:08:51", "2011 NOV29 09:10:15", "2011 NOV29
>         10:44:17", 
>         "2011 NOV29 11:32:05", "2011 NOV29 12:00:52", "2011 NOV29
>         12:32:48", 
>         "2011 NOV29 13:21:50", "2011 NOV29 13:22:12", "2011 NOV29
>         13:22:38", 
>         "2011 NOV29 13:23:35", "2011 NOV29 13:28:38", "2011 NOV29
>         13:29:44", 
>         "2011 NOV29 13:30:25", "2011 NOV29 13:32:13", "2011 NOV29
>         13:33:28", 
>         "2011 NOV29 13:35:59", "2011 NOV29 13:37:01", "2011 NOV29
>         13:39:37", 
>         "2011 NOV29 13:41:33", "2011 NOV29 14:08:07", "2011 NOV29
>         14:09:53", 
>         "2011 NOV29 14:13:15", "2011 NOV29 14:16:16", "2011 NOV29
>         14:17:32", 
>         "2011 NOV29 14:37:05", "2011 NOV29 14:59:51", "2011 NOV29
>         14:59:51", 
>         "2011 NOV29 14:59:54", "2011 NOV29 15:00:10", "2011 NOV29
>         15:13:22", 
>         "2011 NOV29 16:09:50", "2011 NOV29 16:10:17", "2011 NOV29
>         16:31:14", 
>         "2011 NOV29 16:43:20", "2011 NOV30 09:22:09", "2011 NOV30
>         10:19:21", 
>         "2011 NOV30 11:42:04", "2011 NOV30 11:50:11", "2011 NOV30
>         12:21:11", 
>         "2011 NOV30 13:06:39")
>         
>         
>         datetimes <- strptime(datetimes,"%Y %b%d %H:%M:%S")
>         dtdt <- IDateTime(datetimes)
>         setkey(x=dtdt,idate,itime)
>         plot(dtdt$idate,dtdt$itime)#works although I still need to
>         figure out how to get times that I, as a human, understand
>         qplot(x=dtdt$idate,y=dtdt$itime) # gives an error
>         
>         
>         #Error in as.data.frame.default(x[[i]], optional = TRUE) : 
>         #  cannot coerce class '"ITime"' into a data.frame
>         
>         
>         dtdt[,qplot(idate,itime)]#same error
>         
>         
>         
>         
>         qplot(x=dtdt$idate,y=as.POSIXct(dtdt$itime,tz="EST")) #have to
>         specify the time zone otherwise I got evertying at UTC
>         
>         
>         Clearly this has something to do with the way ggplot wants its
>         data. Plot handled it but qplot did not.
>         
>         
>         
>         
>         Farrel Buchinsky
>         
>         
>         
>         
>         On Fri, Dec 16, 2011 at 03:50, Matthew Dowle
>         <mdowle at mdowle.plus.com> wrote:
>                 Hi.
>                 
>                 You should be able to add more than one column in one
>                 step using := by
>                 setting with=FALSE like this :
>                 
>                 > DT = data.table(a=1:3,b=4:6)
>                 > DT
>                     a b
>                 [1,] 1 4
>                 [2,] 2 5
>                 [3,] 3 6
>                 > DT[,c("newcol1","newcol2"):=list(7:9,3.14)]
>                 Error in `[.data.table`(DT, , `:=`(c("newcol1",
>                 "newcol2"),
>                 list(7:9,  :
>                  LHS of := must be a single column name, when
>                 with=TRUE. When
>                 with=FALSE the LHS may be a vector of column names or
>                 positions.
>                 
>                 so set with=FALSE :
>                 
>                 >
>                 DT[,c("newcol1","newcol2"):=list(7:9,3.14),with=FALSE]
>                 Error in `[.data.table`(DT, , `:=`(c("newcol1",
>                 "newcol2"),
>                 list(7:9,  :
>                  Logical error. Can only add new columns one at a time
>                 currently
>                 >
>                 
>                 I intended that to work so I'll take a look. But, the
>                 intention is at
>                 some point to enable this :
>                 
>                   DT[,{ newcol1 := ....
>                         newcol2 := ....
>                         newcol3 := newcol1 + newcol2
>                       }, by=... ]
>                 
>                 where not only can you add multiple columns by
>                 reference and by group,
>                 but they can 'see' previous columns just added. That's
>                 going to be quite
>                 tricky to implement so don't hold your breath!
>                 
>                 In the meantime it's either one at a time with :=, or
>                 use (slower) [<-
>                 method to add multiple columns (via at least one full
>                 table copy).
>                 
>                 Re plotting, I'm hoping to learn ggplot2 myself soon,
>                 so bear that in
>                 mind, but this is the way I thought should work :
>                 
>                  ape.ph[,qplot(date, timei)]
>                 
>                 If that gives the same error I'm not sure why it would
>                 try to coerce
>                 ITime into a data.frame. Does anyone else know?  If
>                 you can provide some
>                 example data it might help save us time to reproduce
>                 it.
>                 
>                 Matthew
>                 
>                 
>                 
>                 On Thu, 2011-12-15 at 19:25 -0500, Farrel Buchinsky
>                 wrote:
>                 > I was able to take a column of dates and times that
>                 looked like this
>                 >
>                 > 2011 NOV18 09:29:16
>                 >
>                 >
>                 > I then tried this
>                 >
>                 > ape.ph[,date:=IDateTime(strptime(ape.ph$time,"%Y %b%
>                 d %H:%M:%S"))[[1]]]
>                 > ape.ph[,timei:=as.ITime(strptime(ape.ph$time,"%Y %b%
>                 d %H:%M:%S"))]
>                 >
>                 >
>                 > I had to create one column called date to accept the
>                 first column of
>                 > the IDateTime output. There does not appear to be a
>                 way to use the
>                 > IDateTime function and take both of its columns and
>                 stick it into the
>                 > data.table since one can only add one new column at
>                 a time as far as I
>                 > can tell. Anyway, I then created a variable timei to
>                 accept the time.
>                 >
>                 >
>                 > I wanted to plot the times that something happened
>                 on the y axis and
>                 > then on the x axis I wanted to have the various
>                 dates
>                 >
>                 >
>                 > library(ggplot2)
>                 > qplot(ape.ph[,date], ape.ph[,timei])
>                 >
>                 >
>                 > but alas I got this
>                 >
>                 >
>                 > Error in as.data.frame.default(x[[i]], optional =
>                 TRUE) :
>                 > cannot coerce class '"ITime"' into a data.frame
>                 >
>                 >
>                 > I am eventually hoping for this
>                 > qplot(ape.ph[,date], ape.ph[,timei]) +
>                 >    scale_y_datetime(format="%H:%M:%S") +
>                 >            xlab("Day") + ylab("Hour")
>                 >
>                 >
>                 > Do you have any examples of dates and times used in
>                 a data.table and
>                 > then being used to plot graphs? I would love to see
>                 some examples. I
>                 > could not find any
>                 >
>                 >
>                 >
>                 >
>                 > Farrel Buchinsky
>                 > Google Voice Tel: (412) 567-7870
>                 >
>                 >
>                 
>                 > _______________________________________________
>                 > datatable-help mailing list
>                 > datatable-help at lists.r-forge.r-project.org
>                 >
>                 https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>                 
>                 
>         
>         
> 
> 




More information about the datatable-help mailing list