[datatable-help] IDateTime and plotting thereof

Fri Dec 16 18:02:02 CET 2011

I discovered that if I wanted control of the dates on the x axis I had to
convert the IDate to as.POSIXct as well otherwise strange and undesirable
things started happening - for instance all the dates were the same if I
tried to format the dates in a particular way. So here is what I finished
with my original data set.

qplot(data=ape.ph,x=as.POSIXct(date,tz="EST"),
y=as.POSIXct(timei,tz="EST")) +
   scale_y_datetime(name="time",format="%H:%M") +
   scale_x_datetime(name="November",major="1 day", format = "%a %d")

Farrel Buchinsky
Google Voice Tel: (412) 567-7870

On Fri, Dec 16, 2011 at 10:35, Farrel Buchinsky <fjbuch at gmail.com> wrote:

> Dear Matthew
>
> Thank you very much for your help.
> I found something fascinating. I do not know the details of POSIXct and
> iTime but look at this.
>
> library(data.table)
> library(ggplot2)
>
> datetimes <- c("2011 NOV18 09:29:16", "2011 NOV18 10:42:40", "2011 NOV18
> 10:47:12",
> "2011 NOV18 11:06:01", "2011 NOV18 11:35:34", "2011 NOV18 11:51:09",
> "2011 NOV18 12:23:16", "2011 NOV18 12:30:01", "2011 NOV18 12:34:28",
> "2011 NOV18 12:51:45", "2011 NOV18 12:56:37", "2011 NOV18 13:28:17",
>  "2011 NOV18 13:37:41", "2011 NOV18 13:49:07", "2011 NOV18 14:47:04",
> "2011 NOV18 15:16:39", "2011 NOV18 15:17:10", "2011 NOV18 15:19:19",
> "2011 NOV19 08:28:00", "2011 NOV19 09:20:55", "2011 NOV19 10:41:25",
> "2011 NOV19 10:41:41", "2011 NOV19 12:53:15", "2011 NOV20 11:22:59",
> "2011 NOV20 19:53:25", "2011 NOV21 07:16:35", "2011 NOV21 08:50:45",
>  "2011 NOV21 09:32:59", "2011 NOV21 09:43:01", "2011 NOV21 10:29:52",
> "2011 NOV21 10:34:23", "2011 NOV21 10:37:09", "2011 NOV21 10:55:18",
> "2011 NOV21 11:43:46", "2011 NOV21 11:46:50", "2011 NOV21 12:10:16",
> "2011 NOV21 12:16:44", "2011 NOV21 13:53:42", "2011 NOV21 13:53:43",
> "2011 NOV21 14:06:06", "2011 NOV21 14:23:39", "2011 NOV21 15:14:21",
>  "2011 NOV21 15:35:53", "2011 NOV21 15:46:29", "2011 NOV21 15:49:37",
> "2011 NOV21 16:45:32", "2011 NOV21 20:36:31", "2011 NOV22 08:56:03",
> "2011 NOV22 09:33:12", "2011 NOV22 09:39:52", "2011 NOV22 10:40:46",
> "2011 NOV22 11:19:48", "2011 NOV22 11:23:32", "2011 NOV22 12:04:16",
> "2011 NOV22 12:07:04", "2011 NOV22 12:23:31", "2011 NOV22 12:28:54",
>  "2011 NOV22 12:31:14", "2011 NOV22 12:53:31", "2011 NOV22 12:54:11",
> "2011 NOV22 13:41:54", "2011 NOV22 15:34:11", "2011 NOV23 07:03:44",
> "2011 NOV23 07:04:57", "2011 NOV23 08:01:42", "2011 NOV23 08:37:14",
> "2011 NOV23 08:57:55", "2011 NOV23 09:00:34", "2011 NOV23 09:10:38",
> "2011 NOV23 09:21:19", "2011 NOV23 09:23:58", "2011 NOV23 10:48:10",
>  "2011 NOV23 11:23:58", "2011 NOV23 11:49:14", "2011 NOV23 12:19:44",
> "2011 NOV23 12:31:34", "2011 NOV23 12:48:00", "2011 NOV23 13:22:08",
> "2011 NOV23 13:30:36", "2011 NOV23 14:01:56", "2011 NOV23 14:38:15",
> "2011 NOV23 14:54:02", "2011 NOV23 15:33:31", "2011 NOV23 15:49:12",
> "2011 NOV25 09:21:42", "2011 NOV25 09:22:25", "2011 NOV25 09:24:22",
>  "2011 NOV25 10:07:42", "2011 NOV25 15:03:15", "2011 NOV28 08:12:32",
> "2011 NOV28 09:20:39", "2011 NOV28 09:26:09", "2011 NOV28 10:25:47",
> "2011 NOV28 11:19:40", "2011 NOV28 11:54:54", "2011 NOV28 14:33:53",
> "2011 NOV28 15:08:25", "2011 NOV28 15:09:25", "2011 NOV28 15:23:07",
> "2011 NOV28 15:23:30", "2011 NOV28 15:50:35", "2011 NOV29 08:22:39",
>  "2011 NOV29 09:08:51", "2011 NOV29 09:10:15", "2011 NOV29 10:44:17",
> "2011 NOV29 11:32:05", "2011 NOV29 12:00:52", "2011 NOV29 12:32:48",
> "2011 NOV29 13:21:50", "2011 NOV29 13:22:12", "2011 NOV29 13:22:38",
> "2011 NOV29 13:23:35", "2011 NOV29 13:28:38", "2011 NOV29 13:29:44",
> "2011 NOV29 13:30:25", "2011 NOV29 13:32:13", "2011 NOV29 13:33:28",
>  "2011 NOV29 13:35:59", "2011 NOV29 13:37:01", "2011 NOV29 13:39:37",
> "2011 NOV29 13:41:33", "2011 NOV29 14:08:07", "2011 NOV29 14:09:53",
> "2011 NOV29 14:13:15", "2011 NOV29 14:16:16", "2011 NOV29 14:17:32",
> "2011 NOV29 14:37:05", "2011 NOV29 14:59:51", "2011 NOV29 14:59:51",
> "2011 NOV29 14:59:54", "2011 NOV29 15:00:10", "2011 NOV29 15:13:22",
>  "2011 NOV29 16:09:50", "2011 NOV29 16:10:17", "2011 NOV29 16:31:14",
> "2011 NOV29 16:43:20", "2011 NOV30 09:22:09", "2011 NOV30 10:19:21",
> "2011 NOV30 11:42:04", "2011 NOV30 11:50:11", "2011 NOV30 12:21:11",
> "2011 NOV30 13:06:39")
>
> datetimes <- strptime(datetimes,"%Y %b%d %H:%M:%S")
> dtdt <- IDateTime(datetimes)
> setkey(x=dtdt,idate,itime)
> plot(dtdt$idate,dtdt$itime)#works although I still need to figure out how
> to get times that I, as a human, understand
> qplot(x=dtdt$idate,y=dtdt$itime) # gives an error
>
> #Error in as.data.frame.default(x[[i]], optional = TRUE) :
> #  cannot coerce class '"ITime"' into a data.frame
>
> dtdt[,qplot(idate,itime)]#same error
>
>
> qplot(x=dtdt$idate,y=as.POSIXct(dtdt$itime,tz="EST")) #have to specify the
> time zone otherwise I got evertying at UTC
>
> *Clearly this has something to do with the way ggplot wants its data.
> Plot handled it but qplot did not.*
>
>
> Farrel Buchinsky[?]
>
>
>
>
> On Fri, Dec 16, 2011 at 03:50, Matthew Dowle <mdowle at mdowle.plus.com>wrote:
>
>> Hi.
>>
>> You should be able to add more than one column in one step using := by
>> setting with=FALSE like this :
>>
>> > DT = data.table(a=1:3,b=4:6)
>> > DT
>>     a b
>> [1,] 1 4
>> [2,] 2 5
>> [3,] 3 6
>> > DT[,c("newcol1","newcol2"):=list(7:9,3.14)]
>> Error in `[.data.table`(DT, , `:=`(c("newcol1", "newcol2"),
>> list(7:9,  :
>>  LHS of := must be a single column name, when with=TRUE. When
>> with=FALSE the LHS may be a vector of column names or positions.
>>
>> so set with=FALSE :
>>
>> > DT[,c("newcol1","newcol2"):=list(7:9,3.14),with=FALSE]
>> Error in `[.data.table`(DT, , `:=`(c("newcol1", "newcol2"),
>> list(7:9,  :
>>  Logical error. Can only add new columns one at a time currently
>> >
>>
>> I intended that to work so I'll take a look. But, the intention is at
>> some point to enable this :
>>
>>   DT[,{ newcol1 := ....
>>         newcol2 := ....
>>         newcol3 := newcol1 + newcol2
>>       }, by=... ]
>>
>> where not only can you add multiple columns by reference and by group,
>> but they can 'see' previous columns just added. That's going to be quite
>> tricky to implement so don't hold your breath!
>>
>> In the meantime it's either one at a time with :=, or use (slower) [<-
>> method to add multiple columns (via at least one full table copy).
>>
>> Re plotting, I'm hoping to learn ggplot2 myself soon, so bear that in
>> mind, but this is the way I thought should work :
>>
>>  ape.ph[,qplot(date, timei)]
>>
>> If that gives the same error I'm not sure why it would try to coerce
>> ITime into a data.frame. Does anyone else know?  If you can provide some
>> example data it might help save us time to reproduce it.
>>
>> Matthew
>>
>>
>>
>> On Thu, 2011-12-15 at 19:25 -0500, Farrel Buchinsky wrote:
>> > I was able to take a column of dates and times that looked like this
>> >
>> > 2011 NOV18 09:29:16
>> >
>> >
>> > I then tried this
>> >
>> > ape.ph[,date:=IDateTime(strptime(ape.ph$time,"%Y %b%d %H:%M:%S"))[[1]]]
>> > ape.ph[,timei:=as.ITime(strptime(ape.ph$time,"%Y %b%d %H:%M:%S"))]
>> >
>> >
>> > I had to create one column called date to accept the first column of
>> > the IDateTime output. There does not appear to be a way to use the
>> > IDateTime function and take both of its columns and stick it into the
>> > data.table since one can only add one new column at a time as far as I
>> > can tell. Anyway, I then created a variable timei to accept the time.
>> >
>> >
>> > I wanted to plot the times that something happened on the y axis and
>> > then on the x axis I wanted to have the various dates
>> >
>> >
>> > library(ggplot2)
>> > qplot(ape.ph[,date], ape.ph[,timei])
>> >
>> >
>> > but alas I got this
>> >
>> >
>> > Error in as.data.frame.default(x[[i]], optional = TRUE) :
>> > cannot coerce class '"ITime"' into a data.frame
>> >
>> >
>> > I am eventually hoping for this
>> > qplot(ape.ph[,date], ape.ph[,timei]) +
>> >    scale_y_datetime(format="%H:%M:%S") +
>> >            xlab("Day") + ylab("Hour")
>> >
>> >
>> > Do you have any examples of dates and times used in a data.table and
>> > then being used to plot graphs? I would love to see some examples. I
>> > could not find any
>> >
>> >
>> >
>> >
>> > Farrel Buchinsky
>> > Google Voice Tel: (412) 567-7870
>> >
>> >
>> > _______________________________________________
>> > datatable-help mailing list
>> > datatable-help at lists.r-forge.r-project.org
>> >
>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20111216/ea4689f0/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 868 bytes
Desc: not available
URL: <http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20111216/ea4689f0/attachment-0001.gif>