From binal.patel at jvion.com Sat Jun 20 02:10:57 2015 From: binal.patel at jvion.com (binalpatel) Date: Fri, 19 Jun 2015 17:10:57 -0700 (PDT) Subject: [datatable-help] Error in Melt Message-ID: <1434759057325-4708879.post@n4.nabble.com> Hi, I've started receiving this error today. Error in melt.data.table(dt, id.vars = c(gender, targetVar), measure.vars = c(6:length(dt)), : attempt to set index 1851513568/1851513568 in SET_STRING_ELT When attempting to melt a data.table object. After trying to debug my code, and quality check the data for a couple hours, I still can't figure out why it's failing. I was hoping for guidance on where I should look, or possible reasons why this error may occur. Thanks! - Binal -- View this message in context: http://r.789695.n4.nabble.com/Error-in-Melt-tp4708879.html Sent from the datatable-help mailing list archive at Nabble.com. From aragorn168b at gmail.com Sat Jun 20 10:24:58 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Sat, 20 Jun 2015 10:24:58 +0200 Subject: [datatable-help] Error in Melt In-Reply-To: <1434759057325-4708879.post@n4.nabble.com> References: <1434759057325-4708879.post@n4.nabble.com> Message-ID: Always include your `sessionInfo()` while posting about issues. Also, please test it with the?latest devel version?from the project page if you?ve not already.Lastly, if the issue exists in the latest devel, then please provide a reproducible example for me to test. --? Arun On 20 Jun 2015 at 02:32:02, binalpatel (binal.patel at jvion.com) wrote: Hi, I've started receiving this error today. Error in melt.data.table(dt, id.vars = c(gender, targetVar), measure.vars = c(6:length(dt)), : attempt to set index 1851513568/1851513568 in SET_STRING_ELT When attempting to melt a data.table object. After trying to debug my code, and quality check the data for a couple hours, I still can't figure out why it's failing. I was hoping for guidance on where I should look, or possible reasons why this error may occur. Thanks! - Binal -- View this message in context: http://r.789695.n4.nabble.com/Error-in-Melt-tp4708879.html Sent from the datatable-help mailing list archive at Nabble.com. _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From aragorn168b at gmail.com Sun Jun 21 10:58:46 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Sun, 21 Jun 2015 10:58:46 +0200 Subject: [datatable-help] Restoring data.frame functionality for selecting columns Message-ID: I?ve created an issue here on not having to need ?with=FALSE? for subsetting things the data.frame way:?https://github.com/Rdatatable/data.table/issues/1188 It?d be nice to know what others think about this. I believe it?s a hinderance to new users trying to learn data.table, being confronted with a difference in the most basic data.frame operation up-front. And there?s no real harm in doing it. Let us know what you think :-) --? Arun -------------- next part -------------- An HTML attachment was scrubbed... URL: From gsee000 at gmail.com Sun Jun 21 16:57:20 2015 From: gsee000 at gmail.com (G See) Date: Sun, 21 Jun 2015 09:57:20 -0500 Subject: [datatable-help] Restoring data.frame functionality for selecting columns In-Reply-To: References: Message-ID: Agreed that it seems innocuous, but my only request is that we get a stable version on CRAN before making a change like that. :-P Garrett On Sun, Jun 21, 2015 at 3:58 AM, Arunkumar Srinivasan wrote: > I?ve created an issue here on not having to need ?with=FALSE? for subsetting > things the data.frame way: > https://github.com/Rdatatable/data.table/issues/1188 > > It?d be nice to know what others think about this. I believe it?s a > hinderance to new users trying to learn data.table, being confronted with a > difference in the most basic data.frame operation up-front. And there?s no > real harm in doing it. > > Let us know what you think :-) > > -- > Arun > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help From fperickson at wisc.edu Wed Jun 24 01:41:39 2015 From: fperickson at wisc.edu (Frank Erickson) Date: Tue, 23 Jun 2015 19:41:39 -0400 Subject: [datatable-help] by after a merge is useless except for by=.EACHI? Message-ID: I wanted to do X[ Y, newx := yvar[1], by=yvar2 ] Does what I'm trying to do here somehow not make sense? This seems to work fine: X[ Y, newx := yvar[1], by=.EACHI ] My full example is below. Thanks, Frank DT <- data.table(Address=c('A','B','C','A','D','C','E'), Transaction=c(1,1,2,3,3,4,4)) setkey(DT,Address) t0DT <- DT[,.(t0=Transaction[1]),by=Address] DT[t0DT,Entity:=t0[1],by=Transaction] based on http://stackoverflow.com/questions/31011741/creating-an-entity-id-using-data-table -------------- next part -------------- An HTML attachment was scrubbed... URL: From aragorn168b at gmail.com Thu Jun 25 18:39:39 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Thu, 25 Jun 2015 18:39:39 +0200 Subject: [datatable-help] Restoring data.frame functionality for selecting columns In-Reply-To: References: Message-ID: Hi Joe, very glad to hear that `with = FALSE` wasn?t a *real hurdle* :-).? I think it?s better to follow up / post further thoughts directly here:?https://github.com/Rdatatable/data.table/issues/1188 I?m less convinced that this would be a positive change at this moment :-(. --? Arun On 25 Jun 2015 at 18:31:56, Joe Matusik (jmatusik at leadid.com) wrote: Hi Arun,? Sorry for the massively late response, Of all the data.table hurdles that I had to jump, the 'with = FALSE' issue was one of the easiest to overcome. Having said that, removing any barriers to entry for new users without taking away from data.table's superior functionality is probably never going to be a bad thing. I'd say go for it. ? Thanks, Joe On Sun, Jun 21, 2015 at 4:58 AM, Arunkumar Srinivasan wrote: I?ve created an issue here on not having to need ?with=FALSE? for subsetting things the data.frame way:?https://github.com/Rdatatable/data.table/issues/1188 It?d be nice to know what others think about this. I believe it?s a hinderance to new users trying to learn data.table, being confronted with a difference in the most basic data.frame operation up-front. And there?s no real harm in doing it. Let us know what you think :-) --? Arun _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From niparisco at gmail.com Thu Jun 25 21:46:40 2015 From: niparisco at gmail.com (Nicolas Paris) Date: Thu, 25 Jun 2015 15:46:40 -0400 Subject: [datatable-help] [data.table] Aggregate as vector Message-ID: Hi, Here an example what I want to get : > dt <- data.table(c(1,2,3,4),c("a","b","b","a")) > dt V1 V2 1: 1 a 2: 2 b 3: 3 b 4: 4 a What I want to get is a V3 column, containing an aggregated vector : > dt V1 V2 V3 1: 1 a c(1,4) 2: 2 b c(2,3) That does not work : dt[,V3 := as.list(c(V1)),by=V2] Is there a way to do so ? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperickson at wisc.edu Thu Jun 25 21:48:54 2015 From: fperickson at wisc.edu (Frank Erickson) Date: Thu, 25 Jun 2015 15:48:54 -0400 Subject: [datatable-help] [data.table] Aggregate as vector In-Reply-To: References: Message-ID: Hi, I think you want dt[,.(list(V1)),by=V2] # V2 V1 # 1: a 1,4 # 2: b 2,3 Using the .() notation makes it easier to keep track of what columns are being made, I find. --Frank On Thu, Jun 25, 2015 at 3:46 PM, Nicolas Paris wrote: > Hi, > > Here an example what I want to get : > > > dt <- data.table(c(1,2,3,4),c("a","b","b","a")) > > dt > V1 V2 > 1: 1 a > 2: 2 b > 3: 3 b > 4: 4 a > > What I want to get is a V3 column, containing an aggregated vector : > > dt > V1 V2 V3 > 1: 1 a c(1,4) > 2: 2 b c(2,3) > > That does not work : > > dt[,V3 := as.list(c(V1)),by=V2] > > Is there a way to do so ? > > Thanks > > > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperickson at wisc.edu Thu Jun 25 21:50:40 2015 From: fperickson at wisc.edu (Frank Erickson) Date: Thu, 25 Jun 2015 15:50:40 -0400 Subject: [datatable-help] [data.table] Aggregate as vector In-Reply-To: References: Message-ID: Oh, sorry, didn't notice you were making a new column, in that case: dt[,V3:=list(list(V1)),by=V2] On Thu, Jun 25, 2015 at 3:48 PM, Frank Erickson wrote: > Hi, > > I think you want > > dt[,.(list(V1)),by=V2] > # V2 V1 > # 1: a 1,4 > # 2: b 2,3 > > > Using the .() notation makes it easier to keep track of what columns are > being made, I find. > > --Frank > > On Thu, Jun 25, 2015 at 3:46 PM, Nicolas Paris > wrote: > >> Hi, >> >> Here an example what I want to get : >> >> > dt <- data.table(c(1,2,3,4),c("a","b","b","a")) >> > dt >> V1 V2 >> 1: 1 a >> 2: 2 b >> 3: 3 b >> 4: 4 a >> >> What I want to get is a V3 column, containing an aggregated vector : >> > dt >> V1 V2 V3 >> 1: 1 a c(1,4) >> 2: 2 b c(2,3) >> >> That does not work : >> >> dt[,V3 := as.list(c(V1)),by=V2] >> >> Is there a way to do so ? >> >> Thanks >> >> >> >> _______________________________________________ >> datatable-help mailing list >> datatable-help at lists.r-forge.r-project.org >> >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aragorn168b at gmail.com Thu Jun 25 21:53:36 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Thu, 25 Jun 2015 21:53:36 +0200 Subject: [datatable-help] [data.table] Aggregate as vector In-Reply-To: References: Message-ID: To add to Frank?s reply, I think `dt[, V3 := .(list(V1)), by=V2]` doesn?t work with `1.9.4`, but has been fixed in 1.9.5. The first list is for the syntax - (cols) := list(values). Cases like `DT[, A := 1:5]` and is internally wrapped by `list()` for convenience, as they are very common use cases. --? Arun On 25 Jun 2015 at 21:51:09, Frank Erickson (fperickson at wisc.edu) wrote: Oh, sorry, didn't notice you were making a new column, in that case: dt[,V3:=list(list(V1)),by=V2] On Thu, Jun 25, 2015 at 3:48 PM, Frank Erickson wrote: Hi, I think you want dt[,.(list(V1)),by=V2] # ? ?V2 ?V1 # 1: ?a 1,4 # 2: ?b 2,3 Using the .() notation makes it easier to keep track of what columns are being made, I find. --Frank On Thu, Jun 25, 2015 at 3:46 PM, Nicolas Paris wrote: Hi, Here an example what I want to get : > dt <- data.table(c(1,2,3,4),c("a","b","b","a")) > dt ? ?V1 V2 1: ?1 ?a 2: ?2 ?b 3: ?3 ?b 4: ?4 ?a What I want to get is a V3 column, containing an aggregated vector : > dt ? ?V1 V2 V3 1: ?1 ?a ? c(1,4) 2: ?2 ?b ? c(2,3) That does not work : dt[,V3 := as.list(c(V1)),by=V2] Is there a way to do so ? Thanks _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From niparisco at gmail.com Thu Jun 25 22:02:27 2015 From: niparisco at gmail.com (Nicolas Paris) Date: Thu, 25 Jun 2015 16:02:27 -0400 Subject: [datatable-help] [data.table] Aggregate as vector In-Reply-To: References: Message-ID: Thanks a lot guys, I was sure that was possible ? > ? > The first list is for the syntax - (cols) := list(values)? ?Ok, I forgot this point? 2015-06-25 15:53 GMT-04:00 Arunkumar Srinivasan : > To add to Frank?s reply, I think `dt[, V3 := .(list(V1)), by=V2]` doesn?t > work with `1.9.4`, but has been fixed in 1.9.5. > The first list is for the syntax - (cols) := list(values). Cases like > `DT[, A := 1:5]` and is internally wrapped by `list()` for convenience, as > they are very common use cases. > -- > Arun > > On 25 Jun 2015 at 21:51:09, Frank Erickson (fperickson at wisc.edu) wrote: > > Oh, sorry, didn't notice you were making a new column, in that case: > dt[,V3:=list(list(V1)),by=V2] > > > On Thu, Jun 25, 2015 at 3:48 PM, Frank Erickson > wrote: > >> Hi, >> >> I think you want >> >> dt[,.(list(V1)),by=V2] >> # V2 V1 >> # 1: a 1,4 >> # 2: b 2,3 >> >> >> Using the .() notation makes it easier to keep track of what columns are >> being made, I find. >> >> --Frank >> >> On Thu, Jun 25, 2015 at 3:46 PM, Nicolas Paris >> wrote: >> >>> Hi, >>> >>> Here an example what I want to get : >>> >>> > dt <- data.table(c(1,2,3,4),c("a","b","b","a")) >>> > dt >>> V1 V2 >>> 1: 1 a >>> 2: 2 b >>> 3: 3 b >>> 4: 4 a >>> >>> What I want to get is a V3 column, containing an aggregated vector : >>> > dt >>> V1 V2 V3 >>> 1: 1 a c(1,4) >>> 2: 2 b c(2,3) >>> >>> That does not work : >>> >>> dt[,V3 := as.list(c(V1)),by=V2] >>> >>> Is there a way to do so ? >>> >>> Thanks >>> >>> >>> >>> _______________________________________________ >>> datatable-help mailing list >>> datatable-help at lists.r-forge.r-project.org >>> >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >>> >> >> > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From my.r.help at gmail.com Tue Jun 30 03:06:42 2015 From: my.r.help at gmail.com (Michael Smith) Date: Tue, 30 Jun 2015 09:06:42 +0800 Subject: [datatable-help] Integration with magrittr Message-ID: <5591EBA2.1060505@gmail.com> All, I love data.table and I also love magrittr. However, some things don't work, e.g. DT <- data.table(a = 1:3, b = 4:6) DT[, a %<>% `*`(2)] # instead of DT[, a := a %>% `*`(2)] Is there some way to make it work or am I missing something? I understand it's because how data.table is essentially a DSL and parses things in its own ways. Is there a workaround or should I post a feature request? Thanks, M From michael.nelson at sydney.edu.au Tue Jun 30 03:18:55 2015 From: michael.nelson at sydney.edu.au (Michael Nelson) Date: Tue, 30 Jun 2015 01:18:55 +0000 Subject: [datatable-help] Integration with magrittr In-Reply-To: <5591EBA2.1060505@gmail.com> References: <5591EBA2.1060505@gmail.com> Message-ID: <692C53CA-6073-4048-9609-3D874FA7A234@sydney.edu.au> := isn't actually defined as a function by data table., it only works within a call to [.data.table as the j argument, and is recognised by parsing the j expression (within the function call). This I would think it would need to be a feature request. > On 30 Jun 2015, at 11:07 am, Michael Smith wrote: > > All, > > I love data.table and I also love magrittr. However, some things don't work, e.g. > > DT <- data.table(a = 1:3, b = 4:6) > DT[, a %<>% `*`(2)] # instead of DT[, a := a %>% `*`(2)] > > Is there some way to make it work or am I missing something? I understand it's because how data.table is essentially a DSL and parses things in its own ways. Is there a workaround or should I post a feature request? > > Thanks, > M > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help From lianoglou.steve at gene.com Tue Jun 30 22:06:47 2015 From: lianoglou.steve at gene.com (Steve Lianoglou) Date: Tue, 30 Jun 2015 13:06:47 -0700 Subject: [datatable-help] Integration with magrittr In-Reply-To: <5591EBA2.1060505@gmail.com> References: <5591EBA2.1060505@gmail.com> Message-ID: Hi, > I love data.table and I also love magrittr. I really love both of these, too. > However, some things don't work, > e.g. > > DT <- data.table(a = 1:3, b = 4:6) > DT[, a %<>% `*`(2)] # instead of DT[, a := a %>% `*`(2)] In my personal opinion, though, I feel like this might be going a stretch too far. Although this might just be a contrived example, it's hard for me to divine an instance where this would look any less contrived. I mean, is: DT[, a %<>% `*`(2)] ## or DT[, a := a %>% `*`(2)] Really any more useful/simpler than: DT[, a := a * 2] Can't think of when I'd prefer the magrittr'ized version ... perhaps when the column name (`a`) is actually a much longer? (so that %<>% saves you some typing), or? -steve -- Steve Lianoglou Computational Biologist Genentech