From mel at mbacou.com Tue Aug 4 11:39:34 2015 From: mel at mbacou.com (Bacou, Melanie) Date: Tue, 4 Aug 2015 05:39:34 -0400 Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! Message-ID: <55C08856.40903@mbacou.com> Hi, something very strange today as I'm trying to simply read in the attached CSV into R (it's small). I seem to be losing all decimals in the last column. I tried to set `colClasses`, but made no difference. Not sure what's wrong with me today! Can any one replicate? I suppose it's in the input file, will keep digging. Thx, --Mel. E.g. this is what I see when opening the CSV in emacs: And then this is what shows up in R: ```r pcn <- fread("PovCalServlet_15.08.03.csv") sapply(pcn, class) # country povLine mean hcr gap sev # "character" "numeric" "numeric" "numeric" "numeric" "numeric" # watts popM yearNum # "numeric" "numeric" "numeric" # => looks as expected, but then no decimal... pcn08$yearNum # [1] 2008 2012 2009 2009 2006 2008 2007 2008 2011 2004 2006 2011 2008 2010 2005 # [16] 2003 2006 2008 2007 2012 2002 2005 2010 2007 2010 2010 2010 2008 2008 2012 # [31] 2006 2009 2010 2011 2010 2011 2010 2011 2006 2011 2011 2009 2010 2012 2011 # [46] 2008 2012 2009 2010 2008 2012 2009 2009 2006 2008 2007 2008 2011 2004 2006 # [61] 2011 2008 2010 2005 2003 2006 2008 2007 2012 2002 2005 2010 2007 2010 2010 # [76] 2010 2008 2008 2012 2006 2009 2010 2011 2010 2011 2010 2011 2006 2011 2011 # [91] 2009 2010 2012 2011 2008 2012 2009 2010 sessionInfo() # R version 3.2.1 (2015-06-18) # Platform: x86_64-w64-mingw32/x64 (64-bit) # Running under: Windows 7 x64 (build 7601) Service Pack 1 # # locale: # [1] LC_COLLATE=English_United States.1252 # [2] LC_CTYPE=English_United States.1252 # [3] LC_MONETARY=English_United States.1252 # [4] LC_NUMERIC=C # [5] LC_TIME=English_United States.1252 # # attached base packages: # [1] stats graphics grDevices utils datasets methods base # # other attached packages: # [1] rgdal_1.0-4 sp_1.1-1 data.table_1.9.4 rj_2.0.3-2 # # loaded via a namespace (and not attached): # [1] magrittr_1.5 plyr_1.8.3 tools_3.2.1 rj.gd_2.0.0-1 # [5] reshape2_1.4.1 Rcpp_0.11.6 stringi_0.5-5 grid_3.2.1 # [9] stringr_1.0.0 chron_2.3-47 lattice_0.20-33 ``` -- Melanie BACOU International Food Policy Research Institute Snr. Program Manager, Spatial Data and Analytics Work +1(202)862-5699 E-mail m.bacou at cgiar.org Visit www.harvestchoice.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dhechbei.png Type: image/png Size: 20307 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PovCalServlet_15.08.03.csv Type: text/csv Size: 5716 bytes Desc: not available URL: From kpm.nachtmann at gmail.com Tue Aug 4 13:16:46 2015 From: kpm.nachtmann at gmail.com (nachti) Date: Tue, 4 Aug 2015 04:16:46 -0700 (PDT) Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! In-Reply-To: <55C08856.40903@mbacou.com> References: <55C08856.40903@mbacou.com> Message-ID: <1438687006319-4710727.post@n4.nabble.com> Hi, copying your code, everything works as expected for me. Maybe you just referenced to a wrong object (pcn08)? ``` > library(data.table) data.table 1.9.4 For help type: ?data.table *** NB: by=.EACHI is now explicit. See README to restore previous behaviour. > pcn <- fread("PovCalServlet_15.08.03.csv") sapply(pcn, class) pcn <- fread("PovCalServlet_15.08.03.csv") > sapply(pcn, class) country povLine mean hcr gap sev "character" "numeric" "numeric" "numeric" "numeric" "numeric" watts popM yearNum "numeric" "numeric" "numeric" > pcn08$yearNum Error: object 'pcn08' not found > pcn$yearNum [1] 2008.50 2011.50 2009.25 2009.00 2006.00 2007.50 2007.00 2008.00 2011.00 [10] 2004.00 2005.50 2011.00 2008.00 2010.50 2005.00 2003.00 2005.50 2008.00 [19] 2007.00 2012.00 2002.00 2005.40 2010.00 2007.00 2010.00 2010.23 2010.00 [28] 2008.00 2008.00 2012.00 2006.00 2008.64 2009.50 2011.00 2009.83 2010.83 [37] 2010.00 2011.00 2006.50 2011.00 2010.67 2009.00 2009.50 2011.80 2011.00 [46] 2008.00 2012.50 2009.30 2010.00 2008.50 2011.50 2009.25 2009.00 2006.00 [55] 2007.50 2007.00 2008.00 2011.00 2004.00 2005.50 2011.00 2008.00 2010.50 [64] 2005.00 2003.00 2005.50 2008.00 2007.00 2012.00 2002.00 2005.40 2010.00 [73] 2007.00 2010.00 2010.23 2010.00 2008.00 2008.00 2012.00 2006.00 2008.64 [82] 2009.50 2011.00 2009.83 2010.83 2010.00 2011.00 2006.50 2011.00 2010.67 [91] 2009.00 2009.50 2011.80 2011.00 2008.00 2012.50 2009.30 2010.00 > sessionInfo() R version 3.2.1 (2015-06-18) Platform: x86_64-suse-linux-gnu (64-bit) Running under: openSUSE 13.1 (Bottle) (x86_64) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] data.table_1.9.4 loaded via a namespace (and not attached): [1] compiler_3.2.1 magrittr_1.5 plyr_1.8.3 tools_3.2.1 reshape2_1.4.1 [6] Rcpp_0.11.6 stringi_0.5-5 stringr_1.0.0 chron_2.3-4 ``` ~g -- View this message in context: http://r.789695.n4.nabble.com/Apparent-loss-of-decimals-when-reading-a-numeric-column-with-fread-tp4710722p4710727.html Sent from the datatable-help mailing list archive at Nabble.com. From mel at mbacou.com Tue Aug 4 13:54:50 2015 From: mel at mbacou.com (Bacou, Melanie) Date: Tue, 4 Aug 2015 07:54:50 -0400 Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! In-Reply-To: <1438687006319-4710727.post@n4.nabble.com> References: <55C08856.40903@mbacou.com> <1438687006319-4710727.post@n4.nabble.com> Message-ID: <55C0A80A.3030402@mbacou.com> Hi, Thx, I see I have another problem, not related to data.table (sorry). R seems to truncate numbers in the console. Not sure what's going on. --Mel. ```{r} > 1-0.5 [1] 0.5 > 2008-0.05 [1] 2008 > 45-0.5 [1] 44.5 > 100-0.5 [1] 99.5 > 1000-0.5 [1] 1000 > 10000-0.5 [1] 10000 ``` On 8/4/2015 7:16 AM, nachti wrote: > Hi, > copying your code, everything works as expected for me. > Maybe you just referenced to a wrong object (pcn08)? > > ``` >> library(data.table) > data.table 1.9.4 For help type: ?data.table > *** NB: by=.EACHI is now explicit. See README to restore previous behaviour. >> pcn <- fread("PovCalServlet_15.08.03.csv") > sapply(pcn, class) > pcn <- fread("PovCalServlet_15.08.03.csv") >> sapply(pcn, class) > country povLine mean hcr gap sev > "character" "numeric" "numeric" "numeric" "numeric" "numeric" > watts popM yearNum > "numeric" "numeric" "numeric" >> pcn08$yearNum > Error: object 'pcn08' not found >> pcn$yearNum > [1] 2008.50 2011.50 2009.25 2009.00 2006.00 2007.50 2007.00 2008.00 2011.00 > [10] 2004.00 2005.50 2011.00 2008.00 2010.50 2005.00 2003.00 2005.50 2008.00 > [19] 2007.00 2012.00 2002.00 2005.40 2010.00 2007.00 2010.00 2010.23 2010.00 > [28] 2008.00 2008.00 2012.00 2006.00 2008.64 2009.50 2011.00 2009.83 2010.83 > [37] 2010.00 2011.00 2006.50 2011.00 2010.67 2009.00 2009.50 2011.80 2011.00 > [46] 2008.00 2012.50 2009.30 2010.00 2008.50 2011.50 2009.25 2009.00 2006.00 > [55] 2007.50 2007.00 2008.00 2011.00 2004.00 2005.50 2011.00 2008.00 2010.50 > [64] 2005.00 2003.00 2005.50 2008.00 2007.00 2012.00 2002.00 2005.40 2010.00 > [73] 2007.00 2010.00 2010.23 2010.00 2008.00 2008.00 2012.00 2006.00 2008.64 > [82] 2009.50 2011.00 2009.83 2010.83 2010.00 2011.00 2006.50 2011.00 2010.67 > [91] 2009.00 2009.50 2011.80 2011.00 2008.00 2012.50 2009.30 2010.00 >> sessionInfo() > R version 3.2.1 (2015-06-18) > Platform: x86_64-suse-linux-gnu (64-bit) > Running under: openSUSE 13.1 (Bottle) (x86_64) > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] data.table_1.9.4 > > loaded via a namespace (and not attached): > [1] compiler_3.2.1 magrittr_1.5 plyr_1.8.3 tools_3.2.1 > reshape2_1.4.1 > [6] Rcpp_0.11.6 stringi_0.5-5 stringr_1.0.0 chron_2.3-4 > ``` > > ~g > > > > -- > View this message in context: http://r.789695.n4.nabble.com/Apparent-loss-of-decimals-when-reading-a-numeric-column-with-fread-tp4710722p4710727.html > Sent from the datatable-help mailing list archive at Nabble.com. > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help From mel at mbacou.com Tue Aug 4 14:06:39 2015 From: mel at mbacou.com (Bacou, Melanie) Date: Tue, 4 Aug 2015 08:06:39 -0400 Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! In-Reply-To: <55C0A80A.3030402@mbacou.com> References: <55C08856.40903@mbacou.com> <1438687006319-4710727.post@n4.nabble.com> <55C0A80A.3030402@mbacou.com> Message-ID: <55C0AACF.7050407@mbacou.com> In case that happens to others. Seems related to R global `digits` options, maybe something changed in R.3.2.1. --Mel. ``` options(digits=3) 1000-0.5 # [1] 1000 options(digits=5) 1000-0.5 # [1] 999.5 ``` On 8/4/2015 7:54 AM, Bacou, Melanie wrote: > Hi, > Thx, I see I have another problem, not related to data.table (sorry). > R seems to truncate numbers in the console. Not sure what's going on. > > --Mel. > > ```{r} > > 1-0.5 > [1] 0.5 > > 2008-0.05 > [1] 2008 > > 45-0.5 > [1] 44.5 > > 100-0.5 > [1] 99.5 > > 1000-0.5 > [1] 1000 > > 10000-0.5 > [1] 10000 > ``` > > On 8/4/2015 7:16 AM, nachti wrote: >> Hi, >> copying your code, everything works as expected for me. >> Maybe you just referenced to a wrong object (pcn08)? >> >> ``` >>> library(data.table) >> data.table 1.9.4 For help type: ?data.table >> *** NB: by=.EACHI is now explicit. See README to restore previous >> behaviour. >>> pcn <- fread("PovCalServlet_15.08.03.csv") >> sapply(pcn, class) >> pcn <- fread("PovCalServlet_15.08.03.csv") >>> sapply(pcn, class) >> country povLine mean hcr gap sev >> "character" "numeric" "numeric" "numeric" "numeric" "numeric" >> watts popM yearNum >> "numeric" "numeric" "numeric" >>> pcn08$yearNum >> Error: object 'pcn08' not found >>> pcn$yearNum >> [1] 2008.50 2011.50 2009.25 2009.00 2006.00 2007.50 2007.00 2008.00 >> 2011.00 >> [10] 2004.00 2005.50 2011.00 2008.00 2010.50 2005.00 2003.00 2005.50 >> 2008.00 >> [19] 2007.00 2012.00 2002.00 2005.40 2010.00 2007.00 2010.00 2010.23 >> 2010.00 >> [28] 2008.00 2008.00 2012.00 2006.00 2008.64 2009.50 2011.00 2009.83 >> 2010.83 >> [37] 2010.00 2011.00 2006.50 2011.00 2010.67 2009.00 2009.50 2011.80 >> 2011.00 >> [46] 2008.00 2012.50 2009.30 2010.00 2008.50 2011.50 2009.25 2009.00 >> 2006.00 >> [55] 2007.50 2007.00 2008.00 2011.00 2004.00 2005.50 2011.00 2008.00 >> 2010.50 >> [64] 2005.00 2003.00 2005.50 2008.00 2007.00 2012.00 2002.00 2005.40 >> 2010.00 >> [73] 2007.00 2010.00 2010.23 2010.00 2008.00 2008.00 2012.00 2006.00 >> 2008.64 >> [82] 2009.50 2011.00 2009.83 2010.83 2010.00 2011.00 2006.50 2011.00 >> 2010.67 >> [91] 2009.00 2009.50 2011.80 2011.00 2008.00 2012.50 2009.30 2010.00 >>> sessionInfo() >> R version 3.2.1 (2015-06-18) >> Platform: x86_64-suse-linux-gnu (64-bit) >> Running under: openSUSE 13.1 (Bottle) (x86_64) >> >> locale: >> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C >> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 >> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 >> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C >> [9] LC_ADDRESS=C LC_TELEPHONE=C >> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C >> >> attached base packages: >> [1] stats graphics grDevices utils datasets methods base >> >> other attached packages: >> [1] data.table_1.9.4 >> >> loaded via a namespace (and not attached): >> [1] compiler_3.2.1 magrittr_1.5 plyr_1.8.3 tools_3.2.1 >> reshape2_1.4.1 >> [6] Rcpp_0.11.6 stringi_0.5-5 stringr_1.0.0 chron_2.3-4 >> ``` >> >> ~g >> >> >> >> -- >> View this message in context: >> http://r.789695.n4.nabble.com/Apparent-loss-of-decimals-when-reading-a-numeric-column-with-fread-tp4710722p4710727.html >> Sent from the datatable-help mailing list archive at Nabble.com. >> _______________________________________________ >> datatable-help mailing list >> datatable-help at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >> > From mel at mbacou.com Tue Aug 4 14:54:28 2015 From: mel at mbacou.com (Bacou, Melanie) Date: Tue, 4 Aug 2015 08:54:28 -0400 Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! In-Reply-To: <55C0AACF.7050407@mbacou.com> References: <55C08856.40903@mbacou.com> <1438687006319-4710727.post@n4.nabble.com> <55C0A80A.3030402@mbacou.com> <55C0AACF.7050407@mbacou.com> Message-ID: <55C0B604.5030002@mbacou.com> Starting to wonder if something else is going on with my R install. Using the same CSV file, I'm not getting what I expect with `.I` (I would expect an index by group). ``` pcn08 <- fread("./data/PovCalServlet_15.08.03.csv") # and then grouping by 2 fields pcn08[, test := .I, by=list(country, povLine)] pcn08$test # [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # [28] 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 # [55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 # [82] 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 # but there are 86 groups dim(pcn08[, .N, by=list(country, povLine)]) # [1] 86 3 ``` On 8/4/2015 8:06 AM, Bacou, Melanie wrote: > In case that happens to others. Seems related to R global `digits` > options, maybe something changed in R.3.2.1. > --Mel. > > ``` > options(digits=3) > 1000-0.5 > # [1] 1000 > options(digits=5) > 1000-0.5 > > # [1] 999.5 > > ``` > > On 8/4/2015 7:54 AM, Bacou, Melanie wrote: >> Hi, >> Thx, I see I have another problem, not related to data.table (sorry). >> R seems to truncate numbers in the console. Not sure what's going on. >> >> --Mel. >> >> ```{r} >> > 1-0.5 >> [1] 0.5 >> > 2008-0.05 >> [1] 2008 >> > 45-0.5 >> [1] 44.5 >> > 100-0.5 >> [1] 99.5 >> > 1000-0.5 >> [1] 1000 >> > 10000-0.5 >> [1] 10000 >> ``` >> >> On 8/4/2015 7:16 AM, nachti wrote: >>> Hi, >>> copying your code, everything works as expected for me. >>> Maybe you just referenced to a wrong object (pcn08)? >>> >>> ``` >>>> library(data.table) >>> data.table 1.9.4 For help type: ?data.table >>> *** NB: by=.EACHI is now explicit. See README to restore previous >>> behaviour. >>>> pcn <- fread("PovCalServlet_15.08.03.csv") >>> sapply(pcn, class) >>> pcn <- fread("PovCalServlet_15.08.03.csv") >>>> sapply(pcn, class) >>> country povLine mean hcr gap sev >>> "character" "numeric" "numeric" "numeric" "numeric" "numeric" >>> watts popM yearNum >>> "numeric" "numeric" "numeric" >>>> pcn08$yearNum >>> Error: object 'pcn08' not found >>>> pcn$yearNum >>> [1] 2008.50 2011.50 2009.25 2009.00 2006.00 2007.50 2007.00 >>> 2008.00 2011.00 >>> [10] 2004.00 2005.50 2011.00 2008.00 2010.50 2005.00 2003.00 2005.50 >>> 2008.00 >>> [19] 2007.00 2012.00 2002.00 2005.40 2010.00 2007.00 2010.00 2010.23 >>> 2010.00 >>> [28] 2008.00 2008.00 2012.00 2006.00 2008.64 2009.50 2011.00 2009.83 >>> 2010.83 >>> [37] 2010.00 2011.00 2006.50 2011.00 2010.67 2009.00 2009.50 2011.80 >>> 2011.00 >>> [46] 2008.00 2012.50 2009.30 2010.00 2008.50 2011.50 2009.25 2009.00 >>> 2006.00 >>> [55] 2007.50 2007.00 2008.00 2011.00 2004.00 2005.50 2011.00 2008.00 >>> 2010.50 >>> [64] 2005.00 2003.00 2005.50 2008.00 2007.00 2012.00 2002.00 2005.40 >>> 2010.00 >>> [73] 2007.00 2010.00 2010.23 2010.00 2008.00 2008.00 2012.00 2006.00 >>> 2008.64 >>> [82] 2009.50 2011.00 2009.83 2010.83 2010.00 2011.00 2006.50 2011.00 >>> 2010.67 >>> [91] 2009.00 2009.50 2011.80 2011.00 2008.00 2012.50 2009.30 2010.00 >>>> sessionInfo() >>> R version 3.2.1 (2015-06-18) >>> Platform: x86_64-suse-linux-gnu (64-bit) >>> Running under: openSUSE 13.1 (Bottle) (x86_64) >>> >>> locale: >>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C >>> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 >>> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 >>> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C >>> [9] LC_ADDRESS=C LC_TELEPHONE=C >>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C >>> >>> attached base packages: >>> [1] stats graphics grDevices utils datasets methods base >>> >>> other attached packages: >>> [1] data.table_1.9.4 >>> >>> loaded via a namespace (and not attached): >>> [1] compiler_3.2.1 magrittr_1.5 plyr_1.8.3 tools_3.2.1 >>> reshape2_1.4.1 >>> [6] Rcpp_0.11.6 stringi_0.5-5 stringr_1.0.0 chron_2.3-4 >>> ``` >>> >>> ~g >>> >>> >>> >>> -- >>> View this message in context: >>> http://r.789695.n4.nabble.com/Apparent-loss-of-decimals-when-reading-a-numeric-column-with-fread-tp4710722p4710727.html >>> Sent from the datatable-help mailing list archive at Nabble.com. >>> _______________________________________________ >>> datatable-help mailing list >>> datatable-help at lists.r-forge.r-project.org >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >>> >> > From fperickson at wisc.edu Tue Aug 4 15:02:43 2015 From: fperickson at wisc.edu (Frank Erickson) Date: Tue, 4 Aug 2015 09:02:43 -0400 Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! In-Reply-To: <55C0B604.5030002@mbacou.com> References: <55C08856.40903@mbacou.com> <1438687006319-4710727.post@n4.nabble.com> <55C0A80A.3030402@mbacou.com> <55C0AACF.7050407@mbacou.com> <55C0B604.5030002@mbacou.com> Message-ID: .I was never an index by group. You'd have to make that more manually, like 1:.N On Tue, Aug 4, 2015 at 8:54 AM, Bacou, Melanie wrote: > Starting to wonder if something else is going on with my R install. Using > the same CSV file, I'm not getting what I expect with `.I` (I would expect > an index by group). > > ``` > pcn08 <- fread("./data/PovCalServlet_15.08.03.csv") > > # and then grouping by 2 fields > pcn08[, test := .I, by=list(country, povLine)] > > pcn08$test > # [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 > 24 25 26 27 > # [28] 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 > 50 51 52 53 54 > # [55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 > 77 78 79 80 81 > # [82] 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 > > # but there are 86 groups > dim(pcn08[, .N, by=list(country, povLine)]) > # [1] 86 3 > > ``` > > > On 8/4/2015 8:06 AM, Bacou, Melanie wrote: > >> In case that happens to others. Seems related to R global `digits` >> options, maybe something changed in R.3.2.1. >> --Mel. >> >> ``` >> options(digits=3) >> 1000-0.5 >> # [1] 1000 >> options(digits=5) >> 1000-0.5 >> >> # [1] 999.5 >> >> ``` >> >> On 8/4/2015 7:54 AM, Bacou, Melanie wrote: >> >>> Hi, >>> Thx, I see I have another problem, not related to data.table (sorry). R >>> seems to truncate numbers in the console. Not sure what's going on. >>> >>> --Mel. >>> >>> ```{r} >>> > 1-0.5 >>> [1] 0.5 >>> > 2008-0.05 >>> [1] 2008 >>> > 45-0.5 >>> [1] 44.5 >>> > 100-0.5 >>> [1] 99.5 >>> > 1000-0.5 >>> [1] 1000 >>> > 10000-0.5 >>> [1] 10000 >>> ``` >>> >>> On 8/4/2015 7:16 AM, nachti wrote: >>> >>>> Hi, >>>> copying your code, everything works as expected for me. >>>> Maybe you just referenced to a wrong object (pcn08)? >>>> >>>> ``` >>>> >>>>> library(data.table) >>>>> >>>> data.table 1.9.4 For help type: ?data.table >>>> *** NB: by=.EACHI is now explicit. See README to restore previous >>>> behaviour. >>>> >>>>> pcn <- fread("PovCalServlet_15.08.03.csv") >>>>> >>>> sapply(pcn, class) >>>> pcn <- fread("PovCalServlet_15.08.03.csv") >>>> >>>>> sapply(pcn, class) >>>>> >>>> country povLine mean hcr gap sev >>>> "character" "numeric" "numeric" "numeric" "numeric" "numeric" >>>> watts popM yearNum >>>> "numeric" "numeric" "numeric" >>>> >>>>> pcn08$yearNum >>>>> >>>> Error: object 'pcn08' not found >>>> >>>>> pcn$yearNum >>>>> >>>> [1] 2008.50 2011.50 2009.25 2009.00 2006.00 2007.50 2007.00 2008.00 >>>> 2011.00 >>>> [10] 2004.00 2005.50 2011.00 2008.00 2010.50 2005.00 2003.00 2005.50 >>>> 2008.00 >>>> [19] 2007.00 2012.00 2002.00 2005.40 2010.00 2007.00 2010.00 2010.23 >>>> 2010.00 >>>> [28] 2008.00 2008.00 2012.00 2006.00 2008.64 2009.50 2011.00 2009.83 >>>> 2010.83 >>>> [37] 2010.00 2011.00 2006.50 2011.00 2010.67 2009.00 2009.50 2011.80 >>>> 2011.00 >>>> [46] 2008.00 2012.50 2009.30 2010.00 2008.50 2011.50 2009.25 2009.00 >>>> 2006.00 >>>> [55] 2007.50 2007.00 2008.00 2011.00 2004.00 2005.50 2011.00 2008.00 >>>> 2010.50 >>>> [64] 2005.00 2003.00 2005.50 2008.00 2007.00 2012.00 2002.00 2005.40 >>>> 2010.00 >>>> [73] 2007.00 2010.00 2010.23 2010.00 2008.00 2008.00 2012.00 2006.00 >>>> 2008.64 >>>> [82] 2009.50 2011.00 2009.83 2010.83 2010.00 2011.00 2006.50 2011.00 >>>> 2010.67 >>>> [91] 2009.00 2009.50 2011.80 2011.00 2008.00 2012.50 2009.30 2010.00 >>>> >>>>> sessionInfo() >>>>> >>>> R version 3.2.1 (2015-06-18) >>>> Platform: x86_64-suse-linux-gnu (64-bit) >>>> Running under: openSUSE 13.1 (Bottle) (x86_64) >>>> >>>> locale: >>>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C >>>> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 >>>> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 >>>> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C >>>> [9] LC_ADDRESS=C LC_TELEPHONE=C >>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C >>>> >>>> attached base packages: >>>> [1] stats graphics grDevices utils datasets methods base >>>> >>>> other attached packages: >>>> [1] data.table_1.9.4 >>>> >>>> loaded via a namespace (and not attached): >>>> [1] compiler_3.2.1 magrittr_1.5 plyr_1.8.3 tools_3.2.1 >>>> reshape2_1.4.1 >>>> [6] Rcpp_0.11.6 stringi_0.5-5 stringr_1.0.0 chron_2.3-4 >>>> ``` >>>> >>>> ~g >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://r.789695.n4.nabble.com/Apparent-loss-of-decimals-when-reading-a-numeric-column-with-fread-tp4710722p4710727.html >>>> Sent from the datatable-help mailing list archive at Nabble.com. >>>> _______________________________________________ >>>> datatable-help mailing list >>>> datatable-help at lists.r-forge.r-project.org >>>> >>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >>>> >>> >>> >> > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mel at mbacou.com Tue Aug 4 15:06:06 2015 From: mel at mbacou.com (Bacou, Melanie) Date: Tue, 4 Aug 2015 09:06:06 -0400 Subject: [datatable-help] Apparent loss of decimals when reading a numeric column with fread() ! In-Reply-To: References: <55C08856.40903@mbacou.com> <1438687006319-4710727.post@n4.nabble.com> <55C0A80A.3030402@mbacou.com> <55C0AACF.7050407@mbacou.com> <55C0B604.5030002@mbacou.com> Message-ID: <55C0B8BE.1080002@mbacou.com> Right, thanks Frank. Sorry, I think fatigue is piling up here! --Mel. On 8/4/2015 9:02 AM, Frank Erickson wrote: > .I was never an index by group. You'd have to make that more manually, > like 1:.N > > On Tue, Aug 4, 2015 at 8:54 AM, Bacou, Melanie > wrote: > > Starting to wonder if something else is going on with my R > install. Using the same CSV file, I'm not getting what I expect > with `.I` (I would expect an index by group). > > ``` > pcn08 <- fread("./data/PovCalServlet_15.08.03.csv") > > # and then grouping by 2 fields > pcn08[, test := .I, by=list(country, povLine)] > > pcn08$test > # [1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 > 21 22 23 24 25 26 27 > # [28] 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 > 48 49 50 51 52 53 54 > # [55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 > 75 76 77 78 79 80 81 > # [82] 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 > > # but there are 86 groups > dim(pcn08[, .N, by=list(country, povLine)]) > # [1] 86 3 > > ``` > > > On 8/4/2015 8:06 AM, Bacou, Melanie wrote: > > In case that happens to others. Seems related to R global > `digits` options, maybe something changed in R.3.2.1. > --Mel. > > ``` > options(digits=3) > 1000-0.5 > # [1] 1000 > options(digits=5) > 1000-0.5 > > # [1] 999.5 > > ``` > > On 8/4/2015 7:54 AM, Bacou, Melanie wrote: > > Hi, > Thx, I see I have another problem, not related to > data.table (sorry). R seems to truncate numbers in the > console. Not sure what's going on. > > --Mel. > > ```{r} > > 1-0.5 > [1] 0.5 > > 2008-0.05 > [1] 2008 > > 45-0.5 > [1] 44.5 > > 100-0.5 > [1] 99.5 > > 1000-0.5 > [1] 1000 > > 10000-0.5 > [1] 10000 > ``` > > On 8/4/2015 7:16 AM, nachti wrote: > > Hi, > copying your code, everything works as expected for me. > Maybe you just referenced to a wrong object (pcn08)? > > ``` > > library(data.table) > > data.table 1.9.4 For help type: ?data.table > *** NB: by=.EACHI is now explicit. See README to > restore previous behaviour. > > pcn <- fread("PovCalServlet_15.08.03.csv") > > sapply(pcn, class) > pcn <- fread("PovCalServlet_15.08.03.csv") > > sapply(pcn, class) > > country povLine mean hcr gap sev > "character" "numeric" "numeric" "numeric" > "numeric" "numeric" > watts popM yearNum > "numeric" "numeric" "numeric" > > pcn08$yearNum > > Error: object 'pcn08' not found > > pcn$yearNum > > [1] 2008.50 2011.50 2009.25 2009.00 2006.00 2007.50 > 2007.00 2008.00 2011.00 > [10] 2004.00 2005.50 2011.00 2008.00 2010.50 2005.00 > 2003.00 2005.50 2008.00 > [19] 2007.00 2012.00 2002.00 2005.40 2010.00 2007.00 > 2010.00 2010.23 2010.00 > [28] 2008.00 2008.00 2012.00 2006.00 2008.64 2009.50 > 2011.00 2009.83 2010.83 > [37] 2010.00 2011.00 2006.50 2011.00 2010.67 2009.00 > 2009.50 2011.80 2011.00 > [46] 2008.00 2012.50 2009.30 2010.00 2008.50 2011.50 > 2009.25 2009.00 2006.00 > [55] 2007.50 2007.00 2008.00 2011.00 2004.00 2005.50 > 2011.00 2008.00 2010.50 > [64] 2005.00 2003.00 2005.50 2008.00 2007.00 2012.00 > 2002.00 2005.40 2010.00 > [73] 2007.00 2010.00 2010.23 2010.00 2008.00 2008.00 > 2012.00 2006.00 2008.64 > [82] 2009.50 2011.00 2009.83 2010.83 2010.00 2011.00 > 2006.50 2011.00 2010.67 > [91] 2009.00 2009.50 2011.80 2011.00 2008.00 2012.50 > 2009.30 2010.00 > > sessionInfo() > > R version 3.2.1 (2015-06-18) > Platform: x86_64-suse-linux-gnu (64-bit) > Running under: openSUSE 13.1 (Bottle) (x86_64) > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets > methods base > > other attached packages: > [1] data.table_1.9.4 > > loaded via a namespace (and not attached): > [1] compiler_3.2.1 magrittr_1.5 plyr_1.8.3 tools_3.2.1 > reshape2_1.4.1 > [6] Rcpp_0.11.6 stringi_0.5-5 stringr_1.0.0 > chron_2.3-4 > ``` > > ~g > > > > -- > View this message in context: > http://r.789695.n4.nabble.com/Apparent-loss-of-decimals-when-reading-a-numeric-column-with-fread-tp4710722p4710727.html > Sent from the datatable-help mailing list archive at > Nabble.com. > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > > > > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From statquant at outlook.com Wed Aug 5 13:47:47 2015 From: statquant at outlook.com (statquant3) Date: Wed, 5 Aug 2015 04:47:47 -0700 (PDT) Subject: [datatable-help] bug is fread ? Message-ID: <1438775267348-4710776.post@n4.nabble.com> Trying to read the following csv: date,time,name,propertyValue,programName 20150804,07:17:22.928,aerhqer,strjhwr,adtrhadtr 20150804,07:17:22.928,A__z,"*SHLVL="*TMOUT,datrhnaetrh 20150804,07:17:22.928,adrhae,true,adthad R) fread('test.csv') Error in fread("/u/delta1/colinu/test/freadTest/PropertiesLog.csv") : Field 4 on line 3 starts with quote (") but then has a problem. It can contain balanced unescaped quoted subregions but if it does it can't contain embedded \n as well. Check for unbalanced unescaped quotes: "*SHLVL="*TMOUT,datrhnaetrh read.csv2 works... R) setDT(read.csv2(file='/u/delta1/colinu/test/freadTest/PropertiesLog.csv',sep=','))[] date time name propertyValue programName 1: 20150804 07:17:22.928 aerhqer strjhwr adtrhadtr 2: 20150804 07:17:22.928 A__z *SHLVL=*TMOUT datrhnaetrh 3: 20150804 07:17:22.928 adrhae true adthad Can you guys reproduce, can that be considered a bug ? If not how can I trick fread in reproducing read.csv2 ? -- View this message in context: http://r.789695.n4.nabble.com/bug-is-fread-tp4710776.html Sent from the datatable-help mailing list archive at Nabble.com. From aragorn168b at gmail.com Wed Aug 5 13:52:00 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Wed, 5 Aug 2015 13:52:00 +0200 Subject: [datatable-help] bug is fread ? In-Reply-To: <1438775267348-4710776.post@n4.nabble.com> References: <1438775267348-4710776.post@n4.nabble.com> Message-ID: Yes. (Known) bug. Will fix. --? Arun On 5 Aug 2015 at 13:50:31, statquant3 (statquant at outlook.com) wrote: Trying to read the following csv: date,time,name,propertyValue,programName 20150804,07:17:22.928,aerhqer,strjhwr,adtrhadtr 20150804,07:17:22.928,A__z,"*SHLVL="*TMOUT,datrhnaetrh 20150804,07:17:22.928,adrhae,true,adthad R) fread('test.csv') Error in fread("/u/delta1/colinu/test/freadTest/PropertiesLog.csv") : Field 4 on line 3 starts with quote (") but then has a problem. It can contain balanced unescaped quoted subregions but if it does it can't contain embedded \n as well. Check for unbalanced unescaped quotes: "*SHLVL="*TMOUT,datrhnaetrh read.csv2 works... R) setDT(read.csv2(file='/u/delta1/colinu/test/freadTest/PropertiesLog.csv',sep=','))[] date time name propertyValue programName 1: 20150804 07:17:22.928 aerhqer strjhwr adtrhadtr 2: 20150804 07:17:22.928 A__z *SHLVL=*TMOUT datrhnaetrh 3: 20150804 07:17:22.928 adrhae true adthad Can you guys reproduce, can that be considered a bug ? If not how can I trick fread in reproducing read.csv2 ? -- View this message in context: http://r.789695.n4.nabble.com/bug-is-fread-tp4710776.html Sent from the datatable-help mailing list archive at Nabble.com. _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleynes+r at gmail.com Wed Aug 5 20:24:16 2015 From: gleynes+r at gmail.com (Gene Leynes) Date: Wed, 5 Aug 2015 13:24:16 -0500 Subject: [datatable-help] Any date for 1.9.6 push to CRAN? In-Reply-To: References: Message-ID: I was also hoping to find out the next push date. There have been some significant bug fixes since last October! Any way that someone (maybe even me) can help on the outstanding issues? I went to the valgrind site hoping to find the issues, but I guess there's no way to see the tests unless you're the admin? Thanks, Gene PS: valgrind should link to data.table too! http://valgrind.org/gallery/users.html#scientific > Contact valgrind at valgrind.org if you are happy to have your project added > to this list. Please provide a URL and a short description of your project. On Thu, Jul 23, 2015 at 9:44 AM, Matt Dowle wrote: > Hi Damian, > valgrind is reporting 3 issues and there's a https:// issue in the > vignette when run on win-builder. But all cran + bioconductor > packages downstream are checked ok and we're passing big endian as > emulated thanks to qemu so we can be sure to pass solaris sparc. > Hard to give an eta. > Matt > > > On Wed, Jul 22, 2015 at 3:10 AM, Damian Betebenner > wrote: > > Hi all, > > > > It was mentioned recently that 1.9.6 would be pushed to CRAN soon. Any > rough dates when that might occur? > > > > Alway appreciative for the great work! > > > > Damian > > _______________________________________________ > > datatable-help mailing list > > datatable-help at lists.r-forge.r-project.org > > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattjdowle at gmail.com Wed Aug 5 21:07:25 2015 From: mattjdowle at gmail.com (Matt Dowle) Date: Wed, 5 Aug 2015 12:07:25 -0700 Subject: [datatable-help] Any date for 1.9.6 push to CRAN? In-Reply-To: References: Message-ID: That would be great. Here's how to run through valgrind on Ubuntu. R CMD build data.table R CMD INSTALL data.table_1.9.5.tar.gz sudo apt-get install valgrind R -d "valgrind --tool=memcheck --leak-check=full" --vanilla require(data.table) require(bit64) test.data.table() options(datatable.verbose=TRUE) test.data.table() Matt On Wed, Aug 5, 2015 at 11:24 AM, Gene Leynes wrote: > I was also hoping to find out the next push date. There have been some > significant bug fixes since last October! > > Any way that someone (maybe even me) can help on the outstanding issues? > > I went to the valgrind site hoping to find the issues, but I guess there's > no way to see the tests unless you're the admin? > > > Thanks, > > Gene > > PS: valgrind should link to data.table too! > http://valgrind.org/gallery/users.html#scientific >> >> Contact valgrind at valgrind.org if you are happy to have your project added >> to this list. Please provide a URL and a short description of your project. > > > > On Thu, Jul 23, 2015 at 9:44 AM, Matt Dowle wrote: >> >> Hi Damian, >> valgrind is reporting 3 issues and there's a https:// issue in the >> vignette when run on win-builder. But all cran + bioconductor >> packages downstream are checked ok and we're passing big endian as >> emulated thanks to qemu so we can be sure to pass solaris sparc. >> Hard to give an eta. >> Matt >> >> >> On Wed, Jul 22, 2015 at 3:10 AM, Damian Betebenner >> wrote: >> > Hi all, >> > >> > It was mentioned recently that 1.9.6 would be pushed to CRAN soon. Any >> > rough dates when that might occur? >> > >> > Alway appreciative for the great work! >> > >> > Damian >> > _______________________________________________ >> > datatable-help mailing list >> > datatable-help at lists.r-forge.r-project.org >> > >> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >> _______________________________________________ >> datatable-help mailing list >> datatable-help at lists.r-forge.r-project.org >> >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > > From jmtruppia at gmail.com Thu Aug 6 15:13:07 2015 From: jmtruppia at gmail.com (Juan Manuel Truppia) Date: Thu, 06 Aug 2015 13:13:07 +0000 Subject: [datatable-help] Any date for 1.9.6 push to CRAN? In-Reply-To: References: Message-ID: Last time I installed data.table from source on Windows (7) I didn't have any issue. Do you have the commands you would like me to try? Or just install from source the latest master from Github? On Thu, Aug 6, 2015 at 7:00 AM < datatable-help-request at lists.r-forge.r-project.org> wrote: > Send datatable-help mailing list submissions to > datatable-help at lists.r-forge.r-project.org > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > or, via email, send a message with subject or body 'help' to > datatable-help-request at lists.r-forge.r-project.org > > You can reach the person managing the list at > datatable-help-owner at lists.r-forge.r-project.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of datatable-help digest..." > > > Today's Topics: > > 1. bug is fread ? (statquant3) > 2. Re: bug is fread ? (Arunkumar Srinivasan) > 3. Re: Any date for 1.9.6 push to CRAN? (Gene Leynes) > 4. Re: Any date for 1.9.6 push to CRAN? (Matt Dowle) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 5 Aug 2015 04:47:47 -0700 (PDT) > From: statquant3 > To: datatable-help at lists.r-forge.r-project.org > Subject: [datatable-help] bug is fread ? > Message-ID: <1438775267348-4710776.post at n4.nabble.com> > Content-Type: text/plain; charset=us-ascii > > Trying to read the following csv: > > date,time,name,propertyValue,programName > 20150804,07:17:22.928,aerhqer,strjhwr,adtrhadtr > 20150804,07:17:22.928,A__z,"*SHLVL="*TMOUT,datrhnaetrh > 20150804,07:17:22.928,adrhae,true,adthad > > R) fread('test.csv') > Error in fread("/u/delta1/colinu/test/freadTest/PropertiesLog.csv") : > Field 4 on line 3 starts with quote (") but then has a problem. It can > contain balanced unescaped quoted subregions but if it does it can't > contain > embedded \n as well. Check for unbalanced unescaped quotes: > "*SHLVL="*TMOUT,datrhnaetrh > > read.csv2 works... > > R) > > setDT(read.csv2(file='/u/delta1/colinu/test/freadTest/PropertiesLog.csv',sep=','))[] > date time name propertyValue programName > 1: 20150804 07:17:22.928 aerhqer strjhwr adtrhadtr > 2: 20150804 07:17:22.928 A__z *SHLVL=*TMOUT datrhnaetrh > 3: 20150804 07:17:22.928 adrhae true adthad > > Can you guys reproduce, can that be considered a bug ? > If not how can I trick fread in reproducing read.csv2 ? > > > > > -- > View this message in context: > http://r.789695.n4.nabble.com/bug-is-fread-tp4710776.html > Sent from the datatable-help mailing list archive at Nabble.com. > > > ------------------------------ > > Message: 2 > Date: Wed, 5 Aug 2015 13:52:00 +0200 > From: Arunkumar Srinivasan > To: statquant3 , > datatable-help at lists.r-forge.r-project.org > Subject: Re: [datatable-help] bug is fread ? > Message-ID: > Content-Type: text/plain; charset="utf-8" > > Yes. (Known) bug. Will fix. > > --? > Arun > > On 5 Aug 2015 at 13:50:31, statquant3 (statquant at outlook.com) wrote: > > Trying to read the following csv: > > date,time,name,propertyValue,programName > 20150804,07:17:22.928,aerhqer,strjhwr,adtrhadtr > 20150804,07:17:22.928,A__z,"*SHLVL="*TMOUT,datrhnaetrh > 20150804,07:17:22.928,adrhae,true,adthad > > R) fread('test.csv') > Error in fread("/u/delta1/colinu/test/freadTest/PropertiesLog.csv") : > Field 4 on line 3 starts with quote (") but then has a problem. It can > contain balanced unescaped quoted subregions but if it does it can't > contain > embedded \n as well. Check for unbalanced unescaped quotes: > "*SHLVL="*TMOUT,datrhnaetrh > > read.csv2 works... > > R) > > setDT(read.csv2(file='/u/delta1/colinu/test/freadTest/PropertiesLog.csv',sep=','))[] > date time name propertyValue programName > 1: 20150804 07:17:22.928 aerhqer strjhwr adtrhadtr > 2: 20150804 07:17:22.928 A__z *SHLVL=*TMOUT datrhnaetrh > 3: 20150804 07:17:22.928 adrhae true adthad > > Can you guys reproduce, can that be considered a bug ? > If not how can I trick fread in reproducing read.csv2 ? > > > > > -- > View this message in context: > http://r.789695.n4.nabble.com/bug-is-fread-tp4710776.html > Sent from the datatable-help mailing list archive at Nabble.com. > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20150805/bf9f3a17/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Wed, 5 Aug 2015 13:24:16 -0500 > From: Gene Leynes > To: Matt Dowle > Cc: Damian Betebenner , > "datatable-help at lists.r-forge.r-project.org" > > Subject: Re: [datatable-help] Any date for 1.9.6 push to CRAN? > Message-ID: > < > CAOBARViLW7MSXQFN9KwM0pbzA+kJSUVi_QJ4Wa1AxR8M21+u7A at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > I was also hoping to find out the next push date. There have been some > significant bug fixes since last October! > > Any way that someone (maybe even me) can help on the outstanding issues? > > I went to the valgrind site hoping to find the issues, but I guess there's > no way to see the tests unless you're the admin? > > > Thanks, > > Gene > > PS: valgrind should link to data.table too! > http://valgrind.org/gallery/users.html#scientific > > > Contact valgrind at valgrind.org if you are happy to have your project > added > > to this list. Please provide a URL and a short description of your > project. > > > > On Thu, Jul 23, 2015 at 9:44 AM, Matt Dowle wrote: > > > Hi Damian, > > valgrind is reporting 3 issues and there's a https:// issue in the > > vignette when run on win-builder. But all cran + bioconductor > > packages downstream are checked ok and we're passing big endian as > > emulated thanks to qemu so we can be sure to pass solaris sparc. > > Hard to give an eta. > > Matt > > > > > > On Wed, Jul 22, 2015 at 3:10 AM, Damian Betebenner > > wrote: > > > Hi all, > > > > > > It was mentioned recently that 1.9.6 would be pushed to CRAN soon. Any > > rough dates when that might occur? > > > > > > Alway appreciative for the great work! > > > > > > Damian > > > _______________________________________________ > > > datatable-help mailing list > > > datatable-help at lists.r-forge.r-project.org > > > > > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > _______________________________________________ > > datatable-help mailing list > > datatable-help at lists.r-forge.r-project.org > > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.r-forge.r-project.org/pipermail/datatable-help/attachments/20150805/ec1af34a/attachment-0001.html > > > > ------------------------------ > > Message: 4 > Date: Wed, 5 Aug 2015 12:07:25 -0700 > From: Matt Dowle > To: gleynes+r at gmail.com > Cc: Damian Betebenner , > "datatable-help at lists.r-forge.r-project.org" > > Subject: Re: [datatable-help] Any date for 1.9.6 push to CRAN? > Message-ID: > RC10+haV-b5OibL23q4sm7EADMv5w-gF-Kz7v0gA at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > That would be great. Here's how to run through valgrind on Ubuntu. > > R CMD build data.table > R CMD INSTALL data.table_1.9.5.tar.gz > sudo apt-get install valgrind > R -d "valgrind --tool=memcheck --leak-check=full" --vanilla > require(data.table) > require(bit64) > test.data.table() > options(datatable.verbose=TRUE) > test.data.table() > > Matt > > > > On Wed, Aug 5, 2015 at 11:24 AM, Gene Leynes wrote: > > I was also hoping to find out the next push date. There have been some > > significant bug fixes since last October! > > > > Any way that someone (maybe even me) can help on the outstanding issues? > > > > I went to the valgrind site hoping to find the issues, but I guess > there's > > no way to see the tests unless you're the admin? > > > > > > Thanks, > > > > Gene > > > > PS: valgrind should link to data.table too! > > http://valgrind.org/gallery/users.html#scientific > >> > >> Contact valgrind at valgrind.org if you are happy to have your project > added > >> to this list. Please provide a URL and a short description of your > project. > > > > > > > > On Thu, Jul 23, 2015 at 9:44 AM, Matt Dowle > wrote: > >> > >> Hi Damian, > >> valgrind is reporting 3 issues and there's a https:// issue in the > >> vignette when run on win-builder. But all cran + bioconductor > >> packages downstream are checked ok and we're passing big endian as > >> emulated thanks to qemu so we can be sure to pass solaris sparc. > >> Hard to give an eta. > >> Matt > >> > >> > >> On Wed, Jul 22, 2015 at 3:10 AM, Damian Betebenner > >> wrote: > >> > Hi all, > >> > > >> > It was mentioned recently that 1.9.6 would be pushed to CRAN soon. Any > >> > rough dates when that might occur? > >> > > >> > Alway appreciative for the great work! > >> > > >> > Damian > >> > _______________________________________________ > >> > datatable-help mailing list > >> > datatable-help at lists.r-forge.r-project.org > >> > > >> > > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > >> _______________________________________________ > >> datatable-help mailing list > >> datatable-help at lists.r-forge.r-project.org > >> > >> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > > > > > > > > ------------------------------ > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > End of datatable-help Digest, Vol 66, Issue 4 > ********************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From statquant at outlook.com Tue Aug 11 14:18:17 2015 From: statquant at outlook.com (statquant3) Date: Tue, 11 Aug 2015 05:18:17 -0700 (PDT) Subject: [datatable-help] rbind lists of list of data table Message-ID: <1439295497444-4710977.post@n4.nabble.com> Say I have a list of list of data.tables l <- list( data.table(a=rnorm(3), b=rnorm(3)) , data.table(c=rnorm(12), d=rnorm(12)) ) ll <- list() ll[[1]] <- l ll[[2]] <- l Is there a smart use of rbindlist or use in conjusction with *apply/foreach that can efficiently bind this list of list to a list of data.table element by element (here we would get a list of 2 data.table with resp. 2columns, 6 rows and 2 columns 24 rows) -- View this message in context: http://r.789695.n4.nabble.com/rbind-lists-of-list-of-data-table-tp4710977.html Sent from the datatable-help mailing list archive at Nabble.com. From eduard.antonyan at gmail.com Tue Aug 11 16:12:09 2015 From: eduard.antonyan at gmail.com (Eduard Antonyan) Date: Tue, 11 Aug 2015 09:12:09 -0500 Subject: [datatable-help] rbind lists of list of data table In-Reply-To: <1439295497444-4710977.post@n4.nabble.com> References: <1439295497444-4710977.post@n4.nabble.com> Message-ID: apply(as.data.table(ll), 1, rbindlist) or if you don't care about keeping the original list as a list, use setDT instead of as.data.table On Tue, Aug 11, 2015 at 7:18 AM, statquant3 wrote: > Say I have a list of list of data.tables > > l <- list( data.table(a=rnorm(3), b=rnorm(3)) , data.table(c=rnorm(12), > d=rnorm(12)) ) > ll <- list() > ll[[1]] <- l > ll[[2]] <- l > > Is there a smart use of rbindlist or use in conjusction with *apply/foreach > that can efficiently bind this list of list to a list of data.table element > by element (here we would get a list of 2 data.table with resp. 2columns, 6 > rows and 2 columns 24 rows) > > > > -- > View this message in context: > http://r.789695.n4.nabble.com/rbind-lists-of-list-of-data-table-tp4710977.html > Sent from the datatable-help mailing list archive at Nabble.com. > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From statquant at outlook.com Tue Aug 18 14:54:12 2015 From: statquant at outlook.com (statquant3) Date: Tue, 18 Aug 2015 05:54:12 -0700 (PDT) Subject: [datatable-help] Idiomatic way of using expression in i Message-ID: <1439902452478-4711229.post@n4.nabble.com> Say I have a data.table like: dt <- data.table(x=rnorn(1000),y=rnorm(1000)) I have to construct a i statement dynamically, for example the result could be iExpr <- 'x>1 & y<.045' How can I evaluate dt[x>1 & y<.045] using iExpr ? -- View this message in context: http://r.789695.n4.nabble.com/Idiomatic-way-of-using-expression-in-i-tp4711229.html Sent from the datatable-help mailing list archive at Nabble.com. From statquant at outlook.com Tue Aug 18 14:57:27 2015 From: statquant at outlook.com (statquant3) Date: Tue, 18 Aug 2015 05:57:27 -0700 (PDT) Subject: [datatable-help] Idiomatic way of using expression in i In-Reply-To: <1439902452478-4711229.post@n4.nabble.com> References: <1439902452478-4711229.post@n4.nabble.com> Message-ID: <1439902647094-4711230.post@n4.nabble.com> DT[eval(parse(text=iExpr))] is working, but it is the "correct" way -- View this message in context: http://r.789695.n4.nabble.com/Idiomatic-way-of-using-expression-in-i-tp4711229p4711230.html Sent from the datatable-help mailing list archive at Nabble.com. From gsee000 at gmail.com Tue Aug 18 15:07:35 2015 From: gsee000 at gmail.com (G See) Date: Tue, 18 Aug 2015 08:07:35 -0500 Subject: [datatable-help] Idiomatic way of using expression in i In-Reply-To: <1439902647094-4711230.post@n4.nabble.com> References: <1439902452478-4711229.post@n4.nabble.com> <1439902647094-4711230.post@n4.nabble.com> Message-ID: I would do it like this (per FAQ 1.6) dt <- data.table(x=rnorm(1000),y=rnorm(1000)) iExpr <- quote(x>1 & y<.045) dt[eval(iExpr)] On Tue, Aug 18, 2015 at 7:57 AM, statquant3 wrote: > DT[eval(parse(text=iExpr))] is working, but it is the "correct" way > > > > -- > View this message in context: http://r.789695.n4.nabble.com/Idiomatic-way-of-using-expression-in-i-tp4711229p4711230.html > Sent from the datatable-help mailing list archive at Nabble.com. > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help From statquant at outlook.com Tue Aug 18 15:51:42 2015 From: statquant at outlook.com (statquant3) Date: Tue, 18 Aug 2015 06:51:42 -0700 (PDT) Subject: [datatable-help] Idiomatic way of using expression in i In-Reply-To: References: <1439902452478-4711229.post@n4.nabble.com> <1439902647094-4711230.post@n4.nabble.com> Message-ID: <1439905902323-4711237.post@n4.nabble.com> But iExpr has to be constructed dynamically... Is it what you are doing ? BTW I red FAQ 1.6 and could not understand it, I might be worth rewriting it using i and j expressions and simple character expression like 'x>1 & y<1' -- View this message in context: http://r.789695.n4.nabble.com/Idiomatic-way-of-using-expression-in-i-tp4711229p4711237.html Sent from the datatable-help mailing list archive at Nabble.com. From eduard.antonyan at gmail.com Tue Aug 18 17:46:50 2015 From: eduard.antonyan at gmail.com (Eduard Antonyan) Date: Tue, 18 Aug 2015 10:46:50 -0500 Subject: [datatable-help] Idiomatic way of using expression in i In-Reply-To: <1439905902323-4711237.post@n4.nabble.com> References: <1439902452478-4711229.post@n4.nabble.com> <1439902647094-4711230.post@n4.nabble.com> <1439905902323-4711237.post@n4.nabble.com> Message-ID: If your expression is already put together as a string, then I don't think there are any other options besides eval(parse(text= On Tue, Aug 18, 2015 at 8:51 AM, statquant3 wrote: > But iExpr has to be constructed dynamically... > Is it what you are doing ? > > BTW I red FAQ 1.6 and could not understand it, I might be worth rewriting > it > using i and j expressions and simple character expression like 'x>1 & y<1' > > > > -- > View this message in context: > http://r.789695.n4.nabble.com/Idiomatic-way-of-using-expression-in-i-tp4711229p4711237.html > Sent from the datatable-help mailing list archive at Nabble.com. > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From statquant at outlook.com Wed Aug 19 09:28:03 2015 From: statquant at outlook.com (statquant3) Date: Wed, 19 Aug 2015 00:28:03 -0700 (PDT) Subject: [datatable-help] Can I update the FAQ and other docs on github ? Message-ID: <1439969283855-4711267.post@n4.nabble.com> I realize that I end up coming back to the same things, for most of them there is no or complicated explanation in the FAQ/vignettes. Is it open for modification ? -- View this message in context: http://r.789695.n4.nabble.com/Can-I-update-the-FAQ-and-other-docs-on-github-tp4711267.html Sent from the datatable-help mailing list archive at Nabble.com. From thomas.harrop at ird.fr Thu Aug 20 15:25:06 2015 From: thomas.harrop at ird.fr (Tom Harrop) Date: Thu, 20 Aug 2015 15:25:06 +0200 Subject: [datatable-help] Segfault using data.table 1.9.4 with multtest 2.24.0 on R 3.2.2 Message-ID: <55D5D532.1040006@ird.fr> Hello all, I'm running into a segfault using data.table with the multtest package: > > library(data.table) > data.table 1.9.4 For help type: ?data.table > *** NB: by=.EACHI is now explicit. See README to restore previous > behaviour. > > p <- data.table(pval = rnorm(100, 0.5, 0.1)) > > p[, multtest::mt.rawp2adjp(pval, proc = "BH", alpha = 0.1)] > > *** caught segfault *** > address 0x500000005, cause 'memory not mapped' > > Traceback: > 1: `[.data.table`(x, i) > 2: x[i] > 3: head.data.table(x, topn) > 4: head(x, topn) > 5: rbind(head(x, topn), tail(x, topn)) > 6: print.data.table(x) > 7: function (x, ...) UseMethod("print")(x) > > Possible actions: > 1: abort (with core dump, if enabled) > 2: normal R exit > 3: exit R without saving workspace > 4: exit R saving workspace Am I doing something wrong? The following executes without problems (otherwise I would be contacting the multtest authors!): > > library(data.table) > data.table 1.9.4 For help type: ?data.table > *** NB: by=.EACHI is now explicit. See README to restore previous > behaviour. > > p <- data.table(pval = rnorm(100, 0.5, 0.1)) > > multtest::mt.rawp2adjp(p$pval, proc = "BH", alpha = 0.1) My sessionInfo is at the end. Please let me know if any more information is required. Thanks for reading, Tom > > sessionInfo() > R version 3.2.2 (2015-08-14) > Platform: x86_64-pc-linux-gnu (64-bit) > Running under: Ubuntu 14.04.3 LTS > > locale: > [1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_AU.UTF-8 LC_COLLATE=en_AU.UTF-8 > [5] LC_MONETARY=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8 > [7] LC_PAPER=en_AU.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] data.table_1.9.4 > > loaded via a namespace (and not attached): > [1] MASS_7.3-43 magrittr_1.5 plyr_1.8.3 > [4] parallel_3.2.2 tools_3.2.2 survival_2.38-3 > [7] reshape2_1.4.1 Rcpp_0.12.0 Biobase_2.28.0 > [10] splines_3.2.2 multtest_2.24.0 stringi_0.5-5 > [13] BiocGenerics_0.14.0 stringr_1.0.0 chron_2.3-47 > [16] stats4_3.2.2 > From aragorn168b at gmail.com Sat Aug 22 11:11:11 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Sat, 22 Aug 2015 11:11:11 +0200 Subject: [datatable-help] Segfault using data.table 1.9.4 with multtest 2.24.0 on R 3.2.2 In-Reply-To: <55D5D532.1040006@ird.fr> References: <55D5D532.1040006@ird.fr> Message-ID: v1.9.5?https://github.com/Rdatatable/data.table?doesn?t sefgault, but returns an error (and it?s right). The object returned from `multtest` is a list with 4 elements out of which 3rd and 4th are NULL. Since the list elements are not of equal length, it errors when trying to convert the result to a? data.table. --? Arun On 20 Aug 2015 at 15:25:21, Tom Harrop (thomas.harrop at ird.fr) wrote: Hello all, I'm running into a segfault using data.table with the multtest package: > > library(data.table) > data.table 1.9.4 For help type: ?data.table > *** NB: by=.EACHI is now explicit. See README to restore previous > behaviour. > > p <- data.table(pval = rnorm(100, 0.5, 0.1)) > > p[, multtest::mt.rawp2adjp(pval, proc = "BH", alpha = 0.1)] > > *** caught segfault *** > address 0x500000005, cause 'memory not mapped' > > Traceback: > 1: `[.data.table`(x, i) > 2: x[i] > 3: head.data.table(x, topn) > 4: head(x, topn) > 5: rbind(head(x, topn), tail(x, topn)) > 6: print.data.table(x) > 7: function (x, ...) UseMethod("print")(x) > > Possible actions: > 1: abort (with core dump, if enabled) > 2: normal R exit > 3: exit R without saving workspace > 4: exit R saving workspace Am I doing something wrong? The following executes without problems (otherwise I would be contacting the multtest authors!): > > library(data.table) > data.table 1.9.4 For help type: ?data.table > *** NB: by=.EACHI is now explicit. See README to restore previous > behaviour. > > p <- data.table(pval = rnorm(100, 0.5, 0.1)) > > multtest::mt.rawp2adjp(p$pval, proc = "BH", alpha = 0.1) My sessionInfo is at the end. Please let me know if any more information is required. Thanks for reading, Tom > > sessionInfo() > R version 3.2.2 (2015-08-14) > Platform: x86_64-pc-linux-gnu (64-bit) > Running under: Ubuntu 14.04.3 LTS > > locale: > [1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C > [3] LC_TIME=en_AU.UTF-8 LC_COLLATE=en_AU.UTF-8 > [5] LC_MONETARY=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8 > [7] LC_PAPER=en_AU.UTF-8 LC_NAME=C > [9] LC_ADDRESS=C LC_TELEPHONE=C > [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] data.table_1.9.4 > > loaded via a namespace (and not attached): > [1] MASS_7.3-43 magrittr_1.5 plyr_1.8.3 > [4] parallel_3.2.2 tools_3.2.2 survival_2.38-3 > [7] reshape2_1.4.1 Rcpp_0.12.0 Biobase_2.28.0 > [10] splines_3.2.2 multtest_2.24.0 stringi_0.5-5 > [13] BiocGenerics_0.14.0 stringr_1.0.0 chron_2.3-47 > [16] stats4_3.2.2 > _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.harrop at ird.fr Sat Aug 22 14:08:41 2015 From: thomas.harrop at ird.fr (Tom Harrop) Date: Sat, 22 Aug 2015 14:08:41 +0200 Subject: [datatable-help] Segfault using data.table 1.9.4 with multtest 2.24.0 on R 3.2.2 In-Reply-To: References: <55D5D532.1040006@ird.fr> Message-ID: <9B3EC0B0-3084-48A4-A435-944716902C8A@ird.fr> Hi Arun, Of course. Thanks for pointing that out. Tom > On 22 Aug 2015, at 11:11, Arunkumar Srinivasan wrote: > > v1.9.5 https://github.com/Rdatatable/data.table doesn?t sefgault, but returns an error (and it?s right). > The object returned from `multtest` is a list with 4 elements out of which 3rd and 4th are NULL. > Since the list elements are not of equal length, it errors when trying to convert the result to a > data.table. > > > -- > Arun > > On 20 Aug 2015 at 15:25:21, Tom Harrop (thomas.harrop at ird.fr ) wrote: > >> Hello all, >> >> I'm running into a segfault using data.table with the multtest package: >> >> > > library(data.table) >> > data.table 1.9.4 For help type: ?data.table >> > *** NB: by=.EACHI is now explicit. See README to restore previous >> > behaviour. >> > > p <- data.table(pval = rnorm(100, 0.5, 0.1)) >> > > p[, multtest::mt.rawp2adjp(pval, proc = "BH", alpha = 0.1)] >> > >> > *** caught segfault *** >> > address 0x500000005, cause 'memory not mapped' >> > >> > Traceback: >> > 1: `[.data.table`(x, i) >> > 2: x[i] >> > 3: head.data.table(x, topn) >> > 4: head(x, topn) >> > 5: rbind(head(x, topn), tail(x, topn)) >> > 6: print.data.table(x) >> > 7: function (x, ...) UseMethod("print")(x) >> > >> > Possible actions: >> > 1: abort (with core dump, if enabled) >> > 2: normal R exit >> > 3: exit R without saving workspace >> > 4: exit R saving workspace >> >> Am I doing something wrong? The following executes without problems >> (otherwise I would be contacting the multtest authors!): >> >> > > library(data.table) >> > data.table 1.9.4 For help type: ?data.table >> > *** NB: by=.EACHI is now explicit. See README to restore previous >> > behaviour. >> > > p <- data.table(pval = rnorm(100, 0.5, 0.1)) >> > > multtest::mt.rawp2adjp(p$pval, proc = "BH", alpha = 0.1) >> >> My sessionInfo is at the end. Please let me know if any more information >> is required. >> >> Thanks for reading, >> >> Tom >> >> > > sessionInfo() >> > R version 3.2.2 (2015-08-14) >> > Platform: x86_64-pc-linux-gnu (64-bit) >> > Running under: Ubuntu 14.04.3 LTS >> > >> > locale: >> > [1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C >> > [3] LC_TIME=en_AU.UTF-8 LC_COLLATE=en_AU.UTF-8 >> > [5] LC_MONETARY=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8 >> > [7] LC_PAPER=en_AU.UTF-8 LC_NAME=C >> > [9] LC_ADDRESS=C LC_TELEPHONE=C >> > [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C >> > >> > attached base packages: >> > [1] stats graphics grDevices utils datasets methods base >> > >> > other attached packages: >> > [1] data.table_1.9.4 >> > >> > loaded via a namespace (and not attached): >> > [1] MASS_7.3-43 magrittr_1.5 plyr_1.8.3 >> > [4] parallel_3.2.2 tools_3.2.2 survival_2.38-3 >> > [7] reshape2_1.4.1 Rcpp_0.12.0 Biobase_2.28.0 >> > [10] splines_3.2.2 multtest_2.24.0 stringi_0.5-5 >> > [13] BiocGenerics_0.14.0 stringr_1.0.0 chron_2.3-47 >> > [16] stats4_3.2.2 >> > >> >> >> _______________________________________________ >> datatable-help mailing list >> datatable-help at lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From f_j_rod at hotmail.com Tue Aug 25 18:17:09 2015 From: f_j_rod at hotmail.com (Frank S.) Date: Tue, 25 Aug 2015 18:17:09 +0200 Subject: [datatable-help] Conditional replacement using a single line Message-ID: Hello everyone, I ask a question directly linked with other question which I was answered in this list a year ago. Let the data table be: DT <- data.table(obs=1:7, id=c(1,1,1,4,4,4,4), time=c(3,4,7,5,8,10,15)) Now I add a new column, called "value", which contains all zeros except the first observation within each "id" group, which is equal to 2. One possible solution I was given is the following two code lines: > DT[ , value:=0] > DT[!duplicated(id), value:=2] But I wonder if it is possible to do the same ON A SINGLE CODE LINE. I have tried different options but they doesn't work: 1) DT[ , value:= c(0, 2), by=id] 2) DT[ , value:= ifelse( DT[!duplicated(id)]==T, 2, 0), by=id] Many thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From eduard.antonyan at gmail.com Tue Aug 25 18:25:39 2015 From: eduard.antonyan at gmail.com (Eduard Antonyan) Date: Tue, 25 Aug 2015 11:25:39 -0500 Subject: [datatable-help] Conditional replacement using a single line In-Reply-To: References: Message-ID: DT[, value := c(2, rep(0, .N-1)), by = id] On Tue, Aug 25, 2015 at 11:17 AM, Frank S. wrote: > Hello everyone, > I ask a question directly linked with other question which I was answered > in this list a year ago. Let the data table be: > DT <- data.table(obs=1:7, id=c(1,1,1,4,4,4,4), time=c(3,4,7,5,8,10,15)) > > Now I add a new column, called "value", which contains all zeros except > the first observation within each "id" group, which is equal to 2. > One possible solution I was given is the following two code lines: > > DT[ , value:=0] > > DT[!duplicated(id), value:=2] > > But I wonder if it is possible to do the same ON A SINGLE CODE LINE. I > have tried different options but they doesn't work: > 1) DT[ , value:= c(0, 2), by=id] > 2) DT[ , value:= ifelse( DT[!duplicated(id)]==T, 2, 0), by=id] > > Many thanks! > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From caneff at gmail.com Tue Aug 25 18:27:39 2015 From: caneff at gmail.com (Chris Neff) Date: Tue, 25 Aug 2015 16:27:39 +0000 Subject: [datatable-help] Conditional replacement using a single line In-Reply-To: References: Message-ID: You almost had it right with number 2: DT[,value:=ifelse(!duplicated(id),0,2)] On Tue, Aug 25, 2015 at 12:26 PM Eduard Antonyan wrote: > DT[, value := c(2, rep(0, .N-1)), by = id] > > On Tue, Aug 25, 2015 at 11:17 AM, Frank S. wrote: > >> Hello everyone, >> I ask a question directly linked with other question which I was answered >> in this list a year ago. Let the data table be: >> DT <- data.table(obs=1:7, id=c(1,1,1,4,4,4,4), time=c(3,4,7,5,8,10,15)) >> >> Now I add a new column, called "value", which contains all zeros except >> the first observation within each "id" group, which is equal to 2. >> One possible solution I was given is the following two code lines: >> > DT[ , value:=0] >> > DT[!duplicated(id), value:=2] >> >> But I wonder if it is possible to do the same ON A SINGLE CODE LINE. I >> have tried different options but they doesn't work: >> 1) DT[ , value:= c(0, 2), by=id] >> 2) DT[ , value:= ifelse( DT[!duplicated(id)]==T, 2, 0), by=id] >> >> Many thanks! >> >> _______________________________________________ >> datatable-help mailing list >> datatable-help at lists.r-forge.r-project.org >> >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >> > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperickson at wisc.edu Tue Aug 25 18:37:25 2015 From: fperickson at wisc.edu (Frank Erickson) Date: Tue, 25 Aug 2015 12:37:25 -0400 Subject: [datatable-help] Conditional replacement using a single line In-Reply-To: References: Message-ID: Of course, you can put it on one line by simple chaining, too: DT[,v := 0][ DT[,.I[1],by=id]$V1, v := 2] There's a feature request that would allow simpler syntax: https://github.com/Rdatatable/data.table/issues/571 (not yet possible:) DT[,v := 0][, v[1] := 2, by=id] --Frank E. On Tue, Aug 25, 2015 at 12:27 PM, Chris Neff wrote: > You almost had it right with number 2: > > DT[,value:=ifelse(!duplicated(id),0,2)] > > > On Tue, Aug 25, 2015 at 12:26 PM Eduard Antonyan < > eduard.antonyan at gmail.com> wrote: > >> DT[, value := c(2, rep(0, .N-1)), by = id] >> >> On Tue, Aug 25, 2015 at 11:17 AM, Frank S. wrote: >> >>> Hello everyone, >>> I ask a question directly linked with other question which I was >>> answered in this list a year ago. Let the data table be: >>> DT <- data.table(obs=1:7, id=c(1,1,1,4,4,4,4), time=c(3,4,7,5,8,10,15)) >>> >>> Now I add a new column, called "value", which contains all zeros except >>> the first observation within each "id" group, which is equal to 2. >>> One possible solution I was given is the following two code lines: >>> > DT[ , value:=0] >>> > DT[!duplicated(id), value:=2] >>> >>> But I wonder if it is possible to do the same ON A SINGLE CODE LINE. I >>> have tried different options but they doesn't work: >>> 1) DT[ , value:= c(0, 2), by=id] >>> 2) DT[ , value:= ifelse( DT[!duplicated(id)]==T, 2, 0), by=id] >>> >>> Many thanks! >>> >>> _______________________________________________ >>> datatable-help mailing list >>> datatable-help at lists.r-forge.r-project.org >>> >>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help >>> >> >> _______________________________________________ >> datatable-help mailing list >> datatable-help at lists.r-forge.r-project.org >> >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > > > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help > -------------- next part -------------- An HTML attachment was scrubbed... URL: From f_j_rod at hotmail.com Tue Aug 25 18:37:45 2015 From: f_j_rod at hotmail.com (Frank S.) Date: Tue, 25 Aug 2015 18:37:45 +0200 Subject: [datatable-help] Conditional replacement using a single line In-Reply-To: References: , Message-ID: Thank you Eduard and Chris for your very rapid answers!!! Frank S. From: caneff at gmail.com Date: Tue, 25 Aug 2015 16:27:39 +0000 Subject: Re: [datatable-help] Conditional replacement using a single line To: eduard.antonyan at gmail.com; f_j_rod at hotmail.com CC: datatable-help at lists.r-forge.r-project.org You almost had it right with number 2: DT[,value:=ifelse(!duplicated(id),0,2)] On Tue, Aug 25, 2015 at 12:26 PM Eduard Antonyan wrote: DT[, value := c(2, rep(0, .N-1)), by = id] On Tue, Aug 25, 2015 at 11:17 AM, Frank S. wrote: Hello everyone, I ask a question directly linked with other question which I was answered in this list a year ago. Let the data table be: DT <- data.table(obs=1:7, id=c(1,1,1,4,4,4,4), time=c(3,4,7,5,8,10,15)) Now I add a new column, called "value", which contains all zeros except the first observation within each "id" group, which is equal to 2. One possible solution I was given is the following two code lines: > DT[ , value:=0] > DT[!duplicated(id), value:=2] But I wonder if it is possible to do the same ON A SINGLE CODE LINE. I have tried different options but they doesn't work: 1) DT[ , value:= c(0, 2), by=id] 2) DT[ , value:= ifelse( DT[!duplicated(id)]==T, 2, 0), by=id] Many thanks! _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From f_j_rod at hotmail.com Tue Aug 25 18:40:59 2015 From: f_j_rod at hotmail.com (Frank S.) Date: Tue, 25 Aug 2015 18:40:59 +0200 Subject: [datatable-help] Conditional replacement using a single line In-Reply-To: References: , , , , Message-ID: Thanks Frank for the your ingenious solution! From: f_j_rod at hotmail.com To: caneff at gmail.com; eduard.antonyan at gmail.com Date: Tue, 25 Aug 2015 18:37:45 +0200 CC: datatable-help at lists.r-forge.r-project.org Subject: Re: [datatable-help] Conditional replacement using a single line Thank you Eduard and Chris for your very rapid answers!!! Frank S. From: caneff at gmail.com Date: Tue, 25 Aug 2015 16:27:39 +0000 Subject: Re: [datatable-help] Conditional replacement using a single line To: eduard.antonyan at gmail.com; f_j_rod at hotmail.com CC: datatable-help at lists.r-forge.r-project.org You almost had it right with number 2: DT[,value:=ifelse(!duplicated(id),0,2)] On Tue, Aug 25, 2015 at 12:26 PM Eduard Antonyan wrote: DT[, value := c(2, rep(0, .N-1)), by = id] On Tue, Aug 25, 2015 at 11:17 AM, Frank S. wrote: Hello everyone, I ask a question directly linked with other question which I was answered in this list a year ago. Let the data table be: DT <- data.table(obs=1:7, id=c(1,1,1,4,4,4,4), time=c(3,4,7,5,8,10,15)) Now I add a new column, called "value", which contains all zeros except the first observation within each "id" group, which is equal to 2. One possible solution I was given is the following two code lines: > DT[ , value:=0] > DT[!duplicated(id), value:=2] But I wonder if it is possible to do the same ON A SINGLE CODE LINE. I have tried different options but they doesn't work: 1) DT[ , value:= c(0, 2), by=id] 2) DT[ , value:= ifelse( DT[!duplicated(id)]==T, 2, 0), by=id] Many thanks! _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From f_j_rod at hotmail.com Thu Aug 27 13:58:21 2015 From: f_j_rod at hotmail.com (Frank S.) Date: Thu, 27 Aug 2015 13:58:21 +0200 Subject: [datatable-help] Avoid warnings when obtaining fictitious rows Message-ID: Hello everyone, I can execute my R code but I obtain a warning message and I do not know how to avoid it. Let's consider, just by way of example, the following data table with 2 subjects under study: DT <- data.table(id=c(2,2,2,11,11), start=c(rep(as.Date("2007-01-01"),3),rep(as.Date("2010-09-30"),2)), visit=c("2007-03-01","2008-07-15","2008-11-17","2010-10-15","2011-05-17"), end=c(rep(as.Date("2009-05-01"),3),rep(as.Date("2012-03-25"),2))) I get a new data table which assigns a fictitious visit on each 31 December between the dates "start" and "end" corresponding to each individual: DT.new <- unique(DT[,list(start, visit=as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31"))), end), by=id]) Warning messages: In `[.data.table`(DT, , list(start, visit = as.Date(c(paste0(year(start[1]):(year(end[1]) - : Column 2 of result for group 1 is length 2 but the longest column in this result is 3. Recycled leaving remainder of 1 items. This warning is once only for the first group with this issue. The R code works, but how can I avoid the warning? Many thanks for any suggestion!! -------------- next part -------------- An HTML attachment was scrubbed... URL: From aragorn168b at gmail.com Fri Aug 28 11:55:27 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Fri, 28 Aug 2015 11:55:27 +0200 Subject: [datatable-help] Avoid warnings when obtaining fictitious rows In-Reply-To: References: Message-ID: Wrap the j-exp with print. It will help you identify what the issue is. Arun > On 27 Aug 2015, at 13:58, Frank S. wrote: > > Hello everyone, > > I can execute my R code but I obtain a warning message and I do not know how to avoid it. > Let's consider, just by way of example, the following data table with 2 subjects under study: > > DT <- data.table(id=c(2,2,2,11,11), > start=c(rep(as.Date("2007-01-01"),3),rep(as.Date("2010-09-30"),2)), > visit=c("2007-03-01","2008-07-15","2008-11-17","2010-10-15","2011-05-17"), > end=c(rep(as.Date("2009-05-01"),3),rep(as.Date("2012-03-25"),2))) > > I get a new data table which assigns a fictitious visit on each 31 December between > the dates "start" and "end" corresponding to each individual: > > DT.new <- unique(DT[,list(start, > visit=as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31"))), > end), by=id]) > > Warning messages: > In `[.data.table`(DT, , list(start, visit = as.Date(c(paste0(year(start[1]):(year(end[1]) - : > Column 2 of result for group 1 is length 2 but the longest column in this result is 3. Recycled leaving remainder of 1 items. This warning is once only for the first group with this issue. > > The R code works, but how can I avoid the warning? > > Many thanks for any suggestion!! > _______________________________________________ > datatable-help mailing list > datatable-help at lists.r-forge.r-project.org > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From f_j_rod at hotmail.com Fri Aug 28 17:16:52 2015 From: f_j_rod at hotmail.com (Frank S.) Date: Fri, 28 Aug 2015 17:16:52 +0200 Subject: [datatable-help] Avoid warnings when obtaining fictitious rows In-Reply-To: References: , Message-ID: Hi Arun, First of all, thnak you for answering once again! Following your suggestion, I've tried to wrap my j-expression: > unique(DT[ , print(visit <- as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31")))),by=id]) # [1] "2007-12-31" "2008-12-31" # [1] "2010-12-31" "2011-12-31" # Empty data.table (0 rows) of 1 col: id But i do not have any idea which the problem is. Please, can you give me any additional help? Subject: Re: [datatable-help] Avoid warnings when obtaining fictitious rows From: aragorn168b at gmail.com Date: Fri, 28 Aug 2015 11:55:27 +0200 CC: datatable-help at lists.r-forge.r-project.org To: f_j_rod at hotmail.com Wrap the j-exp with print. It will help you identify what the issue is. Arun On 27 Aug 2015, at 13:58, Frank S. wrote: Hello everyone, I can execute my R code but I obtain a warning message and I do not know how to avoid it. Let's consider, just by way of example, the following data table with 2 subjects under study: DT <- data.table(id=c(2,2,2,11,11), start=c(rep(as.Date("2007-01-01"),3),rep(as.Date("2010-09-30"),2)), visit=c("2007-03-01","2008-07-15","2008-11-17","2010-10-15","2011-05-17"), end=c(rep(as.Date("2009-05-01"),3),rep(as.Date("2012-03-25"),2))) I get a new data table which assigns a fictitious visit on each 31 December between the dates "start" and "end" corresponding to each individual: DT.new <- unique(DT[,list(start, visit=as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31"))), end), by=id]) Warning messages: In `[.data.table`(DT, , list(start, visit = as.Date(c(paste0(year(start[1]):(year(end[1]) - : Column 2 of result for group 1 is length 2 but the longest column in this result is 3. Recycled leaving remainder of 1 items. This warning is once only for the first group with this issue. The R code works, but how can I avoid the warning? Many thanks for any suggestion!! _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From aragorn168b at gmail.com Mon Aug 31 23:08:05 2015 From: aragorn168b at gmail.com (Arunkumar Srinivasan) Date: Mon, 31 Aug 2015 23:08:05 +0200 Subject: [datatable-help] Avoid warnings when obtaining fictitious rows In-Reply-To: References: Message-ID: I?m not sure what you?re doing. I meant to wrap the entire expression with `print()`: DT[, print(list(start, visit=as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31"))),end)), by=id] [[1]] [1] "2007-01-01" "2007-01-01" "2007-01-01" $visit [1] "2007-12-31" "2008-12-31? # <~~~ here?s your issue that results in warning. [[3]] [1] "2009-05-01" "2009-05-01" "2009-05-01" ???? [[1]] [1] "2010-09-30" "2010-09-30" $visit [1] "2010-12-31" "2011-12-31" [[3]] [1] "2012-03-25" "2012-03-25" --? Arun On 28 Aug 2015 at 17:16:59, Frank S. (f_j_rod at hotmail.com) wrote: Hi Arun, ? First of all, thnak you for answering once again! ? Following your suggestion, I've tried to wrap my j-expression: ? >?unique(DT[ , print(visit <- as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31")))),by=id]) #?[1] "2007-12-31" "2008-12-31" # [1] "2010-12-31" "2011-12-31" #?Empty data.table (0 rows) of 1 col: id ? But i do not?have any?idea which the problem is. ? Please, can you give me any? additional help? ? Subject: Re: [datatable-help] Avoid warnings when obtaining fictitious rows From: aragorn168b at gmail.com Date: Fri, 28 Aug 2015 11:55:27 +0200 CC: datatable-help at lists.r-forge.r-project.org To: f_j_rod at hotmail.com Wrap the j-exp with print. It will help you identify what the issue is. Arun On 27 Aug 2015, at 13:58, Frank S. wrote: Hello everyone, ? I can execute my R code but I obtain a warning message and I do not know how to avoid it. Let's consider, just by way of example, the following data table with 2 subjects under study: ? DT <- data.table(id=c(2,2,2,11,11), ? start=c(rep(as.Date("2007-01-01"),3),rep(as.Date("2010-09-30"),2)), ? visit=c("2007-03-01","2008-07-15","2008-11-17","2010-10-15","2011-05-17"), ? end=c(rep(as.Date("2009-05-01"),3),rep(as.Date("2012-03-25"),2))) ? I get a new data table which assigns a fictitious visit on each 31 December between the dates "start" and "end" corresponding to each individual: ? DT.new <- unique(DT[,list(start, ?visit=as.Date(c(paste0(year(start[1]):(year(end[1])-1),"-12-31"))), ?end), by=id]) ? Warning messages: In `[.data.table`(DT, , list(start, visit = as.Date(c(paste0(year(start[1]):(year(end[1]) -? : ? Column 2 of result for group 1 is length 2 but the longest column in this result is 3. Recycled leaving remainder of 1 items. This warning is once only for the first group with this issue. ? The R code works, but how can I avoid the warning? ? Many thanks for any suggestion!! _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: