From power.julian.chen at gmail.com Fri Feb 12 02:32:19 2016 From: power.julian.chen at gmail.com (Julian Chen) Date: Thu, 11 Feb 2016 17:32:19 -0800 (PST) Subject: [datatable-help] Random effects in GAMs Message-ID: <1455240739792-4717371.post@n4.nabble.com> I am now trying to use random effects in GAMs developed by Professor Simon Wood. Prof Wood uses s(...,bs="re") to account for the random effects. Random intercepts models or random slopes models are two different types of mixed linear models or general random effects model (Cameron and Trivedi, 2005). I wonder if the Wood's method includes both random intercepts and random slopes. Based on my understanding, this method does? Anyone can help me clarify this method? The following is the link of Random effects in GAM developed by Prof Wood. https://stat.ethz.ch/R-manual/R-devel/library/mgcv/html/random.effects.html -- View this message in context: http://r.789695.n4.nabble.com/Random-effects-in-GAMs-tp4717371.html Sent from the datatable-help mailing list archive at Nabble.com. From michael.nelson at sydney.edu.au Fri Feb 12 07:25:14 2016 From: michael.nelson at sydney.edu.au (Michael Nelson) Date: Fri, 12 Feb 2016 06:25:14 +0000 Subject: [datatable-help] Random effects in GAMs In-Reply-To: <1455240739792-4717371.post@n4.nabble.com> References: <1455240739792-4717371.post@n4.nabble.com> Message-ID: This is not the mailing list you are looking for. Datatable-help is for help about datatable. That being said Ngo, L. and Wand, M.P. (2004). Smoothing with Mixed Model Software. Volume 9, 2004, Issue 1 of Journal of Statistical Software, 1-54. Provide a great walk through on how additive models can be fitted using mixed models and random effects. On 12 Feb 2016, at 12:57 PM, Julian Chen > wrote: I am now trying to use random effects in GAMs developed by Professor Simon Wood. Prof Wood uses s(...,bs="re") to account for the random effects. Random intercepts models or random slopes models are two different types of mixed linear models or general random effects model (Cameron and Trivedi, 2005). I wonder if the Wood's method includes both random intercepts and random slopes. Based on my understanding, this method does? Anyone can help me clarify this method? The following is the link of Random effects in GAM developed by Prof Wood. https://stat.ethz.ch/R-manual/R-devel/library/mgcv/html/random.effects.html -- View this message in context: http://r.789695.n4.nabble.com/Random-effects-in-GAMs-tp4717371.html Sent from the datatable-help mailing list archive at Nabble.com. _______________________________________________ datatable-help mailing list datatable-help at lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -------------- next part -------------- An HTML attachment was scrubbed... URL: From clephas at vreelandgroep.nl Thu Feb 18 13:32:29 2016 From: clephas at vreelandgroep.nl (LoesClephas) Date: Thu, 18 Feb 2016 04:32:29 -0800 (PST) Subject: [datatable-help] Error in mult %chin% c("first", "last", "all") : Internal error: savetl_init checks failed (0 100 0x7f8b42cc3d60 0x7f8b47c0df90). Message-ID: <1455798749995-4717545.post@n4.nabble.com> I get this Error. About three weeks ago everything worked perfectly fine. Anybody any idea what happened? Already reinstalled data.table package etcetera. Kind regards, Loes -- View this message in context: http://r.789695.n4.nabble.com/Error-in-mult-chin-c-first-last-all-Internal-error-savetl-init-checks-failed-0-100-0x7f8b42cc3d60-0x-tp4717545.html Sent from the datatable-help mailing list archive at Nabble.com. From fluffinator007 at gmail.com Thu Feb 18 20:51:03 2016 From: fluffinator007 at gmail.com (CCRyder) Date: Thu, 18 Feb 2016 11:51:03 -0800 (PST) Subject: [datatable-help] Efficiency help - create variable conditional on row-specific column values Message-ID: <1455825063327-4717563.post@n4.nabble.com> Each row of the table has 15 columns with "ID" and 15 corresponding "ID Indicator" columns. These are alternating in order (i.e., ID1 / IDind1 / ID2 / IDind2 /....). The ID Indicator variable specifies whether the corresponding ID is the "primary ID" for each row/observation (there is only 1 primary ID for each). My goal is to create a new variable that equals the Primary ID for each row. I'm doing this now using for loops (see code below). This works, but I am certain there is a much more efficient way to accomplish the task. /*RData$PrimaryID <- NA_character_ for(i in seq(9:37, by = 2)) { for(j in 1:nrow(RData)) { if(!(is.na(RData[j:i])) & (RData[j,i] == 'Y')) RData[j, 38] <- NPI[j, (i-1)] } }*/ /NOTE: The ID and IDind variables are alternative in positions 9-37 on the dataframe. The 38th columns is the newly created "PrimaryID"./ I am thinking the better way to do this is with apply/sapply/lapply, or by vectorizing the work somehow. I haven't been able to get anything working, though. Any help, tips, or guidance would be very much appreciated! -- View this message in context: http://r.789695.n4.nabble.com/Efficiency-help-create-variable-conditional-on-row-specific-column-values-tp4717563.html Sent from the datatable-help mailing list archive at Nabble.com. From ashokkrish at gmail.com Mon Feb 22 06:33:46 2016 From: ashokkrish at gmail.com (ashruser) Date: Sun, 21 Feb 2016 21:33:46 -0800 (PST) Subject: [datatable-help] Removing rows from a matrix based on column entries Message-ID: <1456119226405-4717649.post@n4.nabble.com> Dear R Users, I have a question about removing rows from a matrix. All matrix entries are either a 0 or a 1. The rows are sorted according to the row sum. Here is an example matrix e1 <- c(0,0,0,1,0,0,0) e2 <- c(1,0,0,0,0,0,0) e3 <- c(0,1,0,0,0,0,0) e4 <- c(0,0,1,0,1,0,0) e5 <- c(1,1,0,0,0,0,0) e6 <- c(1,0,0,0,1,0,0) e7 <- c(0,0,1,0,1,1,0) e8 <- c(0,0,1,0,1,0,1) e9 <- c(1,1,0,1,1,0,0) e10 <- c(0,0,1,1,0,1,1) (E <- rbind(e1, e2, e3, e4, e5, e6, e7, e8, e9, e10)) Which prints > (E <- rbind(e1, e2, e3, e4, e5, e6, e7, e8, e9, e10)) [,1] [,2] [,3] [,4] [,5] [,6] [,7] e1 0 0 0 1 0 0 0 e2 1 0 0 0 0 0 0 e3 0 1 0 0 0 0 0 e4 0 0 1 0 1 0 0 e5 1 1 0 0 0 0 0 e6 1 0 0 0 1 0 0 e7 0 0 1 0 1 1 0 e8 0 0 1 0 1 0 1 e9 1 1 0 1 1 0 0 e10 0 0 1 1 0 1 1 I want to remove rows in the following fashion. If a row has a single 1 then all following rows below that with a 1 in that column position should be removed. So we observe rows e1 e2 and e3 can successively remove rows e5, e6, e9 and e10. Leaving us with rows e1, e2, e3, e4, e7 and e8. for (v in 2:dim(E)[1]) { print(v) print(E[v, 4]) if (E[v, 4] == 1) E <- E[-v,] } Removing rows inside a for-loop is giving me an error. So I thought I ll first find rows (if any) have a rowsum 1 and identified them. Then I try to remove the following rows with a 1 in that position using a for-loop. Once again an error. UnitRowsum <- E[which(rowSums(E) == 1),] UnitRowsum for (v in 1:dim(UnitRowsum)[1]) { print(which(UnitRowsum[v, ] == 1)) } Furthermore I want to continue row removals based on rows with sum greater than one and removes all following rows that have a 1 in all those positions and so on. To example what I mean I have now have a reduced matrix [,1] [,2] [,3] [,4] [,5] [,6] [,7] e1 0 0 0 1 0 0 0 e2 1 0 0 0 0 0 0 e3 0 1 0 0 0 0 0 e4 0 0 1 0 1 0 0 e7 0 0 1 0 1 1 0 e8 0 0 1 0 1 0 1 Row e4 dominates rows e7 and e8 and therefore have to be removed as well. This continues till no more rows can be removed. Could you please help me. Sincerely, Ash -- View this message in context: http://r.789695.n4.nabble.com/Removing-rows-from-a-matrix-based-on-column-entries-tp4717649.html Sent from the datatable-help mailing list archive at Nabble.com. From rodonn at stanford.edu Mon Feb 29 03:50:08 2016 From: rodonn at stanford.edu (rodonn) Date: Sun, 28 Feb 2016 18:50:08 -0800 (PST) Subject: [datatable-help] Magrittr + Data.table = Invalid .internal.selfref detected Message-ID: <1456714207596-4717983.post@n4.nabble.com> Here's a simple example that can reproduce the error. require(data.table) require(magrittr) require(dplyr) data(cars) setDT(cars) cars %<>% filter(Cylinder == 6) %>% setDT cars[, DollarsPerMiles := Price / Mileage] This generates the error: Warning message: In `[.data.table`(cars, , `:=`(DollarsPerMiles, Price/Mileage)) : Invalid .internal.selfref detected and fixed by taking a (shallow) copy of the data.table so that := can add this new column by reference. At an earlier point, this data.table has been copied by R (or been created manually using structure() or similar). Avoid key<-, names<- and attr<- which in R currently (and oddly) may copy the whole data.table. Use set* syntax instead to avoid copying: ?set, ?setnames and ?setattr. Also, in R<=v3.0.2, list(DT1,DT2) copied the entire DT1 and DT2 (R's list() used to copy named objects); please upgrade to R>v3.0.2 if that is biting. If this message doesn't help, please report to datatable-help so the root cause can be fixed. It seems like this either magrittr or dplyr is making an improper copy. I know that there is a way of doing this sort of filtering using the data.table syntax, but I find the dplyr syntax easier for certain manipulations. Thanks! -- View this message in context: http://r.789695.n4.nabble.com/Magrittr-Data-table-Invalid-internal-selfref-detected-tp4717983.html Sent from the datatable-help mailing list archive at Nabble.com. From adi2827 at gmail.com Mon Feb 29 05:40:37 2016 From: adi2827 at gmail.com (akang) Date: Sun, 28 Feb 2016 20:40:37 -0800 (PST) Subject: [datatable-help] Calculate correlation In R Message-ID: <1456720837273-4717984.post@n4.nabble.com> I have a total of 6 files. File1, File2, File3, File4 & File5 have the same structure except for ID's. I want to 1.sum up the values in all rows like rowSums(file1) 2.Match the Id in File1-File5(Column1) to Id in File6 (Column2) and find the correlation between the sum of rows from File1 and column3 of File6 so that for 5 different files I have 5 different values and write it to a file. This is what i tried but of course it doesnt work. Ill appreciate any help.TIA! Geno<-read.table("data1", head=TRUE, row.names=1) Pheno<-read.table("data1_p" ) rowSums(Geno) cor(rowSums(Geno), Pheno$V3) File1 snp1 snp2 snp3 Id1 0.1 2.3 3.2 Id2 0.01 2.2 3.5 Id3 0.004 0.4 1 File2 snp1 snp2 snp3 Id4 0.3 2.0 3.0 Id5 0.1 2.02 3.05 Id7 0.04 0.04 0.1 File3 snp1 snp2 snp3 Id8 0.3 2.0 3.0 Id9 0.1 2.02 3.05 Id11 0.04 0.04 0.1 File4 snp1 snp2 snp3 Id12 0.3 2.0 3.0 Id13 0.1 2.02 3.05 Id14 0.04 0.04 0.1 File5 snp1 snp2 snp3 Id15 0.03 2.1 3.7 Id16 0.01 2.05 3.01 Id17 0.04 0.04 0.01 File6 (No header) 0 Id1 0.03 2.1 3.7 2 Id2 0.01 2.05 3.01 2 Id3 0.04 0.04 0.01 0 Id4 0.03 2.1 3.7 2 Id5 0.01 2.05 3.01 2 Id6 0.04 0.04 0.01 0 Id7 0.03 2.1 3.7 2 Id8 0.01 2.05 3.01 2 Id9 0.04 0.04 0.01 0 Id10 0.03 2.1 3.7 2 Id11 0.01 2.05 3.01 2 Id12 0.04 0.04 0.01 2 Id13 0.01 2.05 3.01 2 Id14 0.04 0.04 0.01 0 Id15 0.03 2.1 3.7 2 Id16 0.01 2.05 3.01 2 Id17 0.04 0.04 0.01 -- View this message in context: http://r.789695.n4.nabble.com/Calculate-correlation-In-R-tp4717984.html Sent from the datatable-help mailing list archive at Nabble.com. From adi2827 at gmail.com Mon Feb 29 05:45:52 2016 From: adi2827 at gmail.com (akang) Date: Sun, 28 Feb 2016 20:45:52 -0800 (PST) Subject: [datatable-help] Calculate correlation In R In-Reply-To: <1456720837273-4717984.post@n4.nabble.com> References: <1456720837273-4717984.post@n4.nabble.com> Message-ID: <1456721152608-4717985.post@n4.nabble.com> I have a total of 6 files. File1, File2, File3, File4 & File5 have the same structure except for ID's. I want to 1.sum up the values in all rows like rowSums(file1) 2.Match the Id in File1-File5(Column1) to Id in File6 (Column2) and find the correlation between the sum of rows from File1 and column3 of File6 so that for 5 different files I have 5 different values and write it to a file. This is what i tried but of course it doesnt work. Ill appreciate any help.TIA! Geno<-read.table("data1", head=TRUE, row.names=1) Pheno<-read.table("data1_p" ) rowSums(Geno) cor(rowSums(Geno), Pheno$V3) File1 snp1 snp2 snp3 Id1 0.1 2.3 3.2 Id2 0.01 2.2 3.5 Id3 0.004 0.4 1 File2 snp1 snp2 snp3 Id4 0.3 2.0 3.0 Id5 0.1 2.02 3.05 Id7 0.04 0.04 0.1 File3 snp1 snp2 snp3 Id8 0.3 2.0 3.0 Id9 0.1 2.02 3.05 Id11 0.04 0.04 0.1 File4 snp1 snp2 snp3 Id12 0.3 2.0 3.0 Id13 0.1 2.02 3.05 Id14 0.04 0.04 0.1 File5 snp1 snp2 snp3 Id15 0.03 2.1 3.7 Id16 0.01 2.05 3.01 Id17 0.04 0.04 0.01 File6 (No header) 0 Id1 0.03 2.1 3.7 2 Id2 0.01 2.05 3.01 2 Id3 0.04 0.04 0.01 0 Id4 0.03 2.1 3.7 2 Id5 0.01 2.05 3.01 2 Id6 0.04 0.04 0.01 0 Id7 0.03 2.1 3.7 2 Id8 0.01 2.05 3.01 2 Id9 0.04 0.04 0.01 0 Id10 0.03 2.1 3.7 2 Id11 0.01 2.05 3.01 2 Id12 0.04 0.04 0.01 2 Id13 0.01 2.05 3.01 2 Id14 0.04 0.04 0.01 0 Id15 0.03 2.1 3.7 2 Id16 0.01 2.05 3.01 2 Id17 0.04 0.04 0.01 -- View this message in context: http://r.789695.n4.nabble.com/Calculate-correlation-In-R-tp4717984p4717985.html Sent from the datatable-help mailing list archive at Nabble.com.