<div dir="ltr"><div><div>Dear all!</div><div>I commited newest version of reshuffle</div><div>Now reshuffle works 2x faster!=)</div><div>Reasons:</div><div><br></div><div> --ostringstream oss: outputs cache</div><div><br>
</div><div> --exclude from cycle's and put them upper<span class="" style="white-space:pre"> </span></div><div> double* buf = new double[per_trait_per_snp];</div><div> char s[30];</div><div><br></div><div> --(int64_t) blablabla instead of (int64_t)bla + (int64_t)bla + (int64_t)bla</div>
<div><br></div><div>To find "hot spots" in reshuffle, I used</div><div> </div><div>GNU Profiler</div><div>GNU Coverage testing tool</div><div><br></div><div>Very useful tools to find right places in programm to optimizate!</div>
<div><br></div><div>Now 5Gb CLAK-GWAS output convert to 16 Gb txt files for 380 sec or 6 minutes.</div><div>Machine: Intel Core i7 930; 8Gb RAM (it is not cluster's node, I think on cluster's node reshuffle's run would be faster=)</div>
<div><br></div><div>There are problems with extract heritability and write slim data.</div><div>I'll check soon</div><div><span style="color:rgb(0,0,0);font-size:12px"> </span></div><div><pre style="margin-top:0px;margin-bottom:0px;color:rgb(0,0,0);font-size:12px">
<code><br></code></pre></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/7/20 ๓ฯฤยฯ ๛มามะฯื <span dir="ltr"><<a href="mailto:sharapovsodbo@gmail.com" target="_blank">sharapovsodbo@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p>Hello! <br>
I'll will check reshuffle tomorrow.</p>
<div class="gmail_quote">20.07.2013 22:26 ะฯฬฺุฯืมิลฬุ "Yurii Aulchenko" <<a href="mailto:yurii.aulchenko@gmail.com" target="_blank">yurii.aulchenko@gmail.com</a>> ฮมะษำมฬ:<div><div class="h5"><br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Another point: apparently you do not check boundaries - e.g. when I try to get results for trait #200,000 (I have 107,000 only) I get the core dump.<div><br></div><div>YA<br><br><div class="gmail_quote">On Sat, Jul 20, 2013 at 5:15 PM, Yurii Aulchenko <span dir="ltr"><<a href="mailto:yurii.aulchenko@gmail.com" target="_blank">yurii.aulchenko@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Sodbo,<div>
</div><div><br></div><div>It seems that reshuffle does not work correctly, at least I can not get to the results with it (see below). I use a dataset with ~107k traits and ~280k SNPs.</div><div><br></div><div>Any idea? - do I do something wrong?</div>
<div><br></div><div>YA</div><div><br></div><div>With perl-extractor I get chi2 of 62</div><div><br></div><div><div>ya567666@cluster:~[167]$ perl extractCell.pl /hpcwork/df938257/natgen/B2 329 209602 | gawk '{print $_,($2/$4)^2}'</div>
<div>-0.165153577923775 0.580845952033997 0.0298683661967516 0.0734809562563896 -0.00155110028572381 62.4845</div><div><br></div><div>But this is not the case with reshuffle (and also I do not get any output with reshuffle /hpcwork/df938257/natgen/B2 --chi=30, while I know there are such chi2's in the results)</div>
<div><br></div><div>ya567666@cluster:~[167]$ reshuffle /hpcwork/df938257/natgen/B2 --snps=209602 --traits=329 --chi</div><div>Finish iout_file read 0.11 sec</div><div>Start_write_chi_data=0.14 sec</div><div>End_write_chi_trait spm_1_AND_spmp_23 0.14 sec</div>
<div>Finish_write_chi_data 0.14 sec</div><div>Finish reshuffling 0.14 sec</div><div>ya567666@cluster:~[168]$ cat chi_data.txt</div><div>SNP Trait beta_1 beta_SNP se_1 se_SNP cov_SNP_1 Chi2</div>
<div>rs4902242 spm_1_AND_spmp_23 -0.00234050769358873 -0.0338250175118446 0.128280490636826 0.0329618416726589 0.0770578160881996 1.05306001371466</div><div>ya567666@cluster:~[169]$</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>-----------------------------------------------------<br>Yurii S. Aulchenko<br><div><br></div><div>[<a href="http://nl.linkedin.com/in/yuriiaulchenko" target="_blank">LinkedIn</a>][ <a href="http://twitter.com/YuriiAulchenko" target="_blank">Twitter</a> ] [ <a href="http://yurii-aulchenko.blogspot.nl/" target="_blank">Blog</a> ]</div>
</div>
</blockquote></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><u>_________________________________</u><div><u><br></u>With best regards<br><br>Sodbo Zh. Sharapov<div>Phone: +79831347688</div><div>Email: <a href="mailto:sharapovsodbo@gmail.com" target="_blank">sharapovsodbo@gmail.com</a></div>
<div> <a href="mailto:sharapov@bionet.nsc.ru" target="_blank">sharapov@bionet.nsc.ru</a></div><div>Skype: sharapovsodbo
</div></div></div>
</div>