Dear Thomas,<br><br>I'm not sure if that would work but it might be worth trying:<br>1- split your data set into many subsets (i.e. 25 subsets with 1,000 individuls each),<br>2- load them as 25 different genind objects,<br>
3-merge the 25 genind objects into a single genind object to have the original data as a single genind object (function 'repool'; the markers have to be the same for all objects to be merged, but
there is no constraint on alleles)<br><br>Cheers,<br><br>Sebastien.<br><br clear="all">*********************<br>Dr. Sébastien Puechmaille<br>Max Planck Institute for Ornithology<br>
Sensory Ecology Group<br>
Eberhard-Gwinner-Straße<br>
Haus Nr. 11<br>
82319 Seewiesen<br>
Germany
<br><br>and<br><br>
UCD School of Biological and Environmental Sciences<br>
University College Dublin (Zoology)<br>
UCD Science and Education Research Center (West)<br>
Belfield<br>
Dublin 4<br>
Ireland<br><br><a href="http://batlab.ucd.ie/%7Espuechmaille/" target="_blank">http://batlab.ucd.ie/~spuechmaille/</a><br><a href="http://www.ucd.ie/research/people/biologyenvscience/drsebastienpuechmaille/home/" target="_blank">http://www.ucd.ie/research/people/biologyenvscience/drsebastienpuechmaille/home/</a><br>
*********************<br><br><div class="gmail_quote">On 6 July 2011 13:44, Thomas, Evert (Bioversity-Colombia) <span dir="ltr"><<a href="mailto:E.Thomas@cgiar.org">E.Thomas@cgiar.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US"><div><p class="MsoNormal"><span style="color: rgb(31, 73, 125);">Dear Thibaut,<u></u><u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125);">Thanks for this. I have tried running several times overnight now but each time get the message:<u></u><u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"> </span><img src="cid:image001.png@01CC3BB0.8AF35EF0" height="206" width="370"><span style="color: rgb(31, 73, 125);"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><u></u> <u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);">I am running windows7 on a 64bit system with 4x 2.4GHz and 4Gb RAM, so I don’t think the problem is related to my PC?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125);">Many thanks for any suggestions you might have…<u></u><u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><u></u> <u></u></span></p><p class="MsoNormal">
<span style="color: rgb(31, 73, 125);">Cheers Evert<u></u><u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><u></u> <u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);">(PS when reading in my CSV is use “stringsAsFactor=F”, so that my marker data is read in as characters –could that be the problem?)<u></u><u></u></span></p>
<div><div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; padding: 3pt 0in 0in;"><p class="MsoNormal"><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> Jombart, Thibaut [mailto:<a href="mailto:t.jombart@imperial.ac.uk" target="_blank">t.jombart@imperial.ac.uk</a>] <br>
<b>Sent:</b> Monday, July 04, 2011 11:33 AM<br><b>To:</b> Thomas, Evert (Bioversity-Colombia); <a href="mailto:adegenet-forum@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum@r-forge.wu-wien.ac.at</a><br><b>Subject:</b> RE: [adegenet-forum] dataset too large? Follow-up<u></u><u></u></span></p>
</div></div><div><div></div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal" style="margin-bottom: 12pt;"><span style="font-size: 10pt; color: black;">Dear Thomas, <br><br>The algorithm for translating your data into individual frequencies is not linear. RAM saturation is likely to cause supplementary delays in any case, but windows is good at having applications freezing/crashing in such cases ("R has stopped working...send a report") . How much memory do you have on your computer? In any case I would recommend running overnight to make sure it just doesn't take ages, but works.<br>
<br>We are looking at a big dataset, but it is merely 2-3 times bigger than eHGDP, which was not such a pain to obtain.<br><br>As for multicore, the package is not available for windows, unfortunately. <br><br>Importing your data from STRUCTURE won't help, it will actually be longer and more RAM-demanding.<br>
<br>On the bright side, once you'll have your data imported, analysis should be slightly less time-consuming.<br><br>Best<br><br>Thibaut<u></u><u></u></span></p><div><p class="MsoNormal" style="margin-bottom: 12pt;">
<span style="font-size: 10pt; color: black;"><u></u> <u></u></span></p>
</div><div><div class="MsoNormal" style="text-align: center;" align="center"><span style="font-size: 12pt; font-family: "Times New Roman","serif"; color: black;"><hr align="center" size="2" width="100%">
</span></div><div><p class="MsoNormal" style="margin-bottom: 12pt;"><b><span style="font-size: 10pt; color: black;">From:</span></b><span style="font-size: 10pt; color: black;"> <a href="mailto:adegenet-forum-bounces@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum-bounces@r-forge.wu-wien.ac.at</a> [<a href="mailto:adegenet-forum-bounces@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum-bounces@r-forge.wu-wien.ac.at</a>] on behalf of Thomas, Evert (Bioversity-Colombia) [<a href="mailto:E.Thomas@CGIAR.ORG" target="_blank">E.Thomas@CGIAR.ORG</a>]<br>
<b>Sent:</b> 04 July 2011 16:18<br><b>To:</b> <a href="mailto:adegenet-forum@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum@r-forge.wu-wien.ac.at</a><br><b>Subject:</b> Re: [adegenet-forum] dataset too large? Follow-up</span><span style="font-size: 12pt; font-family: "Times New Roman","serif"; color: black;"><u></u><u></u></span></p>
</div><div><div><p class="MsoNormal"><span style="color: rgb(31, 73, 125);">Dear,</span><span style="color: black;"><u></u><u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"> </span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125);">The problem does not seem to be related to my commands, since I do get results for subsets of my data (1000 individuals takes 40 seconds), but it does not seem to work for my entire dataset of >25000 individuals (should theoretically take 16.6 minutes, but after 4 hours still no result) … any suggestions? </span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125);"><br>many thanks in advance</span><span style="color: black;"><u></u><u></u></span></p><p class="MsoNormal"><span style="color: rgb(31, 73, 125);"> </span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125);">evert</span><span style="color: black;"><u></u><u></u></span></p><div><div style="border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-color: -moz-use-text-color;">
<p class="MsoNormal"><b><span style="font-size: 10pt; color: black;">From:</span></b><span style="font-size: 10pt; color: black;"> <a href="mailto:adegenet-forum-bounces@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum-bounces@r-forge.wu-wien.ac.at</a> [mailto:<a href="mailto:adegenet-forum-bounces@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum-bounces@r-forge.wu-wien.ac.at</a>] <b>On Behalf Of </b>Thomas, Evert (Bioversity-Colombia)<br>
<b>Sent:</b> Friday, July 01, 2011 1:56 PM<br><b>To:</b> <a href="mailto:adegenet-forum@r-forge.wu-wien.ac.at" target="_blank">adegenet-forum@r-forge.wu-wien.ac.at</a><br><b>Subject:</b> [adegenet-forum] dataset too large?</span><span style="color: black;"><u></u><u></u></span></p>
</div></div><p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;">Dear colleagues,<u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;">I am new to R so apologies for my ignorance, but I have a couple of questions: <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;">I am trying to use adegenet (on a 64bit system, windows7) for analyzing a SSR dataset. It consists 96 loci and I have >25000 individuals (after resampling). I have loaded the database as a dataframe in R, but am not able to convert to genind format (PC physical memory becomes saturated, while only 10% of CPU is used) . Could this be related to the size of my dataset? Any suggestions?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;">On another note: Alternatively, I tried importing my data to genind object from the corresponding file in Structure format. However, my version of Structure (2.3.3.) does not seem to generate .stru or .str files, any solution there?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;">And a last point: I am unable to install/load the R application multicore because it is not among the packages list…<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;">This is what I have done:<u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;">I did a read.csv with “header=T”, and then rownames<-cacaoCSV[,1]<u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal">
<span style="color: black;">The problems occurs with the following command<u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;">cacao<-df2genind(cacaoCSV, sep="/",ind.names=NULL, loc.names=NULL, pop=cacaoCSV[,2], missing=NA, ploidy=2, type="codom")<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal"><span style="color: black;">Many thanks in advance for any advice or suggestion you might have!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p><p class="MsoNormal" style="margin-bottom: 12pt;"><span style="color: black;">Enjoy the weekend<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;">Evert Thomas,</span><i><span style="font-family: "Antique Olive"; color: black;"> PhD</span></i><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; font-family: "Antique Olive"; color: rgb(0, 112, 192);">Associate Expert, Conservation and Use of </span></i><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><i><span style="font-size: 10pt; font-family: "Antique Olive"; color: rgb(0, 112, 192);">Forest Genetic Resources in Latin America</span></i><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; color: black;"> </span><span style="color: black;"><u></u><u></u></span></p><p class="MsoNormal" style="text-align: justify;"><b><span style="font-size: 10pt; font-family: "Antique Olive"; color: rgb(0, 153, 0);" lang="EN-GB">Bioversity International</span></b><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="EN-GB">Regional Office for the Americas</span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="IT">Recta Cali-Palmira Km 17 – CIAT</span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="IT">Cali, Colombia</span><span style="color: black;"><u></u><u></u></span></p><p class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10pt; font-family: "Antique Olive"; color: black;">P.O. Box 6713</span><span style="color: black;"><u></u><u></u></span></p><p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="EN-GB"> </span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align: justify;"><b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="EN-GB">Tel</span></b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="EN-GB">. 57 2 4450048 / 49 Ext 113</span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align: justify;"><b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="FR">Fax</span></b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="FR"> 57 2 4450096</span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="FR">Email</span></b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;" lang="FR">: <a href="mailto:e.thomas@cgiar.org" title="mailto:e.thomas@cgiar.org" target="_blank">e.thomas@cgiar.org</a></span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;">Skype</span></b><span style="font-size: 10pt; font-family: "Antique Olive"; color: black;">: evertthomas</span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><u><span style="font-size: 10pt; font-family: "Antique Olive"; color: green;" lang="EN-GB"><a href="http://UrlBlockedError.aspx" title="blocked::http://www.bioversityinternational.org/
http://www.bioversityinternational.org/" target="_blank">www.bioversityinternational.org</a></span></u><span style="color: black;"><u></u><u></u></span></p><p class="MsoNormal"><span style="font-size: 12pt; font-family: "Times New Roman","serif"; color: black;"> </span><span style="color: black;"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: black;"> <u></u><u></u></span></p></div></div></div></div></div></div></div></div><br>_______________________________________________<br>
adegenet-forum mailing list<br>
<a href="mailto:adegenet-forum@lists.r-forge.r-project.org">adegenet-forum@lists.r-forge.r-project.org</a><br>
<a href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/adegenet-forum" target="_blank">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/adegenet-forum</a><br>
<br></blockquote></div><br>