[adegenet-forum] set.seeds in DAPC

[Manuela]

Hi Caitlin.

Good point!

In fact, I' didn´t notice this tiny  nuance in the rationale behind
cross-validation on using a stratified sampling of 10% of individuals
(validation set sample) in the well-exemplified nancycats datset, through
the ciclic process of PC retention, sampling and DAPC procedures in each
set number of PCAs retained, BUT not the same set of individuals in each
round.

>From the second one based on supplementary individuals used on predicting
results. Also the way they were selected was different. They result from a
split of the original sample into a stratified "testing sample" of X
individuals, BUT using a non-random sample as defined by set.seed()
function.

Later, I'll present you a new set of questions raised by clines for being
thoroughly evaluated on modelling by DAPC.

Cheers,
M.



2014-06-19 1:32 GMT+01:00 Caitlin Collins <caitiecollins at gmail.com>:

> Hi Manuela,
>
> Glad to hear I could help a bit!
>
> I should stress that our use of set.seed() in the tutorial has been mainly
> for the purpose of making the tutorial, as a document, consistent and
> identically reproducible. In an experimental context, however, eg. in the
> case of selecting supplementary individuals, if you are truly attempting to
> test a concept (for example, in validating a model), you would actually
> *want* random behaviour (ie. an effectively random sample). This is
> particularly the case if you are performing repeated sampling, as one often
> does with supplementary individuals. So be careful to only set the seed
> when you do NOT want a random sample; otherwise, just leave out set.seed()
> from the process and let the computer pick a sample at random.
>
> Best,
> Caitlin.
>
>
> On Thu, Jun 19, 2014 at 12:17 AM, Manuela <manuelacorreia2 at gmail.com>
> wrote:
>
>> Dear Caitlin,
>>
>>
>> Thank you  for such a clear response and at same time for being so
>> knowledgeable. It was quiet interesting to have a glimpse on the way how
>> the Adegenet team decided to use the set.seeds to obtain consistent
>> results, as well as (that was just brilliant!) to control the lab. jitter.
>>
>> As you point up with the 3 examples its better to try several set.seeds
>> in order to find out the best labels position with our dataset. And when we
>> reach the final stage of cross-validation we ought to choose one seed to
>> ensure that the training set of supplementary individuals (no matter the
>> number (10%, 20%)) will always made up of  the same set of individuals.
>>
>> Thank you. I've learnt so much with this long response.
>>
>> Cheers,
>> M.
>>
>>
>> 2014-06-18 19:48 GMT+01:00 Caitlin Collins <caitiecollins at gmail.com>:
>>
>> Hi,
>>>
>>> Glad to see you've been reading the tutorial in such detail!
>>>
>>> These are great questions, and the way you have asked them actually
>>> hints at the answer: set.seed() is not inherently linked to multivariate
>>> techniques or datasets, but rather with random number generation (more
>>> specifically, with getting *reproducible* results from "random"
>>> processes). This is probably why you have seen set.seed come up in the
>>> context of bootstrap Monte Carlo procedures!
>>>
>>> Essentially, when R is asked to generate a "random" number, it actually
>>> generates a pseudo-random number by taking some input and generating an
>>> output that seems random. Without being given an input, R does this by
>>> using your computer's clock and using the current time as its starting
>>> point, from which it generates a seemingly random number. You would not get
>>> the same random number at a different time, so we find this adequate to
>>> call the process "random" number generation, BUT if in fact you tried to
>>> generate two "random" numbers at the exact same time (down to the
>>> millisecond), you would actually get the exact same "random" number. (Note:
>>> I have glossed over a lot of really interesting things about this process,
>>> so if you want to know more about random number generation, please read on
>>> here:
>>> http://cran.r-project.org/web/packages/randtoolbox/vignettes/fullpres.pdf
>>> ).
>>>
>>> This potential problem with random number generation can occasionally be
>>> quite useful in cases where we want to run something that requires random
>>> number generation but where we would also like to get the same result each
>>> time.
>>> set.seed() is the way we control this. With set.seed(), the "seed" is
>>> used as the input to our random number generation (instead of the clock),
>>> which allows you to get *reproducible *"random" numbers.
>>>
>>> Try this example:
>>>
>>> rnorm(3)
>>> rnorm(3)
>>>
>>> set.seed(1)
>>> rnorm(3)
>>>
>>> set.seed(1) # note: for set.seed() to work, you need to use it before
>>> every instance of random number generation.
>>> rnorm(3)
>>>
>>> Neat! Having established this, we can now answer your questions about
>>> why we use set.seed() where we do in the DAPC tutorial.
>>>
>>> On page 20, we use it before creating a loading plot. This is just
>>> because we use the argument lab.jitter to move the labels around a bit.
>>> Jitter works by adding random noise, so we can control it with set.seed().
>>> We have chosen to use set.seed(4) simply because it "randomly" put the
>>> labels in a nice enough place. Arguably, set.seed(6) would have done a
>>> better job (next time!), but it's a good thing we didn't use set.seed(2).
>>>
>>> If you would like, you can see for yourself:
>>>
>>> data(H3N2)
>>> pop(H3N2) <- factor(H3N2$other$epid)
>>> dapc.flu <- dapc(H3N2, n.pca=30,n.da=10)
>>>
>>> set.seed(4)
>>> contrib <- loadingplot(dapc.flu$var.contr, axis=2, thres=.07,
>>> lab.jitter=1)
>>>
>>> set.seed(6)
>>> contrib <- loadingplot(dapc.flu$var.contr, axis=2, thres=.07,
>>> lab.jitter=1)
>>>
>>> set.seed(2)
>>> contrib <- loadingplot(dapc.flu$var.contr, axis=2, thres=.07,
>>> lab.jitter=1)
>>>
>>> Finally, we use set.seed(2) on page 39 to get a "random" sample of 20
>>> individuals (you were right about that) to serve as our "supplementary
>>> individuals" for that exercise. Here, the use of set.seed(2) just ensures
>>> that no matter how many times we edit and re-build that tutorial, we will
>>> always get the same set of 20 individuals, which is useful for
>>> consistency's sake.
>>>
>>> All in all, I apologise for the long response that was possibly less
>>> related to DAPC than you might have expected, but I hope that helped answer
>>> your question!
>>>
>>> Best,
>>> Caitlin.
>>>
>>>
>>>
>>>
>>> On Wed, Jun 18, 2014 at 6:51 PM, Manuela <manuelacorreia2 at gmail.com>
>>> wrote:
>>>
>>>> Hi there,
>>>>
>>>>
>>>> I'd like to understand  the role of set.seeds and the criteria chosen
>>>> in the DAPC examples according to the two examples presented in the
>>>> lattested version of DAPC tutorial.
>>>>
>>>> I used to see set. seeds(N?) in the context of significance as well as
>>>> bootstrap Monte Carlo procedures, but not within multivariate techniques or
>>>> even with datasets.
>>>>
>>>> At page 20 from DAPC tutorial there is a set. seed(4) before getting
>>>> the loadingplot. Also, another example at page 39, before split the dataset
>>>> microbov in two parts. And by the way, what is  20 in the sample(e,20....)?
>>>> 20 individuals picked at random from all microbov populations?
>>>>
>>>>
>>>> So, I do have two questions.
>>>> One is  "why to use them?" here in these particular examples?
>>>> The second one "what criteria were behind the choice of the number 4 in
>>>> the former case, and the number 2 in the latter?
>>>>
>>>> How do I know which seed will be the best one for my datased in case I
>>>> need to have the loadingplot?
>>>>
>>>> Thanks in advance,
>>>> M.
>>>>
>>>> _______________________________________________
>>>> adegenet-forum mailing list
>>>> adegenet-forum at lists.r-forge.r-project.org
>>>>
>>>> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/adegenet-forum
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/adegenet-forum/attachments/20140619/948c5e9d/attachment-0001.html>