<div dir="ltr"><p class="MsoNormal">Hello again,</p><p class="MsoNormal"><br></p><p class="MsoNormal">In response to your two questions:</p><p class="MsoNormal"><span style><br></span></p><p class="MsoNormal" style="text-indent:0px"><b><span style>1)</span><span style> </span></b></p><p class="MsoNormal" style="text-indent:0px"><span style><br></span></p><p class="MsoNormal"><span style>The output element “mean and CI for random chance” provides the values that are
used to draw the horizontal solid (mean) and dashed (CI) lines on the plot
generated for cross-validation.</span></p><p class="MsoNormal"><span style><br></span></p><p class="MsoNormal"><span style>In your case, the mean and CI for random
chance was 49% (43%, 60%). The interpretation of this would be that if the highest
success in outcome prediction that you were able to achieve with any model was
between 43% and 60%, then you could be 95% confident that the ability of even the
best model to assign individuals to the correct group does not differ
significantly from the success rate you could achieve by assigning individuals
to a group at random by, say, flipping a coin as a method of determining what group
they belonged to. Ergo, you would not have succeeded in creating a useful
model. </span><span style> </span></p><p class="MsoNormal"><br></p><p class="MsoNormal">However, your results indicate that with 25
PCs retained, your model had a success rate of 69.5%, so you <i>have</i> created a “useful” model. Even
though it is not a particularly successful model, it still has a mean success
rate that is 20% higher than the mean success for the coin toss approach, and
10% higher than the upper limit of the CI for random chance. So you can be 95%
confident that the somewhat modest ability of your best model to discriminate between
groups is not just happening by chance—the model is truly doing something
useful.</p><p class="MsoNormal">------</p>
<p class="" style><b>2)<span style="font-stretch:normal;font-size:7pt;font-family:'Times New Roman'"> </span>2)</b></p><p class="" style><span style> While your interpretation is generally
true, in that group membership is not well-predicted by any model, I think you have
mis-read the results. The way they are laid out, at least in the text you
copied into the e-mail, has skewed the values given for the means to the right
of the number of PCs that they should be corresponding to… With 25 PCs, your
optimal model is actually achieving a mean success of nearly 70%. Still not too
good, but better than 63%. The MSE for 25 PCs is 32.4%, which is indeed quite
high.</span></p>
<p class="">However, the interpretation of this is not
that you can only be “sure” of correctly predicting around 20% to the right
pre-defined group. Rather, you can be “sure” of correctly predicting almost
70%! I think your confusion here may come from your interpretation of what the
random chance values mean. Finding that the mean success for your best model is
20% above the mean success for random chance does not mean you can only be sure
of 20% correct predictions. Rather, you could say that while you can in fact
expect a 70% success rate (your highest mean success), your model is only
providing an improvement of ~ 20% over the success rate you could have achieved
by tossing a coin.</p>
<p class="">This changes the severity of your final
conclusion. First, I should mention that it’s not fair to say that “[your] set
of microsatellites can’t explain well [your] pre-defined groups”. Instead, it
might be more accurate to say, “<i>With</i>
the set of microsatellites available, you are unable to build a <i>model</i> with DAPC that explains well the
variation between your pre-defined groups.” Finally, in light of the points
above, while it is still true that the model does not explain the variation
between groups particularly well, it does explain about 70% of that variation,
so I wouldn’t consider it to be “unsuccessful”. </p>
<p class="">-----</p>
<p class="">Sorry for the long answer, but I hope it
helps a bit at least! </p><p class="">Please let me know if it doesn’t though, or if you have any more
questions. </p>
<p class=""> </p>
<p class="">All the best, <br>
Caitlin. <br>
<br>
<br>
</p></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 16, 2014 at 11:30 PM, Angela Merino <span dir="ltr"><<a href="mailto:Angela.Merino@cawthron.org.nz" target="_blank">Angela.Merino@cawthron.org.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-NZ" link="blue" vlink="purple">
<div><span class="">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks you very much! It was really helpful!
</span><span style="font-size:11.0pt;font-family:Wingdings">J</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Then I understand that my models is not significantly the best model that could be found using my variables (in my case, microsatellites). If I use a model with n.pca=20
or =40 I got pretty the same success of membership prediction (and with the same big root mean squared error).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><u></u> <u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><span>1)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">My last questions (I hope!) to understand the output of the
<i>cross.validation</i> function is what does it mean the Median and Confidence Interval for Random Chance (below in yellow)? I think it means that with a confidence of 95% the value of successful assignment would be a value between 43% and 60%, which therefore
means again that the optimization of my model was “not successful”. (??)<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><span>2)<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">About the global interpretation of this results, I would say that membership of my predefined groups are not well predicted by any model as the mean successful
assignment is not higher than 63% (Maximum when n.pcs=25) and in addition the mean squared errors is quite high (30-40%). I would be “sure” of predicting only around 20% to the right predefined group. In short, my set of microsatellites can’t explain well
my predefined groups.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
</span><p class="MsoNormal"><u></u><img width="631" height="497" src="cid:image002.jpg@01CFE9FD.C12A0D30" align="left" hspace="12" alt="cid:image002.jpg@01CFE7A4.CCC02130"><u></u><b><span style="font-size:9.0pt;background:yellow">$`Median
and Confidence Interval for Random Chance`</span></b><span style="font-size:9.0pt;background:yellow"><u></u><u></u></span></p><span class="">
<p class="MsoNormal"><b><span style="font-size:9.0pt;background:yellow"> 2.5% 50% 97.5%
</span></b><span style="font-size:9.0pt;background:yellow"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;background:yellow">0.4294840 0.4928747 0.5962807</span></b><b><span style="font-size:9.0pt">
</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">$`Mean Successful Assignment by Number of PCs of PCA`</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt"> 5 10 15 20 25 30 35 40
</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">0.5871429 0.6000000 0.5819048 0.6014286 0.6952381 0.6747619 0.6333333 0.6109524
</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">$`Number of PCs Achieving Highest Mean Success`</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">[1] "25"</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">$`Root Mean Squared Error by Number of PCs of PCA`</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt"> 5 10 15 20 25 30 35 40
</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">0.4301795 0.4141872 0.4389381 0.4131429 0.3241735 0.3531491 0.3885084 0.4145894
</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">$`Number of PCs Achieving Lowest MSE`</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt">[1] "25"</span></b><span style="font-size:9.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
</span><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks in advance! I am learning a lot about R and adegenet package and I find really interesting to assess weak genetic population structure.<u></u><u></u></span></p><span class="">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Kind regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">‘Angela<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Caitlin Collins [mailto:<a href="mailto:caitiecollins@gmail.com" target="_blank">caitiecollins@gmail.com</a>]
<br>
<b>Sent:</b> Friday, 17 October 2014 1:28 a.m.<br>
<b>To:</b> Angela Merino<br>
<b>Cc:</b> Collins, Caitlin; Jombart, Thibaut<br>
<b>Subject:</b> Re: Question about how to interpret Cross validation in my analysis. Thanks!<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</span><div>
<p class="MsoNormal">Hi Angela,
<br></p><div><div class="h5">
<br>
Well, I have two pieces of good news for you, and one piece of mediocre news.<br>
<br>
First, there’s nothing to worry about with respect to the “NULL” that you are seeing. It just gets printed when xval.plot=TRUE as an artefact of one of the lines of the printing function. It has no meaning, and certainly does not imply that your model is not
valid. (Given the stress that I now realise this glaring “NULL” may cause, I’ve changed the way the plots print now, so in the next release of adegenet this won’t happen.)<br>
<br>
Second, you are absolutely correct in your interpretation of the results of xvalDapc (which are stored in whatever object you assigned the results to, in your case, “xval”).
<u></u><u></u></div></div><p></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">This brings me to the mediocre news: given that your interpretation is correct, it seems that the best model you can achieve with DAPC, where n.pca=25, is only able to predict the
group membership of validation set individuals in 63% of the cases, with a 32% root mean squared error. Arguably, this is not great. Your final comment on the matter, though, is quite insightful. The fact that you can achieve the same modest level of success
with 20-80 PCs indicates that the optimisation procedure has not been particularly successful. Ideally, one would like to see an arch, with a maximum success point somewhere in the middle. In your case, there is a bit of an arch, but it isn’t particularly
striking. <u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The only thing I might add to your interpretation of this result is that it’s not so much that the model is poor because a similar level of success can be achieved with variable
numbers of PCs. If mean success was virtually constant, but varying around 90%, the interpretation would not be that the model is poor, but rather that most levels of PC retention can compose a model that effectively discriminates between groups.
<br>
<br>
I hope this has helped answer some of your questions. If you have any more, please feel free to ask.
<br>
<br>
Best, <br>
Caitlin. <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div></div></div><div><div class="h5">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Oct 13, 2014 at 11:48 PM, Angela Merino <<a href="mailto:Angela.Merino@cawthron.org.nz" target="_blank">Angela.Merino@cawthron.org.nz</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Caitlin Collins and Thibaut Jombart,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">My name is Angela Parody-Merino and I am a PhD student at Massey University (New Zealand). I am studying the population genetic structure in a migratory bird (the New Zealand Godwit)
with 23 microsatellites. Anyway, maybe this is a very simple question but I really want to understand and be sure about the meaning and interpretation of the output when doing cross-validation. I have been some days looking in the internet and reading explanations
etc…without being able to really understand what’s going on with my analysis. Could you help me please?
<span style="font-family:Wingdings">J</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">This is the script of the analysis:<u></u><u></u></p>
<p class="MsoNormal">> x <- ELpop<u></u><u></u></p>
<p class="MsoNormal">> mat <- as.matrix(na.replace(x, method="mean"))<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Replaced 371 missing values
<u></u><u></u></p>
<p class="MsoNormal">> grp <- pop(x)<u></u><u></u></p>
<p class="MsoNormal">> xval <- xvalDapc(mat, grp, n.pca.max = 40, training.set = 0.9,<u></u><u></u></p>
<p class="MsoNormal">+ result = "groupMean", center = TRUE, scale = FALSE,<u></u><u></u></p>
<p class="MsoNormal">+ n.pca = NULL, n.rep = 500, xval.plot = TRUE)<u></u><u></u></p>
<p class="MsoNormal"><span style="color:red">NULL
</span><b>>>> What does it mean this NULL? Does it mean that the model is not valid?</b><u></u><u></u></p>
<p class="MsoNormal"><u></u><img width="821" height="647" src="cid:image001.jpg@01CFE9FA.D80955E0" align="left" hspace="12"><u></u><b><span style="color:#558ed5">$`Median
and Confidence Interval for Random Chance`</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> 2.5% 50% 97.5%
</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">0.4294840 0.4928747 0.5962807
</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> </span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">$`Mean Successful Assignment by Number of PCs of PCA`</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> 5 10 15 20 25 30 35 40
</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">0.5871429 0.6000000 0.5819048 0.6014286 0.6952381 0.6747619 0.6333333 0.6109524
</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> </span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">$`Number of PCs Achieving Highest Mean Success`</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">[1] "25"</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> </span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">$`Root Mean Squared Error by Number of PCs of PCA`</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> 5 10 15 20 25 30 35 40
</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">0.4301795 0.4141872 0.4389381 0.4131429 0.3241735 0.3531491 0.3885084 0.4145894
</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5"> </span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">$`Number of PCs Achieving Lowest MSE`</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:#558ed5">[1] "25"</span></b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="color:red"> </span></b><u></u><u></u></p>
<p class="MsoNormal"><b>From the screenshot and the output results of the cross validation (in blue), I would say that my model (retaining 25PCs) can predict with a mean of 63% but it is not such a
good model because most of the models that can be obtained by retaining 20, 40, 60, 80 PCs are quite the same successful. Is it my interpretation correct?</b><u></u><u></u></p>
<p class="MsoNormal"><b> </b><u></u><u></u></p>
<p class="MsoNormal"><b> </b><u></u><u></u></p>
<p class="MsoNormal"><b> </b><u></u><u></u></p>
<p class="MsoNormal">Thanks in advance,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Kind regards,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">‘Angela Parody-Merino<u></u><u></u></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><strong><span style="color:red">Attention: </span></strong><br>
This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies
of it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Cawthron reserves the right to monitor all e-mail communications through its networks.
Any opinions expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorised to make that statement.
<u></u><u></u></p>
<p>This e-mail message has been scanned and cleared by <strong><span style="color:#400080">MailMarshal
</span></strong><u></u><u></u></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</blockquote></div><br></div>