<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<font face="Times New Roman, Times, serif">Dear Chris,<br>
This is difficult to answer that questions without knowing what you've
really done. Model parameterizations have a strong influence on model's
outputs. <br>
You talk about discrepancies. I do not see any difference for Species A
but they are indeed differences for species B, especially for GLM and
GBM. Do not beleive that differences of about 0.1 or 0.2 is enormous,
this is nothing. AUC is an index, it is not a statistic. It is also
very sensitive to the prevalence of the species. <br>
</font><br>
Few things to remember:<br>
<br>
1- Models are sensitive to the initial conditions and parameterisation.<br>
=> did you use the same parameterisation than in BIOMOD? for
instance, did you use the library GAM or MGCV? number of degree of
freedom? Which optimisation strategy did you sue for GBM?
Cross-validation or out-of-bag selection? <br>
=> Did you run the models on 100% of your initial data or did you
first calibrate on a random sample and evaluate on the remaining part?
did you keep the prevalence fixed? <br>
=> For GLM for instance: did you use polynomial, quadratic or linear
terms? which level? stepwise regression, backward or forward? <br>
=> Randomforest, GBM, neural networks have some buid-in
stochasticity. They could give different outputs when run several
times. <br>
<br>
There are so many ways to parameterise those models that this is very
difficult without having the data or the code you used. In any way, I
am not surprised. <br>
<br>
In anyway, this is an interesting email. It shows that although we have
been mostly focusing on inter-model variability, there is also a
substantial intra-model variability due to sample size, optimisation
algorithm, and stochasticity.<br>
I know Jane Elith (Australia) started to work on bit on this issue but
this is relatively well know actually. Take SAS, SPSS and R (gam, mgcv)
and run a GAM or neural-network on the same data, you'll get different
results. <br>
<br>
Hope it helps,<br>
Wilfried<br>
<br>
<br>
Christopher A Walter a écrit :
<blockquote
cite="mid:OF96DA7116.D28427F4-ON85257603.006C9285-85257603.006CB421@usgs.gov"
type="cite"><br>
<font face="sans-serif" size="2">Wilfried</font>
<br>
<br>
<font face="sans-serif" size="2">We have been testing the BIOMOD
models
against the stand-alone versions in R (functions glm, gam, gbm, and
randomForest)
and while we understand that there are many different options for
running
them, we cannot seem to make them match even when trying to equalize
the
defaults.</font>
<br>
<br>
<font face="sans-serif" size="2"> Below are tables of AUC and Somer's
rank correlation for two species, ran with BIOMOD and the stand-alone
models.
We are trying to justify using BIOMOD, but we cannot explain why it
creates
such a different output. Any thoughts on why this is happening?</font>
<br>
<br>
<br>
<br>
<br>
<br>
<img src="cid:part1.04090007.09040900@ujf-grenoble.fr"><br>
<br>
<br>
<font face="sans-serif" size="2">All the best</font>
<br>
<br>
<font face="sans-serif" size="2">Chris</font>
<br>
<br>
<font face="sans-serif" size="2">Christopher A. Walter <br>
Biological Science Technician <br>
USGS Leetown Science Center <br>
11649 Leetown Road <br>
Kearneysville, WV 25430 <br>
Phone: 304 724-4479 <br>
<br>
"Its easy not to think when you're not told that much."<br>
-Justin Pierre</font>
<pre wrap="">
<hr size="4" width="90%">
_______________________________________________
Biomod-commits mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Biomod-commits@lists.r-forge.r-project.org">Biomod-commits@lists.r-forge.r-project.org</a>
<a class="moz-txt-link-freetext" href="https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits">https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
--
Wilfried Thuiller
Laboratoire d'Ecologie Alpine, UMR-CNRS 5553
Université J. Fourier
BP 53, 38041 Grenoble Cedex 9, France
Tel: +33 (0)4 76 63 54 53
Fax: +33 (0)4 76 51 42 79
Email: <a class="moz-txt-link-abbreviated" href="mailto:wilfried.thuiller@ujf-grenoble.fr">wilfried.thuiller@ujf-grenoble.fr</a>
Home page: <a class="moz-txt-link-freetext" href="http://www.will.chez-alice.fr">http://www.will.chez-alice.fr</a>
Website: <a class="moz-txt-link-freetext" href="http://www-leca.ujf-grenoble.fr/equipes/tde.htm">http://www-leca.ujf-grenoble.fr/equipes/tde.htm</a>
FP6 European MACIS project: <a class="moz-txt-link-freetext" href="http://www.macis-project.net">http://www.macis-project.net</a>
FP6 European EcoChange project: <a class="moz-txt-link-freetext" href="http://www.ecochange-project.eu">http://www.ecochange-project.eu</a></pre>
</body>
</html>