<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Dear Raquel, </div><div><br></div><div>Sorry for the very late answer.</div><div><br></div><div>I really need to rewrite completely the Ensemble Forecasting function which is not that clear. </div><div><br></div><div><blockquote type="cite"><div>As for question 5, as far as I could gather, the consensus_spp_proj.name<br>gives predictions on the total data (as well as on each repetition), whereas<br>Total_consensus_proj.name gives predictions based on all repetitions (if<br>final.model.out=F) or on all the repetitions and total data (if<br>final.model.out=T). </div></blockquote><div><br></div><div>Right. </div><br><blockquote type="cite"><div>However, the consensus_spp_proj.name_Bin output doesn't<br>seem to give binary predictions for the ensemble median (the manual says<br>that a threshold cannot be calculated), but the<br>Total_consensus_proj.name_Bin output does - on which thresholds are they<br>based? <br></div></blockquote><div><br></div><div>Indeed, this is not clear. For this Total_consensus_proj.name_Bin, the average threshold from the selected models is used. I just changed by the median which is more logical. </div><div>In order words, the same threshold was used for "mean" and "median" until now. </div><div><br></div><div>I'll also modify the function to make sure consensus_spp_proj.name_Bin could also give a binary projection for the median by taking the median threshold. </div><div><br></div><div>Cheers,</div><div>Wilfried</div><div><br></div><div><br></div><div><br></div><br><blockquote type="cite"><div><br>Thanks once again,<br>Raquel <br><br><br>-----Original Message-----<br>From: <a href="mailto:biomod-commits-bounces@lists.r-forge.r-project.org">biomod-commits-bounces@lists.r-forge.r-project.org</a><br>[mailto:biomod-commits-bounces@lists.r-forge.r-project.org] On Behalf Of<br><a href="mailto:biomod-commits-request@lists.r-forge.r-project.org">biomod-commits-request@lists.r-forge.r-project.org</a><br>Sent: 16 December 2010 12:01 PM<br>To: biomod-commits@lists.r-forge.r-project.org<br>Subject: Biomod-commits Digest, Vol 20, Issue 4<br><br>Send Biomod-commits mailing list submissions to<br><span class="Apple-tab-span" style="white-space:pre"> </span>biomod-commits@lists.r-forge.r-project.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br><span class="Apple-tab-span" style="white-space:pre"> </span><br>https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits<br><br>or, via email, send a message with subject or body 'help' to<br><span class="Apple-tab-span" style="white-space:pre"> </span>biomod-commits-request@lists.r-forge.r-project.org<br><br>You can reach the person managing the list at<br><span class="Apple-tab-span" style="white-space:pre"> </span>biomod-commits-owner@lists.r-forge.r-project.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Biomod-commits digest..."<br><br><br>Today's Topics:<br><br> 1. Predictive accuracy in Ensemble.Forecasting and<span class="Apple-tab-span" style="white-space:pre"> </span>Models<br> functions (Raquel A. Garcia)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Wed, 15 Dec 2010 15:54:19 +0100<br>From: "Raquel A. Garcia" <raquel.garcia@mncn.csic.es><br>Subject: [Biomod-commits] Predictive accuracy in Ensemble.Forecasting<br><span class="Apple-tab-span" style="white-space:pre"> </span>and<span class="Apple-tab-span" style="white-space:pre"> </span>Models functions<br>To: biomod-commits@lists.r-forge.r-project.org<br>Message-ID: <20101215155419.42243vxobjhtsk9n@webmail.csic.es><br>Content-Type: text/plain; charset="utf-8"<br><br><br><br>Dear BIOMODers <br><br>I wonder if you can help with a few questions about the predictive accuracy<br>of models. <br><br>1.For the consensus predictions from Ensemble.Forecasting(), the Manual says<br>that the test is performed on the calibration data. So, to confirm, the<br>first column of $test.results reports the ROC scores using all the data, and<br>the following columns the ROC scores using the calibration data for each<br>repetition? (there was a post a few months ago about the possibility of<br>these tests being done on testing data instead, but I am assuming that the<br>current version still uses calibration data) <br><br>2. The help for the Ensemble.Forecasting() function says that the test<br>scores ? are obtained by applying the same ensemble computation on the<br>current predictions as on the future forecasts, and compared with the data<br>input for that species (using the Roc evaluation method)?. So, again to<br>confirm, are they calculated by running new ROC evaluations on the consensus<br>projections? <br><br>3. For the Models() function, the evaluation.results output for each<br>species, as far as I understand, gives in the last columns of $Spp_full the<br>sensitivity and specificity calculated using all the data (the full model).<br>Is it fair then to expect that calculating the Sens-Spec from the observed<br>data and the binary predictions using the total data should lead to similar<br>results as the Sens-Spec from the Models? (for presence-only data species I<br>would use the pseudo-absence data instead of the observed data). When I try<br>these calculations for the individual model projections, I get similar<br>values of specificity but lower values of sensitivity; and when I try it<br>for the mean, weighted mean and median consensus projections I get<br>unrealistically low values of sensitivity. <br><br>And two last ones still about the Ensemble.Forecasting() function: <br><br>4. The Total_consensus_proj.name output generates a ?general ensemble<br>forecast across all the runs? for each consensus method. Whether this<br>ensemble takes into account the final models as well as the repetitions<br>depends on the final.model.out argument, I think ? what is the default for<br>this argument? <br><br>5. In the consensus_spp_proj.name outputs, the second dimension includes the<br>full model and the repetitions. Does this full model refer to ensembles<br>built with the total data only or with total data plus the repetitions? If<br>the latter, is there no output with the consensus predictions based on the<br>total data only? <br><br>Hope this is not too long or unclear - many thanks <br><br>Raquel <br><br>Raquel A. Garcia<br>Biodiversity and Global Change Lab<br>www.biochange-lab.eu/<br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL:<br><http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/201<br>01215/0efa1b00/attachment.html><br><br>------------------------------<br><br>_______________________________________________<br>Biomod-commits mailing list<br>Biomod-commits@lists.r-forge.r-project.org<br>https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits<br><br><br>End of Biomod-commits Digest, Vol 20, Issue 4<br>*********************************************<br><br>_______________________________________________<br>Biomod-commits mailing list<br>Biomod-commits@lists.r-forge.r-project.org<br>https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits<br></div></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><span class="Apple-style-span" style="font-size: 12px; "><div>--------------------------</div><div>Dr. Wilfried Thuiller</div><div>Laboratoire d'Ecologie Alpine, UMR CNRS 5553</div><div>Université Joseph Fourier</div><div>BP53, 38041 Grenoble cedex 9, France</div><div>tel: +33 (0)4 76 51 44 97</div><div>fax: +33 (0)4 76 51 42 79</div><div><br></div><div>Email: <a href="mailto:wilfried.thuiller@ujf-grenoble.fr">wilfried.thuiller@ujf-grenoble.fr</a><br>Home page: <a href="http://www.will.chez-alice.fr/">http://www.will.chez-alice.fr</a><br>Website: <a href="http://www-leca.ujf-grenoble.fr/equipes/tde.htm">http://www-leca.ujf-grenoble.fr/equipes/tde.htm</a><br><br>FP6 European MACIS project: <a href="http://www.macis-project.net/">http://www.macis-project.net</a><br>FP6 European EcoChange project: <a href="http://www.ecochange-project.eu/">http://www.ecochange-project.eu</a></div><div><br></div></span></div></div></span><br class="Apple-interchange-newline"></div></span><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline">
</div>
<br></body></html>