[Biomod-commits] Evaluation.results interpretations

Wilfried Thuiller wilfried.thuiller at ujf-grenoble.fr
Tue Jul 20 14:06:14 CEST 2010


Dear Hedvig,

You can have a look at the Biomod Manual Page 45 where everything is explained. 

> Biomod.Manual()


To answer the question: 

For $Sp1_full_rep1

The first column is the score on the remaining 20% of the data after calibration of the model. The last four columns are determined with the 80% used for calibration and the 20% leftovers combined (not the full model, they are simply combined). 
The cut-off is estimated using the combined data. 

For $Sp1_full

The first column is the average of the cross- validation of all the repetitions. The second one is the score when the model is evaluated on independent data if any is available, and the four following colums are results obtained from the final model itself (using 100% of the data).

The model with 100% (called final model here) is thus calibrated and the predictions saved as well in /Pred/

HIH,
Wilfried

PS: see below for some direct answers to the questions:
> For repetition 1, the first column is the crossvalidation, from comparing predictions on the 20% evaluation data (from the models calibrated on 80% of data) to original distribution data. But how are the 'leftover' columns made? Crossvalidation with the 80%-models applied on 100% of data? Are Cutoffs made by comparing the predictions from 80%-models on 20% evaluation data, to the real distribution (in the same 20% of locations)?
> 

No, the predictions on the 80 and 20% are simply combined, evaluated and then se/sp estimated.
> For $Sp1_full, the first column is the average of all cross-validations (so it's the same value in my case). Are the other columns the model calibrated on 100% of the data, i.e. a 'final model'?

yes
> Are those Cutoffs made by the 100%-models applied on the entire studyarea?

yes
> Values for Sensitivity and Specificity seem to be generally higher than for the repetitions. (The manual says 'using the model built with all data for calibration' which confuses me).
> 

Yes, there are higher most of the time because the model uses all the available data. 
> Finally, does Biomod make a 'final model' trained on 100% of the data?
> 

Yeap. Save in /Pred. 



> Thank you very much,
> Hedvig Nenzén
> 
> One of my species as an example:
> > Evaluation.results.Kappa
> $Sp1_full
> Cross.validation indepdt.data total.score Cutoff Sensitivity Specificity
> ANN 0.688 none 0.7149572 495.00 89.70370 81.40044
> CTA 0.735 none 0.8010009 435.16 94.88889 84.13567
> GAM 0.683 none 0.7108026 526.29 90.59259 79.75930
> GBM 0.738 none 0.7730225 588.36 93.85185 82.38512
> GLM 0.670 none 0.6884088 588.82 87.18519 81.72867
> MARS 0.759 none 0.7729465 454.08 93.03704 83.47921
> FDA 0.733 none 0.7568880 372.35 92.66667 82.16630
> RF 0.745 none 1.0000000 440.00 100.00000 100.00000
> SRE 0.545 none 0.5607607 10.00 91.18519 62.69147
> 
> $Sp1_full_rep1
> Cross.validation indepdt.data total.score Cutoff Sensitivity Specificity
> ANN 0.688 none 0.7166007 486.71 89.92593 81.29103
> CTA 0.735 none 0.8030243 407.90 93.77778 85.88621
> GAM 0.683 none 0.7103411 546.15 90.22222 80.19694
> GBM 0.738 none 0.7740735 572.40 94.66667 81.40044
> GLM 0.670 none 0.6887384 589.41 86.96296 82.05689
> MARS 0.759 none 0.7715822 502.25 92.51852 84.02626
> FDA 0.733 none 0.7554794 355.14 93.03704 81.50985
> RF 0.745 none 0.9494617 480.00 98.29630 96.49891
> SRE 0.545 none 0.5783646 10.00 90.66667 65.20788
> 
> 
> 
> 
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits

--------------------------
Dr. Wilfried Thuiller
Laboratoire d'Ecologie Alpine, UMR CNRS 5553
Université Joseph Fourier
BP53, 38041 Grenoble cedex 9, France
tel: +33 (0)4 76 51 44 97
fax: +33 (0)4 76 51 42 79

Email: wilfried.thuiller at ujf-grenoble.fr
Home page: http://www.will.chez-alice.fr
Website: http://www-leca.ujf-grenoble.fr/equipes/tde.htm

FP6 European MACIS project: http://www.macis-project.net
FP6 European EcoChange project: http://www.ecochange-project.eu





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20100720/dfc935b9/attachment.htm>


More information about the Biomod-commits mailing list