[Biomod-commits] Optimized threshold by ROC

[Wilfried Thuiller]

Dear Anne,

No there are not strictly the same. 

The one you mentioned is: 

maximising[(SE+SP)/2]

the one is BIOMOD is:

minimising[abs(SE-SP)]

Both are used in the SDM field. Although there are not strictly equivalent, the associated thresholds are often very similar. 

You can see the difference running the script below:

library(BIOMOD)
library(PresenceAbsence)

data(Sp.Env)

Initial.State(Response=Sp.Env[,12:13], Explanatory=Sp.Env[,4:10], 
IndependentResponse=NULL, IndependentExplanatory=NULL)

Models(GAM = TRUE, NbRunEval = 1, DataSplit = 80,
   Yweights=NULL, Roc=TRUE, Optimized.Threshold.Roc=TRUE, Kappa=F, TSS=F, KeepPredIndependent = FALSE, VarImport=0,
   NbRepPA=0, strategy="circles", coor=CoorXY, distance=2, nb.absences=1000)


load("pred/Pred_Sp277")

data=cbind(Sp.Env[,1], Sp.Env[,13], Pred_Sp277[,3,1,1]/1000)

plotroc <- roc.plot.calculate(data)

### Plot the change in sensitivity in function of the threshold
plot(plotroc$threshold, plotroc$sensitivity, type="l", col="blue ")
### Plot the change in specificity in function of the threshold
lines(plotroc$threshold, plotroc$specificity)
### Plot the change in one criteria to select the threshold: (SE+SP)/2 in function of the threshold
lines(plotroc$threshold, (plotroc$specificity+plotroc$sensitivity)/2, col="red")
### Add a red circle when maximising one criteria to select the threshold: (SE+SP)/2 
points(plotroc$threshold[which((plotroc$specificity+plotroc$sensitivity)/2 ==max((plotroc$specificity+plotroc$sensitivity)/2))], max((plotroc$specificity+plotroc$sensitivity)/2), col="red")
### Plot the change in one criteria to select the threshold: min(abs((SE-SP)) in function of the threshold. This is the one in BIOMOD
lines(plotroc$threshold, abs(plotroc$specificity-plotroc$sensitivity), col="orange")
### Add a orange circle when minimising to select the threshold: min(abs((SE-SP)). This is the one in BIOMOD
points(plotroc$threshold[which(abs(plotroc$specificity-plotroc$sensitivity) ==min(abs(plotroc$specificity-plotroc$sensitivity)))], max((plotroc$specificity+plotroc$sensitivity)/2), col="orange")


Hope it helps,

Wilfried




Le 11 août 2010 à 13:48, Anne Blach Overgaard a écrit :

> Dear BIOMOD group,
>  
> On page 46 in the BIOMOD manual it is explained that selecting the "Optimized threshold by ROC" gives a cutoff representing "the best probability threshold maximising the percentage of presence and absence correctly predicted for the evaluation data". Going through the SDM litterature different terms are used for these thresholds and I would just like to ask if any one could tell me if this threshold is equivalent to the one named "Maximising the sum of sensitivity and specificity" or "Maximum sensitivity plus specificity"
>  
> Best regards,
>  
> Anne
> 
> 
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits

--------------------------
Dr. Wilfried Thuiller
Laboratoire d'Ecologie Alpine, UMR CNRS 5553
Université Joseph Fourier
BP53, 38041 Grenoble cedex 9, France
tel: +33 (0)4 76 51 44 97
fax: +33 (0)4 76 51 42 79

Email: wilfried.thuiller at ujf-grenoble.fr
Home page: http://www.will.chez-alice.fr
Website: http://www-leca.ujf-grenoble.fr/equipes/tde.htm

FP6 European MACIS project: http://www.macis-project.net
FP6 European EcoChange project: http://www.ecochange-project.eu





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20100811/6e915c3f/attachment.htm>