[Biomod-commits] biomod2/ Maxent run

[Damien Georges]

Dear Semra,

In fact the results of variables importances obtained with biomod2 (the 
first table) and directly with MAXENT (the second one) differs because 
they are not based on the same metrics.

What is calculated in biomod2 correspond more to Permutation importance 
(2nd column) than to Percent of contribution (1st column).

In both case the prediction of the model with calibrating+testing data 
is compared to the one obtained with one variable randomly shuffled.

In MAXENT, the AUC decreasing is used to quantify the variable 
importances whereas we used the correlation between 'normal' predictions 
and 'shuffled' ones in biomod2. (one of reasons not to take AUC 
decreasing for variables importance calculation is that it's 
uncomputable for SRE models)

That's mainly why the results differs.

Moreover, the shuffling process is repeated 'VarImport' times in biomod2 
(the mean is returned) whereas (if I remember well) it's only done one 
time in MAXENT.

If we decide to recalculate our own variables importance and evaluation 
for MAXENT models in biomod2 it's because we want to be able to compare 
all models with each other.

That being said, keep in mind that variable importances calculation is 
an heuristic metric that can vary a lot between different runs. This 
metric is also realy dependent of variables correlations. That's why, 
you might be really careful when you want to make conclusion using it.

Hope that's understandable and helpful,

Best,

Damien



On 27/07/2012 14:58, semra yalcin wrote:
> Dear Damien and BIOMOD-users,
>
> I would like to ask a question about MaxEnt results on biomod2. I 
> followed each step and run the models of Myocastor from vignette/biomod2.
>
> When I compare the results of biomod2-maxent (on R) with the results 
> in the folder of MaxEnt model 
> ("Myocastor\models\Myocastor_PA1_RUN1_MAXENT"), I saw that the order 
> of importance of the variance and AUC values of the model are different.
>
> For example;
>
> getModelsVarImport(myBiomodModelOut) returns as
>
> ....
> , , RUN1, PA1
>
>        SRE    RF MAXENT
> Var1 0.048 0.343  0.531
> Var2 0.048 0.270  0.140
> Var3 0.140 0.187  0.134
> Var4 0.162 0.328  0.568
> Var5 0.074 0.115  0.126
> ....
>
> while " Myocastor_PA1_RUN1.html" shows the order of the variables as
>
> Variable 	Percent contribution 	Permutation importance
> bio_3(Var1) 	47 	41.1
> bio_11(Var4) 	24.4 	39.7
> bio_4(Var2) 	19.4 	3
> bio_12(Var5) 	6.5 	11
> bio_7(Var3) 	2.8 	5.2
>
>
>
> As you see, the order of the variance by "getModelsVarImport" is Var4/ 
> Var1/ Var2/ Var3/ Var5 whereas the order of the variance in the MaxEnt 
> results is Var1/ Var4/ Var2/ Var5/Var3. So what could be the reason 
> for this differences?
>
> Moreover, there is a difference between ROC(AUC) values between 
> biomod2-maxent and MaxEnt documents. Is this because that biomod2 
> calculates ROC of testing data ("myBiomodModelEval["ROC",,"MAXENT",,]"?
>
> By the way I run models with my own data and I have same differences 
> in the order of importance of the variance in the maxent results.
>
> Thank you in advance.
>
> Semra.
>
>
>
>
>
>
>
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/biomod-commits/attachments/20120727/0af8517a/attachment.html>