[Biomod-commits] Comparison among variable importance

[Wilfried Thuiller]

Hi there all,

This is, indeed, a tricky question. 
The way we perform variable importance, as do RandomForest, is really without unit. As Viorel mentioned, this is 1 minus the mean of correlations (depending on the number of variable importance runs). This is done independently (as Maxent is also doing as far as I understood). I do not think there is a sound statistical way of standardizing them. You may do as Damien suggest but I fear this is not really sound and should be interpreted without caution (but I am happy to be convinced otherwise).
As Viorel suggested, box plots would do the same more or less the same but in a more sound way to me. 

Best,
Wilfried





Le 16 mai 2013 à 16:44, Viorel Popescu <vioreldpopescu at gmail.com> a écrit :

> Hi everyone, I had a similar question a few weeks ago, and from my
> understanding the varImportance scores are drawn from correlation
> coefficients between different models (with and without the respective
> variable permuted), namely, 1 minus the correlation (r). Hence, they cannot
> add up to 1, the way variable importance is performed in MaxEnt. In that
> case, standardizing the scores would not a be very good way to deal
> with the varImportance. Am I missing something here?
> 
> The way I dealt with them (with advice from Wilfried) was to show boxplots
> of varImportance score for each variable across modeling techniques (and
> taxa if you need to make such comparisons), and refer to median scores
> (distribution of scores was far from normal).
> 
> Cheers,
> Viorel
> 
> 
> On Thu, May 16, 2013 at 3:39 AM, Damien Georges
> <damien.georges2 at gmail.com>wrote:
> 
>> Dear Antonio,
>> 
>> I guess you are right and you need to make  a kind of standardisation of
>> scores to make them all comparable. Make them sum to 1 is a good way to do
>> it.
>> 
>> Cheers,
>> Damien.
>> 
>> 
>> 
>> On 16/05/2013 12:09, Antonio Canepa wrote:
>> 
>>> Dear list,
>>> I would like to know if you can directly compare among different models
>>> the
>>> variable importance of each variable selected?
>>> My guess is not, because when I got the importance of each variable (
>>> getModelsVarImport()), the sum of all of them for each model was not = 1.
>>> I don't know if I'm doing something wrong or there is another way to
>>> "standardise" the output from getModelsVarImport() and thus compare
>>> throughout models ?
>>> 
>>> 
>> ______________________________**_________________
>> Biomod-commits mailing list
>> Biomod-commits at lists.r-forge.**r-project.org<Biomod-commits at lists.r-forge.r-project.org>
>> https://lists.r-forge.r-**project.org/cgi-bin/mailman/**
>> listinfo/biomod-commits<https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits>
>> 
> 
> 
> 
> -- 
> Viorel D. Popescu
> David H. Smith Conservation Research Fellow
> University of California - Santa Cruz &
> Simon Fraser University, Biological Sciences
> (604) 340 4228
> https://sites.google.com/site/vioreldpopescu/
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits

-----------------------------
Dr. Wilfried Thuiller
Laboratoire d'Ecologie Alpine, UMR CNRS 5553
Université Joseph Fourier
BP53, 38041 Grenoble cedex 9, France
tel: +33 (0)4 76 51 44 97
fax: +33 (0)4 76 51 42 79

Email: wilfried.thuiller at ujf-grenoble.fr
Personal website: http://www.will.chez-alice.fr
Team website: http://www-leca.ujf-grenoble.fr/equipes/emabio.htm

ERC Starting Grant TEEMBIO project: http://www.will.chez-alice.fr/Research.html