[Biomod-commits] Comparison among variable importance
Joaquin DL
jduquelazo at gmail.com
Mon May 20 18:24:46 CEST 2013
I think this is a really interesting discussion!
But how can we do those boxplots?
If we include the variable importance output within all the runs, we will
consider"bad" model inside our variable comparison and include variables
that might be high correlated with this bad prediction disturbing our
result... I am correct?
Could it be posible to compare variables between models according to their
rank ( the variable that is more correlated with the prediction would be
classified as the most importance, rank 1) insted of their correlation
(weight)... I wont be able to identify the weight but can say which
variable influence more in the prediction... I might be wrong
I found a paper where they did it like that. have a look to: Syphard, A.D.
& Franklin, J. (2009) Differences in spatial predictions among species
distribution modeling methods vary with species traits and environmental
predictors. Ecography, 32, 907-918.
Cheers
Joaquin
2013/5/17 <biomod-commits-request at lists.r-forge.r-project.org>
> Send Biomod-commits mailing list submissions to
> biomod-commits at lists.r-forge.r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
> or, via email, send a message with subject or body 'help' to
> biomod-commits-request at lists.r-forge.r-project.org
>
> You can reach the person managing the list at
> biomod-commits-owner at lists.r-forge.r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Biomod-commits digest..."
>
>
> Today's Topics:
>
> 1. Comparison among variable importance (Antonio Canepa)
> 2. Re: Comparison among variable importance (Damien Georges)
> 3. Re: Comparison among variable importance (Viorel Popescu)
> 4. Re: Comparison among variable importance (Wilfried Thuiller)
> 5. Re: Comparison among variable importance (Antonio Canepa)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 16 May 2013 12:09:48 +0200
> From: Antonio Canepa <ajcanepa at gmail.com>
> To: biomod-commits at lists.r-forge.r-project.org
> Subject: [Biomod-commits] Comparison among variable importance
> Message-ID:
> <
> CAN6VVbSW5nkzmeYcmOB+brDLU43+vwL2Et91t8o8RSctDzUDSg at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Dear list,
> I would like to know if you can directly compare among different models the
> variable importance of each variable selected?
> My guess is not, because when I got the importance of each variable (
> getModelsVarImport()), the sum of all of them for each model was not = 1.
> I don't know if I'm doing something wrong or there is another way to
> "standardise" the output from getModelsVarImport() and thus compare
> throughout models ?
>
> --
> Antonio Canepa Oneto
> PhD student
> Dept. Biologia Marina i Oceanografia
> Institut de Ciencies Del Mar CMIMA, CSIC
> Ps. Mar?tim de la Barceloneta,37-49, c.p. 08003
> BARCELONA, Catalunya,Spain
> Phone: +34 932 30 9500 (ext. 6067)
> Mobile: +34 628 320 778
> e-mail: canepa at icm.csic.es
> ajcanepa at gmail.com
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 16 May 2013 12:39:27 +0200
> From: Damien Georges <damien.georges2 at gmail.com>
> To: Antonio Canepa <ajcanepa at gmail.com>
> Cc: biomod-commits at lists.r-forge.r-project.org
> Subject: Re: [Biomod-commits] Comparison among variable importance
> Message-ID: <5194B75F.3070301 at gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Dear Antonio,
>
> I guess you are right and you need to make a kind of standardisation of
> scores to make them all comparable. Make them sum to 1 is a good way to
> do it.
>
> Cheers,
> Damien.
>
>
> On 16/05/2013 12:09, Antonio Canepa wrote:
> > Dear list,
> > I would like to know if you can directly compare among different models
> the
> > variable importance of each variable selected?
> > My guess is not, because when I got the importance of each variable (
> > getModelsVarImport()), the sum of all of them for each model was not = 1.
> > I don't know if I'm doing something wrong or there is another way to
> > "standardise" the output from getModelsVarImport() and thus compare
> > throughout models ?
> >
>
>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 16 May 2013 07:44:25 -0700
> From: Viorel Popescu <vioreldpopescu at gmail.com>
> To: Damien Georges <damien.georges2 at gmail.com>
> Cc: biomod-commits at lists.r-forge.r-project.org
> Subject: Re: [Biomod-commits] Comparison among variable importance
> Message-ID:
> <CANX0Vu9109LPiNa0a7rr5gXLcfEF2nEzE+4_d0Pu=
> 0PewXgcyA at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi everyone, I had a similar question a few weeks ago, and from my
> understanding the varImportance scores are drawn from correlation
> coefficients between different models (with and without the respective
> variable permuted), namely, 1 minus the correlation (r). Hence, they cannot
> add up to 1, the way variable importance is performed in MaxEnt. In that
> case, standardizing the scores would not a be very good way to deal
> with the varImportance. Am I missing something here?
>
> The way I dealt with them (with advice from Wilfried) was to show boxplots
> of varImportance score for each variable across modeling techniques (and
> taxa if you need to make such comparisons), and refer to median scores
> (distribution of scores was far from normal).
>
> Cheers,
> Viorel
>
>
> On Thu, May 16, 2013 at 3:39 AM, Damien Georges
> <damien.georges2 at gmail.com>wrote:
>
> > Dear Antonio,
> >
> > I guess you are right and you need to make a kind of standardisation of
> > scores to make them all comparable. Make them sum to 1 is a good way to
> do
> > it.
> >
> > Cheers,
> > Damien.
> >
> >
> >
> > On 16/05/2013 12:09, Antonio Canepa wrote:
> >
> >> Dear list,
> >> I would like to know if you can directly compare among different models
> >> the
> >> variable importance of each variable selected?
> >> My guess is not, because when I got the importance of each variable (
> >> getModelsVarImport()), the sum of all of them for each model was not =
> 1.
> >> I don't know if I'm doing something wrong or there is another way to
> >> "standardise" the output from getModelsVarImport() and thus compare
> >> throughout models ?
> >>
> >>
> > ______________________________**_________________
> > Biomod-commits mailing list
> > Biomod-commits at lists.r-forge.**r-project.org<
> Biomod-commits at lists.r-forge.r-project.org>
> > https://lists.r-forge.r-**project.org/cgi-bin/mailman/**
> > listinfo/biomod-commits<
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
> >
> >
>
>
>
> --
> Viorel D. Popescu
> David H. Smith Conservation Research Fellow
> University of California - Santa Cruz &
> Simon Fraser University, Biological Sciences
> (604) 340 4228
> https://sites.google.com/site/vioreldpopescu/
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 16 May 2013 18:34:18 +0200
> From: Wilfried Thuiller <wilfried.thuiller at ujf-grenoble.fr>
> To: Viorel Popescu <vioreldpopescu at gmail.com>
> Cc: biomod-commits at r-forge.wu-wien.ac.at
> Subject: Re: [Biomod-commits] Comparison among variable importance
> Message-ID: <BAC3D53E-CA4A-44A7-BFD1-C34BA4C99874 at ujf-grenoble.fr>
> Content-Type: text/plain; charset=iso-8859-1
>
> Hi there all,
>
> This is, indeed, a tricky question.
> The way we perform variable importance, as do RandomForest, is really
> without unit. As Viorel mentioned, this is 1 minus the mean of correlations
> (depending on the number of variable importance runs). This is done
> independently (as Maxent is also doing as far as I understood). I do not
> think there is a sound statistical way of standardizing them. You may do as
> Damien suggest but I fear this is not really sound and should be
> interpreted without caution (but I am happy to be convinced otherwise).
> As Viorel suggested, box plots would do the same more or less the same but
> in a more sound way to me.
>
> Best,
> Wilfried
>
>
>
>
>
> Le 16 mai 2013 ? 16:44, Viorel Popescu <vioreldpopescu at gmail.com> a ?crit
> :
>
> > Hi everyone, I had a similar question a few weeks ago, and from my
> > understanding the varImportance scores are drawn from correlation
> > coefficients between different models (with and without the respective
> > variable permuted), namely, 1 minus the correlation (r). Hence, they
> cannot
> > add up to 1, the way variable importance is performed in MaxEnt. In that
> > case, standardizing the scores would not a be very good way to deal
> > with the varImportance. Am I missing something here?
> >
> > The way I dealt with them (with advice from Wilfried) was to show
> boxplots
> > of varImportance score for each variable across modeling techniques (and
> > taxa if you need to make such comparisons), and refer to median scores
> > (distribution of scores was far from normal).
> >
> > Cheers,
> > Viorel
> >
> >
> > On Thu, May 16, 2013 at 3:39 AM, Damien Georges
> > <damien.georges2 at gmail.com>wrote:
> >
> >> Dear Antonio,
> >>
> >> I guess you are right and you need to make a kind of standardisation of
> >> scores to make them all comparable. Make them sum to 1 is a good way to
> do
> >> it.
> >>
> >> Cheers,
> >> Damien.
> >>
> >>
> >>
> >> On 16/05/2013 12:09, Antonio Canepa wrote:
> >>
> >>> Dear list,
> >>> I would like to know if you can directly compare among different models
> >>> the
> >>> variable importance of each variable selected?
> >>> My guess is not, because when I got the importance of each variable (
> >>> getModelsVarImport()), the sum of all of them for each model was not =
> 1.
> >>> I don't know if I'm doing something wrong or there is another way to
> >>> "standardise" the output from getModelsVarImport() and thus compare
> >>> throughout models ?
> >>>
> >>>
> >> ______________________________**_________________
> >> Biomod-commits mailing list
> >> Biomod-commits at lists.r-forge.**r-project.org<
> Biomod-commits at lists.r-forge.r-project.org>
> >> https://lists.r-forge.r-**project.org/cgi-bin/mailman/**
> >> listinfo/biomod-commits<
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
> >
> >>
> >
> >
> >
> > --
> > Viorel D. Popescu
> > David H. Smith Conservation Research Fellow
> > University of California - Santa Cruz &
> > Simon Fraser University, Biological Sciences
> > (604) 340 4228
> > https://sites.google.com/site/vioreldpopescu/
> > _______________________________________________
> > Biomod-commits mailing list
> > Biomod-commits at lists.r-forge.r-project.org
> >
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
> -----------------------------
> Dr. Wilfried Thuiller
> Laboratoire d'Ecologie Alpine, UMR CNRS 5553
> Universit? Joseph Fourier
> BP53, 38041 Grenoble cedex 9, France
> tel: +33 (0)4 76 51 44 97
> fax: +33 (0)4 76 51 42 79
>
> Email: wilfried.thuiller at ujf-grenoble.fr
> Personal website: http://www.will.chez-alice.fr
> Team website: http://www-leca.ujf-grenoble.fr/equipes/emabio.htm
>
> ERC Starting Grant TEEMBIO project:
> http://www.will.chez-alice.fr/Research.html
>
>
>
> ------------------------------
>
> Message: 5
> Date: Thu, 16 May 2013 22:07:36 +0200
> From: Antonio Canepa <ajcanepa at gmail.com>
> To: Wilfried Thuiller <wilfried.thuiller at ujf-grenoble.fr>
> Cc: biomod-commits at r-forge.wu-wien.ac.at
> Subject: Re: [Biomod-commits] Comparison among variable importance
> Message-ID:
> <CAN6VVbQOfCsVetfSaFSrOZCE=
> 71LFNNgNw_WjPxDtMKWH7s0_A at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi there!,
> Many thanks for your help.
> It's great that thsi list works so good.
>
> Cheers,
>
> Antonio
>
>
> On 16 May 2013 18:34, Wilfried Thuiller
> <wilfried.thuiller at ujf-grenoble.fr>wrote:
>
> > Hi there all,
> >
> > This is, indeed, a tricky question.
> > The way we perform variable importance, as do RandomForest, is really
> > without unit. As Viorel mentioned, this is 1 minus the mean of
> correlations
> > (depending on the number of variable importance runs). This is done
> > independently (as Maxent is also doing as far as I understood). I do not
> > think there is a sound statistical way of standardizing them. You may do
> as
> > Damien suggest but I fear this is not really sound and should be
> > interpreted without caution (but I am happy to be convinced otherwise).
> > As Viorel suggested, box plots would do the same more or less the same
> but
> > in a more sound way to me.
> >
> > Best,
> > Wilfried
> >
> >
> >
> >
> >
> > Le 16 mai 2013 ? 16:44, Viorel Popescu <vioreldpopescu at gmail.com> a
> ?crit
> > :
> >
> > > Hi everyone, I had a similar question a few weeks ago, and from my
> > > understanding the varImportance scores are drawn from correlation
> > > coefficients between different models (with and without the respective
> > > variable permuted), namely, 1 minus the correlation (r). Hence, they
> > cannot
> > > add up to 1, the way variable importance is performed in MaxEnt. In
> that
> > > case, standardizing the scores would not a be very good way to deal
> > > with the varImportance. Am I missing something here?
> > >
> > > The way I dealt with them (with advice from Wilfried) was to show
> > boxplots
> > > of varImportance score for each variable across modeling techniques
> (and
> > > taxa if you need to make such comparisons), and refer to median scores
> > > (distribution of scores was far from normal).
> > >
> > > Cheers,
> > > Viorel
> > >
> > >
> > > On Thu, May 16, 2013 at 3:39 AM, Damien Georges
> > > <damien.georges2 at gmail.com>wrote:
> > >
> > >> Dear Antonio,
> > >>
> > >> I guess you are right and you need to make a kind of standardisation
> of
> > >> scores to make them all comparable. Make them sum to 1 is a good way
> to
> > do
> > >> it.
> > >>
> > >> Cheers,
> > >> Damien.
> > >>
> > >>
> > >>
> > >> On 16/05/2013 12:09, Antonio Canepa wrote:
> > >>
> > >>> Dear list,
> > >>> I would like to know if you can directly compare among different
> models
> > >>> the
> > >>> variable importance of each variable selected?
> > >>> My guess is not, because when I got the importance of each variable (
> > >>> getModelsVarImport()), the sum of all of them for each model was not
> =
> > 1.
> > >>> I don't know if I'm doing something wrong or there is another way to
> > >>> "standardise" the output from getModelsVarImport() and thus compare
> > >>> throughout models ?
> > >>>
> > >>>
> > >> ______________________________**_________________
> > >> Biomod-commits mailing list
> > >> Biomod-commits at lists.r-forge.**r-project.org<
> > Biomod-commits at lists.r-forge.r-project.org>
> > >> https://lists.r-forge.r-**project.org/cgi-bin/mailman/**
> > >> listinfo/biomod-commits<
> >
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
> > >
> > >>
> > >
> > >
> > >
> > > --
> > > Viorel D. Popescu
> > > David H. Smith Conservation Research Fellow
> > > University of California - Santa Cruz &
> > > Simon Fraser University, Biological Sciences
> > > (604) 340 4228
> > > https://sites.google.com/site/vioreldpopescu/
> > > _______________________________________________
> > > Biomod-commits mailing list
> > > Biomod-commits at lists.r-forge.r-project.org
> > >
> >
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
> >
> > -----------------------------
> > Dr. Wilfried Thuiller
> > Laboratoire d'Ecologie Alpine, UMR CNRS 5553
> > Universit? Joseph Fourier
> > BP53, 38041 Grenoble cedex 9, France
> > tel: +33 (0)4 76 51 44 97
> > fax: +33 (0)4 76 51 42 79
> >
> > Email: wilfried.thuiller at ujf-grenoble.fr
> > Personal website: http://www.will.chez-alice.fr
> > Team website: http://www-leca.ujf-grenoble.fr/equipes/emabio.htm
> >
> > ERC Starting Grant TEEMBIO project:
> > http://www.will.chez-alice.fr/Research.html
> >
> > _______________________________________________
> > Biomod-commits mailing list
> > Biomod-commits at lists.r-forge.r-project.org
> >
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
> >
>
>
>
> --
> Antonio Canepa Oneto
> PhD student
> Dept. Biologia Marina i Oceanografia
> Institut de Ciencies Del Mar CMIMA, CSIC
> Ps. Mar?tim de la Barceloneta,37-49, c.p. 08003
> BARCELONA, Catalunya,Spain
> Phone: +34 932 30 9500 (ext. 6067)
> Mobile: +34 628 320 778
> e-mail: canepa at icm.csic.es
> ajcanepa at gmail.com
>
>
> ------------------------------
>
> _______________________________________________
> Biomod-commits mailing list
> Biomod-commits at lists.r-forge.r-project.org
> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/biomod-commits
>
> End of Biomod-commits Digest, Vol 49, Issue 6
> *********************************************
>
More information about the Biomod-commits
mailing list