[Basta-users] BaSTA model and covariate selection: DIC, K-L, and/of confidence intervals

[Diederik Strubbe]

Dear BaSTA people,

Thanks to your help we are making progress concerning our analysis of
post-fledging survival of a tropical passerine. We have now identified a
‘best model’, using MultiBaSTA.R and based on the DIC values. The best
model is a Weibull-Makeham, characterized by a DIC value of 351.6 (our
'base-model').

We are now exploring the influence of covariates, but stumble upon
something we cannot immediately understand. When we perform univariate
tests (i.e. base model + one covariate), we get the follow results:

Covariate 1: developmental age:
DIC: 336.7
Estimate: -0.117
StdErr:    0.0918
Lower95%CI: -0.295
Upper95%CI: 0.0628

Covariate 2: body condition:
DIC: 337.2
Estimate: 0.0076
StdErr:    0.128
Lower95%CI: -0.246
Upper95%CI: 0.250

Covariate 3: group size:
DIC: 362.8
Estimate: -0.211
StdErr:    0.084
Lower95%CI: -0.394
Upper95%CI: -0.063


According to the DIC criterion, Covariate1 and Covariate2 results in a
better model fit (DIC values about 15 points lower than the
Weibull-Makeham model without covariates). Covariate3 results in a worse
fit (DIC values about 10 points higher).

However, when looking at the estimates and standard errors (and
confidence intervals), Covariate3 seems to be best supported: estimate >
standard error, 0 not included in confidence interval. In contrast,
estimate2 has a low (ie good) DIC value, but its estimate is much
smaller than the standard deviation, and the confidence intervals surely
includes 0. This is counterintuitive.

After reading some recent papers using BaSTA, I think I come to the
following conclusion about model and variable selection

-    DIC is used to select between different survival models (e.g.
through MultiBaSTA). Once the ‘best’ survival model is selected,
covariate selection proceeds through:
-    Categorical variables: Kullback-Leibler distances (using 0.65 as
approximat threshold)
-    Continuous variables: assess mean and standard deviation, and
whether 0 is included in the 95%confidence interval.

Following this logical, in the example above, I’d conclude that survival
for our tropical birds is influenced by group size, that there is a weak
trend for developmental age and no relationship with body condition.

I’d appreciate any thoughts or comments about this interpretation of
BaSTA results!
Many thanks in advance,

Diederik


PS: I should note we found that DIC values can differ somewhat between
model run (ie each time an identical model was run, DIC values are not
identical). I notice that such differences are to be expected
(Spiegelhalter et al 2002, p607). Therefore, all values reported above
are averaged obtained from 100 model runs.

-- 
Dr.Diederik Strubbe
Evolutionary Ecology Group
Department of Biology
University of Antwerp
Middelheimcampus GV310
Groenenborgerlaan 171
2020 Antwerpen, Belgium
office: +32 3 265 34 69
mobile phone: +32 477445568
skype user name: lakrinn