[Mediation-information] Moderated mediation in `mediation` package: A way around arbitrary moderator values?

[Kosuke Imai]

I would not rely on the arbitrary p-value threshold, which as you point out
is not a god practice.  You can plot these effects with confidence
intervals for various moderator values.

On Sun, Aug 27, 2017 at 9:41 AM Mark White <markhwhiteii at gmail.com> wrote:

> Hello,
>
> I am working on a moderated mediation model.
>
> I had originally fit it calculating Hayes's index of moderated mediation
> (i.e., the product of coefficients method, then bootstrapping it—I adapted
> it for R, here:
> https://github.com/markhwhiteii/processr/blob/master/R/model7.R).
>
> I got a review from a journal that wanted me to do a sensitivity analysis
> for this using the `mediation` package, and to use your method in general.
> I ran into a problem, though: The significance of my moderated mediation
> model *depends on the values I choose of the moderator. *
>
> *(Attached is some .R code with my data `dput()` for replicability.)*
>
> Consider the code:
>
> mod_m <- lm(ent ~ cond*angi, dat)
> mod_y <- lm(legit ~ cond*angi + ent, dat)
> m_out <- mediate(mod_m, mod_y, treat="cond", mediator="ent")
> modmed_out <- test.modmed(m_out, covariates.1=list(angi=lo_angi),
> covariates.2=list(angi=hi_angi))
>
> The significance of the difference between the two mediation effects
> depends on what values I chose for low and high values of the moderator (in
> this case, `angi`).
>
> Do you have more of a continuous test that I could perform? That is, the
> product of coefficients method is essentially the slope for the moderator
> predicting the indirect effect (
> http://www.tandfonline.com/doi/full/10.1080/00273171.2014.962683). If I
> choose +/- 1SD, then my p-value is .052; if I choose +/- 1.5SD, then it is
> .038.
>
> This seems like it is far too easy to p-hack and overly dependent on
> arbitrary values. My moderator is on a Likert scale, so there is no real
> meaningful values of it in a non-arbitrary sense.
>
> Is there any way to get a more general test of "does the ACME *depend* on
> the moderator in general"?
>
> Thank you for your time,
> Mark
>
-- 
Kosuke Imai Professor, Department of Politics Center for Statistics and
Machine Learning Princeton University http://imai.princeton.edu Sent from
my mobile phone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.r-forge.r-project.org/pipermail/mediation-information/attachments/20170902/fab630c2/attachment.html>