With vs. without a mediation hypothesis |
|
|
|
TE power False-positive psychology |
|
Statistical significance of the TE is irrelevant for testing mediation |
A mediation hypothesis is required before testing for indirect effects The test of IE often has greater power than the test of the TE1, and requiring TE1 be significant ignores this, as well as the risk of opposing direct and indirect |
The basis of both NHST and theory-driven research necessitate that tests are done carefully, and statistical significance is not “fished” for |
|
|
The TE should be significant before testing mediation |
The basis of both NHST and theory-driven research necessitate that tests are done carefully, and statistical significance is not “fished” for by moving to a IE test after non-significance of TE1 test |
Does not apply to exploratory research conducted in a manner that is in keeping with best practices |
|
Specific Effects vs. Global Model |
|
|
|
SEM vs. regression Network models |
|
Effects perspective focuses on individual parameters, with inferences based on their presence and/or strength |
Often it is only necessary to show that an effect exists, generally in support of a given theoretical framework |
Model-constraints that may bias parameters are avoided, but runs the risk of overfitting and poor replication rates due to larger standard errors |
|
|
Statistical models are the focus, with mediation effect(s) a subset of the global set of relationships |
Effects do not occur in isolation, and are instead a part of a larger set of causes, effects, and boundary conditions |
Higher precision and better replication if model assumptions hold, but biased parameter estimates if they do not. Interpretation of the effects is also conditional on the global model used |
|
Significance testing vs. Effect sizes |
|
|
|
Theory building vs. practical application |
|
Null hypothesis significance testing (NHST) is used as the criteria to evaluate an effect |
Priority is to distinguish an effect from noise |
Ignores gradation of effects and does not address proportion of outcome variance explained |
|
|
Effect sizes and confidence are used to evaluate an effect |
Establishing presence is insufficient because uncertainty and magnitude affect both replication and relevance of a given effect |
Meaningful effects may be ignored because they seem too small to matter in practice, but are nonetheless important when aggregated across many people or for theory building |
|
Direct vs. Indirect Effects |
|
|
|
Appropriateness of indirect effects |
|
A directness perspective focuses establishing direct causes |
For some problems and situations one may want to know direct causes, intermediate steps leave room for interference |
Has the advantage that one does not need to count on uncertain intermediate steps, but potentially ignores suppression effects due to other variables |
|
|
An indirectness perspective focuses on understanding the intermediate steps between IV and DV, between a cause and its effect |
For almost any given cause and effect encountered there are intermediate effects, and explaining and understanding these effects provides more information about the original cause and effect mechanisms which may themselves be more relevant to the outcome |
Provides a fuller picture of the relationship between X and Y, but at much greater risk of making incorrect inferences and explanations |
|
Hypothesized vs. Alternative explanations |
|
|
|
False-positive psychology Parameter sensitivity |
|
Focus is on confirmatory evidence and supporting a mediation hypothesis |
Past research should guide future research, increasing speed and ideally efficiency of research |
Restricted explanation search that runs the risk of attempting to support previous findings and neglecting more accurate alternative explanations |
|
|
Focus is on testing alternative hypotheses that conflict with a specific mediation hypothesis |
Evidence in support of a mediation claim is stronger if it can be shown that plausible alternative explanations do not hold |
Infinite number of alternative explanations, both reasonable and unreasonable, and often no clear stopping rule for what constitutes an adequate search |
|