Table 1.
Comparisons of assumptions and properties of massively univariate analysis and Bayesian multilevel modeling.
| Massively Univariate Analysis | Bayesian Multilevel Modeling | |
|---|---|---|
| Number of models | number of units plus correction for multiple comparisons | one |
| Sharing of information | each unit is independent | units are exchangeable and loosely regularized |
| Focus of error control | overall type I (i.e., FPR) | type S (sign) and type M (magnitude) |
| Strategy for multiplicity | FPR correction (control for inflated statistical evidence) | partial pooling (control for inflated effect sizes) |
| Effect uncertainty | epistemic (effect is intrinsic and fixed with uncertainty from measurement error, etc.) | aleatoric (effect has inherent variability) |
| Effect inferences | effect: locally unbiased with no calibration; uncertainty: uninterpretable at unit level and dichotomized at the clique level | effect: locally biased and globally calibrated; uncertainty: expressed via posterior distribution |
| Framing of hypotheses | P(data | H0): estimate the “surprise” of having the observed data under the null hypothesis H0 scenario | P(HR | data): find the evidence for research hypothesis HR given the observed data |
| Inference method | perform NHST with a binary decision based on an FPR-adjusted threshold | assess statistical evidence P(HR | data) through posterior distribution: highlight but no hide |
| Model efficiency | local (e.g., unbiasedness of each unit, statistical power) | global (cross-validations, posterior predictive checks) |