Table 2.
Methodological approaches to consider in the synthesis of heterogeneous data
| Problem | Possible methodological solution | Selected key caveats |
|---|---|---|
| High statistical heterogeneity* | Random effects | Does not explain heterogeneity, small study effects, limited data |
| Meta-regression | Choice of variables, ecological fallacy, limited data | |
| Bayesian meta-analysis | Prior specification | |
| Bayesian meta-regression | Similar to meta-regression and Bayesian meta-analysis | |
| Meta-analysis of individual level data | Unavailable individual level information | |
| Different interventions compared | Merge interventions in same class | Unrecognised heterogeneity |
| Network meta-analysis | Inconsistency in direct versus indirect comparisons | |
| Different metrics of same outcome | Conversion formulas | Difficulties in clinical interpretation |
| Different outcomes, same construct | Standardised effects | Difficulties in clinical interpretation |
| Different outcomes | Meta-analysis of multiple outcomes | Specification of correlations |
| Observational data | Generalised evidence synthesis | Spurious precision, confounding, selective reporting |
| Cluster randomised trials | Account for clustering correlation | Unavailable sufficient information |
| Crossover trials | Account for period or carry-over effect | Unavailable sufficient information |
| Other study design issues | Same as for high statistical heterogeneity | As for high statistical heterogeneity above |
| Different participants or settings | Same as for high statistical heterogeneity | As for high statistical heterogeneity above |
| Many counts per participant | Meta-analysis of multiple period or follow-up | Unavailable sufficient information |
| Limited data | Standard meta-analysis methods | Caution needed as for any meta-analysis |
Popular software such as RevMan can accommodate only random effects calculations, while Comprehensive Meta-Analysis also accommodates simple meta-regressions. Bayesian models and models of multiple treatments or outcomes can be run in WinBugs. Most models can also be run in STATA or R.
*The approach used for high statistical heterogeneity may also be applicable to situations where clinical heterogeneity is considered high because of differences in interventions, metrics, outcomes, designs, participants, or settings.