Table 1.

Summary of the Limitations of the Meta-analysis Review (2014) by Foxcroft et al.

Research questions and objectives

Obtaining overall MI effect sizes was the primary aim. Heterogeneous samples, settings, interventions, designs, and follow-ups were included and combined, without examining between-study heterogeneity in effect sizes. Potentially meaningful subgroups for MI were not explored in the presence of substantial statistical heterogeneity beyond chance.

Study identification, screening, and eligibility evaluation procedures

Search strategies were not comprehensive. The qualifications of reviewers (expertise and training background) were undisclosed. Screening was based on titles and abstracts but not full-text articles. Multiple studies reporting the same trial data were treated as if they were independent studies. Disagreements between reviewers and coders were undocumented.

Data abstraction and coding procedures

Data from eligible follow-ups from eligible studies were omitted. For studies with multiple follow-ups at 4 months and more, outcome data from the final follow-up were chosen over short-term follow-up. Inaccurate data extraction and entry: – Sample sizes were coded inconsistently and inaccurately; – No bias correction was done for small samples; – Data for intervention and control groups were switched; – Mean and standard deviation estimates were inaccurately coded; and – Follow-up time periods were coded incorrectly and data combined incorrectly.

Data analysis, data presentation, and interpretation

As a result of the flaws noted from the previous stages, effect size estimates and standard errors may not be accurate. Data extracted from original studies may be correct but still provide inaccurate data (see Marsden et al. [2006] in Figure 2). Complex data structures – multiple related interventions, subgroups, and outcomes – were unaccounted, raising concerns of nonindependence among estimates and inflated sample size.