Table 2.
Example content and steps for meta-reviews, synthesized (a) narratively and (b) quantitatively.
| a. Narrative Content Analysis for Meta-Reviews (generated from example provided by Protogerou & Johnson, 2014). | ||
|---|---|---|
| Review aim: Identify factors underlying success of behavioral HIV-prevention interventions for adolescents | ||
| (1) Preparation | • Select appropriate/representative unit of analysis from the ‘universe’ • Data immersion |
• Literature search and selection: Identify and retrieve reviews of HIV-prevention interventions for adolescents; involves a precise search, clear inclusion criteria, clear screening process • Eligible study designs: meta-analyses and qualitative systematic reviews that looked at intervention efficacy and intervention features • Read included reviews in-depth, multiple times |
| (2) Organization | • Open coding (iterative process) • Creating categories • Abstraction |
•Writing notes and headings in margins: identify intervention features of interest • Group intervention features into distinct categories • Detail labels and definitions for groupings so that features hang together: • “behavior change techniques” included abstinence messages, emotion management training, and communication/negotiation skills training • “recipient features” included vulnerable samples, ethnic/race relevant, and gender relevant |
| (3) Reporting | • Report findings in a way that makes sense of the literature for the reader | • Categorization matrix that includes names of groupings, relevant elements in each group, and how often features occurred in reports as well as their co-occurrence with other features. |
| b. Example “Semi-quantitative” Analysis for Meta-Reviews, Their Aims, and Analytic Steps. | ||
| Citation(s) | Aim | Analytic steps |
| (A) Chow & Ekholm (2018). (B) Polanin, Tanner-Smith, & Hennessy (2016) |
Estimate extent of publication bias | 1. Aggregate published studies to form one pooled effect size 2. Aggregate unpublished studies to form one pooled effect size 3. Calculate Z-statistic for the difference between published and unpublished studies 4. Meta-analysis of Z-statistic using method to account for dependencies |
| (C) Ebrahim, Bance, Athale, Malachowski, & Ioannidis (2016) | Estimate bias due to industry involvement bias | 1. Extract data on funding sources and other key characteristics 2. Calculate proportions of reviews funded by different sources 3. Use exact tests to evaluate whether review conclusions differ by funding source 4. Report ORs, 95% CIs, p-values |
| (D) Johnson, Low, & MacDonald (2015) | Examine analytic practices of assessing and evaluating whether the quality of primary studies affects the results meta-analyses generate. | 1. Extract data on primary study quality and other key characteristics 2. Calculate descriptive summaries 3. Calculate correlations |