Table 1.
Modified GRADE approach for evaluating the quality of evidence of meta-analysis data subgroups
| Criteria | GRADE Points | Explanation |
|---|---|---|
| Downgrading criteria | ||
| 1. Individual study risk-of-bias rating and reporting limitations | One point deducted for each criterion where conditions are met. Sensitivity analysis considered to have appreciable impact if range of values changed estimates by >20 % or changed significance of overall effect. | |
| a) >50 % of trials had an unclear or high overall risk-of-bias rating | a) = −1 b) = −1 | |
| b) Key assumptions/imputed values due to reporting limitations had appreciable impact on results in sensitivity analysis | ||
| 2. Inconsistency of direction and heterogeneity of findings among studies | Heterogeneity in the results was measured by I 2. Consistency considered when the individual study estimates in the meta-analysis all show the same direction of effect. | |
| a) Consistent direction of effect, but significant heterogeneity | a) = −1 b) = −2 | |
| b) Inconsistent direction of effect and significant heterogeneity | ||
| 3. Imprecision of effect estimates | a) = −1 | Power calculations conducted assuming α = 0.05 and β = 0.2. For continuous outcomes, power calculated for a difference in means of 0.5 using a range of representative standard deviations from the meta-analysis subgroup. For dichotomous outcomes, power calculated using a relative risk reduction of 30 % and median control group risk from the meta-analysis subgroup. |
| a) The total number of participants in the meta-analysis subgroup is less than that required by a conventional sample size calculation for a single adequately powered controlled trial | ||
| 4. Indirectness of individual study parameter as representative of target parameter | Indirectness indicates studies did not directly measure the target parameter of interest to the review question (e.g. food safety outcomes only reported as part of a combined score/scale with other constructs such as nutrition). | |
| a) >50 % of trials indirectly measure the intervention, population, comparison, or outcome | a) = −1 b) = −2 | |
| a) >50 % of trials measure two or more of the above parameters indirectly | ||
| 5. Publication bias | a) = −1 | This criterion can only be evaluated if publication bias assessment is possible based on the nature of the data (i.e. ≥10 studies, non-significant heterogeneity, and at least some of the studies have significant results). |
| a) Detected or suspected in data subset | ||
| Upgrading criteria | ||
| 1. Large magnitude of effect | a) = +1 | Large effect considered at least a 2-fold reduction in risk. |
| a) Large effect in the absence of plausible confounders and major threats to validity | ||
| 2. Results may have been underestimated due to the study design (e.g. population sampled) | a) = +1 | E.g. intervention was tested only on individuals with prior food safety knowledge/training, and it is likely that a stronger effect would have been found if the intervention was tested in the general consumer population. |
| a) Criterion present | ||
| 3. Dose-response gradient | a) = +1 | Meta-regression dose variable represents >1 training course/session or multifaceted messaging interventions vs. a single course/session or provision of messaging materials through a single medium or exposure type. |
| a) >50 % of trials identified a dose-response relationship OR dose identified as significant in meta-regression. | ||