Table 3.
Main methodological differences between the GRADE approach [54], and approaches taken by the 3rd World Cancer Research Fund/ American Institute for Cancer Research Expert report [8], and the USDA Dietary Guidelines for Americans 2015–2020 [7] to rate the certainty of evidence
| Macro-level | Micro-level | GRADE | 3rd WCRF/AICR Reporta | USDA DGA 2015–2020b | |||
|---|---|---|---|---|---|---|---|
| Y/N/U | Explanation | Y/N/U | Explanation | Y/N/U | Explanation | ||
| A priori assumption | Body of evidence (RCTs and cohort studies) begins as high certainty | N | Using NOS | Y | N/A | Y | N/A |
| Y | Using ROBINS-I | ||||||
| Certainty criteria | Risk of bias (aka study quality) | Y |
Cochrane RoB tool for RCTs; NOS or ROBINS-I for NRS |
U | No tool; some criteria reported: confounding, measurement error and selection bias (but assessment unclear) | Y | Tool adapted from Cochrane: selection bias, performance bias, detection bias, attrition bias |
| Inconsistency | Y | Similarity of point estimates, extent of overlap of 95% CI, statistical criteria including tests of heterogeneity, and I2 | Y | I2 | U | Description vague: e.g., consistent in direction and size of effect or degree of association and statistical significance with very minor exceptions | |
| Indirectness | Y | According to PICO criteria | N | N/A | Y | According to PICO criteria | |
| Imprecision | Y | Examination of 95% CI, OIS | N | N/A | U | Large sample size | |
| Publication bias | Y | Funnel plot | N | N/A | N | N/A | |
| Large effect | Y |
RR: > 2 or < 0.5 (large effect) RR: > 5 or < 0.2 (very large effect) |
Y | RR: > 2 (large effect) | U | Description vague: e.g., clinically meaningful effect size | |
| Dose–response | Y | Linear, non-linear | Y | Linear, non-linear | N | N/A | |
| Plausible residual confounding | Y | Plausible confounders would decrease an effect, or would create a spurious effect when results suggest no effect | N | N/A | N | N/A | |
| Transparent Summary | Summary of Findings Table | Y | Certainty rating (and criteria) for each outcome and the estimate of effect | N | N/A | N | N/A |
| Overall rating | Certainty of evidence | High, Moderate, Low, Very low | Convincing, Probable, Limited-suggestive, Limited-no conclusion, Substantial effect on risk unlikely | Strong, Moderate, Limited, Grade not assignable | |||
95% CI 95% confidence interval, I2 inconsistency, N no, N/A not applicable, NOS Newcastle Ottawa Scale, OIS optimal information size, PICO population, intervention, comparison, outcome, ROBINS-I risk of bias in non-randomized studies of interventions, RR risk ratio, U unclear; Y: yes
aWorld Cancer Research Fund/American Institute for Cancer Research report is the most comprehensive global research project on diet, and physical activity and cancer risk or survival [8]
bThe United States Department of Agriculture’s Dietary Guidelines for Americans 2015–2020 are the most comprehensive guidance for healthy eating in the US and worldwide [7]