Table 2.

Summary of quality of evidence and strength of evidence evaluation criteria

Evaluation factors	Summary of criteria
Quality downgrading factors
Risk of bias	Study limitations – a substantial risk of bias across body of evidence
Indirectness	Evidence was not directly comparable to the question of interest (i.e. population, exposure, comparator, outcome)
Inconsistency	Widely different estimates of effect in similar populations (heterogeneity or variability in results)
Imprecision	Studies had few participants and few events (wide confidence intervals)
Publication bias	Studies missing from body of evidence, resulting in an over or underestimate of true effects from exposure
Quality upgrading factors
Large magnitude of effect	Upgraded if modelling suggested confounding alone unlikely to explain associations that were judged to be of large magnitude
Dose response	Upgraded if consistent relationship between dose and response in one or multiple studies, and/or dose response across studies
Confounding minimizes effect	Upgraded if consideration of all plausible residual confounders or biases would underestimate the effect or suggest a spurious effect when results show no effect
Strength considerations
Quality	Overall quality rating of the body of evidence (from above)
Effect estimate	Direction of the relationship seen between exposure and outcome
Confidence	Confidence in the effect estimate and likelihood that new studies would change the conclusion
Other	Any additional aspects of the data that may influence certainty