Table 5.

Evidence grading tools used by included systematic reviews

Tool used	Number of reviews using approach	Number of reviews that used published modified versions	Number of reviews that modified/ deviated from guidelines	Originally developed for	Initial grading based on study type	Main considerations	Reproductive/ children’s health- related considerations of directness, heterogeneity, confounding	Timing of exposure and outcome assessment considered?	Considerations of absence of effects
GRADE [12, 98, 99, 102, 103]	8 [71, 72, 77, 78, 80, 82–84]	2 [82, 83]	1 [71]	Assessing quality of evidence in clinical practice	Yes (observational studies start at low, or moderate in modified version by WHO)	Study design, risk of bias among primary studies, inconsistency, imprecision, indirectness, publication bias, magnitude, dose–response association, residual opposing confounding (see modified versions for additional considerations) [99, 100]	Evidence in adults vs. children mentioned under “indirectness” [67] Provided child-specific examples regarding confounding	Timing of outcome assessment (general) considered under “indirectness” [67]	No explicit guidance provided
Navigation Guide [64, 104]	4 [5, 74, 76, 79]	0	0	Systematic reviews in environmental health	Yes (observational studies start at moderate)	Study design, risk of bias among primary studies, inconsistency, imprecision, indirectness, publication bias, magnitude, dose–response association, residual opposing confounding)	Provided child-specific examples regarding confounding	Yes, both considered as part of rating quality and strength of evidence	Yes, in terms of publication bias and in terms of “strength of evidence (i.e., evidence for no effect)”
Office of Health Assessment and Translation (OHAT) [15, 66]	3 [73, 75, 81]	0	1 [81]	Literature-based risk evaluations of environmental substances	Presence of study-design features considered (Controlled exposures, exposure prior to outcome, individual outcome data, comparison groups used)	Study-design features, risk of bias, inconsistency, imprecision, indirectness, publication bias, magnitude of effect, dose–response, residual opposing confounding, consistency (across species, populations, study designs), other	For inconsistency, the lifestage at exposure and assessment considered	Timing of exposure and outcome mentioned (in general). For inconsistency and indirectness, exposure duration and timing relative to outcome considered	Yes, in translating quality to level of evidence: “evidence of no health effect”
Preamble for monographs by the International Agency for Research on Cancer (IARC) (2006) [100]	1 [68]	0	1 [68]	Identification and evaluation of potential carcinogens (etiology)	No, but limitations of different study designs are considered in relation to research question	Assignment based on study design, quantity, and quality of included studies, statistical power, and consistency between studies Preceded by considerations including exposure assessment methods, temporal effects of exposure, use of biomarkers, criteria for causality (based on Bradford Hill criteria) [105]. In the most recent version from 2019, this is replaced by “considerations for assessing the body of epidemiological evidence” [13, 21, 105]	No	Temporal effects and latency considered prior to evidence grading	Yes (assigned for several unbiased, consistent, precise study results based on exposures covering the full expected range in humans, with adequate length of follow-up available)
Centre for Evidence-Based Medicine (CEBM) guidelines (2009, 2011) [95, 96]	1 [70]	0	0	Grading quality of evidence for use in clinical decision-making	Yes (experimental studies are rated higher than observational ones)	Study design and quality, precision, consistency, directness, magnitude of effect	No	Length of follow-up considered (in general)	No
Scottish Intercollegiate Guidelines Network (SIGN) (2011) [97]	1 [69]	0	1 [69]	Guideline development in clinical care research	Yes (observational studies cannot score higher than “B”)	Previously determined level of evidence (based on study design, risk of bias, and likelihood that “relationship is causal”), consistency of studies, and applicability of the evidence to the target population	No	No	No
The Best Evidence Synthesis (BES) System [101]	1 [78]	0	0	Clinical management guidelines for acute lower back problems	No	Number, relevance, and quality of available studies	No	No	No

Abbreviations: BES Best Evidence Synthesis, CEBM Centre for Evidence Based-Medicine, GRADE Grading of Recommendations, Assessment, Development, and Evaluations, IARC International agency for research on cancer, OHAT Office of Health Assessment and Translation, SIGN Scottish Intercollegiate Guidelines Network