. 2020 Sep 14;128(9):095002. doi: 10.1289/EHP6980

Table 1.

Comparing risk of bias tools.

RoB within individual studies	Study name
RoB within individual studies	ROBINS-I^a	Newcastle Ottawa scale^b	Morgan (GRADE)^c	Navigation guide^d	OHAT^e
RCT/target experiment as ideal study design	Yes	No	Yes	No	No
Consider direction or magnitude of bias, and importance for effect estimate	Optional, but not formally incorporated into tool	No	Optional^f	No^g	Optional, but not formally incorporated into tool
Assign highest domain risk of bias to entire study	Yes	No (but commonly done when used by summing stars/scores across domains)	Yes	No study-level bias summary	No, but used to assign to tiers in study synthesis
Consider statistical methodology as a separate domain	No	No	No	No	Optional
Evidence synthesis
Rank observational studies as inherently suffering from bias	Not applicable (no formal presentation of evidence synthesis)	Not applicable (no formal presentation of evidence synthesis)	Yes, indirectly because of RCT comparison, but under development	Yes, start at moderate certainty	Yes, start at low to moderate certainty
Possibly reject some studies based on bias	Not applicable (no formal presentation of evidence of synthesis)	Not applicable (no formal presentation of evidence of synthesis)	Yes, although may be allowed in sensitivity analysis	Yes, although may be included in sensitivity analysis	Yes, although may be included in sensitivity analyses

Note: Tools included in this table are risk of bias tools for individual studies with an algorithm-based component. GRADE, Grading of Recommendations Assessment, Development and Evaluation; OHAT, Office of Health Assessment and Translation; RCT, randomized controlled trial; RoB, risk of bias.

^{^a}

Sterne et al. 2016.

^{^b}

http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.

^{^c}

Morgan et al. 2019.

^{^d}

Woodruff and Sutton 2014. The risk of bias tool used in Navigation Guide comes from a combination of methods described by Viswanathan et al. (2008) and Higgins and Green (2011).

^{^e}

NTP 2019.

^{^f}

Direction of bias considered, but not magnitude or eventual impact on effect estimate.

^{^g}

Not mentioned in five published case studies (https://prhe.ucsf.edu/navigation-guide), nor in original paper by Woodruff and Sutton 2014.