Table 2.
Quality appraisal and risk of bias of observational cohort and cross-sectional studies.
| NIH Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies | Research Question | Study Population | Participant Rate at Least 50% from Eligible | Participants from Same/Similar Population | Sample Size Justification | Exposure(s) Measured Prior to the Outcome(s) | Sufficient Time Frame | Examination of Different Levels of Exposure as Related to the Outcome | Definition and Validation of the Exposure Measures | Exposure(s) Assessed More than Once | Definition of Outcome Measures | Blinded Assessors | Loss to Follow-Up after Baseline 20% or Less | Statistical Measure and Adjustment of Key Confounding Variables | Overall Rating |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Acosta et al., 1998 [31] | Y | N | ? | Y | N | Y | Y | Y | ? | Y | Y | N | Y | Y | Good |
| Aldámiz-Echevarría et al., 2013 [33] | Y | N | ? | Y | N | Y | Y | Y | Y | Y | Y | N | ? | N | Fair |
| Aldámiz-Echevarría et al., 2014 [32] | Y | Y | ? | Y | N | Y | Y | N | Y | Y | Y | N | ? | N | Fair |
| Aldámiz-Echevarría et al., 2015 [34] | Y | N | ? | Y | N | Y | Y | N | Y | Y | Y | N | Y | N | Fair |
| Alm et al., 1986 [27] | Y | Y | ? | N | N | Y | Y | NA | ? | Y | Y | N | Y | N | Poor |
| Dobbelaere et al., 2003 [37] | Y | N | ? | Y | N | N | N | N | Y | N | Y | N | NA | N | Poor |
| Evans et al., 2017 [38] | Y | Y | ? | Y | N | Y | Y | N | Y | Y | Y | N | NA | N | Fair |
| Evans et al., 2018 [53] | Y | Y | ? | Y | N | Y | Y | Y | Y | Y | Y | N | Y | N | Good |
| Evans et al., 2019 [39] | Y | N | ? | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Fair |
| Evers et al., 2018 [28] | Y | Y | ? | Y | N | Y | Y | N | Y | Y | Y | N | Y | N | Fair |
| Green et al., 2019 [42] | Y | N | ? | Y | N | N | N | Y | Y | N | Y | N | NA | N | Poor |
| Hoeksma et al., 2005 [43] | Y | Y | ? | Y | N | Y | Y | N | Y | Y | Y | N | Y | Y | Good |
| Huemer et al., 2007 [44] | Y | N | ? | Y | N | Y | Y | N | Y | Y | Y | N | Y | N | Fair |
| Kindt et al., 1983 [45] | Y | Y | Y | Y | N | Y | Y | Y | N | Y | N | N | Y | N | Poor |
| MacDonald et al., 1996 [57] | Y | N | ? | ? | N | Y | Y | Y | N | Y | Y | N | Y | N | Fair |
| Pinto et al., 2019 [29] | Y | N | ? | Y | N | Y | Y | N | Y | Y | Y | N | Y | N | Fair |
| Ponzone et al., 2008 [47] | Y | N | ? | Y | N | Y | Y | Y | Y | Y | Y | N | Y | Y | Fair |
| Rocha et al., 2012 & 2013 [25,26] | Y | Y | ? | Y | N | N | N | N | Y | N | Y | N | NA | N | Poor |
| Rohde et al., 2014b [49] | Y | N | ? | N | N | Y | NA | Y | N | N | Y | N | NA | N | Poor |
| Rohde et al., 2015 [48] | Y | N | ? | Y | N | N | N | Y | Y | N | Y | N | ? | N | Poor |
| Schulpis et al., 2013 [50] | Y | N | ? | Y | N | Y | N | Y | Y | Y | Y | N | NA | Y | Fair |
| Stockler-Ipsiroglu [58] | Y | N | ? | Y | N | Y | Y | Y | Y | Y | Y | N | Y | N | Fair |
| Thiele et al., 2017 [51] | Y | Y | Y | Y | N | Y | Y | N | N | Y | Y | N | N | Y | Fair |
| van Spronsen et al., 2009 [55] | Y | Y | Y | Y | N | Y | Y | N | Y | Y | Y | N | Y | N | Fair |
| Wendel et al., 1990 [56] | Y | Y | Y | Y | N | Y | Y | N | Y | Y | N | N | N | N | Poor |
Each item was rated as low-risk (“yes” = Y, green), unclear (“cannot determine/not reported” = ?, yellow), or high-risk (“no” = N, red) for the type of bias. NA, not applicable.