Skip to main content
. 2021 May 5;2021(5):CD013491. doi: 10.1002/14651858.CD013491.pub2

3. Detailed risk of bias and applicability assessment.

  Study
Backs‐Dermott 2010 Berlanga 1999 Johansson 2015 Judd 2016 Klein 2018 Pintor 2009 Ruhe 2019 Van Loo 2015 Van Loo 2018 Van Loo 2020 Wang 2014
Type of study Development Development Development Development Development Validation Development Development Development Validation Development Validation Development Validation
Domain 1: Participants
A. Risk of bias
1.1. Appropriate data sources? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
1.2. Appropriate inclusions and exclusion? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
Risk of bias Low Low Low Low Low Low Low Low Low Low Low Low Low Low
B. Applicability
Concern about applicability Low Unclear Low Low Low Low Low Low Low Low Low Low Low Low
Domain 2: Predictors
A. Risk of bias
2.1. Defined and assessed in similar way for all participants? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
2.2. Assessments made without knowledge of outcome? Probably yes Probably yes Yes Yes Yes Yes No information Yes Probably yes Probably yes No information No information Probably yes Probably yes
2.3. All available at time of model’s intended use? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes
Risk of bias Low Low Low Low Low Low Unclear Low Low Low Unclear High Low Low
B. Applicability
Concern about applicability Low Low Low Low Low Low Low Low Low Low Low Low Low Low
Domain 3: Outcome
A. Risk of bias
3.1. Determined appropriately? Yes Yes Yes Probably yes Yes Yes No information Yes Yes Yes Yes Yes Probably yes Probably yes
3.2. Pre‐specified or standard definition? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
3.3. Predictors excluded from outcome definition? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
3.4. Defined and determined similar for all participants? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
3.5. Determined without knowledge of predictors? No information No information No information Probably yes Yes Yes No information No information No information No information No information No information No information No information
3.6. Appropriate time interval between predictor assessment and outcome determination? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
Risk of bias Unclear Unclear Unclear Low Low Low Unclear Unclear Unclear Unclear Unclear Unclear Unclear Unclear
B. Applicability
Concern about applicability Low Low Low Low Low Low Low Low Low Low Low Low Low Low
Domain 4: Analysis
4.1. Reasonable number of participants with outcome? Probably yes No No No Probably yes Yes No No No No No Probably yes No information Yes
4.2. Predictors handled appropriately? Yes Probably yes Yes No Probably yes Probably yes No Probably no No No Probably yes Probably yes No No
4.3. All enrolled participants included in analysis? No No Yes No Yes Yes Yes No Probably yes Probably yes No Yes Yes Yes
4.4. Missing data handled appropriately? No information No information No information Yes Yes Yes No information No No No Yes Yes Probably no Probably no
4.5. Univariable analysis avoided? No No No No Yes NA No Yes Yes NA Yes NA No NA
4.6. Complexities in data accounted for? Probably yes Probably yes Probably yes Yes Yes Yes Probably yes Yes Probably yes Probably yes Yes Probably yes Probably yes Probably yes
4.7. Relevant performance measures? No No No No Yes Yes No No No No No No No No
4.8. Overfitting and optimism accounted for? No No No No Yes NA No No Yes NA Yes NA Yes NA
4.9. Final model corresponds to multivariable analysis? No information No information Probably yes No information Yes NA No information No information Probably no NA Probably yes NA No information NA
Risk of bias High High High High Low Low High High High High High High High High
Overall assessment of risk of bias High High High High Low Low High High High High High High High  High
Overall concern for applicability Low Unclear Low Low Low Low Low Low Low Low Low Low Low Low

Validation refers to external validation