Table 2.
Summary of bias assessment and applicability as per PROBAST tool mainly and adding CHARMS input where necessary
| Study ID | Participants | Predictors | Outcome | Sample size and participant flow | Model development and evaluation | External validation | Overall bias High Moderate Low |
Applicability High concern Moderate concern Low concern |
Comments for bias |
|---|---|---|---|---|---|---|---|---|---|
| UKPDS OM1 [17] 2004 | L | L | L | H | H | Yes | H | H |
Low EPV Did not report relevant performance measuresa |
| J. Leal et al. [18] 2013 | L | L | L | H | L | NA | H | H | Low EPV |
| UKPDS 68 update [19] 2015 | L | L | L | L | L | No | M | H | High number of missing values (up to 57%) but dealt with multiple imputations. No EV after update |
| UKPDS OM2 [22] 2013 | L | L | L | H | H | Yes | H | H | Missing data dealt with LOCF, no multiple imputations |
| Validation of the UKPDS 82 [23] 2015 | L | L | L | H | L | NA | H | H | No. of outcomes and sample size not reported |
| JJ risk engine [24] 2013 | L | H | L | L | L | No | H | M |
Categorised continuous predictor (BMI) lacks external validation |
| Soedamah-Muthu et al. [20] 2014 | L | L | L | H | L | Yes | H | H | Very low EPV and fewer number of outcomes |
| Lagani et al. [27] 2015 | L | L | L | H | L | Yes | H | H | Low EPV |
| Harris et al. [35] 2013 | L | H | L | L | H | No | H | L |
Did not report relevant performance measuresa Lacks external validation |
| Hippisley-Cox et al. [28] 2015 | L | L | L | L | L | Yes | Mb | H | High number of missing values, but dealt with multiple imputations |
| Icelandic model [29] 2011 | L | L | L | L | L | Yes | Lb | L | |
| Van der Heijden [25] 2014 | L | L | L | H | L | NA | H | L | Low number of outcomes (<100) |
| Soto-Pedre et al. [30] 2015 | L | L | L | H | L | NA | H | L | Low number of outcomes (<100) |
| Holtzer-Goor [26] 2015 | L | L | L | H | L | NA | H | L | Low number of outcomes (<100) |
| Lund et al. [8] 2016 | L | L | L | L | L | NA | Lb | L | |
| H. Ng Keunen et al. [21] 2016 | L | L | L | H | L | NA | H | L | Low number of outcomes (<100) |
| Danish model [31] 2012 | L | L | L | L | H | No | H | L |
Did not report relevant performance measuresa No validation |
| Stratton et al. [32] 2013 | L | L | L | L | H | Yes | H | L | Did not report relevant performance measuresa |
| Aldington et al. [33] 2014 | L | L | L | L | L | NA | Unclearb | L | Lack of information as meeting abstract only |
| Stratton Aldington et al. [34] 2014 | L | L | L | L | L | No | Unclearb | L |
Lack of information as meeting abstract only Lacks external validation |
| Scanlon et al. [36] 2015 | L | L | L | H | L | Yes | Mb | L | High amount of missing information and did CCA |
| ISDR [16] 2017 | L | L | L | L | L | No | Mb | L | Lacks external validation |
EPV events per candidate variables, EV external validation, LOCF last observation carried forward, NA not applicable, CCA complete case analysis, L low, M moderate, H high
aCalibration, discrimination and/or classification measures
bSee text