Table 1.

Characteristics of model development and validation studies included in the systematic review

Study ID	Study dates	Participants DM-type source	Predictors (n) candidate final	Outcome follow-up (years)	Sample size (n)	Events (n)	Events per variable	Model performance /evaluation	Presentation
Models development studies with external validation
UKPDS OM1 2004 [17] (D, IV)	1977–1997	T2DM UKPDS trial	14 2	One eye blindness median 10.3y	3642	104	7	Not reported	Equation
Leal et al., 2013 [18] (EV of OM1)	1997–2007	T2DM PTM data	Same		4031	101		C-statistics (95% CI) 0.60 (0.55–0.65)	NA
UKPDS 68 update 2015 [19] (update, IV and EV of OM1)	2000–2009	T2DM (two risk groups) THIN database	14 12	Blindness low risk (LR) 5.1y Intermediate risk (IR) 6.4y	LR 54,169 IR 68,990	LR 176 IR 607	56	C-statistics LR: IV 0.59 and EV 0.57 IR: IV 0.59 and EV 0.58 Calibration plots	Risk groups (low and intermediate)
UKPDS OM2 2013 [22] (D, IV)	1997–2007	T2DM PTM data	24 7	One eye blindness median 17.6y	5102	271	11	Predicted curves in IV	Hazard ratios
UKPDS 82 2015 [23] (EV of OM2)	Not reported	T2DM	Same		Not reported	Not reported		MAPE (%) = 20.31 and R2 = 0.96 Calibration plots	NA
Soedamah-Muthu et al., 2014 [20] (D, IV) EV in the same study three cohorts	1989–1999 1998–1999 1994–2009 2000–2002	T1DM EURODIAB trial data EDC FinnDiane CACTI	15 5 Same	Blindness median 7.4y Median 8.1y Median 7.5y Median 7.3y	1973 554 2999 580	12 29 Not reported 5	<1 (0.8)	Harrell’s C-index: IV 0.74 C-statistics 0.79 0.82 0.73 Calibration plots	Equation score charts Kaplan–Meier plots for high, intermediate and low risk score groups
Lagani et al.,a 2015 [27] (D, IV) EV in the same study	1983–1993 2004–2014	T1DM DCCT, EDIC T1DM and T2DM Chorleywood, UK	51 5 Same	Retinopathy eventb Mean 6.5y	1441(IV) T1DM 36 T2DM 294	969(IV) T1DM 17 T2DM 70	Not applicable	Concordance index: 0.66 (p < 0.001) T1DM 0.72 (p = 0.002) T2DM 0.55 (p = 0.119)	Equation recalibration NA
Hippisley-Cox et al., 2015 [28] (D, IV) EV in the same study	1998–2014	T1, T2DM Q Research CPRD, UK	18 9 Same	Blindness in one or both eyes Total 15y	454,575 (D) 142,419 (IV) 206,050 (EV)	8063 2651 2845	448	Harrell’s C-index Women 0.73 (0.71–0.74) and men 0.75 (0.73–0.77) (IV) 0.73 (0.72–0.75) in both gender (EV) Calibration plots	Web calculator NA
Icelandic model 2011 [29] (D, EV)	Not reported	T1, T2DM Icelandic eye screening /Aarhus diabetic database	7 7	STR f/u not reported	5199	149	21	C-statistics (95% CI) 0.76 (0.74–0.78)  and Calibration graph	Equation nomogram mobile app
Five separate EV studies for Icelandic model*
*van der Heijden 2014 [25]	1998–2010	T2DM, Dutch	Same	STR         Mean 4.4y	3319	76		0.83 (0.74–0.92)	NA
*Soto-Pedre et al., 2015 [30]	Not reported	T1, T2DM, Spain	Same	STR      Median 1.1y	508	16		0.74 (0.62–0.85)	NA
*Holtzer-Goor, 2015 [26]	Not reported	T2DM, Dutch Hospital	Same	STR         Mean 3.3y	888	47		Not reported	NA
*Lund et al., 2016 [8]	2010–2012	T1, T2DM, DESP, UK	Same	STR          Total 2y	9687	531		T1DM 0.70 (0.67–0.73) and T2DM 0.80 (0.78–0.81)	NA
*Ng et al., 2016 [21]	Not reported	T1DM, Dutch Hospital	Same	STR         Mean 6.9y	200	22		0.75 (0.65–0.86)	NA
Stratton et al., 2013 [32] (D)	2005–2010	T1, T2DM DESP, UK	3 3	STR median 2.8y	14,554	803	268	Not reported	Kaplan–Meier plot
Aldington et al., 2014 [33] (EV)	Not reported	Two screening programmes, UK	Same	Total 4y	24,509 32,987	2328 3991		C-statistics 0.76 0.80	NA
Scanlon et al., 2015 [36] (D, IV) EV in the same study three programmes	2005–2011	T1, T2DM DESP, Gloucestershire(D)(IV) East Anglia South London Nottinghamshire	13 4 Same	STR total 5y Median 2.7y 3.8y 4.2y	7012(D) 5778(IV) 17,634 1223 1083	606(D) 490(IV) 845 94 81	47	C-statistics (95% CI) 0.79 (0.76–0.81) IV 0.78 (0.76–0.80) 0.84 (0.78–0.88) 0.82 (0.71–0.90)	Risk score NA

Models development studies with internal validation but no external validation
JJ risk engine 2013 [24] (D, IV)	Not reported	T2DM, Japan Trial data JDCS, and J-EDIT	16 5	Progression of retinopathy Mean 7.2y	1748	178	11	Harrell’s C-index (95% CI) 0.61 (0.52–0.70) Hosmer–Lemeshow p = 0.13	Equation web app
Harris et al., 2013 [35] (D, IV)	2001–2009	DM type not specified Claims database, US	14 14	Progression to PDR Median 1.7y	4617	307	22	Not reported	Equation risk score
Stratton et al., 2014 [34] (D, IV)	Not reported	T1, T2DM DESP, Gloucestershire (D)(IV)	6 5	STR Total 4y	6449(D) 5460 (IV)	555(D) 496(IV)	93	C-statistics (95% CI) 0.79 (0.78–0.81)	Not reported
ISDR 2017 [16] (D, IV)	2009–2014	T1, T2DM DESP, Liverpool and primary care data were combined	19 5	Referable DR Total 5y	11,806	388	20	Harrell’s concordance index: 0.69 at 24 months AUC (95% CI): 0.91 (0.87–0.94) Calibration curve	Equation

Models development studies with no clear validation (performance testing only)
Danish model 2012 [31] (D) Unclear if validated	1994–2007	T1, T2DM Arhus database, Denmark	7 7	Treatment requiring DR f/u not reported	5311 1372	559 208	80	Not reported	Equation

PTM post-trial monitoring data, DESP diabetes eye screening programme, T1DM type 1 diabetes mellitus, T2DM type 2 diabetes mellitus, D development, IV internal validation, EV external validation, U model update, STR sight-threatening retinopathy, PDR proliferative diabetic retinopathy, MAPE mean absolute percentage error, R² coefficient of determination/validation, THIN The Health Improvement Network, CPRD Clinical Practice Research Datalink, DCS Diabetes Care System West-Friesland, Sn sensitivity, Sp specificity, LR low risk, IR intermediate risk

^aFigures from DCCT trial and in electronic supplementary material (not given in the publication)

^bAny worsening in the retina condition that lasted at least 6 months