Abstract
Background
Many prognostic models of diabetic microvascular complications have been developed, but their performances still varies. Therefore, we conducted a systematic review and meta-analysis to summarise the performances of the existing models.
Methods
Prognostic models of diabetic microvascular complications were retrieved from PubMed and Scopus up to 31 December 2020. Studies were selected, if they developed or internally/externally validated models of any microvascular complication in type 2 diabetes (T2D).
Results
In total, 71 studies were eligible, of which 32, 30 and 18 studies initially developed prognostic model for diabetic retinopathy (DR), chronic kidney disease (CKD) and end stage renal disease (ESRD) with the number of derived equations of 84, 96 and 51, respectively. Most models were derived-phases, some were internal and external validations. Common predictors were age, sex, HbA1c, diabetic duration, SBP and BMI. Traditional statistical models (i.e. Cox and logit regression) were mostly applied, otherwise machine learning. In cohorts, the discriminative performance in derived-logit was pooled with C statistics of 0.82 (0.73‑0.92) for DR and 0.78 (0.74‑0.83) for CKD. Pooled Cox regression yielded 0.75 (0.74‑0.77), 0.78 (0.74‑0.82) and 0.87 (0.84‑0.89) for DR, CKD and ESRD, respectively. External validation performances were sufficiently pooled with 0.81 (0.78‑0.83), 0.75 (0.67‑0.84) and 0.87 (0.85‑0.88) for DR, CKD and ESRD, respectively.
Conclusions
Several prognostic models were developed, but less were externally validated. A few studies derived the models by using appropriate methods and were satisfactory reported. More external validations and impact analyses are required before applying these models in clinical practice.
Systematic review registration
PROSPERO CRD42018105287
Supplementary Information
The online version contains supplementary material available at 10.1186/s13643-021-01841-z.
Keywords: Meta-analysis, Microvascular complications, Prognostic model, Type 2 diabetes
Background
Type 2 diabetes (T2D) has increased rapidly over the past 30 years becoming worldwide public health problem with prevalence in adults of 463 million (9.3%) in 2019. It is estimated to be 700 million (10.9%) by 2045 [1], in which currently about 79% of people have diabetes living in low- and middle-income countries [1, 2]. Furthermore, diabetic progression due to its complications-increased disability, impaired quality of life and leading cause of premature death, which accounted for 11.3% of the global mortality [1, 3].
Two life threatening microvascular complications in T2D are diabetic retinopathy (DR) and diabetic nephropathy (DN). DN, known as chronic kidney disease (CKD), characterised by proteinuria and rapidly declined glomerular filtration rate (GFR) [2, 4], accounted for approximately 20‑40% of diabetic populations [5]. DR is the major cause of blindness [6] through fractional retinal detachment, preretinal or vitreous haemorrhage and central vision impairment, with the prevalence of 25% globally [7].
Diabetic microvascular complications commonly occurs in working ages [8], thus declining productivity, increasing tremendous social cost and high burden in healthcare [9, 10]. Therefore, early identification of high-risk patients to prevent occurrence of microvascular complications is very important. Many prognostic models have been developed (e.g. DR [11–18], CKD [19–25] and ESRD models [12, 26–29]) using various statistical methods. A lot of prognostic models were externally validated [12, 17, 19, 20, 22, 26, 29, 30], whilst other models were not [13, 15, 16, 21, 27, 28]. Nonetheless, the best prognostic model for each complication was still inconclusive. Hence, we conducted a systematic review to summarise all prognostic models for diabetic microvascular complications (including DR, CKD and ESRD) that are available and their performances in prediction of complications.
Methods
Protocol registration
This study was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [31] and in accordance with CHARMS checklist [32]. The review protocol was registered at PROSPERO (CRD42018105287).
Search strategy
Studies were identified from PubMed and Scopus up to 31 December 2020. Search terms were constructed based on patients, interventions and outcomes, see details in Additional files 1 and 2.
Study selection
Studies, published in any language, were eligible if they studied in adult T2D, developed or validated any multivariable prognostic models of microvascular complications in T2D with applying any traditional statistical modelling (e.g. logit or Cox regression etcetera) or machine learning (ML), and reported model performance. We also included the studies from reference list of relevant publications.
Data extraction
Data extractions were performed by one reviewer (SAS) and checked by OP. Extracted data were characteristics of study and patients (i.e. country, study design, settings, data source, sample size and number of events, ethnicity, age, percent male and diabetic duration), study phase (i.e. derivation or validation), statistical methods, predictors, missing data and outcomes (i.e. DR, CKD and ESRD). In addition, two related properties of model performances (calibration and discrimination) were also extracted.
Risk of bias assessment
Risk of bias assessment was assessed by using Prediction Model Risk of Bias Assessment Tool (PROBAST) [32]. Each item was rated as low, high or unclear. The overall validity was low and high risk if all domains were low risk, and at least one domain was high risk, respectively. Discrepancies were solved by consensus between the team.
Statistical analysis
Characteristics of each prognostic model and predictive performances (including calibration and discrimination) were described. Discrimination was assessed according to original included studies, in which C-statistic was mostly used. If studies reported C-statistic without variance, it was estimated using equations in the previous guidelines [32–34]. Calibration was assessed [35] using calibration plot, goodness-of-fit testing (i.e. Hosmer and Lemeshow χ2 test), calibration slopes or the observed/expected (O/E) ratio.
A meta-analysis was applied for pooling C statistics across studies stratified by study’s design/phase, statistical model and T2D complications. A random-effect model by DerSimonian-Laird [36, 37] was used if heterogeneity was present (p value < 0.10 or I2 > 25%); otherwise, a fixed-effect model was used. A heterogeneity was assessed by Cochrane Q test and I2 statistic. Publication bias in external validation was assessed using funnel-plot [38] and Egger’s test [39]. All statistical analyses were performed using STATA 16 [40]. A p value less than 0.05 was considered as statistically significant, except for heterogeneity which used 0.10.
Results
A total of 32/1009 and 44/3321 studies were eligible for DR and DN, respectively, see Table S1 (Figs. S1‑S2). Amongst them, 205 prognostic equations were derivative, some of them performed internal and external validations. Most studies reported C statistics, but only a few-portions reported calibrations (Table S2).
Risk of bias assessment
Risk of bias assessment of all included studies was presented in Table S3. Amongst 71 studies, about 86 to 95% of studies were determined as low risk of bias for study participants, selection of predictors and outcome measurement. About 23% and 40% of studies were rated as high risk for sample size, participant flow and statistical analysis, respectively. As a result, 35% of studies were overall low risk of bias (Fig. S3).
Diabetic retinopathy
Thirty-two [11–18, 24, 30, 41–62] studies were identified for predictions of DR including 1,120,278 diabetic patients with 128,129 (11.4%) events. Of which, 26 [11, 12, 14, 18, 24, 30, 41 50, 53, 62] and 6 [13, 15–17, 51, 52] studies applied traditional statistical modelling and ML, respectively. Twenty-eight [11–18, 24, 41, 42, 45, 47–62] studies derived 84 original prognostic equations with varied sample sizes of 18 to 254,896. Mean age was 44.6 to 66.6 years, percent male was 27.0 to 61.9, and diabetic duration varied from 1.4 to 15.8 years. Most studies were conducted in Europe and America regions with only 11 [17, 42, 43, 45, 47, 49, 52–55, 59] (34%) studies in Asia. Twenty-four [11–16, 18, 24, 30, 41–43, 45–48, 51, 52, 54, 56, 58–60, 62] and 8 [17, 44, 49, 50, 53, 55, 57, 61] studies were hospital and community-based settings respectively with confirmation of T2D diagnosis from medical records, laboratory tests or use of diabetic drugs. DR was mostly diagnosed by using fundus examination. Follow-up time ranged varied from 1.0 [56] to 20 [11] years with a median of 5 years. Only 5 [18, 24, 30, 50, 58] studies reported percent loss to follow-up which ranged from 2.4 to 31.3%. Eighteen [11–13, 15, 30, 41–45, 48, 50, 51, 54, 55, 57, 59, 61] studies used various methods for dealing with missing data, in which about a half of them used multiple imputations (Table S1). Four [45, 47, 54, 57] and 3 [13, 18, 49] studies provided simplified scoring system and presented nomograms, or else only used regression coefficients/odds ratio to calculate the score.
As for phase of prediction, 4 [12, 17, 41, 42], 2 [11, 45], 4 [30, 43, 44, 46], 15 [13, 15, 16, 18, 48-54, 56‑59] and 6 [14, 24, 55, 60, 62, 63] studies were respectively determined as derived-internal-external (D/I/E), derived-external (D/E), external (E), derived-internal (D/I) and only derived (D) phases (Table S2). Amongst 8 [12, 17, 30, 41–44, 46] external-validation studies, 5 [12, 17, 41, 42, 44] validated their own derived models in the same ethnicity (i.e. Asian [17, 42], mixed ethnicity [12, 41, 44]), except 3 [30, 43, 46] that validated other author’s models that were originally in Caucasians [11] and Asians [45, 47]. Most studies were cohorts/RCT’s, and their DR incidence varied from 1.5% [48] to 42.9% [56] whereas it was 14.2% [43] to 57.2% [54] in cross-sectional studies.
Seventeen [13, 14, 17, 42, 43, 45, 47, 49, 51–56, 58, 61, 64] and 9 [11, 12, 18, 24, 41, 44, 46, 59, 60] studies used logit and Cox whereas 6 [13, 15–17, 51, 52] applied MLs. Various predictors were considered (Fig. S4), in which the most commonly used were diabetic duration, age, HbA1c, SBP and BMI, which these were mainly included in the models as continuous predictors. A total number of included predictors in the conventional statistical models and MLs were not much different with a median of 8 (range 2‑37). Interestingly, few studies used image/signal analytic [56, 58] and genetic variables [42, 55, 62], which were incorporated with conventional clinical data. Two [56, 58] studies predicted specific DR site using multifocal electroretinogram incorporated with traditional clinical factors [56, 58].
Three [42, 55, 62] studies derived genetic risk score (GRS) based on different genetic polymorphisms (range 2‑76). Traditional prognostic factors (i.e. age, sex, diabetic duration, HbA1c and hypertension/SBP) were also retained in the model with GRS.
C statistic varied from 0.50 [13] to 0.95 [58], 0.52 [17] to 0.92 [58] and 0.59 [12] to 0.83 [30] for derived, internal and external validations. Those prognostic equations had been externally validated with moderate to good performance (Table S4). Discrimination performance of logit equations varied from moderate to high with the C statistics of 0.70 [45, 47, 49, 62] to 0.95 [58] and 0.63 [47] to 0.92 [58] in derived and internal validation; likewise, for support vector ML [17] in these corresponding phases of 0.83 and 0.81.
Pooled C statistics of the derived-logit models across cohorts [13, 51, 56–58, 61], cross-sectional [14, 17, 45, 47, 49, 52, 53] and case-control genetic [42, 55, 62] studies were 0.82 (0.73‑0.92; I2 = 99.47%), 0.77 (0.72‑0.82; I2 = 93.21%) and 0.74 (0.71‑0.77; I2 = 36.73%), respectively (Fig. 1). Fixed effect model was observed on pooled derived-Cox [11, 18, 24] models in cohort studies which yielded 0.75 (0.74‑0.77; I2 = 0.0%).
Fig. 1.
Forest plot of C statistics in derivative phase for DR
Pooled C statistics of logit equations across cohorts [13, 51, 56–58] and cross-sectional [17, 47, 49, 52–54] studies for internal validation were 0.83 (0.76‑0.90; I2 = 95.11%) and 0.74 (0.68‑0.81; I2 = 82.46%), respectively. Of those, the external validation for three [17, 45, 47] equations yielded 0.81 (0.78‑0.83; I2 = 8.05%) by cross-sectional studies of logit regression (Table S5). Funnel plot and Egger’s test (p > 0.293) suggested no publication bias by the absence of small study effects, no correlation between sample size and the magnitude of C statistics in external validation studies (Fig. S5).
Model’s calibration in derived phase [11, 12, 14, 18, 49, 54, 59, 61], internal [12, 18, 41, 49, 54, 59] and external [12, 30, 41, 44, 46] validations mostly demonstrated perfect O/E. Nine studies [14–17, 24, 52, 56, 58, 61] might have overfitted model as the ratio of an event per variable (EPV) numbers ranged from 1 [58] to 9 [24, 52], whereas the other 2 [12, 45] models might have underfitted with a ratio as high as 297 [12] to 403 [45].
Diabetic nephropathy
CKD
Thirty [12, 13, 19–25, 44, 59, 65–83] studies purely derived 96 equations including 244,934 diabetic patients with 44,023 (17.9%) events of CKD. CKD incidence ranged from 12.1 to 37.3% for 5 RCT’s [12, 59, 70, 74, 77], and 0.7% to 47.6% for 22 [12, 13, 20, 22–25, 65, 66, 68, 69, 71–73, 75, 76, 78–83] cohorts. Eleven studies [19, 22, 25, 59, 65–67, 69, 76, 78, 82] (36.7%) were conducted in Asians. Sixteen [13, 21, 23, 66–69, 71, 72, 75, 76, 78, 79, 81–83] and 16 [12, 19, 20, 22, 24, 25, 44, 59, 65, 69–71, 73, 74, 77, 80] studies diagnosed CKD based on eGFR and albuminuria, respectively. Median (range) follow-up time was 5.4 (1‑10) years and percent lost to follow up was 0.8% to 35.7%. Twelve [12, 13, 20, 22, 25, 44, 59, 65, 66, 69, 73, 80] (38.7%) studies reported methods for dealing with missing data, 5 [12, 22, 44, 59, 80] had used multiple imputations and only a few reported percent missing data [12, 13, 20, 44].
Five [12, 19, 23, 68, 71], 2 [20, 25], 2 [44, 69], 12 [13, 22, 59, 65–67, 70, 72–74, 78, 79] and 9 [21, 24, 75–77, 80–83] studies were D/I/E, D/E, E, D/I and only D-phases, respectively. Of 9 E-phases, eight [12, 19, 20, 23, 25, 44, 68, 71] studies validated their own models in different datasets, and one [69] validated others author’s model (i.e. QKIDNEY risk score), which developed in general populations. Half of the studies were validated in Asians [19, 20, 25, 43], and 9 [12, 20, 23, 25, 44, 68, 69, 71] used data from cohorts/RCTs. Their mean age ranged from 44.0 to 67.3 years, whereas the percent of male varied from 32.5 to 76.0 with a median follow-up time of 4.9 years.
Out of 28 studies, 96 derived models consisted of 79 traditional statistical models (i.e. logit (n = 16) [13, 19-21, 23, 25, 65‑68, 70, 72, 73, 78, 79, 82] and Cox (n = 11) [12, 22, 24, 59, 74–77, 80, 81, 83]), whereas 17 models (n = 4) [13, 70, 71, 73] performed various MLs algorithms (Table S2). Three [13, 65, 66] studies provided nomograms, whereas 2 [25, 74] studies simplified risk score.
Ninety-two derived models reported C statistics, with 55 (59.7%) internal and 19 (20.6%) external validations. Their discriminative performance varied from 0.50 [13] to 0.93 [21, 66], 0.50 [13] to 0.91 [73] and 0.57 [19] to 0.85 [44] in derived, internal and external phases, respectively (Table S2), which were explicitly described (Table S4). Common predictors were SBP, HbA1c, sex, diabetic duration and eGFR. Two [19, 67] studies combined genetic factors with clinical factors which yielded better discrimination of 0.78 (0.75‑0.81) relative to considered conventional models 0.75 (0.72‑0.78), see Table S6.
Out of twenty-eight [12, 13, 19–25, 59, 65–68, 70–83] derived studies, 14 [20, 22, 23, 59, 65–68, 72–74, 78–80] studies reported acceptable calibration model with O/E ratio ranged from 0.77 [79] to 1.11 [73]. Only seven [12, 23, 67, 73, 74, 78, 79] and six [12, 20, 22, 44, 68, 69] out of 21 validated-studies had O/E ratio of 0.93 [78] to 1.14 [12] and 0.97 [44] to 1.31 [12] in I and E phases (Table S2).
For cohorts, the pooled C statistics for logit (n = 11) [13, 20, 23, 25, 65, 66, 68, 72, 78, 79, 82] and Cox (n = 7) [22, 24, 75, 76, 80, 81, 83] in D phase were 0.78 (0.74‑0.83; I2 = 96.91%) and 0.78 (0.74‑0.82; I2 = 91.78%), respectively (Fig. 2). Cox regression in derived RCTs (n = 3) [59, 74, 77] yielded pooled C statistics of 0.73 (0.62‑0.84; I2 = 95.53%).
Fig. 2.
Forest plot of C statistics in derivative phase for CKD
Derived logit (n = 8) [13, 23, 65, 66, 68, 72, 78, 79] from cohorts were internally validated yielding the pooled C statistics of 0.79 (0.74‑0.83; I2 = 95.53%) which performed closely to the D phases, but poorer in externally validated [20, 23, 25, 68] with pooled C statistic of 0.75 (0.67‑0.84; I2 = 94.68%). Funnel plot and Egger’s test (p > 0.710) showed no publication bias by the absence of small study effects (Fig. S6). There is no correlation between studies for smaller cohorts with higher C statistics.
ESRD
Eighteen studies [12, 26–29, 44, 72, 74, 84–93] originally derived 46 models in 366,210 diabetic patients with the ESRD incidence of 57,294 (15.65%). Of them, 13 [26, 28, 29, 44, 72, 84–87, 89–92] and 5 [12, 27, 74, 88, 93] were cohorts and RCTs, respectively. A half of them were conducted in Asia [28, 29, 84, 85, 89, 90, 92] and the USA [12, 27, 44, 72, 86, 87, 91]. Thirteen [12, 26–29, 72, 74, 84, 88–91, 93] and 5 [44, 85–87, 92] studies were hospital-based and community-based settings respectively, where ESRD was mostly confirmed by dialysis [12, 26–28, 44, 72, 88–92]. Mean follow-up times ranged from 1.5 to 14 years. Only 9 studies (50%) reported methods for dealing with missing data, in which 5 [12, 27, 29, 44, 84] used multiple imputations. Two studies [74, 84] developed simplified risk score based on the Framingham Heart study [94].
Briefly, 1 [12], 2 [26, 29], 1 [44], 5 [28, 72, 74, 84, 85] and 9 [27, 86–93] studies showed D/I/E, D/E, E, D/I, and only D phases, respectively (Table S1). Three [12, 26, 44] studies have externally validated their own models within the same studies, whereas 1 [29] study validated other models’ studies [26, 74].
C statistics varied from 0.76 [89] to 0.97 [27, 85] in derivative phases of 17 studies (n = 51), 0.60 [12] to 0.96 [85] in internal validations (n = 11) and 0.54 [12] to 0.92 [26] in external validations (n = 13), see Table S2. Prognostic model for ESRD was mainly derived by Cox equation in 16 (95%) studies [12, 26‑29, 74, 84-93].
The pooled C statistics in cohorts using Cox were 0.87 (0.84‑0.89; I2 = 92.15%), 0.91 (0.86‑0.96; I2 = 94.86%) for derived and internal validations suggesting discrimination in validations were not much different compared with derived phases. The pooled C statistics in derived RCTs were 0.88 (0.78‑0.98; I2 = 96.82%), see Fig. 3. Moreover, pooled C statistics in external validation demonstrated good performance of 0.86 (0.85‑0.88) in 3 [26, 29, 44] cohorts (Table S5). Funnel plot and Egger’s test (p > 0.513) showed no publication bias by the absence of small study effects in external validation studies for predicting ESRD (Fig. S7).
Fig. 3.
Forest plot of C statistics in derivative phase for ESRD
Common predictors for ESRD were age, sex, HbA1c, eGFR and BMI (Table S6). Predictive models of 6 [27, 28, 86, 87, 91] studies might have over-fitted as events/variable for applying Cox resulted in 6 [28] to 9 [92], whereas their ratios in 4 [12, 29, 89, 90] other studies observed a rule of thumb of 10‑20 (Fig. 3, Table S2).
Discussion
This review summarised prognostic models that were developed and validated for predicting microvascular complications (i.e. DR, CKD and ESRD) in T2D patients. Model performances were described prognostic models separately by derived, internal and external validation.
Seven predictors were commonly used in predictive models of DR, DN and ESRD including age, sex, BMI, diabetic duration, HbA1c, SBP and eGFR. The DR models showed well discriminated with pooled C statistics of 0.82, 0.83 and 0.81 in D, I and E validations, respectively. Model performance was only moderate‑good in CKD for all phases (i.e. the corresponding pooled C statistics about 0.78, 0.79 and 0.75) but quite excellence for ESRD models (i.e. 0.87, 0.91 and 0.86, respectively).
Only a few prognostic models were externally validated with moderate to good discrimination performance, which are applicable in clinical practice. For instance, a few DR-models [11, 12, 41] had good discrimination and calibration in external validations. Three [12, 20, 68] DN models had good discrimination with fair calibration. Other three [12, 26, 29] ESRD models with very large size cohorts were generalisable with good discriminations and were even developed in different ethnicities. Calibration performance was less reported relative to discrimination, although both parameters should be reported for prognostic model development [95–97]. Particularly for observed to expected (O/E) ratio was reported in very few studies, which prevented meta-analysis of calibration.
Currently many prediction models are available by online calculators, or differently presented simplified risk scores or nomograms. Some online risk-calculators have been developed to simplify knowledge translation in clinical practice (i.e. DR [11, 12, 41, 59], CKD [12, 20] and ESRD [26]). However, very few of them have been applied due to the absence of some predictors and users’ interpretations in routine health practice.
We found various clinical settings and developed equations, but only few of them were externally validated with insufficiently reported with a wide range of CKD definitions. Amongst them, there might be potentially over-optimistic as EPV was less than ten by the rule of thumb in a regression model. None of the studies performed impact assessments by applying prognostic models into clinical practice.
Numerous predictors were simultaneously included in the prognostic models with a median of 8 (IQR, 5‑10) predictors. In brief, demographics, biomarkers and clinical features were commonly considered for derived-models of micro-vascular complications. Medical treatment (including anti-hypertensive and diabetic drug control) and some comorbidities were included into some derived equations. Likewise, DR itself might be a predictor of DN [20, 22, 66, 74, 75]. Interestingly, nonconventional predictors (i.e. genetic and image processing [56, 58]) could also predict DR [42, 55, 62] and DN [19, 67].
Missing data in clinical settings particularly for routine datasets are unavoidable. Frequently, the investigators only performed complete-case analysis. Handling missing data is vitally important to prevent biassed results and lost power in generalisations [98]. Additionally, categorisation of continuous predictors or dichotomisation may result in missing information, significant misleading [99], incorrect variable selection and may decrease prediction accuracy [100, 101].
Cohort or RCT should be the most appropriated design for developing prognostic model, whereas a cross-sectional study could be used for external validation. Exceptionally, nested case-control and case-cohort studies were still applicable [96]. The rule of thumb suggested that a number of 10‑20 events should be available for one predictor in a multivariable logit/Cox regression [96, 102, 103]. For instance, seven [14, 17, 24, 52, 56, 58, 61] studies in DR had EPV ratio of 1 [58] to 9 [24, 52], which might cause overfitted model. In DN, eight [21, 22, 68, 73, 76, 77, 79, 83] and seven [27, 28, 85–87, 91, 92] studies might be over-optimistic with the EPV ratio less than 10 for CKD and ESRD, respectively. Overfitting may result in poorer performance in external validation compared with derived-performance. As a result, performances of the traditional statistical models (i.e. logit, Cox) were quite varied across studies. However, ML may be better particularly when predictors themselves have collinearity and high-dimensional interaction amongst predictors. With the rapid era of big data, digitalisation and modern electronic medical records may increase used of ML techniques in derived and validation model.
As the backbone of big data analysis, ML provides the new insight and valuable algorithm in which traditional statistical models are often inadequate. Likewise, using image/signal [56, 58] analysis for predicting DR, some investigators also applied classical ML (e.g. decision trees, random forest, Naïve Bayes and neural network) to predict DN [13, 70, 73]. Nonetheless, the results of ML are black boxes, which are often difficult to interpret due to its characteristics and algorithm complexities [104, 105].
Few other factors may also influence on external-validation performance, e.g. availability of predictors, sources of data (i.e. primary data collection, survey-data or administrative/hospital-claims data), outcome rate and assessment and also population characteristics. However, only about 20 studies (25% of derived models) were externally validated. We therefore strongly suggest that those derived models should be externally validated or updated models where appropriate. Then, impact analysis should next be performed to be more confident in applying in clinical practice.
Conclusions
This study was conducted to systematically review prognostic models of diabetic microvascular complications. Weaknesses and strengths of those prognostic models for each complication were described and commented. Some prognostic models for microvascular complications were good in discrimination in external validations, but in practice none of them performed clinical impact. The existing prognostic models for DR and CKD still need further external validation or update where appropriate. In addition, the new prognostic models should be derived using ML techniques to improve prognostic performance where required.
Supplementary Information
Additional file 1: Search strategy & results from PubMed database.
Additional file 2: Search strategy & results from Scopus database.
Additional file 3: Table S1. Characteristics of studies included in systematic review. Table S2. Describe discrimination and calibration performances of prognostic models. Table S3. Risk of bias assessment using PROBAST. Table S4. Prognostic equations that were externally validated. Table S5. Summary of pooled C-statistic of prognostic model. Table S6. Describe variables that were included in the derivative equations.
Additional file 4: Figure S1. PRISMA diagram for diabetic retinopathy. Figure S2. PRISMA diagram for diabetic nephropathy. Figure S3. Risk-of-bias using PROBAST. Figure S4. Overview top 20 predictors included in derived models. Figure S5. Funnel plot (C-statistics) for DR in external validation. Figure S6. Funnel plot (C-statistics) for CKD in external validation. Figure S7. Funnel plot (C-statistics) for ESRD in external validation.
Acknowledgements
This study was an important part of Sigit Ari Saputro’s training in PhD (Data Science for Healthcare), Faculty of Medicine Ramathibodi Hospital, Mahidol University, Thailand. The authors would like to thank Stephen John Pinder for proofreading and conducting comprehensive English language review.
Abbreviations
- BMI
Body mass index
- CHARMS
Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies
- CKD
Chronic kidney disease
- D
Derivative phase
- DN
Diabetic nephropathy
- DR
Diabetic retinopathy
- E
External validation phase
- eGFR
Estimated glomerular filtration rate
- EPV
Event per variables
- ESRD
End-stage renal disease
- GRS
Genetic risk score
- HbA1c
Glycated haemoglobin
- I
Internal validation phase
- ML
Machine learning
- PRISMA
-
Preferred Reporting Items for Systematic Reviews and Meta-Analysis
PROBAST
Prediction model risk of bias assessment tool
- RCT
Randomised controlled trials
- SBP
Systolic blood pressure
- T2D
Type 2 diabetes
Authors’ contributions
SAS, OP and AT conceptualised the study questions and developed protocol and search strategy for the systematic review. SAS conducted the review (screening, data extraction, analyse the data) and wrote the manuscript. SAS and SK did the risk of bias assessment and wrote interpretation. AT, OP and AP edited and revised the manuscript. All authors have read and approved the final version of the manuscript to be published.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.
Availability of data and materials
All data included in this study were available in the supplementary information files.
Declarations
Ethics approval and consent to participate
Not applicable
Consent for publication
All authors give their consent for publication.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.International Diabetes Federation . IDF Diabetes Atlas Ninth Edition 2019. Brussels: International Diabetes Federation; 2019. [Google Scholar]
- 2.World Health Organization . Classification of diabetes mellitus. Geneva: World Health Organization; 2019. [Google Scholar]
- 3.Dougherty T, Heile M. Type 2 diabetes in the US managed care setting: the burden of disease and rationale for an oral glucagon-like peptide-1 receptor agonist. Am J Manag Care. 2020;26(16 Suppl):S325–Ss34. doi: 10.37765/ajmc.2020.88552. [DOI] [PubMed] [Google Scholar]
- 4.Ahmad J. Management of diabetic nephropathy: recent progress and future perspective. Diabetes Metab Syndr. 2015;9(4):343–358. doi: 10.1016/j.dsx.2015.02.008. [DOI] [PubMed] [Google Scholar]
- 5.American Diabetes Association Microvascular complications and foot care: standards of medical care in diabetes-2020. Diabetes Care. 2020;43(Suppl 1):S135–Ss51. doi: 10.2337/dc20-S011. [DOI] [PubMed] [Google Scholar]
- 6.Savelieff MG, Callaghan BC, Feldman EL. The emerging role of dyslipidemia in diabetic microvascular complications. Curr Opin Endocrinol Diabetes Obes. 2020;27(2):115–123. doi: 10.1097/MED.0000000000000533. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Faselis C, Katsimardou A, Imprialos K, Deligkaris P, Kallistratos M, Dimitriadis K. Microvascular complications of type 2 diabetes mellitus. Curr Vasc Pharmacol. 2020;18(2):117–124. doi: 10.2174/1570161117666190502103733. [DOI] [PubMed] [Google Scholar]
- 8.Farran B, Channanath AM, Behbehani K, Thanaraj TA. Predictive models to assess risk of type 2 diabetes, hypertension and comorbidity: machine-learning algorithms and validation using national health data from Kuwait—a cohort study. BMJ Open. 2013;3(5):e002457. doi: 10.1136/bmjopen-2012-002457. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.American Diabetes Association. Economic costs of diabetes in the U.S. in 2017. Diabetes Care. 2018;41(5):917–28. [DOI] [PMC free article] [PubMed]
- 10.Seuring T, Archangelidi O, Suhrcke M. The economic costs of type 2 diabetes: a global systematic review. Pharmacoeconomics. 2015;33(8):811–831. doi: 10.1007/s40273-015-0268-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Aspelund T, Thornorisdottir O, Olafsdottir E, Gudmundsdottir A, Einarsdottir AB, Mehlsen J, et al. Individual risk assessment and information technology to optimise screening frequency for diabetic retinopathy. Diabetologia. 2011;54(10):2525–2532. doi: 10.1007/s00125-011-2257-7. [DOI] [PubMed] [Google Scholar]
- 12.Basu S, Sussman JB, Berkowitz SA, Hayward RA, Yudkin JS. Development and validation of risk equations for complications of type 2 diabetes (RECODe) using individual participant data from randomised trials. Lancet Diabetes Endocrinol. 2017;5(10):788–798. doi: 10.1016/S2213-8587(17)30221-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Dagliati A, Marini S, Sacchi L, Cogni G, Teliti M, Tibollo V, et al. Machine learning methods to predict diabetes complications. J Diabetes Sci Technol. 2018;12(2):295–302. doi: 10.1177/1932296817706375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kengne AP, Gouking P, Wawo E, Koki G, Balti E, Hakapoka H, et al. Cardiovascular risk profile, diabetes specific factors, and prevalent microvascular eye complications in sub-Saharan Africans with type 2 diabetes. Int J Diabetes Developing Countries. 2015;35:349–355. doi: 10.1007/s13410-014-0283-y. [DOI] [Google Scholar]
- 15.Ogunyemi O, Kermah D. Machine learning approaches for detecting diabetic retinopathy from clinical and public health records. AMIA Annual Symposium Proc AMIA Symposium. 2015;2015:983–990. [PMC free article] [PubMed] [Google Scholar]
- 16.Ogunyemi O, Teklehaimanot S, Patty L, Moran E, George S. Evaluating predictive modeling’s potential to improve teleretinal screening participation in urban safety net clinics. Stud Health Technol Informatics. 2013;192:162–165. [PMC free article] [PubMed] [Google Scholar]
- 17.Oh E, Yoo TK, Park EC. Diabetic retinopathy risk prediction for fundus examination using sparse learning: a cross-sectional study. BMC Medical Informatics Decision Making. 2013;13:106. doi: 10.1186/1472-6947-13-106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Semeraro F, Parrinello G, Cancarini A, Pasquini L, Zarra E, Cimino A, et al. Predicting the risk of diabetic retinopathy in type 2 diabetic patients. J Diabetes Complications. 2011;25(5):292–297. doi: 10.1016/j.jdiacomp.2010.12.002. [DOI] [PubMed] [Google Scholar]
- 19.Blech I, Katzenellenbogen M, Katzenellenbogen A, Wainstein J, Rubinstein A, Harman-Boehm I, et al. Predicting diabetic nephropathy using a multifactorial genetic model. PLoS One. 2011;6(4):e18743. doi: 10.1371/journal.pone.0018743. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Dunkler D, Gao P, Lee SF, Heinze G, Clase CM, Tobe S, et al. Risk prediction for early CKD in type 2 diabetes. Clin J Am Soc Nephrol. 2015;10(8):1371–1379. doi: 10.2215/CJN.10321014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Klisic A, Kavaric N, Ninic A. Retinol-binding protein 4 versus albuminuria as predictors of estimated glomerular filtration rate decline in patients with type 2 diabetes. J Res Med Sci. 2018;23:44. doi: 10.4103/jrms.JRMS_893_17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Miao DD, Pan EC, Zhang Q, Sun ZM, Qin Y, Wu M. Development and validation of a model for predicting diabetic nephropathy in Chinese people. Biomed Environ Sci. 2017;30(2):106–112. doi: 10.3967/bes2017.014. [DOI] [PubMed] [Google Scholar]
- 23.Nowak N, Skupien J, Smiles AM, Yamanouchi M, Niewczas MA, Galecki AT, et al. Markers of early progressive renal decline in type 2 diabetes suggest different implications for etiological studies and prognostic tests development. Kidney International. 2018;93(5):1198–1206. doi: 10.1016/j.kint.2017.11.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Welsh P, Woodward M, Hillis GS, Li Q, Marre M, Williams B, et al. Do cardiac biomarkers NT-proBNP and hsTnT predict microvascular events in patients with type 2 diabetes? Results from the ADVANCE trial. Diabetes Care. 2014;37(8):2202–2210. doi: 10.2337/dc13-2625. [DOI] [PubMed] [Google Scholar]
- 25.Wu M, Lu J, Zhang L, Liu F, Chen S, Han Y, et al. A non-laboratory-based risk score for predicting diabetic kidney disease in Chinese patients with type 2 diabetes. Oncotarget. 2017;8(60):102550–102558. doi: 10.18632/oncotarget.21684. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Elley CR, Robinson T, Moyes SA, Kenealy T, Collins J, Robinson E, et al. Derivation and validation of a renal risk score for people with type 2 diabetes. Diabetes Care. 2013;36(10):3113–3120. doi: 10.2337/dc13-0190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Garlo KG, White WB, Bakris GL, Zannad F, Wilson CA, Kupfer S, et al. Kidney biomarkers and decline in eGFR in patients with type 2 diabetes. Clin J Am Soc Nephrol. 2018;13(3):398–405. doi: 10.2215/CJN.05280517. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Li HY, Lin HA, Nien FJ, Wu VC, Jiang YD, Chang TJ, et al. Serum vascular adhesion protein-1 predicts end-stage renal disease in patients with type 2 diabetes. PLoS One. 2016;11(2):e0147981. doi: 10.1371/journal.pone.0147981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Wan EYF, Fong DYT, Fung CSC, Yu EYT, Chin WY, Chan AKC, et al. Prediction of new onset of end stage renal disease in Chinese patients with type 2 diabetes mellitus - a population-based retrospective cohort study. BMC Nephrol. 2017;18(1):257. doi: 10.1186/s12882-017-0671-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.van der Heijden AA, Walraven I. van ‘t Riet E, Aspelund T, Lund SH, Elders P, et al. Validation of a model to estimate personalised screening frequency to monitor diabetic retinopathy. Diabetologia. 2014;57(7):1332–1338. doi: 10.1007/s00125-014-3246-4. [DOI] [PubMed] [Google Scholar]
- 31.Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356:i6460. doi: 10.1136/bmj.i6460. [DOI] [PubMed] [Google Scholar]
- 33.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 34.Altman DG, Bland JM. How to obtain the confidence interval from a P value. Bmj. 2011;343:d2090. doi: 10.1136/bmj.d2090. [DOI] [PubMed] [Google Scholar]
- 35.Debray TP, Vergouwe Y, Koffijberg H, Nieboer D, Steyerberg EW, Moons KG. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol. 2015;68(3):279–289. doi: 10.1016/j.jclinepi.2014.06.018. [DOI] [PubMed] [Google Scholar]
- 36.DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials. 2015;45(Pt A):139–45. doi: 10.1016/j.cct.2015.09.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Thorlund K, Wetterslev J, Awad T, Thabane L, Gluud C. Comparison of statistical inferences from the DerSimonian-Laird and alternative random-effects model meta-analyses - an empirical assessment of 920 Cochrane primary outcome meta-analyses. Res Synth Methods. 2011;2(4):238–253. doi: 10.1002/jrsm.53. [DOI] [PubMed] [Google Scholar]
- 38.Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882–893. doi: 10.1016/j.jclinepi.2005.01.016. [DOI] [PubMed] [Google Scholar]
- 39.Song F, Khan KS, Dinnes J, Sutton AJ. Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. Int J Epidemiol. 2002;31(1):88–95. doi: 10.1093/ije/31.1.88. [DOI] [PubMed] [Google Scholar]
- 40.StataCorp. Stata Statistical Software: Release 16. College Station: StataCorp LLC; 2019.
- 41.Hippisley-Cox J, Coupland C. Development and validation of risk prediction equations to estimate future risk of blindness and lower limb amputation in patients with diabetes: cohort study. Bmj. 2015;351:h5441. doi: 10.1136/bmj.h5441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Liao WL, Lin JM, Chen WL, Hsieh MC, Wu CM, Chang YW, et al. Multilocus genetic risk score for diabetic retinopathy in the Han Chinese population of Taiwan. Scientific Reports. 2018;8(1):14535. doi: 10.1038/s41598-018-32916-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Yusufi FNK, Ahmed A, Ahmad J. Modelling and developing diabetic retinopathy risk scores on Indian type 2 diabetes patients. Int J Diabetes Developing Countries. 2019;39(1):29–38. doi: 10.1007/s13410-018-0652-z. [DOI] [Google Scholar]
- 44.Basu S, Sussman JB, Berkowitz SA, Hayward RA, Bertoni AG, Correa A, et al. Validation of risk equations for complications of type 2 diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care. 2018;41(3):586–95. [DOI] [PMC free article] [PubMed]
- 45.Hosseini SM, Maracy MR, Amini M, Baradaran HR. A risk score development for diabetic retinopathy screening in Isfahan-Iran. J Res Med Sci. 2009;14(2):105–110. [PMC free article] [PubMed] [Google Scholar]
- 46.Soto-Pedre E, Pinies JA, Hernaez-Ortega MC. External validation of a risk assessment model to adjust the frequency of eye-screening visits in patients with diabetes mellitus. J Diabetes Complications. 2015;29(4):508–511. doi: 10.1016/j.jdiacomp.2014.12.020. [DOI] [PubMed] [Google Scholar]
- 47.Wang J, Chen H, Zhang H, Yang F, Chen RP, Li YB, et al. The performance of a diabetic retinopathy risk score for screening for diabetic retinopathy in Chinese overweight/obese patients with type 2 diabetes mellitus. Ann Med. 2014;46(6):417–423. doi: 10.3109/07853890.2013.878977. [DOI] [PubMed] [Google Scholar]
- 48.García-Fiñana M, Hughes DM, Cheyne CP, Broadbent DM, Wang A, Komárek A, et al. Personalized risk-based screening for diabetic retinopathy: a multivariate approach versus the use of stratification rules. Diabetes Obes Metab. 2019;21(3):560–568. doi: 10.1111/dom.13552. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Mo R, Shi R, Hu Y, Hu F. Nomogram-based prediction of the risk of diabetic retinopathy: a retrospective study. J Dia Res. 2020;2020. 10.1155/2020/7261047. [DOI] [PMC free article] [PubMed]
- 50.Ochs A, McGurnaghan S, Black MW, Leese GP, Philip S, Sattar N, et al. Use of personalised risk-based screening schedules to optimise workload and sojourn time in screening programmes for diabetic retinopathy: a retrospective cohort study. PLoS Med. 2019;16(10):e1002945. doi: 10.1371/journal.pmed.1002945. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Ogunyemi OI, Gandhi M, Tayek C. Predictive models for diabetic retinopathy from non-image teleretinal screening data. AMIA Jt Summits Transl Sci Proc. 2019;2019:472–477. [PMC free article] [PubMed] [Google Scholar]
- 52.Tsao HY, Chan PY, Su EC. Predicting diabetic retinopathy and identifying interpretable biomedical features using machine learning algorithms. BMC Bioinformatics. 2018;19(Suppl 9):283. doi: 10.1186/s12859-018-2277-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Zhu X, Wu H, Ge C, Chen Y, Zhao X, Kong M, et al. Establishment and verification of a risk prediction model of diabetic retinopathy in patients with type 2 diabetes in the community. Chin Gen Pract. 2020;23(6):712–715. [Google Scholar]
- 54.Azizi-Soleiman F, Heidari-Beni M, Ambler G, Omar R, Amini M, Hosseini SM. Iranian risk model as a predictive tool for retinopathy in patients with type 2 diabetes. Can J Diabetes. 2015;39(5):358–363. doi: 10.1016/j.jcjd.2015.01.290. [DOI] [PubMed] [Google Scholar]
- 55.Chong YH, Fan Q, Tham YC, Gan A, Tan SP, Tan G, et al. Type 2 diabetes genetic variants and risk of diabetic retinopathy. Ophthalmology. 2017;124(3):336–342. doi: 10.1016/j.ophtha.2016.11.016. [DOI] [PubMed] [Google Scholar]
- 56.Han Y, Schneck ME, Bearse MA, Jr, Barez S, Jacobsen CH, Jewell NP, et al. Formulation and evaluation of a predictive model to identify the sites of future diabetic retinopathy. Investigative Ophthalmol Visual Sci. 2004;45(11):4106–4112. doi: 10.1167/iovs.04-0405. [DOI] [PubMed] [Google Scholar]
- 57.Mendoza-Herrera K, Quezada AD, Pedroza-Tobias A, Hernandez-Alcaraz C, Fromow-Guerra J, Barquera S. A diabetic retinopathy screening tool for low-income adults in Mexico. Preventing Chronic Dis. 2017;14:E95. doi: 10.5888/pcd14.170157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Ng JS, Bearse MA, Jr, Schneck ME, Barez S, Adams AJ. Local diabetic retinopathy prediction by multifocal ERG delays over 3 years. Investigative Ophthalmol Visual Sci. 2008;49(4):1622–1628. doi: 10.1167/iovs.07-1157. [DOI] [PubMed] [Google Scholar]
- 59.Tanaka S, Tanaka S, Iimuro S, Yamashita H, Katayama S, Akanuma Y, et al. Predicting macro- and microvascular complications in type 2 diabetes: the Japan Diabetes Complications Study/the Japanese Elderly Diabetes Intervention Trial risk engine. Diabetes Care. 2013;36(5):1193–1199. doi: 10.2337/dc12-0958. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Bresnick GH, Palta M. Predicting progression to severe proliferative diabetic retinopathy. Arch Ophthalmol. 1987;105(6):810–4. doi: 10.1001/archopht.1987.01060060096041. [DOI] [PubMed] [Google Scholar]
- 61.Cichosz SL, Johansen MD, Knudsen ST, Hansen TK, Hejlesen O. A classification model for predicting eye disease in newly diagnosed people with type 2 diabetes. Diabetes Res Clin Practice. 2015;108(2):210–215. doi: 10.1016/j.diabres.2015.02.020. [DOI] [PubMed] [Google Scholar]
- 62.Mogilevskyy. Predicting the development of diabetic retinopathy based on identifcation of rs759853 and rs9640883 in the AKR1B1 gene. J Ophthalmol 2017;IV:477.
- 63.Cichosz SL, Johansen MD, Hejlesen O. Toward big data analytics: review of predictive models in management of diabetes and its complications. J Diabetes Sci Technol. 2015;10(1):27–34. doi: 10.1177/1932296815611680. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Ganjifrockwala FA, Joseph JT, George G. Evaluation of kidney function and risk factors of retinopathy in type 2 diabetes mellitus people in South Africa. Diabetes Res Clin Pract. 2017;127:218–223. doi: 10.1016/j.diabres.2017.03.022. [DOI] [PubMed] [Google Scholar]
- 65.Hu Y, Shi R, Mo R, Hu F. Nomogram for the prediction of diabetic nephropathy risk among patients with type 2 diabetes mellitus based on a questionnaire and biochemical indicators: a retrospective study. Aging (Albany NY). 2020;12(11):10317–10336. doi: 10.18632/aging.103259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Jiang S, Fang J, Yu T, Liu L, Zou G, Gao H, et al. Novel model predicts diabetic nephropathy in type 2 diabetes. Am J Nephrol. 2020;51(2):130–138. doi: 10.1159/000505145. [DOI] [PubMed] [Google Scholar]
- 67.Liao LN, Li TC, Li CI, Liu CS, Lin WY, Lin CH, et al. Genetic risk score for risk prediction of diabetic nephropathy in Han Chinese type 2 diabetes patients. Scientific Reports. 2019;9(1):19897. doi: 10.1038/s41598-019-56400-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Peters KE, Davis WA, Ito J, Bringans SD, Lipscombe RJ, Davis TME. Validation of a protein biomarker test for predicting renal decline in type 2 diabetes: the Fremantle diabetes study phase II. J Diabetes Complications. 2019;33(12):107406. doi: 10.1016/j.jdiacomp.2019.07.003. [DOI] [PubMed] [Google Scholar]
- 69.Qian YS, Ming MF. Predicting the risk of chronic kidney disease among type 2 diabetes mellitus patients in a primary care setting: an evaluation of the qkidney model. Malays J Med Health Sci. 2019;15(3):67–73. [Google Scholar]
- 70.Rodriguez-Romero V, Bergstrom RF, Decker BS, Lahu G, Vakilynejad M, Bies RR. Prediction of nephropathy in type 2 diabetes: an analysis of the ACCORD trial applying machine learning techniques. Clin Transl Sci. 2019;12(5):519–528. doi: 10.1111/cts.12647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Song X, Waitman LR, Hu Y, Yu ASL, Robbins DC, Liu M. Robust clinical marker identification for diabetic kidney disease with ensemble feature selection. J Am Med Informatics Assoc. 2019;26(3):242–253. doi: 10.1093/jamia/ocy165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Wysham CH, Gauthier-Loiselle M, Bailey RA, Manceur AM, Lefebvre P, Greenberg M, et al. Development of risk models for major adverse chronic renal outcomes among patients with type 2 diabetes mellitus using insurance claims: a retrospective observational study. Curr Med Res Opin. 2020;36(2):219–227. doi: 10.1080/03007995.2019.1682981. [DOI] [PubMed] [Google Scholar]
- 73.Goldfarb-Rumyantzev AS, Pappas L. Prediction of renal insufficiency in Pima Indians with nephropathy of type 2 diabetes mellitus. Am J Kidney Dis. 2002;40(2):252–264. doi: 10.1053/ajkd.2002.34503. [DOI] [PubMed] [Google Scholar]
- 74.Jardine MJ, Hata J, Woodward M, Perkovic V, Ninomiya T, Arima H, et al. Prediction of kidney-related outcomes in patients with type 2 diabetes. Am J Kidney Dis. 2012;60(5):770–778. doi: 10.1053/j.ajkd.2012.04.025. [DOI] [PubMed] [Google Scholar]
- 75.Jenks SJ, Conway BR, McLachlan S, Teoh WL, Williamson RM, Webb DJ, et al. Cardiovascular disease biomarkers are associated with declining renal function in type 2 diabetes. Diabetologia. 2017;60(8):1400–1408. doi: 10.1007/s00125-017-4297-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Kim JH, Oh SY, Kim EH, Lee MJ, Jeon YK, Kim BH, et al. Addition of nonalbumin proteinuria to albuminuria improves prediction of type 2 diabetic nephropathy progression. Diabetol Metabolic Syndrome. 2017;9:68. doi: 10.1186/s13098-017-0267-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Lindhardt M, Persson F, Zurbig P, Stalmach A, Mischak H, de Zeeuw D, et al. Urinary proteomics predict onset of microalbuminuria in normoalbuminuric type 2 diabetic patients, a sub-study of the DIRECT-Protect 2 study. Nephrol Dialysis Transplantation. 2017;32(11):1866–1873. doi: 10.1093/ndt/gfw292. [DOI] [PubMed] [Google Scholar]
- 78.Low S, Lim SC, Zhang X, Zhou S, Yeoh LY, Liu YL, et al. Development and validation of a predictive model for chronic kidney disease progression in type 2 diabetes mellitus based on a 13-year study in Singapore. Diabetes Res Clin Pract. 2017;123:49–54. doi: 10.1016/j.diabres.2016.11.008. [DOI] [PubMed] [Google Scholar]
- 79.Peters KE, Davis WA, Ito J, Winfield K, Stoll T, Bringans SD, et al. Identification of novel circulating biomarkers predicting rapid decline in renal function in type 2 diabetes: the Fremantle diabetes study phase II. Diabetes Care. 2017;40(11):1548–1555. doi: 10.2337/dc17-0911. [DOI] [PubMed] [Google Scholar]
- 80.Riphagen IJ, Kleefstra N, Drion I, Alkhalaf A, van Diepen M, Cao Q, et al. Comparison of methods for renal risk prediction in patients with type 2 diabetes (ZODIAC-36) PLoS One. 2015;10(3):e0120477. doi: 10.1371/journal.pone.0120477. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Saulnier PJ, Gand E, Velho G, Mohammedi K, Zaoui P, Fraty M, et al. Association of circulating biomarkers (adrenomedullin, TNFR1, and NT-proBNP) with renal function decline in patients with type 2 diabetes: a French prospective cohort. Diabetes Care. 2017;40(3):367–374. doi: 10.2337/dc16-1571. [DOI] [PubMed] [Google Scholar]
- 82.Yang JK, Wang YY, Liu C, Shi TT, Lu J, Cao X, et al. Urine proteome specific for eye damage can predict kidney damage in patients with type 2 diabetes: a case-control and a 5.3-year prospective cohort study. Diabetes care. 2017;40(2):253–260. doi: 10.2337/dc16-1529. [DOI] [PubMed] [Google Scholar]
- 83.Zobel EH, von Scholten BJ, Reinhard H, Persson F, Teerlink T, Hansen TW, et al. Symmetric and asymmetric dimethylarginine as risk markers of cardiovascular disease, all-cause mortality and deterioration in kidney function in persons with type 2 diabetes and microalbuminuria. Cardiovasc Diabetol. 2017;16(1):88. doi: 10.1186/s12933-017-0569-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Lin CC, Li CI, Liu CS, Lin WY, Lin CH, Yang SY, et al. Development and validation of a risk prediction model for end-stage renal disease in patients with type 2 diabetes. Sci Rep. 2017;7(1):10177. doi: 10.1038/s41598-017-09243-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Yang XL, So WY, Kong AP, Clarke P, Ho CS, Lam CW, et al. End-stage renal disease risk equations for Hong Kong Chinese patients with type 2 diabetes: Hong Kong Diabetes Registry. Diabetologia. 2006;49(10):2299–2308. doi: 10.1007/s00125-006-0376-3. [DOI] [PubMed] [Google Scholar]
- 86.Foster MC, Inker LA, Hsu CY, Eckfeldt JH, Levey AS, Pavkov ME, et al. Filtration markers as predictors of ESRD and mortality in Southwestern American Indians with type 2 diabetes. Am J Kidney Dis. 2015;66(1):75–83. doi: 10.1053/j.ajkd.2015.01.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87.Fufaa GD, Weil EJ, Nelson RG, Hanson RL, Bonventre JV, Sabbisetti V, et al. Association of urinary KIM-1, L-FABP, NAG and NGAL with incident end-stage renal disease and mortality in American Indians with type 2 diabetes mellitus. Diabetologia. 2015;58(1):188–198. doi: 10.1007/s00125-014-3389-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88.Lambers Heerspink HJ, Gansevoort RT, Brenner BM, Cooper ME, Parving HH, Shahinfar S, et al. Comparison of different measures of urinary protein excretion for prediction of renal events. J Am Soc Nephrol. 2010;21(8):1355–1360. doi: 10.1681/ASN.2010010063. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Mise K, Hoshino J, Ueno T, Hazue R, Hasegawa J, Sekine A, et al. Prognostic value of tubulointerstitial lesions, urinary N-acetyl-beta-d-glucosaminidase, and urinary beta2-microglobulin in patients with type 2 diabetes and biopsy-proven diabetic nephropathy. Clin J Am Soc Nephrol. 2016;11(4):593–601. doi: 10.2215/CJN.04980515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90.Mise K, Yamaguchi Y, Hoshino J, Ueno T, Sekine A, Sumida K, et al. Paratubular basement membrane insudative lesions predict renal prognosis in patients with type 2 diabetes and biopsy-proven diabetic nephropathy. PLoS One. 2017;12(8):e0183190. doi: 10.1371/journal.pone.0183190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 91.Niewczas MA, Gohda T, Skupien J, Smiles AM, Walker WH, Rosetti F, et al. Circulating TNF receptors 1 and 2 predict ESRD in type 2 diabetes. J Am Soc Nephrol. 2012;23(3):507–515. doi: 10.1681/ASN.2011060627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.Pavkov ME, Knowler WC, Hanson RL, Williams DE, Lemley KV, Myers BD, et al. Comparison of serum cystatin C, serum creatinine, measured GFR, and estimated GFR to assess the risk of kidney failure in American Indians with diabetic nephropathy. Am J Kidney Dis. 2013;62(1):33–41. doi: 10.1053/j.ajkd.2012.11.044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Keane WF, Zhang Z, Lyle PA, Cooper ME, de Zeeuw D, Grunfeld JP, et al. Risk scores for predicting outcomes in patients with type 2 diabetes and nephropathy: the RENAAL study. Clin J Am Soc Nephrol. 2006;1(4):761–767. doi: 10.2215/CJN.01381005. [DOI] [PubMed] [Google Scholar]
- 94.Sullivan LM, Massaro JM, D’Agostino RB., Sr Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631–1660. doi: 10.1002/sim.1742. [DOI] [PubMed] [Google Scholar]
- 95.Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017;318(14):1377–1384. doi: 10.1001/jama.2017.12126. [DOI] [PubMed] [Google Scholar]
- 96.Moons KG, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 2012;98(9):683–690. doi: 10.1136/heartjnl-2011-301246. [DOI] [PubMed] [Google Scholar]
- 97.Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Bmj. 2015;350:g7594. doi: 10.1136/bmj.g7594. [DOI] [PubMed] [Google Scholar]
- 98.Jakobsen JC, Gluud C, Wetterslev J, Winkel P. When and how should multiple imputation be used for handling missing data in randomised clinical trials - a practical guide with flowcharts. BMC Med Res Methodol. 2017;17(1):162. doi: 10.1186/s12874-017-0442-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99.Lagakos SW. Effects of mismodelling and mismeasuring explanatory variables on tests of their association with a response variable. Stat Med. 1988;7(1-2):257–274. doi: 10.1002/sim.4780070126. [DOI] [PubMed] [Google Scholar]
- 100.Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25(1):127–141. doi: 10.1002/sim.2331. [DOI] [PubMed] [Google Scholar]
- 101.Subramanian V, Mascha EJ, Kattan MW. Developing a clinical prediction score: comparing prediction accuracy of integer scores to statistical regression models. Anesth Analg. 2021;132(6):1603–13. 10.1213/ANE.0000000000005362. [DOI] [PubMed]
- 102.Núñez E, Steyerberg EW, Núñez J. Regression modeling strategies. Revista Española de Cardiología (English Edition). 2011;64(6):501–507. doi: 10.1016/j.rec.2011.01.017. [DOI] [PubMed] [Google Scholar]
- 103.Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995;48(12):1495–1501. doi: 10.1016/0895-4356(95)00510-2. [DOI] [PubMed] [Google Scholar]
- 104.Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. 2016;18(12):e323. doi: 10.2196/jmir.5870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 105.Stevens LM, Mortazavi BJ, Deo RC, Curtis L, Kao DP. Recommendations for reporting machine learning analyses in clinical research. Circ Cardiovasc Qual Outcomes. 2020;13(10):e006556. doi: 10.1161/CIRCOUTCOMES.120.006556. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 1: Search strategy & results from PubMed database.
Additional file 2: Search strategy & results from Scopus database.
Additional file 3: Table S1. Characteristics of studies included in systematic review. Table S2. Describe discrimination and calibration performances of prognostic models. Table S3. Risk of bias assessment using PROBAST. Table S4. Prognostic equations that were externally validated. Table S5. Summary of pooled C-statistic of prognostic model. Table S6. Describe variables that were included in the derivative equations.
Additional file 4: Figure S1. PRISMA diagram for diabetic retinopathy. Figure S2. PRISMA diagram for diabetic nephropathy. Figure S3. Risk-of-bias using PROBAST. Figure S4. Overview top 20 predictors included in derived models. Figure S5. Funnel plot (C-statistics) for DR in external validation. Figure S6. Funnel plot (C-statistics) for CKD in external validation. Figure S7. Funnel plot (C-statistics) for ESRD in external validation.
Data Availability Statement
All data included in this study were available in the supplementary information files.



