Cardiovascular Risk Prediction Scores in Type 1 Diabetes

Abstract

Background

The extent of the performance and utility of scores for the risk of cardiovascular disease (CVD) in persons with type 1 diabetes (T1DM) largely remains unclear.

Objective

The purpose of this study was to synthesize data on the performance of CVD risk scores in people living with T1DM.

Methods

This study is a systematic review and meta-analysis. PubMed and EMBASE were searched through December 31, 2023. The included studies: 1) were retrospective, prospective, or cross-sectional in design; 2) included persons with T1DM; 3) assessed CVD outcomes; and 4) had data on at least on CVD risk score. Measures of calibration and discrimination qualitatively summarized. Measures of discrimination were combined using random-effects models stratified by type of risk model.

Results

In a meta-analysis of observational studies of CVD risk scores in T1DM individuals, including 11 studies and 73,664 participants (mean age of 34 years, mainly White individuals and male [55%]), we evaluated 12 CVD risk prediction models (7 T1DM-specific, 1 type 2 diabetes–specific, and 4 general population models). Most risk scores had a moderate to excellent discrimination (C-statistic: 0.73-0.85) and predicted CVD risk well when compared to actual clinical events. CVD risk scores specifically developed in T1DM individuals exhibited a higher discriminative performance—pooled C-statistic of 0.81 vs 0.75 for risk scores developed in the general population or those with type 2 diabetes and also showed a better calibration.

Conclusions

Among individuals with T1DM, CVD risk models had a moderate to excellent discrimination, with a better discrimination and accuracy for T1DM-specific scores.

Central Illustration

Among individual with type 1 diabetes (T1DM), cardiovascular disease (CVD) is the leading cause of death.1, 2, 3 In individuals with T1DM, CVD happens at least a decade or more earlier than among individuals in general population or those with type 2 diabetes (T2DM).² Indeed, T1DM is generally diagnosed at younger ages than T2DM, with the time of exposure to diabetes-related CVD risk factors being much longer.⁴ Furthermore, risk factors for CVD seem to operate differently in T1DM than in T2DM, including a more profound effect of hyperglycemia,^1,4,5 a higher risk among women,⁶ a much higher risk of some of the CVD outcomes (eg; heart failure),^7,8 all suggesting a difference in the pathophysiology of CVD in T1DM.

Although patients with T1DM are at increased risk of developing CVD, they are currently undertreated,^9,10 with low rates of treatment for hypertension, dyslipidemia, and microalbuminuria.^11,12 A factor that limits preventing or delaying CVD disease in T1DM is the delayed identification and management of the risk factors. In current guidelines, there are no formal recommendations on using a specific risk tool for predicting the future risk of CVD among those with T1DM.¹³ Thus, there are no risk scores used on a regular basis in clinical practice for assessing the risk of CVD in T1DM. The 2013 revised American College of Cardiology/American Heart Association pooled cohort equations ¹⁴ and the more recent American Heart Association prediction equation for CVD incorporating cardiovascular kidney metabolic health ¹⁵ are not applicable to individuals with T1DM. The development of these risk tools did not specifically include these individuals nor the age range within which patients with T1DM generally fall. Furthermore, risk scores developed for individuals with T2DM, such as the UKDPS Risk Engine,¹⁶ may underestimate the risk of CVD among individuals with T1DM. Indeed, most of the recommendations for managing CVD risk among people with T1DM are extrapolated from data obtained in people with T2DM.¹⁷

A number of studies have reported on the performance of risk scores for predicting CVD developed specifically among individuals with T1DM. These studies have presented variables results and are limited by relatively small sample size.

There is a lack of clarity on the estimated performance of risk scores for predicting CVD among individuals with T1DM. The existing data on the development and assessment of CVD risk scores in people with T1DM have not been adequately synthesized. Therefore, we conducted a systematic review and meta-analysis of all available evidence on the performance of risk scores for the prediction of CVD among people with T1DM. Our overarching goal is to guide clinicians and people with type 1 diabetes to prevent CVD.

Methods

Data sources and search strategy

We searched PubMed and EMBASE from inception up to December 2023. The reference lists of identified studies were manually scanned and cited references were screened through the ISI Web of Knowledge database for additional eligible studies.

Electronic search used key search terms related to T1DM, CVD, and risk prediction. The search terms are shown in the Supplemental Appendix. The inclusion criteria included the following: 1) patients with a diagnosis of T1DM; 2) retrospective or prospective studies with cardiovascular outcomes (eg; MI, CAD, CVD); and 3) available data on a minimum of one cardiac risk score (eg, Steno Risk score, Swedish Risk score).

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guidelines.¹⁸

Data extraction

We extracted prespecified information in duplicate from the publications using standardized forms (performed by AS and MH). The extracted study-level information included study location, study period (years), design, size, proportion of men, race/ethnicity, average of participants (mean or median), proportion with hypertension, proportion of smokers, proportion on lipid-lowering medications, proportion on blood pressure lowering medications, and the mean values of glycosylated hemoglobin (HbA_1C), blood pressure, total cholesterol, LDL-cholesterol, triglycerides, and duration of diabetes.

For each CVD risk prediction model, we extracted the following parameters if available: type of risk model (T1DM-specific, T2DM-specific, general population), outcomes assessed, follow-up period, N cases/events, event rate per 1,000 person-years, measures of discrimination (eg, C-statistic, C-index), measures of model calibration (eg, Hosmer-Lemeshow [HL] test ratio of expected to observed outcomes), any direct comparisons with one or more additional models including measures of reclassification—net reclassification index to evaluate whether a new model adds value in correctly reclassifying individuals compared to an existing model, integrated discrimination improvement, and information on validation. A model discrimination indicates its ability to correctly identifying those who are actually at a high risk of developing a cardiovascular disease and separate them from those who are actually at low risk of future CVD.¹⁹ A model calibration indicates the ability of a risk model to rank order individuals’ risks.¹⁹ The reclassification indices generally indicated the proportion of individuals reclassified from 1 risk stratum (based on a first model) to a different risk stratum (based on a different model with additional variables compared with the first model).¹⁹

We resolved discrepancies by consensus, adjudicated by 2 additional reviewers (S.E. and J.B.E.T.).

Quality of reporting and risk of bias in included studies

We assessed the quality of studies included in the review using the Transparent Reporting of a Multivariable Prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklist.²⁰ A number of the included studies fulfilled several important requirements for reporting of risk prediction models highlighted in the TRIPOD checklist, except for actions taken for blinding assessment of the outcome or predictor variables, which none of the studies reported. Supplemental Table 1 describes the evaluation of included studies using the TRIPOD checklist.

We assessed the risk of bias in included studies using the Newcastle-Ottawa Quality Assessment Scale for cohort studies.²¹ This scale is derived by assigning points to 3 aspects of study design with a maximum total of 10 points: selection of study participants (maximum 5 points), comparability of study groups (maximum 2 points), and ascertainment of the outcome of interest (maximum 3 points). The 3 components of the scoring system related to assessment of the nonexposed cohort and its comparability with the exposed cohort were not relevant to the studies assessing performance of risk scores, making the maximum possible total points 7 (instead of 10) in this review.

Data synthesis

The characteristics of the studies (eg number of participants, % males, % White individuals, % Black participants) are presented in tables and summarized as the range of values. We report the weighted average of these study-level characteristics weighted by the appropriate denominators (N). We took mean and median values as equivalent approximations of central tendencies (average value). Measures of calibration, including expected to observed ratios, and measures of discrimination are presented in tables and qualitatively summarized. Whenever possible, we combined discrimination measures stratified by type of risk model across studies, using random-effects model. We considered area under receiver operator curve reported as C-statistic and Harrel’s C-index are equivalent. The individual studies were weighted by the study size. Where confidence intervals of C-index or C-statistic were reported, we calculated the standard error as the difference between the upper and lower limits divided by 3.92 (2∗1.96). Where neither the standard error nor the confidence intervals of the estimates were reported, we assigned the standard error from another study included in this review that has the closest number of cases as the study missing this value. Such an imputation was necessary for 5 studies lacking the standard error of the estimates of the performance of risk models.22, 23, 24, 25, 26

We conducted sensitivity analysis including the following: 1) pooling the C-statistics using analytical weight based on the number of CVD cases observed in each study; and 2) performing a logit transformation of the C-statistic measures before pooling them across the studies. We report pooled estimates and 95% confidence intervals.

All analyses were performed using Stata software (Stata Corp v15). We reported the findings according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline.¹⁸

Role of the funding source and ethical approval

The funder had no role in the design and conduct of the study; collection, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication. Our study is a meta-analysis of published studies, so it did not require any approval by the Johns Hopkins University Institutional Review Board.

Results

Characteristics of studies

The study selection process generated a total of 3,438 studies based on titles, abstracts, and full texts (Supplemental Figure 1). Of these, we included 10 publications, representing data from 11 independent studies of people with T1DM in the meta-analyses.22, 23, 24, 25, 26, 27, 28, 29, 30, 31

The characteristics of the included studies are shown in Table 1. The studies were conducted predominantly in Europe, with 2 studies from US, and 1 study from Australia. The size of the studies ranged widely from 84 to 33,183 individuals with T1DM (median 1973, total 73,664). In all studies, the ascertainment of T1DM was made based on clinical records; no study used data on antibodies. The average age of participants ranged between 28 and 50 years (weighted average of 34 years) across the studies. The proportions of males ranged between 35% and 60% (weighted average 55%). The participants were predominantly of European ancestry (White individuals), although data on race were largely not reported. The mean HbA_1C ranged from 7.8% to 9.1% (weighted average 8.3%), mean diabetes duration ranged from 11.5 to 22.7 years (weighted average 15.3 years), mean body mass index count ranged from 23.4 to 26.5 kg/m² (weighted average 25.2 kg/m²), mean SBP ranged from 113 to 137 mm Hg (weighted average 125 mm Hg), mean total-cholesterol ranged from 4.6 to 5.2 mmol/L (weighted average 5.9 mmol/L), mean LDL-cholesterol ranged from 2.4 to 3.2 mmol/L (weighted average 2.9 mmol/L), mean HDL-cholesterol ranged from 1.2 to 1.7 mmol/L (weighted average 1.6 mmol/L), and mean triglycerides ranged from 0.7 to 1.3 mmol/L (weighted average 1.0 mmol/L). The proportion of current smokers ranged from 12% to 64% (weighted average 32%), that of those using hypertension medications from 5% to 40% (weighted average 25%), and that of those using lipid-lowering medications ranged from 8% to 41% (weighted average 22%).

Table 1.

Characteristics of Includes Studies

	Zgibor et al, 2006 22	Zgibor et al, 201023		Davis et al, 201027	Cederholm et al, 2011 24		Soedamah et al, 201425				Vistisen et al, 201628		Llaurado et al, 201729	Boscari et al, 202026	Mcgurnaghan, et al, 202130		Tecce et al, 202231
Subset	NA	Deriv.	Valid.	NA	Deriv.	Valid.	Deriv.	Valid.	Valid.	Valid.	Deriv.	Valid.	NA	NA	Deriv.	Valid.	NA
Study period	1986-2001	1986-2001	1989-1999	1993-2006	2002-2007	2003-2007	1989-1999	1986-2001	1994-2009	2000-2002	2001-2013	2001-2013	NR	2013-2014	2008-2018	2002-2013	2002-2019
Study design	PC	PC	RC	PC	RC	RC	PC	PC	PC	PC	PC	PC	X-Sec	RC	RC	RC	RC
Study Place	PA	PA	Europe	Australia	Sweden	Sweden	Europe	PA	Finland	CO	Denmark	Denmark	Spain	Italy	Scotland	Sweden	Italy
N	537	603	2,328	117	3,661	4,484	1973	554	2,999	590	4,306	2,118	84	223	27,527	33,183	456

Cohort	EDC	EDC	EURODIAB PCS	Fremantle Diabetes Study	Swedish National Diabetes Register	Swedish National Diabetes Register	EURODIAB PCS	EDC Original	FinnDiane	CACTI	Steno Diabetes Center	Funen Diabetes Database	_	Italian Popn	Scottish Care Information-Diabetes_	Swedish National Diabetes Register _	Italian popn
% of male	49.9	50.4	48.4	59.8	55.6	54.9	52.0	49.0	51.0	45.0	54.0	57.7	50.0	34.5	57.0	54.6	50.0
Average age, y	27.3	27.2	32.2	44	44.6	44.6	30.3	28.0	37.3	36.3	42.2	44.2	50.1	43.0	35.0	31.9	30.8
Diabetes duration	19.1	19.0	14.3	12.5	28.0	28.1	11.5	18.6	19.0	22.7	16.6	13.6	19.0	22.0	13.0	16.9	14.2
A1C	10.2	10.4	-	8.6	7.9	8.1	8.3	9.1	8.4	8.0	8.4	8.1	7.9	7.8	8.7	8.0	7.92
% smoking	-	22.2	-	26.3	14.8	13.4	31.0	20.0	26.0	12.0	64.1	29.4	31	16.1	53.0	12.6	37.91
Average SBP, mm Hg	113.0	112.9	120.5	137.0	130.0	130	118.0	113	132.0	117.0	132.2	130.1	126.4	119.0	127.0	122.0	119.65
BMI, kg/m2	-	23.6	23.5	26.5	25.4	25.4	23.4	23.8	25.1	26.3	24.8	25.5	26.0	24.8	26.0	24.6	24.5
TC, mg/dL	4.9	5.09	5.3	5.1	5.0	4.9	5.2	4.9	4.9	4.6	4.9	4.8	4.6	-	NR	NR	4.6
HDL-C, mg/dL	1.4	1.4	1.5	1.2	1.6	1.7	1.5	1.4	1.3	1.5	1.7	1.7	1.7	1.7	NR	NR	1.6
LDL-C, mg/dL	NR	3.0	3.4	-	2.9	2.8	3.2	3.0	3.1	2.6	2.7	2.6	2.4	2.5	NR	NR	2.6
TG, mg/dL	1.2			1.3	1.2	1.2	0.9	0.9	1.0	0.9	1.0	1.0	0.7	0.8	NR	NR	0.8
% on lipid meds	NR	NR	NR	7.7	27.2	27.2	NR	NR	NR	NR	9.8	41	-	35.4	34.6	12.6	NR
% on BP meds	NR	NR	NR	26.5	37.7	37.8	5.0	8.0	32.0	34.0	28.8	40.4		19.2	28.9	21.3	13.14

A1C = glycosylated hemoglobin; BMI = body mass index; BP = blood pressure; Deriv. = Derivation; HDL-C = high density lipoprotein cholesterol; LDL-C = low density lipoprotein cholesterol; NR = not reported; PA = Pennsylvania; PC = prospective cohort; Popn = Population; RC = retrospective cohort; SBP = systolic blood pressure; TC = total cholesterol; % = percentage; X-Sec = cross-sectional.

Performance of risk scores for prediction of cardiovascular disease

Together, the studies reported data on the performance of 12 CVD risk prediction models (7 T1DM-specific, 1 T2DM-specific, and 4 general population models) CVD risk prediction models: Steno T1DM risk engine,²⁸ Swedish T1DM risk score,²⁴ ASCVD Risk Equation, Fremantle T1DM Risk Equation,²⁷ silent myocardial infarction (SMI) Risk Model,³⁰ EURODIAB risk score,²⁵ Epidemiology of Diabetes Complication (EDC) risk model (also known as Pittsburgh CHD in T1DM risk model),²² Scottish Care Information (SCI)-Diabetes CVD risk score, United Kingdom Prospective Diabetes Study (UKPDS) Risk Engine,¹⁶ Framingham Risk Score (FRS), Joint Societies ASCVD risk score, QRISK3, and the ESC 2019 Risk Classification (Tables 1 and 2).22, 23, 24, 25, 26, 27, 28, 29, 30, 31 The variables included in each of the examined risk models and the corresponding outcomes assessed are shown in Supplemental Table 2. Overall, the CVD event rates ranged from 3 to 27 per 1000-person years across the studies (weighted average 14 per 1000-person years). Only a few studies performed a comprehensive assessment of model prediction including model calibration and model discrimination (Supplemental Table 2).

Table 2.

Outcome Measured, Event Rate, and Risk Prediction Parameters for Studies Reporting Discrimination And/or Calibration Measures

	Cohort	Sex/Deriv. vs Valid.	Male %	Average Age, y	Composite Outcome	Risk Model	Average Follow-Up, y	No. of Cases	Rate per 1,000 py	Calibration		E:0 Ratio	C-Statistic (95% CI) Result
										Measure	Result
Zgibor, 200622	EDC	All	49.9	27.3	Fatal CHD, nonfatal MI	UKPDS	11.2	36	6.8	H-L	324.1 (P < 0.0001)	NR	0.76
Zgibor, 200622	EDC	Male	NR	NR	Fatal CHD, nonfatal MI, Q waves	FRS	11.2	20	7.5	H-L	310.3 (P < 0.0001)	NR	0.77
Zgibor, 200622	EDC	Female	NR	NR	Fatal CHD, nonfatal MI, Q waves	FRS	11.2	16	6.1	H-L	6,873.9 (P < 0.0001)	NR	0.87
Zgibor, 201023	EDC Male	Male/Deriv.	100	27.1	CHD-related death, fatal/nonfatal MI, Q-waves on EKG	EDCa	8.0	26	10.7	H-L	NR	Graph	0.84
Zgibor, 201023	EDC Female	Female/Deriv.	0	27.3	CHD-related death, fatal/nonfatal MI, Q-waves on EKG	EDCa	8.0	20	8.36	H-L	NR	Graph	0.89
Zgibor, 201023	Eurodiab PCS Male	Male/Deriv.	100	32.1	CHD-related death, fatal/nonfatal MI, Q-waves on EKG	EDCa	7.0	26	3.248	H-L	NR	Graph	0.77
Zgibor, 201023	Eurodiab PCS Female	Female/Deriv.	0	32.2	CHD-related death, fatal/nonfatal MI, Q-waves on EKG	EDCa	7.0	27	3.252	H-L	NR		0.78
Davis, 201027	FDS	All	59.8	44	MI, CVA, Cardiac death, CVA death, sudden death	Fremantle T1D Risk Equation	5.0	6	10.5	H-L	(P = 0.67)	NR	0.84
Cederholm, 201124	SNDR - Der	All/Deriv.	55.6	44.6	CAD, CVA	Swedish T1D Risk Score	4.9	197	11.1	Modified H-L	0.1 (P = 0.90)	1.00	0.83
Cederholm, 201124	SNDR - Der	Male/Deriv.		NR	CAD, CVA	Swedish T1D Risk Score	4.9	120	NR	Modified H-L	0.6 (P = 0.90)	0.96	0.84
Cederholm, 201124	SNDR - Der	Female/Deriv.		NR	CAD, CVA	Swedish T1D Risk Score	4.9	77	NR	Modified H-L	3.6 (P = 0.30)	1.05	0.83
Cederholm, 201124	SNDR-Val	All/Valid.	NR	44.6	CAD, CVA	Swedish T1D Risk Score	3.9	201	NR	Modified H-L	0.2 (P = 0.90)	0.94	0.80
Cederholm, 201124	SNDR-Val	Male/Valid.		NR	CAD, CVA	Swedish T1D Risk Score	3.9	118	NR	Modified H-L	0.1 (P = 0.90)	0.94	0.77
Cederholm, 201124	SNDR-Val	Female/Valid.		NR	CAD, CVA	Swedish T1D Risk Score	3.9	83	NR	Modified H-L	0.1 (P = 0.90)	0.93	0.83
Soedamah, 201425	Eurodiab-PCS	All/Deriv.	52	30.3	CHD/CVA/ESRD/Amputation/Blindness/All-cause Death	EURODIAB	7.4	95	6.51	calibration plots	NR	NR	0.74
Soedamah, 201425	EDC-O	All/Valid.	49	28	CHD/CVA/ESRD/Amputation/Blindness/All-cause Death/nonfatal MI/major Q-waves/fatal and nonfatal CVA	EURODIAB	8.1	98	21.83	calibration plots	NR	NR	0.79
Soedamah, 201425	FinnDiane	All/Valid.	51	37.3	CHD, CVA, ESRD, Amputation, Blindness, All-cause Death	EURODIAB	7.5	315	14.0	calibration plots	NR	NR	0.82
Soedamah, 201425	CACTI	All/Valid.	45	36.3	CHD, CVA, ESRD, Amputation, Blindness, All-cause Death	EURODIAB	7.3	42	9.9	calibration plots	NR	NR	0.73
Vistisen, 201628	Steno	All/Deriv.	54	42.2	CHD, CVA, PVD, HF	Steno T1D Risk Score	6.8	793	27.2	H-L	12.14 (P = 0.136)	NR	0.826 (0.807-0.845)
Vistisen, 201628	Funen	All/Valid.	57.7	44.2	CHD, CVA, PVD, HF	Steno T1D Risk Score	6.6	243	17.8	H-L	10.9 (P = 0.207)	NR	0.803 (0.767-0.839)
Vistisen, 201628	Steno	All	54	42.2	CHD, CVA, PVD, HF	Swedish T1D Risk Score	6.8	793	27.2	H-L	430.4 (P < 0.001)	NR	0.794 (0.772-0.816)
Vistisen, 2016 28	Funen	All	57.7	44.2	CHD, CVA, PVD, HF	Swedish T1D Risk Score	6.6	243	17.8	H-L	206.9 (P < 0.001)	NR	0.78 (0.745-0.815)
Vistisen, 201628	Steno	All	54	42.2	CHD, CVA, PVD, HF	UKPDS Risk Engine	6.8	793	27.2	H-L	711.8 (P < 0.001)	NR	0.766 (0.743-0.789)
Vistisen, 201628	Funen	All	57.7	44.2	CHD, CVA, PVD, HF	UKPDS Risk Engine	6.6	243	17.8	H-L	210.9 (P < 0.001)	NR	0.737 (0.699-0.775)
Vistisen, 201628	Steno	All	54	42.2	CHD, CVA, PVD, HF	ASCVD Risk Equation	6.8	793	27.2	H-L	402.5 (P < 0.001)	NR	0.748 (0.724-0.771)
Vistisen, 201628	Funen	All	57.7	44.2	CHD, CVA, PVD, HF	ASCVD Risk Equation	6.6	243	17.8	H-L	130.3 (P < 0.001)	NR	0.748 (0.713-0.789)
Llaurado, 201729	NR	All	50	50.1	SMI via perfusion stress	SMI Risk Model	NA	10	NA	NR	NR	NR	0.833 (0.692-0.974)
Llaurado, 201729	NR	All	50	50.1	SMI via perfusion stress	FRS	NA	10	NA	NR	NR	NR	0.688 (0.545-0.83)
Llaurado, 201729	NR	All	50	50.1	SMI via perfusion stress	UKPDS Risk Engine	NA	10	NA	NR	NR	NR	0.559 (0.424-0.693)
Llaurado, 201729	NR	All	50	50.1	SMI via perfusion stress	EDCa	NA	10	NA	NR	NR	NR	0.558 (0.352-0.763)
Boscari, 202026	NR	All	34.5	43	Fatal and nonfatal events of IHD, ischemic stroke, HF, and PAD	Steno T1D Risk Score	4.7	3	2.9	NR	NR		0.95
Mcgurnaghan, 202130	SCI-Diabetes	All	57	35	MI/Stroke/UA/TIA/PVD or CAD/CVD/PAD revasc or major amputation or ACS	SCI-Diabetes CVD Risk Score	10	2,790	14	H-L	(P = 0.70)	NR	0.82 (0.81-0.83)
Mcgurnaghan, 202130	SCI-Diabetes	All	57	35	MI/Stroke/UA/TIA/PVD or CAD/CVD/PAD/ACS	Steno T1D Risk Score	10	2,790	14	H-L	NR	1.27	0.82 (0.81-0.83)
Mcgurnaghan, 202130	SCI-Diabetes	All	57	35	MI/Stroke/UA/TIA/PVD or CAD/CVD/PAD/ACS	QRISK3	10	2,790	14	H-L	NR	0.72	0.75 (0.74-0.76)
Mcgurnaghan, 202130	SNDR	All	54.6	31.9	MI/Stroke/UA/TIA/PVD or CAD/CVD/PAD/ACS	SCI-Diabetes CVD Risk Score	8.6	3,262	12.88	H-L	NR	NR	0.85 (0.84-0.86)
Tecce, 202231	NR	All	50	30.8	MI, CABG, stenting, extra coronary bypass, stroke, PAD, major amputations,	Steno T1D Risk Score	8.5	24	NR	NR	NR	NR	NR
Tecce, 202231	NR	All	50	30.8	MI, CABG, stenting, extra coronary bypass, stroke, PAD, major amputations,	ESC 2019 Risk Classification	8.5	24	NR	NR	NR	NR	NR

AUC = area under receiver operator curve; CABG = coronary artery bypass graft; CAD = coronary heart disease, CHD = coronary heart disease; CVD = cardiovascular disease; CVA = cerebrovascular disease, ESRD = end stage renal disease; IHD = ischemic heart disease; H-L = Hosmer–Lemeshow test of goodness of fit; MI = myocardial infarction; NA = not available; NR = not reported; PAD = peripheral arterial disease; PCI = percutaneous coronary intervention; SMI = silent myocardial infarction; TIA = transient ischemic attack.

^aEDC also known as Pittsburgh CHD in T1DM risk model.

Generally, the models performed moderately or excellently in discriminating,¹⁹ with most studies reporting a C statistic (80% the reported estimates) that ranged between 0.73 and 0.85 (Central Illustration, Table 2, Figure 1). The discrimination ability of the T1DM-specific risk models was generally higher than that of the T2DM and general population risk models (Central Illustration, Figures 1 and 2). Specifically, the pooled C-index for the Steno T1DM, Swedish T1DM, EURODIAB, and UKPDS models were as follows: 0.82 (95% CI: 0.81-0.83), 0.80 (95% CI: 0.78-0.82), 0.80 (95% CI: 0.77-0.83), and 0.73 (95% CI: 0.68-0.78), respectively. The pooled C-index of all T1DM-specific risk models (C-statistic: 0.81 [0.80-0.83]) was higher than the pooled C-index of all risk models developed in the general population or among people with T2DM (C-statistic: vs 0.75 [0.74-0.76]) (Figure 2). The sensitivity analyses that included are: 1) pooling the C-statistics using analytical weight based on the number of CVD cases observed in each study (Supplemental Figure 2); or 2) performing a logit transforming the C-statistic measures before pooling them across the studies (Supplemental Figure 3), yielded similar results.

Central Illustration.

Prediction of Cardiovascular Disease Among Individuals With Type 1 Diabetes

Figure 1.

Discrimination of Cardiovascular Disease Risk Scores in People With Type 1 Diabetes

Figure 2.

Discrimination of Cardiovascular Disease Risk Scores in People With T1DM by Model Type (Developed in the General Population/T2DM Individuals vs T1DM-Specific Populations)

In terms of model calibration, the prediction models developed in T1DM cohorts showed good model fit with closely matching observed and expected tests and a nonsignificant H-L test. On the other hand, UKPDS Risk Engine and the general population models were poorly calibrated with actual event rates (observed, O) being significantly higher than predicted event rates (expected, E), with O/E >1 and highly significant H-L test with large chi-square values (Table 2).

In terms of model validation, the authors of Steno T1D Risk score, Swedish T1DM Risk Score, EURODIAB, EDC, and SCI-Diabetes based risk model reported validation studies based on split sample (Swedish T1DM Risk Score) or external validation in one or more T1DM cohorts other than the derivation cohort, which gave comparable results (Table 2).

Data on subgroup analyses of model performance by sex were limited by the number of studies reporting these values but suggested these may also be important factors to consider (Table 2). None of the studies performed an assessment of the comparative performance of several models in the same population with T1DM, using reclassification indices.

The evaluation of the quality of studies in terms of risk of bias using the Newcastle-Ottawa Scale showed that the studies were overall low risk for bias (Supplemental Table 3). None of the examined CVD risk score was studied for its effect on patient management and outcomes in T1DM.

Discussion

In a meta-analysis of observational studies assessing the performance of CVD risk scores among individuals with T1DM, including 11 separate studies involving 73,664 young, predominantly male and White participants, we found that most of the 12 CVD risk scores evaluated had a moderate to excellent discriminatory performance. The models specifically developed in people with T1DM tended to have a higher discriminative ability (which is to say, was more likely to correctly identify individuals who do and do not have a CVD event) and more importantly exhibited a better calibration (ie the ability to accurately predict the risk) than the risk scores initially developed among people with T2DM (eg; UKPDS Risk Engine) or the general population.

That scores for T2DM or the general population exhibited poorer performance suggests that recommendations for managing CVD risk among those with T1DM should not be based on scores developed from individuals with T2DM, as is currently the case.¹⁷ The inappropriate reliance of current guidelines for CVD risk on studies conducted in individuals with T2DM is partially explained by the lack of clinical trial evidence on the CVD benefits of lowering CVD risk factors other than HbA_1C³² and LDL-cholesterol³³ in individuals with T1DM.³⁴ Importantly, these guidelines do not seem to account for the steep gradient of risk associated with most risk factors, even in the seemingly “normal” range (for example; blood pressure³⁵ and triglycerides³⁶) among individuals with T1DM. This suggests that the targets or thresholds for interventions should perhaps be lower than those used in individuals with T2DM or the general population. Using T1DM-specific risk scores can help address these issues, as this could help capture the totality of the risk associated with all risk factors and thus increase the number of high-risk individuals who will benefit from therapies. The results of this rigorous systematic review and meta-analysis on the performance of CVD risk prediction models among individuals with T1DM provides clinicians with much needed data to make informed decisions in the primary prevention of CVD in their T1DM patients.

A limited number of studies have assessed CVD risk prediction score among individuals with T1DM models. The risk scores that were specifically developed among T1DM tended to predict CV risk well, while the ones that were not initially or specifically developed for those with T1DM-specific underpredicted cardiovascular risk. These findings are consistent with current understanding that CVD risk associated with T1DM is driven by factors that are different from that of those in action in people with T2DM or in the general population besides. Some of the T1DM-specific factors may not be adequately captured by general population risk models. Indeed, people with T1DM and T2DM represent 2 very different phenotypes with respect to age at onset, diabetes duration, onset of kidney disease, and lifetime glycemic load.⁴ Other T1DM-specific factors include for example immunological factors.³⁷ Our study is the first-of-its-kind to synthesize available data on the performance of CVD risk scores in people with T1DM, yielding the most comprehensive and largest assessment on this topic to date. The review provided information on 12 different risk prediction scores, which is more than what has been previously reported by any prior single study. Indeed, our systematic review examined scores across different populations including both men and women across age groups, which improved the statistical power to detect smaller effects.

Our results have important clinical implications as the risk of CVD among T1DM is generally understated or overlooked.9, 10, 11, 12 The findings of a good performance of the T1DM-specific risks score suggest that these models may be used to guide clinicians and health policy makers in assessing CVD risk in people with T1DM. Our results will help to raise awareness about elevated CVD risk among people with T1DM and the need for intensifying primary prevention strategies, as well as stimulate future studies on risk prediction models in this high-risk population. There is a need for an analysis of the effect of the use of the examined CVD risk scores on patient outcomes, a neglected but highly important area. Thus far, no study has assessed the impact of applying the examined CVD prediction model in routine clinical management of individuals with T1DM and ultimately on CVD outcomes. Such an evaluation will facilitate an effective use of these CVD risk score for managing T1DM. Indeed, the current recommendations for managing CVD risk among individuals with T1DM do not incorporate any risk assessment tool.^13,17 An extensive validation of the existing T1DM-specific risk models will facilitate their incorporation in clinical guidelines.

Study limitations

The limitations of the study merit further consideration. First, the diagnosis of T1DM was based on physician-reported diagnosis from the medical records and did not include auto-antibody or C-peptide assessment, which are more specific than sensitive. However, prior studies have shown that identifying T1DM using administrative records is valid.³⁸ Second, no data existed on the calibration of the various models, the comparative performance between different models, and thus the reclassification ability of the models. Few studies comprehensive assessed the model’s prediction ability, including both model calibration and model discrimination. Third, there were few studies for each individual risk prediction model compiled in the analysis. Fourth, there was substantial heterogeneity across the studies, with only a few studies contributing data for each risk score, limiting the interpretation of the findings. Most risk scores did not include data on time during which glucose levels remained in range (time-in-range), imaging parameters, lifestyle factors (physical activity or diet), or on biomarkers. Another important predictor of cardiovascular ouctomes that was not always included in the T1DM risk scores is the renal function. This consideration is important, especially in the context of emerging cardiometabolic therapies that are also protective of the kidney. Only limited data were available for non-White populations. For some of the studies, we had to impute the standard error of the estimates of performance of the risk prediction models. The number and design of the included studies limited our ability to conduct relevant subgroups analyses by sex, race/ethnicity, presence or absence of relevant comorbidities.

The limitations of existing studies would need to be addressed in future studies, which would use large-scale data on T1DM, include a more racial or ethnically-mixed populations, simultaneously assess a comprehensive set of CVD risk prediction models, use a larger set of risk prediction measures, and perform head-to-head comparison of the models’ ability to improve CVD risk assessment in T1DM individuals. Performing more external validation studies will also prove useful for models developed in small cohorts. The current evidence could be complemented by modelling studies of CVD outcomes that leverage data from studies such as Diabetes Interventions and Complications Study (DCCT/EDIC) that has an interventional component.^39,40

Conclusions

Our systematic review and meta-analysis included CVD prediction models for risk estimation among people with T1DM, which had a moderate to excellent discrimination in their ability to predict higher versus lower CVD risk. In particular, and perhaps as expected, models developed specifically in cohorts comprised exclusively of individuals with T1DM had better performance metrics. Further studies are required that address whether implementing these models into clinical care improves the quality of care and delays or prevents cardiovascular disease.

Perspectives.

COMPETENCY IN MEDICAL KNOWLEDGE: The performance of CVD disease risk prediction models (T1DM-specific or not) among people with T1DM is unknown. In a systematic review and meta-analysis of 11 observational studies involving 73,664 individuals with T1DM and 12 CVD risk scores, most CVD risk scores had a moderate to excellent discriminative performance, with a better discriminative ability for T1DM-specific scores versus other populations risk scores. T1DM-specific models predicted CVD events with greater accuracy.

TRANSLATIONAL OUTLOOK: The results of this study support the potential utility of CVD risk scores in routine clinical practice for preventing CVD in people with T1DM.

Funding support and author disclosure

Dr Tcheugui was supported by NIH/NHLBI grant K23 HL153774. Dr Erqou is supported by Department of Veterans Affairs, Providence VA Medical Center and Lifespan Cardiovascular Institute. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.