. 2019 Aug 30;9(8):e025579. doi: 10.1136/bmjopen-2018-025579

Table 3.

Characteristics of prediction models when stroke is reported as a part of composite CV outcome and performance measure (C-statistic) is presented for the composite CV outcome

Study	Location	Outcome	No of predictors	Age	Gender	Events (n)/total participants (N)	Duration of follow-up	Modelling Method	Calibration	Discrimination (with CI)	External Validation
Brownrigg et al ¹⁵	England	CVD events (non-fatal MI, coronary revascularisation, congestive cardiac failure, transient ischaemic attack and stroke)	6 (age, sBP, smoking status, LDL-C and HDL-C and peripheral neuropathy)	Mean age of 63.8 years	Both male and female	399/13 043	Total 2.5 years	Probability weighted Cox regression	χ²=121.2, p<0.001	C-statistic=0.661 (0.636 to 0.686) (with PN)	No
Khalili et al ⁵⁰	Iran	CVD events (deﬁnite MI, probable MI, unstable angina, angiographic-proven CHD, stroke, death from CVD)	4 (BMI, waist circumference, WHR, and waist-to-height ratio)	Mean age 55.7 years (male), 52.7 years (female)	Both male and female	188/1010	Median follow-up 8.4 years	Cox proportional hazard model	NR	C-statistic=0.64 (0.58 to 0.70) (for diabetic men with WHR, model 2) and C-statistic=0.70 (0.65 to 0.75) (for diabetic women with WHR, model 2)	No
Cederholm et al ⁵¹	Sweden	Fatal or non-fatal CVD (CHD or stroke, whichever came first)	9 (A1C, age at the onset of diabetes, diabetes duration, sex, BMI, smoking, sBP, antihypertensive drugs and lipid-reducing drugs)	18–70 years	Both male and female	1482/11 646	Mean follow-up 5.64 years	Cox regression	HL test: χ²=4.29 (p=0.83) and the ratio of observed to predicted survival rates=0.999. Excellent calibration	C-statistic=0.70	No
Davis et al ³³	Australia	CVD (hospitalisation for/with MI or stroke, and death from cardiac or cerebrovascular causes or sudden death)	7 (age, sex, prior CVD, ln (urinary albumin : creatinine ratio), lnHbA1c, ln(HDL-C), Southern European ethnic background and aboriginality)	Mean age 64.1 (38.7–83.7) years	Both male and female	185/1240	Mean follow-up 4.5 years	Cox proportional hazards model	HLˆC test, p=0.74	AUC=0.80, p<0.001	Yes
Kengne et al ³⁴	20 Countries (Asia, Australasia, Europe and Canada)	CVD (fatal or non-fatal MI or stroke or CV death)	10 (age at diagnosis, known duration of diabetes, sex, pulse pressure, treated hypertension, atrial fibrillation, retinopathy, HbA1c, log of urinary albumin/creatinine ratio and non-HDL-C at baseline)	Mean age 65.8 (6.3) years	Both male and female	473/7168	4.5 years	Cox regression model	HL test: p=0.76 (ADVANCE cohort)	AUC=0.702 (0.676 to 0.728) (ADVANCE cohort)	Yes
Ofstad et al ²⁵	Norway	Death or first CV event (MI, stroke or hospitalisation for unstable angina pectoris)	11 (age, gender, known CVD, dB, microalbuminuria, serum levels of HDL-C and creatinine); novel risk markers: (IL-6, log Activin A, E/Em, pathol recovery loop)	Mean age 58.5±10.0 (SD) years	Both male and female	36/132	8.6±2.1 years	Cox proportional hazard model	NR	C-statistic: STD model: 0.794; STD + IL-6 model: 0.913; STD + log Activin A model: 0.859; STD + IL-6 + log Activin A model: 0.923; STD + E/Em + pathol recovery loop model: 0.891	No
Looker et al ⁵²	Five cohorts from Europe	CVD (acute CHD or an ischaemic stroke)	14 (age, sex, smoking, sBP and dBP, LDL-C, HDL-C, triacylglycerol, diabetes duration, HbA1C, BMI, height, (eGFR), cohort and current medication (including antihypertensive agents, aspirin, lipid-lowering agents and insulin therapy)). +6 Biomarkers (NT-proBNP apoCIII hsTnT IL-6 sRAGE IL-15)	Median age 68.4 years (controls) and 68.8 years (cases)	Both male and female	1123/2310	Median follow-up 3.2 years for cases and 6.5 years for controls	Forward selection using logistic regression	NR	AUROC=0.72 (full clinical covariate set plus forward selection biomarkers)	No
Mukamal et al ³²	USA	MI, stroke, CV death	7 in basic model (Age, smoking, sBP, total and HDL-C, creatinine and the use of glucose-lowering agents)	Mean age 72.6 years for female and 73.0 years for male	Both male and female	265/782	10 years	Cox proportional hazard model	Basic model: HL p=0.25; basic model+CRP: HL p=0.87; Basic Model+CRP + (ABI, internal carotid wall thickness, ECG left ventricular hypertrophy): HL p=0.65	Basic model: C -statistic=0.64; Basic model+CRP: C- statistic=0.64; Basic model+CRP + (ABI, internal carotid wall thickness, ECG left ventricular hypertrophy): C-statistic=0.68	Yes
Paynter et al ⁵³	USA	MI, ischaemic stroke, coronary revascularisation or CV death	8 in different models (age, sBP, total cholesterol, HDL-C, smoking, CRP, parental history of premature MI and HbA1c)	Median age 55 years for female and 67.8 years for male	Both male and female	125/685 (women); 170/563 (men)	Median follow-up 10.2 years (women); median follow-up 11.8 years (men)	Cox proportional hazards model	NR	C-statistic of model with HbA1c=0.692 (ATP III) and=0.697 (RRS) for women; C-statistic of model with HbA1c=0.602 (ATP III) and=0.605 (RRS) for men.	No
Price et al ⁵⁴	Scotland	All CV events (fatal and non-fatal MI, angina, fatal IHD, fatal and non-fatal stroke and TIA)	18 (age, sex, baseline CVD status, duration, diabetes treatment, lipid-lowering drugs, BP-lowering drugs, smoking status, BMI, sBP, dBP, HbA1c, HDL-C, total cholesterol, eGFR, microalbuminuria and social status +NT-proBNP) (model D)	60–75 years	Both male and female	112/1066	4 years	Cox proportional hazards model	NR	C-statistic=0.748 (0.691 to 0.805) (model D)	No
Selby et al ⁵⁵	USA	Macrovascular and microvascular complications (MI, other ischaemic heart disease, congestive heart failure, cerebrovascular accident, etc)	16 (outpatient diagnoses, inpatient events, age, antihypertensives, serum creatinine, diabetes treatment, mean HbA1c, albuminuria, primary care visits, outpatient diagnoses of obesity, outpatient ID diagnoses, mean total cholesterol, self-report of neuropathy, education, type of diabetes, sex)	Mean age of 60.8 years	Both male and female	1997/28 838	1 year	Logistic regression model	NR	AUC=0.64 (full model)	No
Zethelius et al ³⁵	Sweden	Fatal/non-fatal CVD (the composite of CHD or stroke)	12 (onset age of diabetes, diabetes duration, total-cholesterol-to-HDL-C ratio, HbA1c, sBP, BMI, males sex, smoker, microalbuminuria, macroalbuminuria, atrial fibrillation, previous CVD)	30–74 years	Both male and female	2488/24 288	Mean follow-up of 4.8 years	Cox proportional hazard model	Modified HL χ² statistic=0.13 (p=0.9)	C-statistic=0.71	No
Alrawahi et al ⁴¹	Oman	First fatal or non-fatal CHD, stroke, or PAD	7 (age, diabetes duration, HbA1c, total cholesterol, albuminuria, hypertension, BMI)	54.5±11.4 years	Both male and female	192/2039	Mean follow-up of 5.3 years	Cox regression model	NR	NR	No
Zarkogianni et al ⁵⁶	Greece	Fatal or non-fatal CVD: stroke and CHD	16 (age, diabetes duration, BMI, glycosylated haemoglobin, pulse pressure, fasting glucose, total cholesterol, triglycerides, HDL-C, smoking habit, sex, hypertension, lipid-lowering therapy, aspirin, insulin therapy, parental history of diabetes)	58.56±10.70 years	Both male and female	41/560	5-year follow-up	Machine learning: HWNNs and SOMs	Brier score: 0.08±0.01 (HWNN-based ensemble 4); 0.07±0.01 (SOM-based ensemble 4); 0.007±0.02 (hybrid ensemble)	AUC=0.6764±0.1509 (HWNN-based ensemble 4); AUC=0.7054±0.1372 (SOM-based ensemble 4); AUC=0.7148±0.1573 (hybrid ensemble)	No
Price et al ⁵⁷	Scotland	Fatal or non-fatal MI or stroke, angina, fatal IHD, TIA, coronary intervention	13 (age, sex, smoking, atrial fibrillation, CKD, arthritis, hypertension, BMI, SBP, total HDL-C, social status, baseline CVD status, lipid-lowering medication) in basic model + (ABI, hs-cTnT, GGT, proBNP, g) in full model	60–75 years	Both male and female	205/1066	8 years	Binary logistic regression	HL p=0.97 (basic model); HL p=0.39 (full model). Well calibrated	C-statistic=0.722 (0.681 to 0.763) (basic model); C-statistic=0.74 (0.699 to 0.781) (full model)	No
Wan et al ⁵⁸	China	IHD, MI, coronary death and sudden death, heart failure, fatal and non-fatal stroke	13 (age, eGFR, total cholesterol/HDL-C ratio, urine ACR, smoker, duration of diabetes mellitus, sBP, HbA1c, anti-hypertensive drugs used, dBP, BMI, insulin used, anti-glucose oral drugs used)	18–79 years	Both male and female	Events (n) NR/137 935	Median follow-up of 5 years	Cox proportional hazard regression	Calibration plots: good calibration	Harrell’s C-statistic Male: 0.705 (0.693 to 0.716) (model 1), 0.689 (0.678 to 0.701) (model 2); Female: 0.719 (0.707 to 0.731)(model 1), 0.708 (0.696 to 0.719) (model 2)	No
Young et al ⁵⁹	USA	MACE: non-fatal MI, non-fatal stroke and CVD-related death; MACE-plus: any MACE, hospitalisation for unstable angina, or hospitalisation for congestive heart failure; CVD-related death	12 (age, gender, type of insurance, race, region, diabetes-related hospitalisations, prior CVD diagnoses, chronic pulmonary disease, use of antihypertensive drugs, use of antihyperglycaemic drugs, HbA1c, urine ACR)	50 years or older	Both male and female	13 856 (MACE), 20 100 (MACE-plus)/181 619	Median duration of the at-risk period: 12 months (primary prevention population) and 11 months (secondary prevention population)	Logistic regression	NR	C-statistic=0.70 (MACE); C-statistic=0.72 (MACE-plus); C-statistic=0.77 (CVD-related death)	No
van der Leeuw et al ⁶⁰	The Netherlands	Major CV events (MI, stroke and vascular death)	12 (age at diabetes diagnosis, duration of diagnosed diabetes, sex, smoking, HbA1c, sBP, total cholesterol/ HDL-C ratio, previous CV event, urinary ACR or eGFR) in base model+NT-proBNP, osteopontin, and MMP-3 in multimarker model	Mean age 59±10 years (SMART), 58±7 (EPIC-NL)	Both male and female	248 (SMART), 134 (EPIC-NL)/1002 (SMART), 218 (EPIC-NL)	Median follow-up 9.2 years in SMART and 11.3 years in EPIC-NL	Cox proportional hazard model	Calibration plots	Base model: C-statistic=0.70 (0.67 to 0.74) (SMART), C-statistic=0.69 (0.64 to 0.74) (EPIC-NL); Multimarker model: C-statistic=0.73 (0.68 to 0.79) (SMART), C-statistic=0.72 (0.64 to 0.77) (EPIC-NL)	No
Alshehry et al ⁶¹	20 countries from Asia, Australasia, Europe and North America	Non-fatal MI, non-fatal stroke, and CV death	14 (age, sex, BMI, SBP, glycohaemoglobin, HDL-C, eGFR, diabetes duration, CRP, history of macrovascular disease, history of heart failure, use of antihypertensive medication, use of antiplatelet medication, exercise) in base model + 7 lipid species	Mean age 67 years	Both male and female	698/3779	Median follow-up of 5 years	Weighted Cox regression	NR	Base model: C-statistic=0.68 (0.678 to 0.682); base model + 7 lipid species: C-statistic=0.70 (0.698 to 0.702)	No
Woodward et al The AD-ON Risk Score⁶²	20 countries from Asia, Australasia, Europe and North America	Non-fatal MI, non-fatal stroke or death from any CV cause, renal death or requirement for renal replacement therapy or renal transplantation	13 (age, sex, sBP with and without use of antihypertensives, duration of diabetes, HbA1c, urinary ACR, eGFR and its square, age at completion of formal education, exercise, history of diabetic retinopathy and current or previous atrial fibrillation)	Mean age of 65.8 years	Both male and female	1145/7301	Median follow-up of 9.9 years	Cox regression model	Calibration plots and HL test (p=0.13). Excellent calibration	C-statistic=0.668 (0.651 to 0.685)	No
Parrinello et al ⁶³	USA	Incident CHD, stroke, heart failure, CKD, lower extremity amputation or peripheral vascular bypass	18 (age, sex, race, education, smoking status, alcohol consumption, physical activity, family history of CVD, glucose-lowering medication use, antihypertensive medication use, cholesterol-lowering medication use, recent onset of diabetes, BMI, LDL-C, HDL-C, triglycerides, sBP, HbA1c) + 12 biomarkers	Mean age of 58.1 years	Both male and female	141 (CVD events)/654	Maximum follow-up of 10 years	Fine and Gray model	Calibration plots: well calibrated	C-statistic=0.667 (0.64 to 0.70) (model 1); C-statistic=0.683 (0.65 to 0.71) (model 2); C-statistic=0.694 (0.66 to 0.72) (model 3); C-statistic=0.716 (0.69 to 0.74) (model 4)	No
Colombo et al ⁶⁴	UK and Ireland	Acute CHD (MI, unstable angina, revascularisation or acute CHD death), fatal or non-fatal stroke	8 (age, sex, SBP, total cholesterol, HDL-C, smoking status, apoCIII and NT-proBNP)	Median age of 62.9 years	Both male and female	144/2105	Maximum follow-up of 5 years	Cox proportional hazard model	NR	AUROC=0.661 (0.615 to 0.706) (Framingham covariates alone); AUROC=0.745 (0.701 to 0.789) (full model with additional biomarkers)	No
Elley et al NZ DCS⁴⁰	New Zealand	Fatal or non-fatal CVD event (ischaemic heart disease, cerebrovascular accident/transient ischaemic attack, PAD)	9 (age at diagnosis, diabetes duration, sex, sBP, smoking status, total cholesterol: HDL ratio, ethnicity, glycated HbA1C), urine ACR)	Median age of 59 years	Both male and female	6479/36 127	Median follow-up of 3.9 years	Cox proportional hazards regression models	Calibration plot	AUROC=0.68 (0.67 to 0.70) (CVD)	Yes

ABI, ankle–brachial index; AD-ON, Action in Diabetes and Vascular Disease: Preterax and Diamicron Modified Release Controlled Evaluation-Observational;ADVANCE, Action in Diabetes and Vascular Disease: Preterax and Diamicron Modified Release Controlled Evaluation; apoCIII, Apolipoprotein C-III; ATP, Adult Treatment Panel; AUC, area under the curve; AUROC, area under the receiver operating characteristic curve; BMI, body mass index; CHD, coronary heart disease; CKD, chronic kidney disease; CRP, C reactive protein; CV, cardiovascular; CVD, cardiovascular disease; dBP, diastolic blood pressure; eGFR, estimated glomerular filtration rate; EPIC-NL, European Prospective Investigation into Cancer and Nurition Netherlands;GGT, gamma-glutamyl transferase; HbA1C, haemoglobin A1C; HDL, high-density lipoprotein; HDL-C, high-density lipoprotein cholesterol; HL, Hosmer-Lemeshow; HL^∧C, Hosmer-Lemeshow C-test;hs-cTnT, high-sensitivity cardiac troponin T; HWNNs, hybrid wavelet neural networks; ID, infectious disease; IHD, ischaemic heart disease;IL-6, interleukin 6; LDL-C, low-density lipoprotein cholesterol; MACE, major adverse cardiovascular event; MI, myocardial infarction;MMP-3, matrix metalloproteinase-3; NR, not reported;NT-proBNP, N-terminal pro b-type Natriuretic Peptide; NZ DCS, New Zealand Diabetes Cohort Study; PAD, peripheral artery disease; PN, peripheral neuropathy; RRS, Reynolds risk score; sBP, systolic blood pressure; SMART, second manifestations of arterial disease; SOMs, self-organising maps; STD, standard; TIA, transient ischaemic attack; WHR, waist-to-hip ratio.