Abstract:
Background
Secondary Manifestations of Arterial Disease (SMART) risk score–based projection of risk for cardiovascular (CV) events can help devise risk mitigation strategies.
Objectives
The objective of the study was to analyze the use of the SMART polyvascular disease score to compute the risk for major adverse CV events (MACE) in U.S. patients.
Methods
We accessed the Baylor Scott & White EPIC informatics and data warehouse to identify patients at their first outpatient cardiology evaluation between April 2014 and October 2023 to estimate up to 10-year risk of MACE, a composite of all-cause death, ischemic stroke, and nonfatal myocardial infarction (MI). Cox regression, accelerated failure time model, and survival analyses were used to develop and validate the SMART risk score.
Results
The study population of 259,250 patients (mean age 60.9 ± 15.2 years, 48.6% females) were divided into development (60%) and test (40%) cohorts; median follow-up 2.1 years (IQR: 0.54-4.4). The SMART risk score allowed accurate estimation of MACE. Patients in low (<10%), moderate (10%-<20%), high (20%-<30%), and very high (≥30%) SMART risk score groups had observed MACE events rates of 2.9%, 15.0%, 24.5%, and 56.5%, respectively, in the test cohort (P < 0.0001 for all intergroup comparisons). Most MACE events were all-cause death, with nonfatal myocardial infarction and stroke also being high, in the very high-risk group. The SMART score outperformed an established risk prediction model (TIMI Risk Score for Secondary Prevention [TRS2°P]; C-statistic = 0.811) in the test cohort.
Conclusions
The SMART polyvascular disease risk score can provide accurate estimation of up to 10-year risk of CV events and could be potentially leveraged to develop individualized risk mitigation strategies.
Key words: cardiovascular disease, cardiovascular events, risk score
Central Illustration
The Secondary Manifestations of Arterial Disease (SMART) risk score, developed and validated by Dorresteijn et al,1 was designed to estimate the risk of recurrent cardiovascular (CV) events in patients with pre-existing CV disease (CVD). The SMART risk prediction model is an established calculator to estimate the 10-year risk of nonfatal myocardial infarction (MI), ischemic stroke, or all-cause death, collectively termed as major adverse CV events or MACE from readily available clinical characteristics.1
Recently, our group published an application and validation of the SMART risk score in a cohort of U.S. Veterans.2 The study aimed to redefine CV risk assessment by shifting the focus from a binary category of secondary prevention to a continuous risk spectrum, using the SMART risk score.3 The overarching goal was to evaluate a structured risk assessment strategy to guide potential interventions for lowering the risk of recurrent CV events. To this end, the SMART risk score, which integrates 14 easily obtainable clinical and laboratory parameters, proved to be a highly suitable tool.1 The variables used in computing the SMART risk score include age, sex, smoking, time (in years) since CVD diagnoses to cardiology visit, and clinical parameters (average systolic blood pressure, history of diabetes mellitus [DM], coronary artery disease [CAD], cerebrovascular disease, abdominal aortic aneurysm [AAA], peripheral artery disease [PAD], high-density lipoprotein [HDL] cholesterol, total cholesterol [TC], estimated glomerular filtration rate [eGFR], and high-sensitivity C-reactive protein [hsCRP]).1,2
The aim of the current study was to apply the SMART risk score to an all-comer outpatient U.S. non-Veteran patient population to predict 10-year MACE, a composite of all-cause death, MI, and ischemic stroke.
Methods
Study population
The study included patients from the Baylor Scott & White Health (BSWH) system, which comprises 52 hospitals and over 1,300 sites of care. Between April 1, 2014, and October 31, 2023, a total of 3,461,702 patients visited the health system, as recorded in the EPIC electronic health records. We analyzed 302,689 adult patients (≥18 years) who were referred to cardiology for atherosclerotic CVD, including cerebrovascular disease, CAD, PAD, AAA, or polyvascular disease states. Patients without a recorded time since first CVD diagnosis were excluded. In addition, those with extreme outliers in continuous variables were removed. Figure 1 illustrates formation of the study cohort. The time to first MACE was assessed for patients who experienced a MACE event following their initial cardiology visit. Patients who did not experience a MACE event and lacked 10-year follow-up data were considered left censored. Patients who remained free of MACE for up to 10 years after their cardiology visit were considered right censored. The final study cohort included 259,250 patients. The data set was split into a development cohort (60%, n = 155,550) and a test cohort (40%, n = 103,700). This study was approved by the Baylor University Medical Center Institutional Review Board.
Figure 1.
Study Cohort
Describes development of the study cohort. BSWH = Baylor Scott & White Health; CVD = cardiovascular disease.
Key covariates of interest for SMART risk score
The present study was conducted to evaluate the performance and applicability of the SMART risk score in the BSWH cohort. The SMART risk score variables included in the model were age, sex, current smoking status, time in years since CVD diagnosis to the first cardiology visit, systolic blood pressure, history of DM, CAD, cerebrovascular disease, AAA, PAD, HDL, TC, eGFR, and hsCRP.1,2 Patient age was calculated as the difference between the initial cardiology visit and the date of birth. Sex, current smoking status, and systolic blood pressure was extracted at the date of the cardiology visit. Comorbidities like DM, CAD, cerebrovascular disease, AAA, PAD, and CVD were defined using International Classification of Diseases-9 and -10 and Current Procedure Terminology codes (Supplemental Appendix 1). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guideline adherence document is included as Supplemental Appendix 2. Patient’s time since CVD diagnosis is the first time the patient was diagnosed with CVD before the initial cardiology visits and was obtained from clinical records. Laboratory data were obtained within 365 days of the initial cardiology. All data were directly extracted via EPIC Clarity.
Outcome of interest
The primary outcome was MACE, a composite of all-cause death, ischemic stroke, and nonfatal MI, within 10 years of the first cardiology visit. MI or ischemic stroke occurring after the initial cardiology visit and before the recorded date of death (if any) were defined using International Classification of Diseases 9 and 10 and Current Procedure Terminology codes. All-cause death was identified from EPIC records, including in-hospital deaths and outpatient updates documented after follow-up calls or when clinics were informed. Outcomes were assessed over a 10-year period from the date of the first cardiology visit. A MACE event was considered for the earliest of the 3 outcomes. Censoring was applied at the earliest of the following events: the date of last documented follow-up or the completion of the 10-year follow-up window.
Statistical analysis
Descriptive analysis was performed on the overall cohort. Missing values were imputed separately for each data set after splitting. Initially, approximately 0.57% of patients had hsCRP values, and 3.22% had CRP values. Imputing hsCRP values was a multistep process. First, a linear regression model was used to estimate hsCRP from CRP values giving us a total of 3.79% hsCRP values. Second, the total 3.79% hsCRP values, systolic blood pressure, eGFR, TC, and HDL were imputed by identifying the extent of missing data assessed using Little’s Missing Completely at Random test (statistic = 29,929.66, P < 0.05), which suggested the data were likely missing at random.2,4 Multiple Imputation by Chained Equations was used, enhanced by predictive mean matching, ensuring plausible and consistent imputations for the variables.5,6
The Levene test was performed to assess the homogeneity of error variances. Continuous predictors were trimmed at the 1st and 99th percentiles to limit the impact of outliers. The Cox proportional hazards model was initially explored; however, repeated assessments revealed violations of the proportional hazard’s assumption across multiple covariates. Although, in the accelerated failure time (AFT) framework, the reported HRs reflect the extent to which covariates accelerate or decelerate the time to MACE.7,8 As a result, an AFT model assuming a Weibull distribution was employed to model time to MACE, providing a more appropriate framework for estimating the impact of covariates on event timing. The development data set was used to build the SMART risk model, which was then validated in the test cohort. We also estimated an AFT model excluding hsCRP to evaluate whether omission of hsCRP materially affected the parameter estimates of the remaining covariates.
SMART risk score model evaluation
Considering the differences in the primary cohort where SMART risk score was derived and the intended use population from BSWH system, we did not use the out-of-the box estimates for the model covariates. Instead, model optimization was performed in the randomly selected 60% study development cohort. For this, natural log transformations were applied to the variables “time since CVD diagnosis to cardiology visit" and “hsCRP," as they initially did not meet the linearity assumption. Maximum likelihood estimation was used to fit the AFT model. Patients who remained event-free for 10 years were right censored, whereas those with <10 years of follow-up were left censored. Model fit was evaluated using deviance residuals to identify potential outliers and assess overall model adequacy. Model performance was further assessed by examining the log-likelihood, likelihood ratio chi-square statistics, and Akaike Information Criterion across candidate models to guide model selection and identify the best-fitting approach.
The performance of the SMART risk score was evaluated in the 40% randomly selected test cohort. Discrimination was assessed using the concordance (C)-statistics, calculated separately for the development, test, and overall cohorts. Calibration was examined through calibration plots comparing predicted vs observed risks. The D’Agostino Nam test used to test the deviance of the predicted from the observed outcomes. Clinical significance of the deviation was accessed based on the calibration plot (Figure 2). Kaplan-Meier (K-M) survival estimates were calculated for the risk of MACE across the SMART risk score strata and compared using Log-rank, Breslow, and Tarone-Ware tests. Clinical utility was assessed using decision curve analysis, which quantified the net benefit of prediction-guided decision-making across a range of threshold probabilities. Comparative performance between the previously established TIMI Risk Score for Secondary Prevention (TRS2°P) score and the SMART risk score was examined at 10 years using decision curve analysis, Brier scores, area under the curve (AUC), and C-statistics.3 In addition, an alternative model excluding hsCRP was constructed to evaluate model performance in the absence of this biomarker. Subgroup analyses stratified by sex, race, and ethnicity were performed across the full cohort to assess the robustness and generalizability of the findings. All analyses were performed using SQL and R (R Core Team, v.4.2.2) (http://www.R-project.org) with relevant statistical packages.
Figure 2.
Uncalibrated Plot for the Test Cohort
MACE = major adverse cardiovascular event.
Results
Baseline characteristics of 259,250 patients included in this analysis were stratified into SMART risk score low-, moderate-, high-, and very high-risk groups as shown as Table 1. Nearly 48.6% patients included were women. The mean age was 60.9 ± 15.2 years, with a higher mean age in high- (69.9 ± 9.7) and very high-risk (72.2 ± 9.97) groups. Majority of the patients were White (82.2%), and African Americans constitute nearly 10.9%. The prevalence of DM, CAD, cerebrovascular disease, PAD, and AAA were expectedly higher in the high- and very high-risk groups. Mean values for TC, low-density lipoprotein, and eGFR were 173.4 ± 47.1 mg/dL, 102.5 ± 36.5 mg/dL, and 81.5 ± 25.5 mL/min/1.73 m2, respectively. The entire study cohort (n = 259,250) was randomly divided into development (n = 155,550) and test cohorts (n = 103,700) using random sampling without replacement. All baseline variables were statistically significantly different between each of the groups (low, moderate, and high risk) vs very high-risk group. Supplemental Tables 1 and 2 illustrate the baseline characteristics of the development and test cohorts divided into the 4 SMART risk groups. The median follow-up time was 2.1 years (IQR: 0.54-4.4 years).
Table 1.
Baseline Characteristics: SMART Risk Score Variables
| Overall (N = 259,250) |
Low Risk (n = 134,875) |
Moderate Risk (n = 44,416) |
High Risk (n = 23,746) |
Very High Risk (n = 56,213) |
|
|---|---|---|---|---|---|
| Age (y) | |||||
| Mean (SD) | 60.98 (15.27) | 52.45 (13.99) | 67.88 (10.90) | 69.96 (9.72) | 72.20 (9.97) |
| Median (Q1, Q3) | 62.88 (51.06, 72.32) | 53.25 (42.95, 63.03) | 67.66 (60.52, 75.87) | 71.02 (64.11, 76.77) | 73.03 (66.31, 79.58) |
| Sex | |||||
| Female | 126,016 (48.6%) | 75,905 (56.3%) | 18,657 (42.0%) | 8,378 (35.3%) | 23,076 (41.1%) |
| Male | 133,234 (51.4%) | 58,970 (43.7%) | 25,759 (58.0%) | 15,368 (64.7%) | 33,137 (58.9%) |
| Race | |||||
| White | 203,094 (82.2%) | 97,996 (77.9%) | 36,820 (85.9%) | 20,125 (86.9%) | 48,153 (87.3%) |
| African American | 26,897 (10.9%) | 16,808 (13.4%) | 3,749 (8.7%) | 1,803 (7.8%) | 4,537 (8.2%) |
| Asian | 10,343 (4.2%) | 6,769 (5.4%) | 1,396 (3.3%) | 732 (3.2%) | 1,446 (2.6%) |
| Other | 6,660 (2.7%) | 4,211 (3.3%) | 917 (2.1%) | 488 (2.1%) | 1,044 (1.9%) |
| Ethnicity | |||||
| Hispanic or Latino | 26,117 (10.5%) | 16,341 (12.9%) | 3,766 (8.8%) | 1,844 (8.0%) | 4,166 (7.5%) |
| Not Hispanic or Latino | 221,973 (89.5%) | 110,342 (87.1%) | 39,262 (91.2%) | 21,350 (92.0%) | 51,019 (92.5%) |
| Time since first diagnosis of CVD (y) | |||||
| Mean (SD) | 0.48 (1.28) | 0.39 (1.17) | 0.60 (1.47) | 0.50 (1.28) | 0.62 (1.36) |
| Median (Q1, Q3) | 0.00 (0.00, 0.10) | 0.00 (0.00, 0.10) | 0.00 (0.00, 0.20) | 0.00 (0.00, 0.10) | 0.00 (0.00, 0.40) |
| Diabetes | 57,777 (22.3%) | 10,773 (8.0%) | 11,124 (25.0%) | 9,143 (38.5%) | 26,737 (47.6%) |
| Coronary artery disease | 89,257 (34.4%) | 7,129 (5.3%) | 22,626 (50.9%) | 19,909 (83.8%) | 39,593 (70.4%) |
| Cerebrovascular disease | 38,455 (14.8%) | 20 (0.0%) | 326 (0.7%) | 759 (3.2%) | 37,350 (66.4%) |
| AAA | 10,471 (4.0%) | 2,729 (2.0%) | 1,890 (4.3%) | 1,418 (6.0%) | 4,434 (7.9%) |
| Peripheral vascular disease | 24,294 (9.4%) | 2,771 (2.1%) | 3,692 (8.3%) | 3,160 (13.3%) | 14,671 (26.1%) |
| Systolic BP | |||||
| Mean (SD) | 130.96 (15.37) | 129.89 (15.13) | 131.53 (15.44) | 131.62 (15.24) | 132.83 (15.71) |
| Median (Q1, Q3) | 129.76 (120.00, 140.00) | 128.50 (119.56, 139.00) | 130.00 (120.76, 140.70) | 130.00 (121.00, 140.82) | 131.40 (122.00, 142.46) |
| eGFR (mL/min/1.73 m2) | |||||
| Mean (SD) | 81.46 (25.47) | 85.77 (24.80) | 79.38 (25.07) | 77.67 (25.03) | 74.38 (25.52) |
| Median (Q1, Q3) | 83.13 (63.38, 99.00) | 87.67 (70.00, 102.50) | 81.00 (61.00, 97.00) | 79.50 (59.00, 95.00) | 75.13 (55.00, 92.67) |
| Total cholesterol (mg/dL) | |||||
| Mean (SD) | 173.39 (47.11) | 177.47 (46.48) | 171.96 (47.34) | 168.42 (47.29) | 166.85 (47.33) |
| Median (Q1, Q3) | 171.00 (142.00, 203.00) | 176.00 (146.00, 206.40) | 169.00 (140.00, 201.00) | 165.00 (136.50, 197.00) | 163.00 (135.00, 195.00) |
| LDL (mg/dL) | |||||
| Mean (SD) | 102.51 (36.53) | 111.22 (35.36) | 99.41 (36.52) | 94.56 (35.32) | 92.39 (35.35) |
| Median (Q1, Q3) | 99.00 (76.00, 125.00) | 109.00 (86.00, 133.00) | 95.00 (73.50, 120.00) | 89.00 (69.00, 115.00) | 87.00 (67.00, 112.00) |
| HDL (mg/dL) | |||||
| Mean (SD) | 45.17 (19.05) | 46.35 (18.87) | 45.02 (19.21) | 43.24 (19.21) | 43.28 (19.07) |
| Median (Q1, Q3) | 44.00 (35.00, 55.67) | 45.00 (36.00, 57.00) | 44.00 (34.50, 55.50) | 43.00 (33.00, 54.00) | 43.00 (33.00, 54.00) |
| Triglyceride (mg/dL) | |||||
| Mean (SD) | 137.88 (88.34) | 135.25 (87.86) | 138.75 (89.97) | 141.94 (91.48) | 140.35 (86.67) |
| Median (Q1, Q3) | 115.00 (82.50, 165.00) | 113.00 (80.00, 162.00) | 116.00 (83.00, 164.50) | 119.00 (85.00, 169.00) | 118.00 (85.00, 168.00) |
| CRP (mg/dL) | |||||
| Mean (SD) | 22.62 (43.72) | 18.21 (39.40) | 24.10 (44.23) | 25.10 (47.85) | 27.48 (47.38) |
| Median (Q1, Q3) | 6.12 (2.00, 18.45) | 4.90 (1.60, 14.20) | 6.90 (2.20, 19.55) | 7.08 (2.33, 18.90) | 7.70 (2.50, 27.10) |
| hsCRP (mg/L) | |||||
| Mean (SD) | 13.46 (19.11) | 12.80 (18.07) | 13.73 (19.69) | 14.29 (20.20) | 14.47 (20.46) |
| Median (Q1, Q3) | 7.12 (3.63, 14.20) | 6.89 (3.58, 13.54) | 7.20 (3.70, 14.39) | 7.51 (3.80, 14.98) | 7.53 (3.85, 15.37) |
| Hematocrit (%) | |||||
| Mean (SD) | 40.48 (5.03) | 41.21 (4.53) | 40.25 (5.28) | 39.88 (5.51) | 39.28 (5.38) |
| Median (Q1, Q3) | 40.90 (37.70, 43.80) | 41.40 (38.63, 44.15) | 40.75 (37.20, 43.80) | 40.42 (36.60, 43.60) | 39.80 (36.00, 42.95) |
| Lipoprotein (a) | |||||
| Mean (SD) | 73.15 (64.81) | 85.17 (69.95) | 70.08 (57.38) | 51.00 (50.46) | 60.78 (62.12) |
| Median (Q1, Q3) | 53.00 (18.00, 111.00) | 69.50 (23.00, 145.00) | 57.00 (22.00, 105.50) | 30.00 (10.00, 94.00) | 35.00 (14.00, 95.00) |
Lp(a) were not significantly different between moderate-, high-, vs very high-risk groups.
HDL, triglycerides, CRP, and hsCRP were not significantly different between high- vs very high-risk groups.
All other variables were statistically significantly different between the groups (low, moderate, and high risk) vs very high-risk group (P < 0.05).
AAA = abdominal aortic aneurysm; BP = blood pressure; CRP = high-sensitive C-reactive protein; CVD = cardiovascular disease; eGFR = estimated glomerular filtration rate; HDL = high-density lipoprotein; hsCRP = high-sensitive C-reactive protein; SMART = Secondary Manifestations of Arterial Disease; - = low-density lipoprotein.
The AFT model used to predict MACE based on SMART risk score variables is presented as Table 2. Table 3 summarizes model performance metrics, including measures of discrimination, concordance, and goodness-of-fit across the development, test, and overall cohorts.
Table 2.
Accelerated Failure Time Model: Model Coefficients and HRs With 95% CIs
| Estimate | SE | Z Statistic | P Value | HR (95% CI) | |
|---|---|---|---|---|---|
| Age | −0.041 | 0.001 | −48.326 | <0.001 | 1.028 (1.026-1.030) |
| Male | −0.205 | 0.018 | −11.110 | <0.001 | 1.148 (1.108-1.189) |
| Current smoking | −0.496 | 0.028 | −17.931 | <0.001 | 1.395 (1.321-1.474) |
| Systolic blood pressure | 0.0001 | 0.001 | −0.759 | 0.448 | 1.000 (0.998-1.002) |
| Diabetes mellitus | −0.577 | 0.019 | −30.768 | <0.001 | 1.474 (1.420-1.529) |
| Coronary artery disease | −0.929 | 0.021 | −44.815 | <0.001 | 1.867 (1.791-1.945) |
| Cerebrovascular disease | −2.418 | 0.022 | −111.179 | <0.001 | 5.076 (4.862-5.300) |
| Abdominal aortic aneurysm | −0.101 | 0.033 | −3.023 | 0.003 | 1.070 (1.003-1.142) |
| Peripheral vascular disease | −0.225 | 0.022 | −10.253 | <0.001 | 1.163 (1.114-1.214) |
| Ln time since diagnosis - cardiology visit (years) | −0.070 | 0.007 | −9.558 | <0.001 | 1.048 (1.034-1.063) |
| HDL | −0.001 | 0.001 | −1.723 | 0.085 | 1.001 (0.999-1.003) |
| Total cholesterol | 0.0001 | 0.0001 | 0.138 | 0.890 | 1.0001 (0.998-1.0012) |
| eGFR | 0.004 | 0.0001 | 12.129 | <0.001 | 0.997 (0.997-0.997) |
| Ln hsCRP | 0.007 | 0.009 | 0.736 | 0.462 | 0.995 (0.978-1.013) |
Scale = 1.49.
HRs were derived from the model’s time ratios and scale/shape parameters.
Ln = logarithmic value; other abbreviations as in Table 1.
Scale = 1.52; HR = exp((-estimate)/scale).
Table 3.
Model Performance, Concordance, and Fit Tests: Summary Statistics of Model Performance in the 3 Cohorts
| Dataset | N | Test | Estimate | df | P Value |
|---|---|---|---|---|---|
| Development cohort | 155,550 | Concordance = 0.812 (0.81, 0.814) | |||
| Log-rank test | 42,791 | 3 | <0.001 | ||
| Breslow test | 41,453 | 3 | <0.001 | ||
| Tarone-Ware test | 40,627 | 3 | <0.001 | ||
| Test cohort | 103,700 | Concordance = 0.811 (0.809, 0.813) | <0.001 | ||
| Log-rank test | 22,517 | 3 | <0.001 | ||
| Breslow test | 21,832 | 3 | <0.001 | ||
| Tarone-Ware test | 21,405 | 3 | <0.001 | ||
| Whole cohort | 259,250 | Concordance = 0.811 (0.809, 0.813) | |||
| Log-rank test | 70,661 | 3 | <0.001 | ||
| Breslow test | 68,502 | 3 | <0.001 | ||
| Tarone-Ware test | 67,161 | 3 | <0.001 |
The model demonstrated strong discriminatory ability, with a C-statistic of 0.811 in the overall cohort. Exclusion of hsCRP from the AFT model did not meaningfully alter performance metrics, including Akaike Information Criterion, concordance, or log-likelihood values, across any data set (development, test, or combined), as shown in Supplemental Table 3. Both versions of the model, with and without hsCRP exhibited nearly identical performance across all evaluated criteria. Details of the computational formulas used to estimate 10-year CVD risk under the Weibull AFT model, including specifications for required covariates, model parameterization, and implementation with and without hsCRP, are provided in Supplemental Appendix 3.
Comparative analysis revealed that the SMART Risk Score consistently outperformed the TRS2°P score. Specifically, the SMART score demonstrated a lower Brier score (11.3 vs 15.4), higher AUC (AUC: 94.8% vs 84.7%), and greater concordance (0.812 vs 0.755), indicating superior predictive accuracy and overall model performance (Table 4).
Table 4.
Model Performance, Concordance, and Fit Tests: Comparison of SMART Risk Score and TRS2°P Score
| Measure | Model | Development | Test |
|---|---|---|---|
| Brier | SMART | 11.3 [11.1;11.5] | 11.8 [11.6;12.0] |
| TRS2°P | 15.4 [15.2;15.6] | 15.4 [15.2;15.6] | |
| AUC | SMART | 94.8 [94.7;94.9] | 94.8 [94.7;94.9] |
| TRS2°P | 84.7 [84.6;84.8] | 84.9 [84.8;85.0] | |
| Concordance | SMART | 0.812 [0.809, 0.814] | 0.811 [0.808, 0.814] |
| TRS2°P | 0.755 [0.752, 0.758] | 0.752 [0.749, 0.755] |
AUC = area under the curve; TRS2°P = TIMI Risk Score for Secondary Prevention; other abbreviation as in Table 1.
The calibration plot of 10-year predicted vs observed event-free survival (ie, 1-risk) for the test cohort with predicted vs observed line having minimal deviation with respect to the line of identity (Figure 2). The deviance of the predicted and observed outcomes was significant in the test cohort (D’Agostino Nam test: P < 0.001), attributed to a large sample size.
The event rates of MACE and its components are described across predefined risk categories, as illustrated in Figure 3. Figures 4A and 4B present K-M survival curves for the development and test cohorts, respectively, stratified by the 4 SMART risk categories. These curves demonstrate that observed survival patterns are consistent with model-based risk stratification. K-M event rates for MACE and its individual components, along with mean time to MACE, are detailed in Table 5, Table 6, Table 7 for the overall cohort, as well as the development and test cohorts, respectively. The very high-risk group, as classified by the SMART risk score, exhibited the highest MACE rates, primarily driven by ischemic stroke and all-cause mortality, and had the shortest mean time to MACE. Comparisons between the very high-risk group and the other risk categories (low, moderate, and high) revealed statistically significant differences in all assessed variables (P < 0.05).
Figure 3.
Cardiovascular Outcomes in the Secondary Manifestations of Arterial Disease Risk Score Groups
(A) Entire cohort; (B) development cohort; (C) test cohort. MI = nonfatal myocardial infarction; SMART = Secondary Manifestations of Arterial Disease; other abbreviation as in Figure 2.
Figure 4.
Kaplan-Meier Analysis for Major Adverse Cardiovascular Event
(A) Development cohort; (B) test cohort. Abbreviation as in Figures 2 and 3.
Table 5.
Kaplan-Meir MACE Rates and Time to MACE for the Overall Cohort
| Overall (N = 259,250) |
Low Risk (n = 134,875) |
Moderate Risk (n = 44,416) |
High Risk (n = 23,746) |
Very High Risk (n = 56,213) |
|
|---|---|---|---|---|---|
| Outcomes | |||||
| MI | 16,498 (6.4%) | 1,621 (1.2%) | 3,274 (7.4%) | 3,091 (13.0%) | 8,512 (15.1%) |
| Stroke | 21,097 (8.1%) | 8 (0.0%) | 159 (0.4%) | 338 (1.4%) | 20,592 (36.6%) |
| Death | 19,607 (7.6%) | 2,391 (1.8%) | 3,451 (7.8%) | 2,804 (11.8%) | 10,961 (19.5%) |
| MACE | 48,203 (18.6%) | 3,960 (2.9%) | 6,647 (15.0%) | 5,816 (24.5%) | 31,780 (56.5%) |
| Time to MACE from index cardiology visit (days) | |||||
| Mean (SD) | 725.07 (746.70) | 713.62 (768.80) | 777.04 (780.79) | 884.63 (804.58) | 686.46 (720.72) |
| Median (Q1, Q3) | 443.00 (87.00, 1,209.00) | 399.00 (66.00, 1,198.00) | 507.00 (81.50, 1,334.50) | 714.50 (117.00, 1,472.00) | 405.00 (89.00, 1,125.00) |
Time to MACE was not significantly different between low vs very high-risk group.
All other variables were statistically significantly different between each of the groups (low, moderate, and high risk) vs very high-risk group (P < 0.05).
MACE = major adverse cardiovascular event; MI = myocardial infarction.
Table 6.
Kaplan-Meir MACE Rates and Time to MACE for the Development Cohort
| Overall (N = 155,550) |
Low Risk (n = 80,859) |
Moderate Risk (n = 26,755) |
High Risk (n = 14,362) |
Very High Risk (n = 33,574) |
|
|---|---|---|---|---|---|
| Outcomes | |||||
| MI | 9,952 (6.4%) | 1,011 (1.3%) | 2,015 (7.5%) | 1,819 (12.7%) | 5,107 (15.2%) |
| Stroke | 12,645 (8.1%) | 3 (0.0%) | 91 (0.3%) | 195 (1.4%) | 12,356 (36.8%) |
| Death | 11,849 (7.6%) | 1,378 (1.7%) | 2,106 (7.9%) | 1,720 (12.0%) | 6,645 (19.8%) |
| MACE | 28,951 (18.6%) | 2,358 (2.9%) | 4,056 (15.2%) | 3,473 (24.2%) | 19,064 (56.8%) |
| Time to MACE from index cardiology visit (days) | |||||
| Mean (SD) | 724.91 (747.86) | 708.10 (762.85) | 781.67 (784.39) | 876.67 (808.53) | 687.30 (721.94) |
| Median (Q1, Q3) | 440.00 (87.00, 1,211.00) | 391.50 (65.50, 1,209.00) | 501.00 (82.00, 1,329.00) | 690.00 (106.00, 1,475.00) | 405.00 (89.00, 1,123.00) |
Time to MACE was not significantly different in low and very high risk group.
All other variables were statistically significantly different between each of the groups (low, moderate, and high risk) vs very high-risk group (P < 0.05).
Abbreviation as in Table 5.
Table 7.
Kaplan-Meir MACE Rates and Time to MACE for the Test Cohort
| Overall (N = 103,700) |
Low Risk (n = 18,844) |
Moderate Risk (n = 33,668) |
High Risk (n = 19,085) |
Very High Risk (n = 32,103) |
|
|---|---|---|---|---|---|
| Outcomes | |||||
| MI | 6,546 (6.3%) | 49 (0.3%) | 540 (1.6%) | 1,294 (6.8%) | 4,663 (14.5%) |
| Stroke | 8,452 (8.2%) | 0 (0.0%) | 4 (0.0%) | 53 (0.3%) | 8,395 (26.2%) |
| Death | 7,758 (7.5%) | 71 (0.4%) | 815 (2.4%) | 1,456 (7.6%) | 5,416 (16.9%) |
| MACE | 19,252 (18.6%) | 118 (0.6%) | 1,338 (4.0%) | 2,715 (14.2%) | 15,081 (47.0%) |
| Time to MACE from index cardiology visit (days) | |||||
| Mean (SD) | 725.31 (744.98) | 669.08 (771.13) | 715.11 (786.40) | 783.03 (769.36) | 716.28 (736.08) |
| Median (Q1, Q3) | 449.00 (88.00, 1,205.00) | 422.50 (71.00, 918.00) | 379.00 (62.00, 1,206.00) | 540.00 (90.00, 1,337.00) | 442.00 (90.00, 1,188.00) |
Time to MACE was not significantly different between low vs very high-risk group.
All other variables were statistically significantly different between each of the groups (low, moderate, and high risk) vs very high-risk group (P < 0.05).
Abbreviation as in Table 5.
In decision curve analysis, compared with the TRS2°P score, the SMART risk score would detect an additional 110 events per 1,000 patients screened at a risk threshold of 25% in the test cohort Table 9. The net clinical benefit of SMART risk score (vs TRS2°P) was even greater at the higher risk thresholds of 37.5% and ≥50% as shown in Figure 5. A similar pattern of net clinical benefit for the SMART risk score was observed in the development cohort Table 8.
Table 9.
Net Benefit of the SMART Risk Score for Guiding Secondary Prevention (Development Cohort)
| Treatment Guide | Risk Threshold | Net Benefita | Patients with Net Benefit |
|---|---|---|---|
| Treat all | 15.0% | 0.21 | 33,404 |
| Treat none | 15.0% | 0.00 | 0 |
| SMART risk score | 15.0% | 0.24 | 36,580 |
| TRS2°P risk score | 15.0% | 0.18 | 28,661 |
| Treat all | 25.0% | 0.11 | 17,118 |
| Treat none | 25.0% | 0.00 | 0 |
| SMART risk score | 25.0% | 0.18 | 28,359 |
| TRS2°P risk score | 25.0% | 0.10 | 15,126 |
| Treat all | 33.0% | 0.004 | 589 |
| Treat none | 33.0% | 0.00 | 0 |
| SMART risk score | 33.0% | 0.15 | 23,232 |
| TRS2°P risk score | 33.0% | 0.04 | 6,558 |
Abbreviation as in Table 1.
Net benefit reflects proportion of correctly treated patients with higher values indicating greater clinical usefulness.
Figure 5.
Secondary Manifestations of Arterial Disease Risk and TRS2°P Score Decision Analysis for Major Adverse Cardiovascular Events
For the test cohort. TRS2°P = TIMI Risk Score for Secondary Prevention; other abbreviations as in Figure 3.
Table 8.
Net Benefit of the SMART Risk Score for Guiding Secondary Prevention (Test Cohort)
| Treatment Guide | Risk Threshold | Net Benefita | Patients with Net Benefit |
|---|---|---|---|
| Treat all | 15.0% | 0.21 | 22,042 |
| Treat none | 15.0% | 0.00 | 0 |
| SMART risk score | 15.0% | 0.24 | 25,036 |
| TRS2°P risk score | 15.0% | 0.18 | 19,100 |
| Treat all | 25.0% | 0.11 | 11,154 |
| Treat none | 25.0% | 0.00 | 0 |
| SMART risk score | 25.0% | 0.21 | 21,929 |
| TRS2°P risk score | 25.0% | 0.10 | 9,881 |
| Treat all | 33.0% | 0.001 | 104 |
| Treat none | 33.0% | 0.00 | 0 |
| SMART risk score | 33.0% | 0.18 | 18,303 |
| TRS2°P risk score | 33.0% | 0.04 | 4,143 |
Abbreviation as in Table 1.
Net benefit reflects proportion of correctly treated patients with higher values indicating greater clinical usefulness.
To assess the consistency of performance across sexes, we developed separate SMART risk score models for men and women (Supplemental Figures 1A and 1B), race (Supplemental Figures 2A and 2B) and ethnicity (Supplemental Figures 3A and 3B) and observed consistent trends across subgroups.
Discussion
In this large-scale study of 259,250 patients, we present robust validation of the SMART risk score as a tool for predicting long-term MACE in a real-world cohort of U.S. patients with established CVD. This is one of the largest studies to date evaluating the predictive power of the SMART risk score, which incorporates 14 clinically accessible risk predictors. By offering a comprehensive assessment across a wide spectrum of CV risk—from low to very high, the SMART risk score provides critical insights into patient stratification, facilitating the development of personalized, cost-effective prevention strategies.
Our findings validate the SMART risk score’s ability to predict MACE in both the development cohort and the test cohort. The model demonstrated a strong discriminatory capacity, with a C-statistic of 0.811 across the overall cohort. Comparable performance metrics from prior SMART risk score studies have already been published.2,6,8 Notably, our cohort included a near-equal representation of women (48.6%) and a diverse clinical sample reflective of contemporary U.S. health care practice.
Comparative analyses revealed that the SMART risk score outperforms other established risk models, such as the TRS2°P score, across several key metrics, including Brier score, AUC, and concordance.3 The SMART risk score's superior predictive accuracy, particularly in the very high-risk category, underscores its potential to better guide clinical decision-making for high-risk CVD patients.4
A major strength of this study is its validity within a U.S. health care setting, which differs significantly from the original Dutch cohort in terms of population characteristics, disease burden, and clinical management. The SMART risk score’s ability to predict MACE across different cohorts—spanning from the original Dutch cohort to the Registry of Risk factors and Event occurrence in patients with Atherothrombosis undergoing long-term Care registry, and now to this large U.S. cohort—demonstrates its generalizability and adaptability across diverse clinical environments.7,8 The findings from our study complement previous work, such as the Registry of Risk factors and Event occurrence in patients with Atherothrombosis undergoing long-term Care registry validation and the U.S. Veterans cohort, supporting the score’s applicability to various populations with established CVD.5,6,8 Importantly, definition of MACE in our study included all-cause mortality, in lieu of CV mortality adopted in prior studies.
Furthermore, the decision curve analysis strengthens the clinical utility of the SMART risk score. This analysis revealed that the score offers the highest net benefit across a broad range of threshold probabilities, outperforming both the “treat-all” and “treat-none” strategies.1,2 Such findings endorse the SMART risk score as a valuable tool for optimizing treatment decisions, balancing the benefit-to-harm ratio, and minimizing overtreatment—a key consideration in real-world clinical settings where precision medicine is increasingly prioritized (Central Illustration). In this respect, the SMART risk score outperforms the TRS2°P score.
Central Illustration.
Estimating the Risk of Cardiovascular Events in U.S. Veterans Using the Secondary Manifestations of Arterial Disease Risk Score
AAA = abdominal aortic aneurysm; CAD = coronary artery disease; CeVD = cardiovascular disease; DM = diabetes mellitus; eGFR = estimated glomerular filtration rate; HDL = high-density lipoprotein; hsCRP = high-sensitivity C-reactive protein; MACE = major adverse cardiovascular event; PAD = peripheral artery disease; SMART = Secondary Manifestations of Arterial Disease.
Study limitations
Although this study highlights the SMART risk score’s potential for clinical practice, several limitations warrant consideration. Firstly, the retrospective nature of our data, derived from electronic medical records, introduces the possibility of data inaccuracies, particularly in capturing certain risk factors and medication adherence. Furthermore, although our cohort includes a substantial proportion of women, the study population is predominantly White, limiting the generalizability of the findings to more racially and ethnically diverse groups. Ongoing and future efforts to validate the SMART risk score in diverse populations are essential to confirm its applicability across different demographic subgroups and health care settings. The C-statistic of 0.811 achieved in this study marks an improvement over previous SMART risk score validations, which generally reported C-statistics around 0.67.2 Although this represents a meaningful advancement, further refinement and validation of the model, particularly in external data sets, will be critical to addressing remaining uncertainties, such as the potential for calibration in populations with differing clinical profiles.
Conclusions
The SMART risk score provides an effective, reproducible method for predicting long-term CV risk in patients with established polyvascular disease. Our findings emphasize its clinical utility for improving risk stratification, guiding individualized treatment decisions, and optimizing secondary prevention strategies. Further validation in more diverse cohorts is necessary to fully understand its potential for broad clinical implementation. With additional studies, the SMART risk score has the potential to become a cornerstone of personalized CV care, driving more informed, precision-based interventions that improve long-term outcomes for patients with CVD.
Funding support and author disclosures
Dr Banerjee is a board member of Elsevier and Cardiovascular Innovations foundation; and reports receiving institutional research grant from GE, Esperion, and Novartis; reports Honoraria from AngioSafe, Kaneka, and Terumo. Dr Stoler is a consultant and advisory board member of Medtronic and Boston Scientific Corporation. Dr Pandey received research support from the National Institute of Health, American Heart Association, Applied Therapeutics, Roche, Ultromics, Gilead Sciences, Bayer, and AstraZeneca. Honoraria as an advisor/consultant for Tricog Health Inc, Lilly USA, Rivus, Cytokinetics, Roche Diagnostics, Axon Therapies, Medtronic, Edward Lifesciences, Science37, Novo Nordisk, Bayer, Medical AI, Astra Zeneca, Baylor Scott and White Research Institute, Boehringer Ingelheim, iRhythm Technologies, Tourmaline Bio, Merck, Sarfez Pharmaceuticals, Semler Scientific, Ultromics, Encarda, Boehringer Ingelheim, Tenax Pharma, Alnylam, Abbott, Kieele Health, Anumana, Acorai, Novartis, Antlia Biosciences. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
Acknowledgments
The authors would like to acknowledge research grant support from Esperion Inc., philanthropic support from Bob and Brigitta Smith to the Baylor Heart and Vascular Lipid Clinic, and philanthropic support from Ashley and Greg Arnold to the Baylor Research Institute.
Footnotes
The authors attest they are in compliance with human studies committees and animal welfare regulations of the authors’ institutions and Food and Drug Administration guidelines, including patient consent where appropriate. For more information, visit the Author Center.
Appendix
For an expanded Methods section and supplemental tables and figures, please see the online version of this paper.
Supplementary data
References
- 1.Dorresteijn J.A., Visseren F.L., Wassink A.M., et al. Development and test of a prediction rule for recurrent vascular events based on a cohort study of patients with arterial disease: the SMART risk score. Heart. 2013;99(12):866–872. doi: 10.1136/heartjnl-2013-303640. [DOI] [PubMed] [Google Scholar]
- 2.Banerjee S., Weideman R.A., Jacob D.A., et al. Estimating the risk of cardiovascular events in U.S. veterans using the SMART risk score. JACC Adv. 2024;4(1) doi: 10.1016/j.jacadv.2024.101459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Deo S.V., Althouse A., Al-Kindi S., et al. Validating the SMART2 score in a racially diverse high-risk nationwide cohort of patients receiving coronary artery bypass grafting. J Am Heart Assoc. 2023;12(21) doi: 10.1161/JAHA.123.030757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kaasenbrood L., Bhatt D.L., Dorresteijn J.A.N., et al. Estimated life expectancy without recurrent cardiovascular events in patients with vascular disease: the SMART-REACH model. J Am Heart Assoc. 2018;7(16) doi: 10.1161/JAHA.118.009217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Berg D.D., Moura F.A., Bellavia A., et al. Assessment of atherothrombotic risk in patients with type 2 diabetes mellitus. J Am Coll Cardiol. 2023;81(25):2391–2402. doi: 10.1016/j.jacc.2023.04.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kaasenbrood L., Boekholdt S.M., van der Graaf Y., et al. Distribution of estimated 10-Year risk of recurrent vascular events and residual risk in a secondary prevention population. Circulation. 2016;134(19):1419–1429. doi: 10.1161/CIRCULATIONAHA.116.021314. [DOI] [PubMed] [Google Scholar]
- 7.Castelijns M.C., Helmink M.A.G., Hageman S.H.J., et al. Cohort profile: the Utrecht cardiovascular cohort-second manifestations of arterial disease (UCC-SMART) Study-an ongoing prospective cohort study of patients at high cardiovascular risk in the Netherlands. BMJ Open. 2023;13(2) doi: 10.1136/bmjopen-2022-066952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jeon-Slaughter H., Chen X., Tsai S., Ramanan B., Ebrahimi R. Developing an internally validated veterans affairs women cardiovascular disease risk score using veterans affairs national electronic health records. J Am Heart Assoc. 2021;10(5) doi: 10.1161/JAHA.120.019217. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.