External validation of the SMART2 model for recurrent cardiovascular risk: performance in socioeconomic, ethnic and psychiatric subgroups

Abstract

Background

Secondary prevention is critical for patients with established atherosclerotic cardiovascular diseases (ASCVD). The SMART2 model predicts recurrent ASCVD risk but does not account for psychiatric disorders, socioeconomic deprivation or ethnicity. We aimed to evaluate SMART2 model performance overall and in subgroups defined by these factors.

Methods

SMART2 was externally validated using electronic health records of patients aged 40–80 in the Leiden-The Hague region. Patients hospitalised for cardiovascular disease or coronary interventions between 1 January 2010 and 31 December 2021 were included. Model performance was assessed using discrimination (10-year area under ROC curve (AUC)) and calibration (observed/expected (OE) ratios and plots, adjusted for competing risks), both overall and in subgroups defined by three factors: psychiatric history, socioeconomic status (SES) and ethnicity.

Results

Among 15 528 patients (66% male, mean age 65) with established ASCVD, median follow-up was 6.0 years. Recurrent cardiovascular events occurred in 2220 patients, and 1820 had competing events. Overall AUC was 0.63 (95% CI 0.61 to 0.65). OE ratio was 0.96 (95% CI 0.92 to 1.00). The model underestimated risk in patients with low SES (OE 1.09, 95% CI 1.02 to 1.17) and in non-Western patients (OE 1.16, 95% CI 1.02 to 1.31). Calibration was adequate in patients with psychiatric history, but the model was substantially underestimating for patients with multiple vulnerabilities—combinations of low SES, non-Western ethnicity and psychiatric history—with OE ranging from 1.19 to 1.29, depending on the combination.

Conclusion

The SMART2 model is well-calibrated in the general population but underestimates risk, especially in subgroups with multiple vulnerabilities. Patients in these vulnerable subgroups may require intensified monitoring. Clinicians should consider social, ethnic and psychiatric factors when interpreting risk estimates.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Socioeconomic status, ethnicity and psychiatric diseases influence cardiovascular risk, but prediction models do not take these into account when estimating risk on recurrent events.

WHAT THIS STUDY ADDS

• Performance of prediction model SMART2 for recurrent cardiovascular events was assessed, overall and in vulnerable subgroups. The quantification of underestimation of risk in the most-vulnerable subgroups (observed/expected ratio of 1.19–1.29) is a novel finding.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• These results indicate that social, ethnic and psychiatric factors should be taken into account when interpreting risk estimates for recurrent cardiovascular events.

Introduction

Atherosclerotic cardiovascular diseases (ASCVD), including heart and cerebrovascular diseases, are among the most important causes for mortality and morbidity worldwide.¹ Patients with existing ASCVD are traditionally considered to be at high risk for recurrent cardiovascular events, although individual risk can vary among these patients.^{1 2} As current ASCVD prevention guidelines emphasise the importance of shared decision-making, the SMART2 model has been developed as a tool to estimate an individual’s level of recurrent cardiovascular risk to guide personalised secondary prevention.^{1 3 4}

However, established models such as SMART2 focus on traditional cardiovascular risk factors and do not account for other important predictors of cardiovascular risk, such as psychiatric disorders, socioeconomic deprivation or ethnicity.⁵,⁷ Psychiatric disorders are prevalent in patients with ASCVD—depression affects 30–40% of patients with coronary heart disease—and are associated with an increased risk of cardiovascular events. This risk is likely mediated by stress, undertreatment, underscreening, unhealthy lifestyle, underlying genetic disposition and the use of antidepressant and antipsychotic drugs.⁸,¹⁴ Socioeconomic deprivation and ethnicity are also linked to an increased cardiovascular risk and are likewise associated with psychiatric morbidity.¹⁵,¹⁷

In the general population without cardiovascular disease, it has been shown that the widely implemented SCORE2 and Framingham risk models may benefit from recalibration or updating based on socioeconomic and ethnicity factors.^{7 18} Whether and to what extent psychiatric, socioeconomic and ethnicity factors—and their interaction—influence recurrent cardiovascular event risk has not been studied extensively.

Therefore, this study assesses the external performance of the SMART2 model in an ethnically and socioeconomic diverse population. We specifically aim to evaluate model performance in subgroups defined by socioeconomic status (SES), history of psychiatric disorders and ethnicity to quantify the effect of these factors on the accuracy of predictions for recurrent cardiovascular events.

Methods

Data sources

This study used electronic health records (EHR) from the Extramural LUMC (Leiden University Medical Center) Academic Network (ELAN) and linked socio-demographic registry data from Statistics Netherlands (SN).^{19 20}

The ELAN data warehouse comprises EHR from hospitals and approximately 150 general practices (GPs) located in the cities of The Hague, Leiden and Zoetermeer, as well as surrounding suburban and rural regions, encompassing roughly 33% of the population in these locations. Between 2010 and 2021, 659 322 unique patients had been registered at one point in an ELAN GP. The population is relatively urban and ethnically diverse compared with the general Dutch population.¹⁹ ELAN comprises records on patient characteristics, symptoms, diagnoses, medication and biomarker measurements. Symptoms and diagnoses are coded with ICPC (International Classification of Primary Care) codes, medication prescriptions with ATC (Anatomical Therapeutic Chemical) codes.

From SN, we obtained registry data on mortality, hospital admissions, income and ethnicity. Mortality records contained date and cause of death, coded according to the 10th version of International Statistical Classification of Diseases and related health problems (ICD-10). The data were derived from the mandatory reporting of the cause of death by a physician, pathologist or forensic physician. Hospital records contained nationwide admissions, diagnoses (ICD-9/10 coded) and interventions. Income records contained subject’s disposable household income and asset worth, adjusted for household composition.

All data were pseudonymised and linked on an individual level. SN functioned as a trusted third party and analyses were carried out in a secure remote access environment using R V.4.4.0, packages arrow, data.table, mice, riskRegression and survival.²¹,²⁵ R code is available on GitHub.

Study population

Aligning with the SMART2 development cohort, the study included individuals aged 40–80 years with stable, established ASCVD, defined as prior hospitalisation for coronary artery disease (CAD), cerebrovascular disease (CeVD), abdominal aortic aneurysm (AAA) or peripheral artery disease (PAD) between 1 January 2010 and 31 December 2021 (for diagnosis/intervention codes, see online supplemental tables S1 and S2). Follow-up (t=0) started once a patient was considered stable, operationalised as the completion of a 3-month event-free period (no other hospitalisations for CAD/CeVD/AAA/PAD or non-fatal outcome event, see below) following an inclusion event. If an additional event occurred within this period, the 3-month interval was reset; this process was repeated as often as necessary, with follow-up commencing 3 months after the final event. Patients with a fatal outcome event within these 3 months were excluded.

Patients who were registered with an ELAN GP for less than 6 months at the time of the inclusion event were excluded from the analysis, as we considered this limited registration duration too short for sufficient routine data capture on risk factors such as blood pressure and cholesterol. Patients who experienced multiple ASCVD events were included only after the first time all inclusion criteria were met. Patients of GPs that used an incompatible EHR system that did not share biomarker measurements were excluded.

Outcome and competing events

The SMART2 model estimates the 10-year absolute risk of recurrent cardiovascular events, defined as a composite of non-fatal myocardial infarction, non-fatal stroke and vascular death. Competing events were deaths due to other causes. Events were retrieved from nationwide hospital and death microdata supplied by SN (ICD-9/10 codes in online supplemental table S3).²⁰ follow-up ended on 31 December 2022. The model version calibrated for the European low-risk region was applied.

Model predictors

The SMART2 model uses predictors age, sex, smoking status, four biomarkers (systolic blood pressure (SBP), estimated glomerular filtration rate (eGFR), high sensitive C-reactive protein (hs-CRP) and non-high-density lipoprotein (HDL) cholesterol), four indicators for history of specific types of ASCVD (CAD/CeVD/PAD/AAA), presence of diabetes, years since first ASCVD diagnosis and an indicator for aspirin or equivalent medication. Variables were extracted from EHR from ELAN and a regional hospital, as detailed in online supplemental materials (‘Variable extraction’).

Statistical analysis

Missing data

Non-HDL cholesterol, SBP and eGFR were imputed using multiple imputation using chained equations. The full procedure, predictor matrix and missingness percentages are specified in the supplementary material (‘Missing data’ and online supplemental table S8).

As hs-CRP is not routinely measured, it was largely unavailable, making it unsuitable for multiple imputation. The development cohort median (2.0) was assigned to all patients, which has been shown to be a valid approach when multiple imputation is not feasible.²⁶

Loss to follow-up was minimal and was not otherwise addressed. Outcome ascertainment was independent of GP deregistration, using nationwide hospital and mortality records. Mortality data were complete for Dutch residents but may be incomplete for emigrants, depending on the country. In 2021, less than 1% of the Dutch population emigrated.²⁷ Non-fatal events occurring outside the Netherlands for emigrants were unknown and ignored, given the relatively low emigration rate.

Performance measures

Model performance was assessed with calibration and discrimination. Discrimination was assessed with the area under the ROC curve (AUC_t) after 10 years and calibration with observed/expected (OE) ratios and calibration plots. To account for censoring and competing events, the observed outcome proportion was estimated with the Aalen-Johansen estimator.²⁸ Calibration plots were generated by dividing individuals into octiles based on their SMART2 risk estimate.

Subgroups

Model performance was evaluated in the overall study population and in subgroups defined by a history of psychiatric disorders, ethnicity and SES.

Patients fell in the psychiatric disorder subgroup if at the moment of inclusion they had ever received an ICPC-coded diagnosis of schizophrenia, affective psychosis, bipolar disorder, anxiety disorder, somatisation disorder, depressive disorder, phobia or compulsive disorder or psychosis not otherwise specified, or if they ever had ≥2 prescriptions for antidepressants or antipsychotics in the same year, or ≥1 for lithium (ICPC and ATC list in online supplemental tables S4 and S5). This selection reflects disorders with established links to increased cardiovascular risk.²⁹,³² Severe mental illness was defined as a history of schizophrenia or other psychotic disorders, bipolar disorder or prescriptions for antipsychotics or lithium (see online supplemental table S6), consistent with prior research.³³

Ethnicity was defined as Dutch if the patient and both parents were born in the Netherlands, or Western/non-Western migrant if the patient or at least one parent was born outside of the Netherlands, as detailed in online supplemental table S7.

SES was derived from disposable household income and asset worth, averaged into a single welfare score ranging from 0 to 100. To account for age-related differences in wealth, scores were converted to percentiles within each age. Patients in the lowest third (≤33rd percentile) were classified as low SES, those in the middle third (34–67th percentile) as middle SES and the remaining as high SES.

Finally, an explorative analysis of subgroups with two vulnerabilities was done, that is, patients with low SES and psychiatric disorder, low SES and non-Western ethnicity and psychiatric disorder and non-Western ethnicity.

Sensitivity analyses

As biomarkers in EHR may not be missing at random, we performed a sensitivity analysis with delta-scaling.³⁴ To simulate the effect of missingness not at random, imputed non-HDL cholesterol values—the biomarker with the highest missingness—were scaled by a fixed parameter after every iteration. Because non-HDL values contribute to imputing other missing biomarkers, this effect will carry over to missing eGFR and SBP values. Two separate delta-scaling procedures were performed, one with delta=0.9, simulating the effect that the true unobserved values are 10% lower than the imputed values, and one with delta=1.1, simulating the opposite effect.

As another sensitivity analysis, we excluded hospital-recorded biomarker measurements, retaining only primary care measured biomarkers. This increased missingness percentages, but may reduce missingness not at random, as hospital visitors and hospital-measured biomarkers may not be representative to patients in primary care and their biomarkers.

In the Netherlands, the COVID-19 pandemic began on 11 March 2020, partially overlapping with our follow-up period. To assess its potential impact on outcomes, we conducted a sensitivity analysis in which follow-up was truncated at this date and enrolment was restricted to patients included before 11 March 2019. A 5-year time horizon was used as follow-up was insufficient for the full 10-year horizon.

Results

In total, 15 528 patients were included in this external validation study (see flowchart in online supplemental figure S1). The majority of the study population were men (66%) and the mean age was 65 (SD 10). Baseline characteristics are detailed in table 1. Median follow-up time estimated by the reversed Kaplan-Meier method was 6.0 years (IQR 3.5–8.9), with 2220 patients having a cardiovascular event (observed 10-year risk: 0.21) and 1820 dying from other causes (observed 10-year risk for competing events: 0.19).

Table 1. Baseline characteristics of the validation cohort compared with original SMART2 development cohort.

Variable	Validation cohort (N=15 528)	Missing	Development cohort (N=8355)
Age (years)	65±10		61 (9)
Male sex	10 222 (65.8)		6198 (74)
People who smoke	5571 (35.9)		2504 (25)
Diabetes mellitus	3968 (25.6)		1467 (18)
Using antithrombotic medicine	14 507 (93.4)		6494 (78)
History of coronary artery disease	9547 (61.5)		5215 (62)
History of peripheral artery disease	1032 (6.6)		1459 (17)
History of cerebrovascular disease	5224 (33.6)		2424 (29)
History of abdominal aortic aneurysm	641 (4.1)		706 (8)
Total cholesterol (mmol/L)	4.1 (3.5–4.8)	7147 (46)	4.6 (3.9–5.5)
HDL cholesterol (mmol/L)	1.2 (1.0–1.5)	7200 (46.4)	1.2 (1.0–1.4)
Non-HDL (mmol/L)	2.8 (2.2–3.5)	7209 (46.4)
hs-CRP (mg/dL)	–	15 528 (100.0)	2.0 (1.0–4.4)
SBP (mm Hg)	137±19	4143 (26.7)	139 (20)
eGFR (mL/min/1.73 m2)	74±20	5378 (34.6)	77 (18)
Psychiatric status (any of the following:)	4118 (26.5)
History of schizophrenia	48 (0.3)
History of affective psychosis/bipolar disorder	69 (0.4)
History of anxiety disorder/anxiety state	785 (5.1)
History of somatisation disorder	100 (0.6)
History of depressive disorder	1714 (11.0)
History of phobia/compulsive disorder	105 (0.7)
History of psychosis NOS, other	76 (0.5)
Was prescribed antipsychotics	447 (2.9)
Was prescribed lithium	78 (0.5)
Was prescribed antidepressive medication	3054 (19.7)
SES class
Low	5245 (33.8)
Mid	5142 (33.1)
High	5141 (33.1)
Ethnicity
Dutch	12 088 (77.8)
Western	1700 (10.9)
Non-Western	1740 (11.2)
Event rate per 1000 person-years*	28		24

All data in n (%), mean±SD or median (Q1–Q3).

^*Event rate of fatal+non-fatal (myocardial infarction, stroke) events per 1000 person-years.

eGFR, estimated glomerular filtration rate; HDL, high-density lipoprotein; hs-CRP, high-sensitive C-reactive protein; NOS, not otherwise specified; SBP, systolic blood pressure; SES, socioeconomic status.

Model performance

As detailed in figures1 2, the SMART2 risk model in the entire study population had an AUC_t after 10 years of 0.63 (95% CI 0.61 to 0.65) and OE ratio of 0.96 (95% CI 0.92 to 1.00). The calibration plot (figure 3) shows that the model was well-calibrated across most octiles, with an overestimation for individuals in the highest octile.

Figure 1. Discrimination of the SMART2 model, based on AUC_t after 10 years of index event, in overall study population and subgroups defined by a history of psychiatric disorders, socioeconomic status and ethnicity. N is the number of patients in the corresponding group. AUC, area under ROC curve; SES, socioeconomic status.

Figure 2. Calibration of the SMART2 model in overall study population and subgroups defined by a history of psychiatric disorders, socioeconomic status and ethnicity. OE ratios were obtained by dividing the observed outcome proportion (estimated with Aalen-Johansen estimator) by the average predicted risk of the SMART2 model. N events is the number of observed events in the corresponding group. OE, observed/expected; SES, socioeconomic status.

Figure 3. Calibration plots of the SMART2 model in the overall study population and subgroups defined by a history of psychiatric disorders, socioeconomic status and ethnicity. Patients were divided into approximately equal-sized octiles based on SMART2 risk prediction. The observed outcome proportion was estimated using the Aalen-Johansen estimator. SES, socioeconomic status.

Subgroup analysis

Performance measures of all subgroups are given in figures13 and are described in the coming subsections. Model discrimination in all subgroups was similar to the overall discrimination. In each subgroup, the predicted risk in the highest octile was consistently too high.

Psychiatric disorders

The subgroup of patients with a history of psychiatric disorder consisted of 4118 (26.5%) patients. The majority of these patients were prescribed antidepressants (N=3054, 19.7%) and/or had a history of depression (N=1714, 11.0%), followed by a history of anxiety (N=785, 5.1%). The OE ratio for patients without and with psychiatric disorder was 0.93 (95% CI 0.88 to 0.98) and 1.06 (95% CI 0.97 to 1.15), respectively. There were 512 patients with severe mental illness, with 84 recurrent events. The OE ratio in this group was 1.17 (95% CI 0.93 to 1.48).

Socioeconomic status

SES classes were split into three approximately equal groups. The OE ratio ranged from 1.09 (95% CI 1.02 to 1.17) in the low SES group, indicating too low risk estimates, to 0.84 (95% CI 0.77 to 0.92) in the high SES group, indicating too high estimates.

Ethnicity

The majority (77.8%) of our population were native Dutch, 10.9% had a non-Dutch Western ethnicity and 11.2% had a non-Western ethnicity. The OE ratio was 0.94 (95% CI 0.89 to 0.99) in the native Dutch population, 0.92 (95% CI 0.80 to 1.06) in patients of Western ethnicity and 1.16 (95% CI 1.02 to 1.31) in patients with non-Western ethnicity.

Subgroups with two vulnerabilities

In our study population, 2602 patients (16.8%) fell into at least two of three vulnerability groups, that is, psychiatric disorder, low SES and/or non-Western ethnicity. The model underestimated risk in psychiatric patients with low SES (OE ratio 1.19, 95% CI 1.05 to 1.35), in non-Western migrants with low SES (OE ratio 1.29, 95% CI 1.11 to 1.50) and in psychiatric patients with non-Western ethnicity (OE ratio 1.26, 95% CI 1.02 to 1.56).

Sensitivity analyses

Discrimination results across all subgroups were robust to delta scaling the imputations. The OE ratios increased for a delta of 0.9 and decreased for a delta of 1.1. This can be explained by scaling laboratory values towards lower (healthier) values, the SMART2 predicted risk will decrease, thereby increasing the OE ratio. While not all statistically significant findings remained significant under varying delta, the overall pattern persisted: OE ratios were consistently higher in vulnerable subgroups—patients with psychiatric disorders, low SES or non-Western ethnicity—compared with their less-vulnerable counterparts—those without psychiatric disorder, with high SES or of native Dutch ethnicity (online supplemental table S9).

Excluding hospital measurements from our analysis increased missingness for non-HDL cholesterol from 46.4% to 50.6%, for SBP from 26.7% to 32.1% and for eGFR from 34.6% to 41.3%. The mean and SD of these predictors were similar to the main analysis (online supplemental table S10). Model performance while only using laboratory values from primary care was similar to the performance of the main analysis (online supplemental table S11).

Restricting enrolment of patients and follow-up to the pre-COVID-19 period resulted in a population of 11 495 patients, with a median follow-up time of 4.6 years (IQR 2.6–7.0) and 1364 patients having a cardiovascular event (observed 5-year risk: 0.12) and 1072 dying from other causes (observed 5-year risk for competing events: 0.10). Patient characteristics were similar to the entire cohort (online supplemental table S12). Performance metrics after a time horizon of 5 years had larger CIs but showed similar patterns to the complete cohort with 10 year horizon: OE ratios were consistently higher in vulnerable subgroups compared with their less-vulnerable counterparts (online supplemental table S13).

Discussion

This study evaluated the performance of the SMART2 model for estimating recurrent cardiovascular risk in an urban population in the Netherlands. Although discrimination was lower than in the development cohort (AUC_t 0.63 vs 0.70), this could partly be attributable to differences in population characteristics and data quality, including substantial missing data.

The model was well-calibrated overall. However, calibration differed across vulnerable subgroups. The model slightly underestimated risk in patients with low SES and in non-Western patients, but was well-calibrated in patients with a history of psychiatric disorders. In patients with any combination of two of these three vulnerabilities, the model underestimated risk by 19–29%, depending on the combination. In the less vulnerable groups (ie, patients without psychiatric disorders, of high SES or of Dutch ethnicity), the model slightly overestimated the risk.

Comparison with the literature

The SMART2 model was previously externally validated in a population of patients undergoing coronary artery bypass grafting in the USA.³⁵ The model underestimated risk in the entire population and was better at predicting risk in white patients compared with black patients. In the external validation of SCORE2 for first-ever cardiovascular events within the same source population, markedly poorer calibration was observed among low-SES and non-Dutch groups with OE ratios reaching 2.5 in the most vulnerable subgroups, compared with an overall ratio of 1.5.⁷ The SMART2 model may be able to better capture effects of the vulnerability factors. The impact of a first cardiovascular event may also reduce relative impact of disparities among SES groups and different ethnicities.

In a population without ASCVD, PRIMROSE showed that including severe mental illness as predictor in cardiovascular risk models may be beneficial.¹⁸ However, PRIMROSE defined severe mental illness based on only psychotic or bipolar disorders, while disorders such as anxiety and depression are more frequently occurring in the general population, while also associated with an increased risk of ASCVD.^{9 10} The EUROASPIRE risk model is an alternative model for prediction of recurrent cardiovascular events in patients with coronary heart disease.³⁶ While this model does take into account psychiatric disorders, it only gives risk estimates for a recurrent event within 2 years, compared with the 10-year horizon of SMART2.

In a recent study, adding psychological factors to the SMART2 model did not meaningfully improve discrimination and calibration, concluding that SMART2 reliably estimates the recurrent cardiovascular event risk in patients with or without psychological factors.³⁷ In our study, we observed overall adequate performance in patients with psychiatric disorders, but a slight overestimation in those without psychiatric disorders, suggesting that psychological factors may still affect recurrent cardiovascular risk. We further extend these findings by examining the combined impact of multiple vulnerabilities, including SES and ethnicity, with results indicating that psychiatric factors in combination with these vulnerabilities lead to underestimation of cardiovascular events.

The mechanisms which cause underestimation of cardiovascular risk in patients with low SES, especially in combination with psychiatric conditions or non-Western ethnicity, remain largely unknown. However, studies point in the direction of limited accessibility to cardiovascular risk management for psychiatric patients, or challenges regarding a healthy lifestyle in patients with low SES.³⁸,⁴¹

Strengths and limitations

Several methodological challenges need to be taken into account to interpret our results. First, we have substituted the development median of hs-CRP in our whole study population, as this biomarker is not routinely measured for cardiovascular risk management in the Netherlands.^{26 42} It is expected that model discrimination would be higher with accurate hs-CRP measurements, and the model calibration depends on whether the imputed median corresponds to the true (unobserved) median of the validation data. It may particularly have impacted model performance in the psychiatric subgroup, as elevated levels of this biomarker are associated with increased risk for psychological disorders.⁴³

A second limitation involves the moderate percentage of missing biomarker data in general due to reliance on the EHR as data source. EHR data are often missing not at random (MNAR).³⁴ General practitioners typically order laboratory tests when patients are unwell, meaning missing values may be closer to healthy levels. However, Dutch clinical guidelines recommend regular monitoring for patients with a history of ASCVD, so missing values could also indicate non-compliance, potentially indicating unhealthy levels instead.^{44 45} We used multiple imputation with chained equations to account for missing data, ensuring that patients with missing predictor values were not omitted from the external validation. Delta imputations indicated that the results and conclusions are robust under different MNAR patterns.

Third, some subgroups in our analysis were small, leading to wide CIs, particularly among patients with severe mental illness, non-Dutch ethnicities or multiple vulnerabilities. The small subgroup sizes underscore the need for cautious interpretation of these findings.

Fourth, part of the follow-up overlapped with the COVID-19 pandemic, which disrupted cardiovascular care and may therefore have affected outcome incidence and, subsequently, model calibration.⁴⁶ However, a sensitivity analysis restricted to the pre-pandemic period showed similar results to the main analysis.

Finally, both the Western and non-Western ethnicity categories comprise heterogeneous populations with potentially distinct cardiovascular risk profiles, which may have masked differential model performance within more specific ethnic subgroups. More granular analyses were not feasible due to limited sample size of these subgroups. This classification may also limit transportability to settings using different ethnic categorisations and may become less informative in increasingly diverse populations with mixed ethnic backgrounds.

Our study also has strengths. The study population is more urban and ethnically diverse compared with the Netherlands in general.¹⁹ Traditional cohorts generally have an under-representation of individuals with socioeconomic deprivation, psychiatric disorders or different ethnicities, while in our population-based cohort the prevalence of these vulnerable subgroups allows the necessary analyses. The unique data environment, with detailed information on these factors, allowed for the assessment of model performance in these subgroups, and even to assess interactions of vulnerabilities such as psychiatric disorders in non-Western migrants or people who suffer socioeconomic deprivation. In addition, while the SMART2 model has been externally validated,^{4 35} this is the first study to assess its performance specifically in patients with psychiatric and socioeconomic vulnerability.

Clinical implications and future research

The SMART2 model was well-calibrated in the overall population, as well as in the subgroup of patients with a history of psychiatric disorders. In low SES and in non-Western patients, the model underestimates risk. Furthermore, the SMART2 model underestimates recurrent cardiovascular risk by 19–29% in patients with a combination of two of these three vulnerabilities, which is the case in 17% of the overall study population.

There are currently no universally accepted clinical decision thresholds for intensifying secondary prevention based on the risk of recurrent cardiovascular events. Therefore, it is not feasible to meaningfully quantify how often subgroup-specific miscalibration would result in patients failing to cross a predefined treatment threshold. However, absolute risks of (recurrent) cardiovascular events and the individually predicted effects of preventive pharmacotherapeutics are increasingly being used for shared decision-making in clinical practice, for example, using the U-Prevent online tool.⁴⁷ In these individual cases, the following illustration shows the relevance of miscalibration. Consider a patient of low SES and non-Western ethnicity who may have a SMART2 predicted risk of 23%, but accounting for the underestimation (O/E ratio of 1.29) in their subgroup, their actual risk may be closer to 30%. While both estimates indicate high risk, the lower predicted value may influence both patients’ and clinicians’ perception of urgency and therefore may be less likely to be prioritised for intensified risk factor monitoring or additional screening.

As SES, ethnicity and psychiatric disorders have been associated with cardiovascular risk in cohorts across different healthcare systems,^{5 6 48 49} the underestimation of recurrent risk in vulnerable subgroups is likely not unique to the Dutch setting. However, external validation in other healthcare systems is needed for more robust knowledge on the extent of miscalibration in vulnerable subgroups or clusters of vulnerability, as this may depend on differences in population composition and in the organisation of care for socioeconomically disadvantaged, migrant or psychiatric patients.

To improve risk prediction and reduce disparities in cardiovascular health, future prediction model developers could consider to integrate these factors of vulnerability as predictors. Since the cost of implementing prediction models into clinical practice—especially as decision support software—is high, it may be beneficial to update or recalibrate existing models for vulnerable subgroups, or deploy statistical methods to add risk factors that were not originally used in model development.⁵⁰ Expanding preventive programmes and improving access to care are essential to reducing disparities in cardiovascular health.

Data availability statement

Data may be obtained from a third party and are not publicly available.