Machine-learning based calculator for personalized risk assessment following ST-elevation myocardial infarction

Abstract

Background

Despite recent therapeutic advancements, a high-risk scenario for subsequent adverse events remains following ST-elevation myocardial infarction (STEMI). Risk scores provide important prognostication data following STEMI. However, the most established risk scores are not contemporary, their prognostic accuracy ranges across populations, and they cannot provide patient-centered recommendations. We aimed to develop a machine learning (ML)-based calculator which provides personalized prognostication following STEMI.

Methods

3340 patients with STEMI from tertiary care centers in Israel and Italy between the years 2004–2020 were included. The calculator leveraged CatBoost for supervised ML. The goal was to predict rates of one-year mortality and to reduce risk by generating patient-centered recommendations.

Results

2887 patients were included for calculator training and 453 for testing. Female sex rate (19.3% in the training set vs. 17.8% in the testing set), BMI (29.7±11.3 vs. 28.5±10.0 kg/m²), and baseline creatinine (1.1±2.2 vs. 1.1±0.98 mg/dL) were similar between the groups. Age (61.4±12.7 vs. 59.8±12.4 years, p = 0.03) was different. The calculator accurately predicted one-year mortality (95.6% accuracy, AUC 0.95, 95% CI: 0.91–0.98 in the Israeli center and 93.2% accuracy, AUC 0.90, 95% CI: 0.79–0.98 in the Italian center). In total, left ventricular ejection fraction post-STEMI demonstrated the strongest contribution towards predicted outcome (mean Shapley value of 0.978). The algorithm also generated a personalized assessment tool of risk for each patient.

Conclusions

The ML-derived STEMI calculator provides the variables of highest predictive value in determining outcomes in a personalized manner. This can be leveraged for prognostication and guidance of patients post-STEMI.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12872-025-04896-1.

Background

In patients with ST-elevation myocardial infarction (STEMI), mortality rates have drastically declined since the development of primary percutaneous coronary intervention (PCI) and the use of secondary preventative measures [1–4]. Even so, the one-year mortality rate following STEMI is approximately 10% and the 5-year cardiac mortality rate is approximately 14% [4]. Several significant risk factors have been implicated in patient outcomes following STEMI, including increasing age, diabetes mellitus, smoking status, hypertension, left ventricular ejection fraction (LVEF), among others [5].

Based on these risk factors, risk scores and calculators have been developed to assess the prognosis of patients following these events. The Thrombolysis in Myocardial Infarction (TIMI) is used to estimate prognosis based on clinical and therapeutic measures in patients suffering from STEMI [6]. The TACTICS-TIMI 18 trial demonstrated the utility TIMI score in deciding between invasive and medical management [7]. The benefit of using scores of this nature are that they can be used at the bedside and can aid in initial triage of patients. While TIMI has been validated using multivariate regression, analyzing which factors are of highest importance, it lacks imperative prognostic indicators, including cardiac biomarkers and invasive data. Other calculators, including the dynamic TIMI risk score, uses the TIMI calculator as a framework and indexes based on in-hospital events [8]. This score can also be performed at the bedside and can also be used throughout the course of hospitalization and at discharge.

The training populations for these risk scores have not been validated on contemporary STEMI patients. The development of the TIMI score consisted of patients who suffered unstable angina/NSTEMI events between 1994 and 1998. Similar scores such as the Global Registry of Acute Coronary Events (GRACE), used to risk stratify acute coronary syndrome (ACS), was developed in the early 2000s [9, 10]. Since then, there have been major changes in epidemiological and prognostic trends for patients suffering from MI. Some of these changes include shifts from cardiovascular to non-cardiovascular causes of mortality for post-discharge patients, lower incidence rates of STEMI, and lower rates of death of NSTEMI [11–13]. Given these epidemiological shifts and given the nature of change, it is important to have risk calculators that are not only based on contemporary data, but that can continue to evolve as changes occur.

Therefore, to better understand contemporary-based outcomes and population trends, this study aims to develop and externally validate a machine-learning (ML)-based calculator to predict one-year all-cause mortality following STEMI using modern multi-center data.

Methods

This registry consists of a retrospective observational dataset comprising patients with STEMI who have previously been hospitalized at one of two large, tertiary centers, Rabin Medical Center (RMC) in Israel and Turin Hospital in Italy. The registry for patient in RMC has been used previously in other publications [14–16], while the registry for Turin is new. All patients suffered STEMI between 2004 and 2020 and were at least 21 years of age. Diagnosis of STEMI was in concordance with the fourth universal definition of myocardial infarction (e.g. Type 1 MI) [17]. There were no clinical exclusion criteria, as this registry was designed to include consecutive all-comers with STEMI to the two institutions. Patients were excluded in cases of records with more than 35% of missing entries.

The study protocol and data collection were approved by the local Institutional Review Board committee in compliance with the Declaration of Helsinki.

Immediate and in-hospital events were prospectively collected in the institutional database. During follow-up, patients completed standardized questionnaires for clinical events either by telephone (e.g., with the patient or with a family member) or in the outpatient clinics at 6‐month intervals. When indicated, records from peripheral hospitals were acquired to verify the events in the follow‐up period. All events were further confirmed and adjudicated by the institutional clinical events adjudication committee. The database consisted of baseline demographic data, procedural characteristics, and outcome data. A complete list of variables used in the model is provided in Supplementary Table 1. Glomerular filtration rate (GFR) was calculated by the Modification of Diet in Renal Disease formula. Chronic kidney disease was defined as GFR below 50 mL/min/1.73 m². We used all-cause one-year mortality as our clinical outcome measurement. All-cause mortality was defined as death occurring within 365 days of the indexed hospitalization and survival status at follow‐up was assessed by review of municipal civil registries at 1 year.

The total study data included 3,340 total patients that were divided into two groups, the first for training and building our model, and the second for testing our model. The first group consisted of 2,887 patients, all from RMC. The ML model was created with the utilization of CatBoost, a gradient-boosting model that iteratively builds decision trees and often outperforms traditional tree-based algorithms, especially on heterogeneous clinical datasets. We incorporated all baseline data into the model, including demographic, laboratory, and imaging data. The second group consisted of 453 patients in total, 321 from RMC and 132 patients from Turin Hospital (Fig. 1).

Fig. 1.

Study design. Of the 3,340 patients included, 2,887 were used for training the algorithm, while 453 were used for algorithm testing

The specific configuration of the training model is described as follows:

The model was adjusted to go through 500 rounds of learning the training data, in each iteration building an additional decision tree to improve the model’s accuracy. Specifically, the learning rate was set to 0.1 with a depth of 10, allowing the model to capture intricate patterns in the data. It was also optimized to maximize area under the curve (AUC) for binary classifications tasks. Lastly, to adjust for imbalanced datasets, the model automatically adjusted weights inversely proportionate to class frequencies. With these configurations, the model was optimized for datasets such as the ones created for this study that have complex patterns and imbalanced classes. Additionally, the CatBoost model natively supports missing values and therefore no imputation was required.

We assessed the performance of the ML model using accuracy, receiver operating characteristic (ROC) AUC, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We classified any datapoint with a probability of greater than 0.5 as “1” and less than or equal to 0.5 as “0”, with “1” indicating freedom of mortality and “0” indicating mortality at 1-year. Accuracy is used to measure the model’s percentage of correct predictions while ROC AUC compares the model’s tradeoff between the true positive rate and false positive rate. Sensitivity measures the model’s ability to identify true positives whereas specificity measures its ability to identify true negatives. PPV is the probability that the model’s assignment of free-of-mortality (positive result) will be alive after one year; whereas, NPV is the inverse. SHAP (Shapley Additive exPlanations) summary plots were used to delineate how our model arrived at its predictions.

Lastly, we assessed the calibration of our model using a calibration curve and an associated integrated calibration index (ICI). The classification strategy used for calibration is identical to the one described above.

Results

Our study comprised of 3,340 patients from two tertiary centers, RMC and Turin Hospital. The model was entirely trained on patients from RMC. 10% of patients from RMC and all patients from Turin served as the testing subjects (Fig. 1). Mean age was 61.4 ± 12.7 and 59.8 ± 12.4 years, p = 0.03 for RMC training and testing data, respectively. Age (p = 0.03) and baseline hemoglobin (13.9 ± 1.7 vs. 14.3 ± 5.9, p = 0.01) were the only characteristics that demonstrated significant differences between the RMC training and testing populations. All other baseline characteristics, including body mass index (BMI), GFR, and time from hospital to primary PCI did not significantly differ between the two groups (Table 1).

Table 1.

Baseline characteristics

Parameter	Training Data (n = 2887)	Testing Data (n = 321)	P-value
Age (years)	61.4 ± 12.7	59.8 ± 12.4	0.03
Sex (F)	19.3 ± 39.5	17.8 ± 38.3	0.51
Creatinine (mg/dL)	1.1 ± 2.2	1.1 ± 0.98	0.81
BMI (kg/m2)	29.7 ± 11.3	28.5 ± 10.0	0.08
GFR (mL/min)	86.0 ± 28.3	89.1 ± 39.5	0.08
Killip Class	1.3 ± 0.8	1.4 ± 0.9	0.30
Cardiogenic Shock	6.3 ± 24.3	6.9 ± 25.4	0.68
SBP (mmHg)	133.8 ± 36.6	133.9 ± 27.9	0.96
Time from Hospital to PCI (hours)	1.7 ± 3.9	1.8 ± 3.8	0.87
History of PCI (%)	16.5 ± 37.1	16.3 ± 37.0	0.94
History of MI (%)	13.8 ± 34.5	15.4 ± 36.2	0.46
History of CABG (%)	3.4 ± 18	1.9 ± 14	0.18
Smoking History (%)	45.3 ± 49.8	45.8 ± 49.9	0.86
History of CVA (%)	5.9 ± 23.5	6.1 ± 24.0	0.87
History of PVD (%)	5.1 ± 21.2	3.8 ± 19.2	0.33
WBCs (thousands/mm3)	12.1± 7.6	12.4 ± 5.1	0.57
Hgb (g/dL)	13.9 ± 1.7	14.3 ± 5.9	0.01
Hgb A1c (mg/dL)	7.5 ± 13.4	6.8 ± 2.3	0.64

PCI = percutaneous coronary intervention; MI = myocardial infarction; CABG = coronary artery bypass graft; CVA = cerebrovascular accident; PVD = peripheral vascular disease

Procedural characteristics are shown in Table 2, with no significant differences in any of the parameters, including LVEF.

Table 2.

Procedural characteristics

Parameter	Training Data (n = 2887)	Testing Data (n = 321)	P-value
No. Vessel Disease	1.8 ± 0.80	1.8 ± 0.80	0.76
Calcification (%)	17.2 ± 37.7	14.8 ± 35.6	0.30
Thrombus (%)	76.5 ± 42.4	78.0 ± 41.5	0.56
Thrombectomy (%)	21.2 ± 41.3	23.1 ± 42.2	0.60
Bifurcation (%)	15.6 ± 36.3	17.0 ± 42.2	0.52
Stent Length (mm)	17.7 ± 10.8	18.1 ± 37.7	0.57
Successful PCI (%)	94.6 ± 22.6	95.3 ± 21.2	0.60
Requiring IABP (%)	8.7 ± 28.3	9.1 ± 28.8	0.82
MR Grade	1.28 ± 1.14	1.39 ± 1.16	0.11
LVEF (%)	49.7 ± 12.6	49.6 ± 12.3	0.90

PCI = percutaneous coronary intervention; IABP = intra-aortic balloon pump; MR = mitral regurgitation; LVEF = left ventricular ejection fraction

Using the testing data, the calculator was able to accurately predict one-year mortality with 95.6% accuracy, AUC 0.95 (95% CI: 0.91–0.98) in RMC and 93.2% accuracy, AUC 0.90 (95% CI: 0.79–0.98) for Turin (Figs. 2 and 3). Sensitivity and specificity were similar between RMC and Turin, with 98.32% and 95.9% sensitivity, respectively and 62.5% and 63.6% specificity, respectively. PPV was similar between the two centers while NPV was slightly higher for RMC data, with 97.01% and 75% PPV/NPV at RMC compared to 96.7% and 58.3% PPV/NPV at Turin.

Fig. 2.

Confusion matrices for patients tested in RMC and Turin Hospital. Matrices demonstrate accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. Matrix on the left includes patients from RMC; on the right includes patients from Turin. 0 defines mortality after one-year, while 1 is free of mortality

Fig. 3.

Receiver operating characteristics for patients tested in RMC and Turin Hospital. Area under the ROC curve provides a measure of performance for our algorithm. The ROC curve on the left includes patients from RMC; on the right includes patients from Turin. LVEF = left ventricular ejection fraction; GFR = glomerular filtration rate; WBCs = white blood cells; SBP = systolic blood pressure; CPK = creatinine phosphokinase; DM = diabetes mellitus; BMI = body mass index; MR = mitral regurgitation

Our SHAP summary plots in Figs. 4 and 5 demonstrated that LVEF had the greatest predictive ability on outcome, with lower LVEF contributing to higher mortality. GFR was the second greatest contributor, with lower GFRs similarly leading to higher one-year mortality.

Fig. 4.

SHAP feature importance plot. Features ranked from highest to least important in predicting outcome for total aggregate data. Importance concluded from algorithm creation and measured using SHAP, allowing for feature comparison. LVEF = left ventricular ejection fraction; GFR = glomerular filtration rate; WBCs = white blood cells; SBP = systolic blood pressure; CPK = creatinine phosphokinase; DM = diabetes mellitus; BMI = body mass index; MR = mitral regurgitation

Fig. 5.

Beeswarm SHAP summary plot. The beeswarm plot demonstrates how each feature impacts the model’s output with SHAP used to indicate feature importance. The top row of this plot shows that lower percentages of left ventricular ejection fraction is associated with increased mortality

The calibration curve is demonstrated in Fig. 6, with an ICI score 0.13. This indicates that our model’s prediction, on average, deviates from the true events by ~ 13%.

Fig. 6.

Model’s calibration curve. This calibration curve with an ICI score of 0.13 demonstrates the precise relationship of our model’s predictions against the known outcome

Lastly, a unique feature was the ability of the calculator to provide an individualized feature plot, demonstrating the importance of each variable for each individual patient encounter. Two examples are shown in Fig. 7. The plots illustrate the decision-making process of the model by displaying how each feature contributes to the final prediction or outcome. It provides a step-by-step view, showing the cumulative impact of each feature on the model’s output, providing a graphical representation of the model’s performance and the importance of individual features on the decision process. In these examples, we input two sample patients with the characteristics described in the plots. For the sample patient on the left side of the feature plot, the algorithm determined that LVEF was the most important factor in determining outcome, followed by GFR, in this case LVEF of 30% and GFR of 21mL/min. The model predicted the sample’s probability of one-year survival at 77.2%. For the sample patient on the right side of the feature plot, Killip classification was the most important factor in determining outcome, followed by baseline hemoglobin, and GFR. The sample patient demonstrated a high Killip score of three, significant anemia, and impaired renal function, with a hemoglobin of 7.1 g/dL and GFR of 59 mL/min. This patient’s probability of survival was 80.5% at one year.

Fig. 7.

Individualized feature plots. These plots demonstrate how each unique patient has different features of importance in determining the risk of mortality

Discussion

In this study, we have created an ML-based calculator that can predict mortality a year after STEMI with high accuracy. The model also demonstrated high sensitivity and PPV, but considerably lower specificity and NPV. This suggests potential utility as a screening tool rather than as a definitive risk classifier on a population level. However, on an individual level, our calculator can provide a personalized assessment of risk for each independent patient based on the parameters of their specific risk factors.

In today’s medical practice, most patients receive primary PCI post-STEMI. Factors that dictate secondary prevention after STEMI are wide-ranging, including medication adherence, limiting harmful environmental exposures and optimal lipid and anti-thrombotic management, among others [18–20]. Studies have demonstrated that improved patient education leads to improved patient self-management [21, 22]. If clinicians can further provide more tailored information regarding a patient’s illness, it is reasonable to conclude that patients may benefit from even greater patient-specific management. The TIMI and GRACE scores have been used to assess the mortality risk following ACS and UA/NSTEMI, respectively. However, both models use traditional regression analyses to predict their outcomes [9, 10]. Regression analysis models, while well validated, use variables that remain fixed after model development. ML algorithms, on the other hand, allow for continuous improvement in reliability as new data sets are added and/or implemented. Additionally, ML algorithms are able to incorporate a much larger number of variables and do not require selection of variables, which allows itself to be limited from analytical bias [23]. Other more recent studies have similarly leveraged ML to improve risk prediction in STEMI patients, demonstrating superior performance over traditional models [23–25]. Our model builds on this foundation by incorporating explainability and a personalized assessment of risk (via SHAP) and an external validation cohort than can help in providing more real-world, contemporary insights.

Bai et al. developed a ML model to assess risk of late cardiogenic shock following STEMI [26]. The algorithm constructed used ML to create a nomogram that was accurate in predicting patients who were more likely to develop cardiogenic shock. In our study, we use ML to assess the risk of one-year mortality for patients following STEMI. Importantly and unique to our algorithm, we can provide a personalized assessment of risk for each patient based on their individual clinical variables. Specifically, we used CatBoost, a type of ML model which utilizes gradient boosting on decision trees as well as iteration data to optimize accuracy. We leveraged CatBoost because it has been demonstrated to show improved short-term predictability of outcomes as compared to other, traditional, ML types [23, 25].

The SHAP summary plot allows precise and intuitive assessment of the most impactful variables on a population-level, with LVEF post-STEMI as the strongest contributor in determining mortality after one year. Our findings reaffirm the prognostic value of LVEF as a robust predictor of short and long-term mortality and highlight it as an important variable in post-STEMI care. Guideline-based HF therapies, including medications and ICD implantation are all influenced by LVEF thresholds. With improved risk stratification, future models may help refine these thresholds and support more individualized treatment decisions. Additionally, avoiding cardiogenic shock, which is the leading cause of death post-STEMI, will obviously portend a much favorable prognosis [27]. Renal function or GFR, was the second strongest contributor in prognosis determination, with lower GFRs contributing to higher one-year mortality [28–30]. Renal impairment is clinically challenging because it indicates multi-organ damage in patients suffering from STEMI and it also can prevent utilizing medications geared towards modifying pathologic ventricular remodeling [31, 32].

Most importantly, with the SHAP summary plot, we can provide a personalized decision plot to clinicians which shows the most important features of prognostic determination in each individual patient. To the best of our knowledge, this is the first study to provide this kind of data for a STEMI-based calculator and among contemporary STEMI patients. ML-based prediction models have previously been incorporated into electronic medical records (EMRs), including those that predict ACS and those that predict sepsis [33, 34]. They have been shown to provide rapid information that can help triage and stratify patients based on severity to enhance care. We plan to similarly have this algorithm incorporated into EMRs for similar benefit. While the output of our calculator should not be the only factor in consideration the management of patients post-STEMI, providing cardiologists with the value of these risk factors may help guide personalized decision-making clinical pathways. Additionally, the ability of our calculator to establish tangible individual risk factors may improve patient engagement and compliance in optimizing the undertake of secondary prevention measures post-STEMI [21].

Study limitations

Our study seeks to create a universal calculator that can be leveraged for any patients undergoing STEMI. A major limitation, however, is that our calculator was only trained on datasets from 2 medical centers in Israel and Italy. To become a more universal calculator that account for several demographic variabilities, we need further studies and cohorts to be implemented into the training set. Additionally, our external Turin cohort was small, and larger cohorts will be needed for further generalizability. Second, our calculator was trained on a random split, meaning that the overlapping temporal trends may introduce bias. Third, only one-year all-cause mortality was measured, not differentiating cardiovascular from other causes. Given that we did not differentiate between the two, there is a possibility of confounding on outcomes. Future work may incorporate adjudicated cause-specific mortality. Lastly, we did not differentiate between inpatient and outpatient mortality, which may provide an additional source of confounding. To report high specificity of risk factors on cardiovascular mortality or morbidity, enhanced outcome measurements are required.

Conclusion

The current study demonstrates the feasibility of utilizing ML as a prognostic tool post STEMI for one year mortality. While the model showed promising accuracy, the relatively small external cohort and one-year outcome limit broad clinical applicability. However, with future prospective studies and broadened outcome measurements, these ML tools will be valuable in enhancing the personalization of medicine among STEMI survivors.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

ACS

Acute coronary syndrome

AUC

Area under the curve

BMI

Body mass index

EMR

Electronic medical record

GFR

Glomerular filtration rate

GRACE

Global Registry of Acute Coronary Events

ICI

Integrated calibration index

LVEF

Left ventricular ejection fraction

ML

Machine learning

NPV

Negative predictive value

PCI

Percutaneous coronary intervention

PPV

Positive predictive value

RMC

Rabin Medical Center

ROC

Receiver operating curve

SHAP

Shapley Additive exPlanations

STEMI

ST-elevation myocardial infarction

TIMI

Thrombolysis in Myocardial Infarction

Author contributions

AK, LP, FD, and RK contributed to the project’s conception; OF, KO, AL, TB assisted with the acquisition of the data; NL and LP assisted with software and statistical analysis of the work; AK, LP, NL contributed to the interpretation of the data. AK, NL, and LP drafted the initial manuscript of the work; All authors contributed to the review and editing of the writing.

Funding

The authors have no sources of funding to disclose.

Data availability

The data underlying this article will be shared at a reasonable request to the corresponding author.

Declarations

Ethics approval and consent to participate

The study was conducted according to the Declaration of Helsinki and was approved by the Institutional Review Board at each center. The study protocol was reviewed by the respective local Ethics Committee at Rabin Medical Center. The requirement for written informed consent was waived due to the retrospective nature of this study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Clinical trial number

Not applicable.