Predicting Major Adverse Cardiovascular Events in Acute Coronary Syndrome: A Scoping Review of Machine Learning Approaches

Abstract

Background  Acute coronary syndrome is the topmost cause of death worldwide; therefore, it is necessary to predict major adverse cardiovascular events and cardiovascular deaths in patients with acute coronary syndrome to make correct and timely clinical decisions.

Objective  The current review aimed to highlight algorithms and important predictor variables through examining those studies which used machine learning algorithms for predicting major adverse cardiovascular events in patients with acute coronary syndrome.

Methods  To predict major adverse cardiovascular events in patients with acute coronary syndrome, the preferred reporting items for scoping reviews guidelines were used. In doing so, PubMed, Embase, Web of Science, Scopus, Springer, and IEEE Xplore databases were searched for articles published between 2005 and 2021. The checklist “Quality assessment of machine learning studies” was used to assess the quality of eligible studies. The findings of the studies are presented in the form of a narrative synthesis of evidence.

Results  In total, among 2,558 retrieved articles, 22 studies were qualified for analysis. Major adverse cardiovascular events and mortality were predicted in 5 and 17 studies, respectively. According to the results, 14 (63.64%) studies did not perform external validation and only used registry data. The algorithms used in this study comprised, inter alia, Regression Logistic, Random Forest, Boosting Ensemble, Non-Boosting Ensemble, Decision Trees, and Naive Bayes. Multiple studies ( N  = 20) achieved a high area under the ROC curve between 0.8 and 0.99 in predicting mortality and major adverse cardiovascular events. The predictor variables used in these studies were divided into demographic, clinical, and therapeutic features. However, no study reported the integration of machine learning model into clinical practice.

Conclusion  Machine learning algorithms rendered acceptable results to predict major adverse cardiovascular events and mortality outcomes in patients with acute coronary syndrome. However, these approaches have never been integrated into clinical practice. Further research is required to develop feasible and effective machine learning prediction models to measure their potentially important implications for optimizing the quality of care in patients with acute coronary syndrome.

Background and Significance

Acute coronary syndrome (ACS) is an intense category of coronary heart disease which leads to a complete or incomplete occlusion of the coronary artery. 1 The ACS spectrum includes: ST-segment elevation myocardial infarction (STEMI), non-STEMI, and unstable angina pectoris 2 which are associated with high adverse events and mortality. 3 Patients with ACS are at a high risk of adverse prognosis, 4 and approximately 15% of them experience major adverse cardiovascular events (MACEs), such as death, heart failure, or revascularization 1 year after diagnosis. 5 6 According to the Global Registry of Acute Coronary Events (GRACE), hospital mortality rate in ACS patients is 5.6%, and 1-year mortality rate is roughly 15%. 7 Risk prediction models are usually used in health care services to identify high-risk patients and make the best treatment decisions. 8 Cardiovascular disease risk prediction models have been developed through machine learning (ML) and regression-based approaches with due consideration to prognostic factors. 9 The thrombolysis in myocardial infarction (TIMI) and GRACE risk scores are the most popular risk prediction models for cardiovascular events. 10 Based on patients clinical features at the time of admission, the GRACE score predicts the risk of 6-month mortality after discharge. 11 ML-based approaches can solve the limitations of traditional regression-based prediction models, enhance the prediction accuracy for cardiovascular disease, and prevent unnecessary treatments. 12 In mortality forecasting, these approaches seek to achieve a high accuracy of prediction and attain an excellent ability to process missing and outlier data. 9

ML, as a subset of artificial intelligence, offers a class of models that can repeatedly learn from data, identify complex data patterns, and predict results. 13 It uses various computational algorithms to describe patterns applied for learning the existing information in datasets in a process called training. 14 15 In fact, ML approaches automatically learn the relationships from the predictor features (training data) and provide insightful knowledge which is then used to make predictions or decisions. 16 17

ML is generally divided into three types, i.e., supervised, unsupervised, and semi-supervised learning methods. 18 Labeled and unlabeled data are used in supervised and unsupervised learning, respectively, while both types of data can be employed in semi-supervised learning. 19 However, supervised ML, due to the heterogeneity of the medical data, is preferred in medical settings. 20

Actually, ML approaches use known data to predict outcomes for unlabeled data. Hence, the accuracy of a model depends on both the accuracy of its output and model training. 19 The performance of ML is enhanced according to the number of high-quality samples. 14 18 19 20 Suitable ML approaches offer generalizable analysis and interpretation of complex variables. 21 In fact, ML approaches apply algorithms to detect trends and patterns not identified through traditional statistical approaches. 22

Nowadays, studies on ML techniques have gained a lot of attention from medical researchers addressing clinical problems. 23 Therefore, it is necessary to expand the strengths and generalization of ML models to health care environment. 24 The main challenge for this issue is the assessment of ML-based predictive models in real health care settings. 20 In some studies, ML-based prediction models had limited application due to poor study design and inappropriate reporting of the results. Nonetheless, if these models are appropriately developed, validated, implemented, and assessed in real settings, they can improve patient benefit. 25

Several studies have compared the performances of ML models for predicting medical outcomes. 26 For example, Benedetto et al compared the discrimination accuracy of ML and regression models and found that ML models provide better discrimination in predicting mortality following cardiac surgery, yet the extent and clinical effect of this improvement is not clear. 27 Consequently, it is necessary to use ML approaches for early prediction and detection of important cardiovascular complications. Due to the importance of this issue, many studies have designed and created variety of prediction algorithms through ML. Yet the implementation of ML approaches in predicting mortality and MACEs in ACS patients for making correct and timely clinical decisions remains a challenge. However, to date, to the best of our knowledge, no scoping review has specifically reviewed studies on the use of ML algorithms for predicting MACEs in ACS patients.

Objectives

The current study intended to synthesize the studies which used ML algorithms for predicting MACEs in ACS patients and highlight algorithms and important predictor variables.

Methods

This scoping review was conducted from March 2020 to May 2021 by using Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist.

Information Sources

In this study, Scopus, PubMed (Medline), Web of Science, EMBASE, and IEEE databases were searched for articles published in English between 2005 and 2021.

Search Strategy

A combination of keywords and Medical Subject Headings (MESH terms) associated with ACS, ML, and MACEs, as well as Boolean AND, OR operators, Truncation operator, (asterisk *), Quotation search, (quotation mark “ ”) was used to search article titles and abstracts. The complete search strategy is presented in Supplementary Material A (available in the online version).

In addition to the evaluation of the full text articles, their references were manually searched to find other suitable articles.

Eligibility Criteria

1. Inclusion criteria

   Articles with one or more of the following criteria were included in the analysis:

   • Articles applying ML algorithms to predict MACEs in patients with ACS.

   • Articles published between 2005 and 2021.

   • Original research articles.

   • Articles published in peer reviewed journals.

   • Articles reporting at least one evaluation index.

2. Exclusion criteria

   Articles with one or more of the following criteria were excluded from the analysis:

   • Articles using classical statistical methods to predict MACEs.

   • Articles focusing only on electrocardiography (ECG) interpretation to predict MACEs.

   • Articles not published in English.

   • Articles without available full texts.

Selection of the Sources of Evidence

All retrieved studies were carefully examined and the duplicates were eliminated. Then, two authors (S.C. and S.S.) independently screened the titles and abstracts against the inclusion and exclusion criteria and removed the irrelevant studies. Any disagreements were resolved through discussion. The reviewers also agreed on the results of the studies.

Data Extraction and Appraisal

The full texts of relevant articles were independently examined by two authors (S.C. and S.S.). The data were gathered by using a data extraction form which was designed based of the Critical Appraisal Checklist and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) in an Excel spreadsheet. The necessary information (such as the sources of data, participants, predicted outcomes, candidate predictors, sample size, missing data, model development, model performance, model evaluation, results, interpretations, and discussions) was extracted from each study and recorded in the form.

Quality Assessment

Two reviewers (F.S. and S.S.) independently evaluated the quality of studies with the quality assessment tool proposed by Qiao. 24 The tool consists of five categories: unmet needs (limits in current non machine-learning approach), reproducibility (feature engineering methods, platforms/packages, and hyper-parameters), robustness (valid methods to overcome over-fit, the stability of results), generalizability (external data validation), and clinical significance (predictors explanation and suggested clinical use). Based on the results, the studies were classified as low, intermediate, and high quality if they obtained less than five, five to seven, or more than eight positive responses, respectively.

Synthesis of the Results

The findings of the studies were presented in the form of a narrative synthesis of evidence. The included studies were categorized based on different characteristics, including the characteristics of ML algorithms and adverse events.

Results

Selection of Sources of Evidence

The results obtained from the search strategy in selected databases, as well as the process of identifying and selecting studies (based on the PRISMA flow diagram for the scoping review process) are presented in Fig. 1 . Altogether, among 2,558 retrieved articles, 1,262 were duplicates. The screening of the titles and abstracts of the articles led to the elimination of 1,245 more articles. The full-texts of 15 remaining studies and seven more articles from bibliographic search were examined based on our inclusion criteria, and finally 22 studies were incorporated into our review.

Fig. 1.

PRISMA flow diagram for the scoping review process of the literature search and study selection. PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses.

Characteristics of the Sources of Evidence

The general characteristics of the studies are presented in Table 1 . According to the results, ML algorithm has increasingly been applied to predict MACEs in ACS. That is, the number of studies on the application of ML algorithm in MACEs between 2018 and 2021 comprised 19 (86.36%) studies, whereas only three (13.63%) studies were published on the subject before 2018. Notably, the number of studies published by May 2021 was greater than the total number of studies published during 2016 and 2017.

Table 1. Extraction of information based on machine learning classification algorithms.

Id	First author, Year of publication, Reference	Outcome	ML algorithms	Source of data	No. of patient's size of training set test set validation set	Evaluation (AUC)
			Comparison
1	Sherazi et al, 2021 10	MACEs	Soft Voting Ensemble classifier (SVE)	Korea Acute Myocardial Infarction Registry KAMIR	N  = 11,189 STEMI ( N  = 5,389) NSTEMI ( N  = 5,800) Split Training (70%) Testing (30%) fivefolds cross validation	SVE 99.61 RF 98.96 ET 99.54 GBM 98.92
			RF Extra Tree (ET) GBM
2	D'Ascenzo et al,2021 4	One-year mortality	Adaptive Boosting PRAISE	BleeMACS and RENAMI Registry	N  = 19,826 adult patients ACS split training 80% validation 20%	PRAISE 0.82 NB 0.82 KNN 0.80 RF 0.83
			NB KNN RF
3	Borracci et al, 2021 43	In-hospital mortality	ANN (One-MLP, Two- MLP) Radial Basis Function Network (RBFN)	General Hospital	N  = 1,255 randomly split 70% training 30% validation	ROC area (CI 95%) One-MLP 0.890 Two-MLP 0.858 RBFN 0.841 LR 0.753
			GRACE LR equation
4	Bai et al et al, 2021 29	One-year mortality	LR KNN CatBoost RF XGBoost	General Hospital of Zunyi Medical University	N  = 656 SMOTEEN hybrid sampling algorithm validated by 10-fold cross validation	RF 0.99 CatBoost 0.99 XGBoost 0.98 LR 0.95 KNN 0.96 GRACE 0.80
			GRACE
5	Khera et al et al, 2021 28	In-hospital mortality	XGBoost ANN LASSO-LR	American College of Cardiology CP-MI Registry	N  = 755,402 AMI Derivation cohort ( n  = 564,918) Validation cohort ( n  = 190,48) Validated 25%	LR 0.888 LASSO 0.886 NN 0.885 XGBoost 0.898
			LR
6	Aziz et al, 2021 49	In-hospital 30 days one-year mortality	RF SVM LR	(NCVD-ACS) Registry	N  = 12,368 STEMI split Training70% Validation 30% 10-fold cross	RF In-hospital 0.86 30 days 0.83 1 year 0.78 SVM In-hospital 0.86 30 days 0.87 1 year 0.84 LR In-hospital 0.88 30 days 0.85 1 year 0.76 TIMI in-hospital 0.81 30 days 0.80 1 year 0.76
			TIMI
7	Lee et al, 2021 40	In-hospital 3-month one-year mortality	RF SVM XGBoost Lasso LR Ridge LR Elastic net LR	Korea Acute Myocardial Infarction Registry KAMIR	STEMI Survival Death N  = 5,155 N  = 402 NSTEMI Survival Death N  = 8,011 N  = 615 Random sampling training set (80%) Test set (20%) 10-fold cross	STEMI
						in-hospital Lasso 0.923 Ridge 0.923 Elastic net 0.923 RF 0.924 SVM 0.875 XGBoost 0.938
						3 month Lasso 0.777 Ridge 0.779 Elastic net 0.777 RF 0.763 SVM 0.667 XGBoost 0.784
						1 year Lasso 0.789 Ridge 0.789 Elastic net 0.917 RF 0.924 SVM 0.848 XGBoost 0.911
			ACTION TIMI GRACE			NSTEMI
						in-hospital Lasso 0.916 LR Ridge 0.918 Elastic net 0.923 RF 0.924 SVM 0.875 XGBoost 0.938
						3-month Lasso 0.849 Ridge 0.826 Elastic net 0.849 RF 0.799 SVM 0.715 XGBoost 0.824
						1 year Lasso 0.815 Ridge 0.809 Elastic net 0.814 RF 0.792 SVM 0.721 XGBoost 0.808
8	Lee et al, 2020 65	One-year mortality	RF (Bootstrap decision Forest and Bootstrap DTs model)	Korea Acute Myocardial Infarction Registry KAMIR	N  = 22,182 training 80% testing 20%	RF 0.924 KAMIR 0.918
			KAMIR
9	Sherazi et al, 2020 9	One-year mortality	DNN GBM GLM RF	Korea Acute Myocardial Infarction Registry KAMIR	N  = 8,227 80.297% training testing 19.703% random sampling	DNN 0.898 GBM 0.898 GLM 0.873 RF 0.883 GRACE 0.810
			GRACE
10	Li et al, 2020 66	One-year mortality	Gaussian NB LR KNN DTs RF XGBoost	General Hospital Western China Hospital Sichuan University	N  = 1,244 10-fold cross-validation	XGBoost 0.942 LR 0.931 NB 0.924 KNN 0.709 DT 0.772 RF 0.932 GRACE -
			GRACE
11	Kwon et al, 2019 6	In-hospital mortality one-year mortality	Deep-learning-based risk stratification for the mortality of patients with AMI (DAMI)	Korea Acute Myocardial Infarction Registry	N  = 25,977 random sampling Training 60% (36 hospitals 40% (23 hospitals)	STEMI DAMI 0.905 LR 0.873 RF 0.890 GRACE 0.851 TIMI 0.852 ACTION 0.781 NSTEMI DAMI 0.870 LR 0.845 RF 0.851 GRACE 0.810 TIMI 0.806 ACTION 0.593
			GRACE ACTION TIMI RF LR
12	Duan et al, 2019 67	MACEs	DNN (Dynamic)	Chinese PLA General Hospital	N  = 2,930 ACS patient samples train and test set with a ratio of 4:1 fourfold of data as the training set and the remaining onefold as the test set	Dynamic 0.713 LR 0.637 RMTM 0.700
			LR Boosted-RMTM
13	Hu et al, 2019 35	MACEs	Ensemble (Rough Set Theory and Dempster-Shafer Theory (RST/DST)	Chinese PLA General hospital	N  = 2,930 ACS patient fivefold cross validation	Ensemble (RST/DST) 0.7 SVM 0.707 L1-LR 0.707 CART 0.630 GRACE 0.636 Bagging 0.700 AdaBoost 0.678
			SVM CART GRACE Ensemble Bagging Ensemble AdaBoost LR
14	Payrovnaziri et al, 2019 44	One-year mortality	DNN	MIMIC-III dataset	N  = 5,436 10-fold-cross 90% training 10% testing	DNN 0.928 Simple Logistic 0.723 LMT 0.724
			Simple Logistic logistic model trees (LMT)
15	Kim et al, 2019 50	MACEs	DNN	Korea Acute Myocardial Infarction Registry	Random sampling split Training 60% Validation 20% Test 20% 10-fold Cross	DNN 1 M 0.97 6 m 0.94 12 m 0.96 GBM 1 m 0.96 6 m 0.95 12 m 0.95 GLM 1 m 0.76 6 m 0.67 12 m 0.72 GRACE 1 m 0.75 6 m 0.72 12 m 0.76
			GBM GLM GRACE
16	Raza et al, 2019 45	One-year mortality	ANN NB SVM DTs	Gulf Registry of Acute Coronary Events	N  = 6,847 10-fold cross validation randomly split 80:20	NN 0.746 NB 0.832 SVM 0.840 DT 0.602 LR 0.843
			LR
17	Piros et al, 2019 42	30-day mortality one-year mortality	DTs ANNs	Registry HUMIR	N  = 47,391 Resampling bootstrap proportion of 7:3 in training and validation	30 d DT 0.788 NN 0.837 LR 0.836 1 y DT 0.754 NN 0.8194 LR 0.8191
			LR
18	Hernesniemi et al, 2019 41	6-month mortality	LR XGBoost	MADDEC – database comprises EHR KARDIO-registry	N  = 9,066 ACS patients Training (70%) and validation (30%)	LR 0.867 XGBoost 0.890 GRACE 0.822
			GRACE
19	Pieszko et al, 2019 36	In-hospital mortality	Dominance-based Rough Set Rough Rule Ensemble (DRSA-BRE)	Local Cardiology Unit	N  = 5,678 patients Fivefold cross-validation	LR 68 XGBoost 78 DRSA-BRE 81.0
			LR XGBoost
20	Li et al, 2017 64	In-hospital mortality	Logistic regression (LR) stepwise Cox CHAID RF NB Bayes network	Chinese Acute Myocardial Infarction (CAMI) Registry	N  = 18,744 patients hospitalized in 2014 training set (9,619 patients) patients hospitalized in 2013 testing set (9,125 patients)	LR 0.843 LR stepwise 0.843 Cox 0.842 Cox stepwise 0.839 CHAID 0.801 RF 0.846 NB 0.825 Bayes Network 0.835 GRACE 0.809 TIMI 0.774
			GRACE TIMI
21	Mansoor et al, 2017 68	In-hospital mortality	Multivariate logistic regression (MLR)	National Inpatient Sample (NIS)	N  = 9,637 patients 80/20% random sample split threefold cross validation	MLR 0.84 RF 0.81
			RF
22	Hu et al, 2016 69	MACEs	SVM RF NB LR	Chinese PLA General Hospital	N  = 2,930 Controls Patient 2,178 Samples with MACE 752 fivefold cross validation	RBMLP SVM 0.703 RF 0/724 NB 0.695 LR 0.705 GRACE 0.636 TIMI 0.579 CRFs SVM 0.705 RF0 0.723 NB 0.695 LR 0.706 GRACE 0.641 TIMI 0.576
			GRACE TIMI

Abbreviations: ACS, acute coronary syndrome; ANN, artificial neural networks; DNN, deep neural network; DT, Decision Tree; GBM, gradient boosting machine; GLM, generalized linear model; GRACE, Global Registry of Acute Coronary Events; KNN, K-nearest neighbor; MACE, major adverse cardiovascular event; NB, Naïve Bayes; RF, Random Forest; SVM, support vector machine; TIMI, thrombolysis in myocardial infarction; XGBoost, Extreme Gradient Boosting.

Based on the aim of the study, the results were divided into two main categories, including the investigation of ML algorithms and essential predictor variables. As indicated in Table 2 , the most common adverse event outcomes were related to 1-year mortality ( N  = 10), while 3 and 6-month mortality were each predicted in only one study. The most common ML method was Logistic Regression (LR), followed by Random Forest (RF), and Boosting Ensemble which were used in 17, 12, and 11 studies, respectively.

Table 2. General characteristics of the included studies.

Adverse event	No. of studies	Percent
One-year mortality	10	45.45 %
In-hospital mortality	8	36.36 %
MACEs	5	22.72 %
30-day mortality	2	9%
6-month mortality	1	4.5 %

Moreover, registry data, hospital record data, and national database were used in 14 (63.63%), seven (31.81%), and three (13.63%) studies, in that order. Only one study used information of randomized trial participants.

The population size used in the selected studies were greatly different. The largest population, 755,402 patients with acute myocardial infarction (MI) from nationwide registry, was recruited in a study by Khera et al, 28 and the smallest population comprised 656 patients with STEMI who participated in a study in China. 29 Twelve studies were conducted on a population of fewer than 10,000 patients, and 10 studies were performed on a population of over 10,000.

The number of predictor variables used in ML models was also different. For instance, one study used models with several input variables fewer than 10, in 14 studies there were 11 to 50 input variables in the models, and in seven studies the models included more than 50 input variables. The extracted variables were divided into three categories, i.e., demographic, clinical, and therapeutic features. Majority of studies ( N  = 20) achieved a high area under the ROC curve (AUC), between 0.8 and 0.99, in predicting mortality and MACEs. All studies were retrospective, but none of them reported the integration of ML models into clinical practice. However, the risk score calculator for each outcome was available online in only one study. 4

Prediction of the Outcomes

In this review, 22 studies used ML algorithms to predict adverse events in ACS patients. The measured outcomes included mortality (77.28%) and MACEs (22.72%). As shown in Table 1 , four studies (18.18%) predicted multiple outcomes which were then utilized to compare the performances of different ML models. Short-term mortality, in-hospital and 30-day mortality were predicted in eight and two studies, in that order, while long-term and 1-year mortality were predicted in 10 (45.45%) studies. Six-month mortality was predicted only in one study (4.5%).

Machine Learning Algorithm

The ML algorithms used in the selected studies are shown in Fig. 2 . The algorithms are divided into supervised, unsupervised, and semi-supervised ML. 30

Fig. 2.

Machine learning algorithm used in the selected studies.

Supervised ML attempts to develop an algorithm in data with known outcomes. 31 In contrast, in unsupervised ML unlabeled data are used. 32 Semi-supervised learning attempts to develop an algorithm when a few samples are labeled. 33

Supervised learning algorithms include LR, K-nearest neighbors (KNN), support vector machine (SVM), Naive Bayes (NB), Decision Trees (DTs), RF, Boosting Ensemble method, and artificial neural networks (ANNs). 31 Deep neural network (DNN) can categorize both supervised and unsupervised ML. In fact, deep learning process occurs through understanding the connections between input and output variables in supervised learning, or between subsets of variables in unsupervised learning. 34

According to the results, supervised learning techniques have been used in all reviewed articles. The most common ML method was LR used in 17 (77.27%) articles, whereas the Ensemble methods, such as Bagging, 35 Voting, 10 and Rough set-based DT Ensemble algorithm 35 36 37 were used in four (18.18%) studies.

RF is a Bagging-type Ensemble that uses multiple DTs models to obtain more accurate results 38 ; it was used in 12 (54.54%) studies. Boosting is another Ensemble technique which sequentially combines multiple ML models with high bias models to correct the predictions of models and obtain better predictions. 39 This Ensemble type was used in 11 (50%) studies; gradient boosting machine and Extreme Gradient Boosting (XGBoost) were used in six and three studies, respectively. CatBoost and AdaBoost were each used in one study. In total, 19 (86.36%) articles used Ensemble techniques, but for a better investigation each Ensemble type is reported separately.

ML methods like NB, DNN, ANNs, DTs, and SVM were each used in five (22.72%) studies. However, fewest number of methods were used in those studies which included KNN (13.63%) and generalized linear model (4.50%). Multiple ML algorithms were employed in 12 (54.54%) studies. The sample size in the selected studies varied from hundreds to thousands. With regards to validation methods, k -fold cross-validation, bootstrap, and random split of data were used in 12 (54.54%), three (9%), and 12 (54.54%) studies, in that order.

The ratio of training and test datasets was not mentioned in four (18.18%) studies. To evaluate the performances of the applied algorithms, various evaluation indices, such as accuracy, sensitivity, specificity, AUC, positive predictive value, and negative predictive value, were reported. The reported quantity of AUC index related to different algorithms is presented in Table 1 .

Model Performance

Table 3 summarizes the AUC performance of models. The highest AUC (99.61% with 36 features) for MACEs prediction models was achieved by the soft voting ensemble (SVE) classifier in a study by Sherazi et al. 10 The table also highlights the performance of the best models along with the number of predictor variables used in each study. In-hospital mortality and top-performing AUC (92%) were achieved by RF with 36 features in a study by Lee et al. 40 With regard to 1-year mortality, the best performance was achieved in a study by Bai et al where the AUC achieved by CatBoost and RF models was 99% after optimizing the sampling technique with 37 features. 29 As for 6-month mortality, the best performance achieved by XGBoost was 89% with 76 features. 41 Fig. 3 shows the best AUC-based outcomes. It can be seen that Ensemble methods ( N  = 10) with RF ( N  = 5) and Boosting Ensemble technique (specially XGBoost) ( N  = 3) has the best performance.

Table 3. Performance of the best model along with the number of predictor variables in each study.

Study	No. of predictors features	ML algorithms	AUC
MACEs
Sherazi et al, 2021 10	36	SVE	99.61
Duan et al, 2019 67	22	DNN (Dynamic)	71.13
Kim et al, 2019 50	51	DNN	97
Hu et al, 2019 35	22	Ensemble (RST/DST)	71.5
Hu et al, 2016 69	286	RF	72.4
In-hospital mortality
Aziz et al, 2021 49	50	LR	88
Khera et al, 2021 28	22	XGBoost	89.8
Borracci et al, 2021 43	8	ANN(One-MLP)	89
Lee et al, 2021 40	55	RF	92
Kwon et al, 2019 6	40	DNN (DAMI) STEMI / NSTEMI	90.05 / 87
Li et al, 2017 64	17	RF	84.6
Pieszko et al, 2018 36	29	Ensemble (DRSA-BRE)	81
Mansoor et al, 2017 68	11	MLR	84
One-year mortality
Ascenzo et al, 2021 4	25	Boosting PRAISE	82
Aziz et al, 2021 49	50	RF	87
Lee et al, 2021 40	55	XGBoost STEMI LR Lasso NSTEMI	91 81.5
Lee et al, 2021 65	95	RF	92.4
Bai et al, 2021 29	37	RF / CatBoost	99
Sherazi et al, 2020 9	69	DNN GBM	89.8
Li et al, 2020 66	59	XGBoost	94.2
Payrovnaziri et al, 2019 44	279	DNN	92.8
Raza et al, 2019 45	24	LR	84.3
Piros et al, 2019 42	23	ANN	81.94
6-month mortality
Hernesniemi et al, 2019 41	76	XGBoost	89
3-month mortality
Lee et al, 2021 40	55	RF STEMI LR Elastic Net NSTEMI	87 84.9
30-day mortality
Aziz et al, 2021 49	50	RF	85
Piros et al, 2019 42	23	ANN	83.7

Abbreviations: ANN, artificial neural networks; DNN, deep neural network; DT, Decision Tree; GBM, gradient boosting machine; KNN, K-nearest neighbor; LR, Logistic Regression; MLR, Multivariate logistic regression; NB, Naïve Bayes; RF, Random Forest; STEMI, ST-segment elevation myocardial infarction; SVM, support vector machine; TIMI, thrombolysis in myocardial infarction; XGBoost, Extreme Gradient Boosting.

Fig. 3.

Performance of the best models based of outcome.

Comparisons

The most frequently used comparator was GRACE which was used in 11 studies. The other regression-based prediction tools, such as TIMI, acute coronary treatment and intervention outcomes network, and Korean acute myocardial infarction registry were used in five, two, and one study, respectively. Ten (45.45%) studies compared the performances of various ML algorithms followed by LR and RF which were used as comparators in six (54.54%) and five (54.54%) studies, in that order. Nevertheless, in two studies by Khera et al 28 and Piros et al, 42 ML models compared with logistic regression did not show increased performance.

Important Predictor Variables

The number and type of predictor variables in ML models were also different. The largest number of input variables ( N  = 286) was used by Hu et al, 35 and the smallest number ( N  = 8) was used by Borracci et al. 43 Overall, five studies used models with 20 or less input variables and eight studies used models with more than 50 input variables. However, the type of variables used by Payrovnaziri et al 44 was unknown. The extracted variables of the selected studies were divided into three categories, i.e., demographic, clinical, and therapeutic features. Therapeutic features were not used in nine studies.

The important variables included: hypertension (HTN), diabetes mellitus, age, creatinine, sex, systolic blood pressure (SBP), fasting blood sugar (FBS), heart rate (HR), post percutaneous coronary intervention (PCI), history of congestive heart failure (CHF), ECG, current smoking, diastolic blood pressure (DBP), post coronary artery bypass graft (CABG), history of stroke, maximum troponin T, and Killip Class which were all mentioned in nine or more studies. However, some biomarkers, such as estimated glomerular filtration rate (EGFR), history of chronic obstructive pulmonary disease, and creatinine clearance were reported only in one study. The frequency of the extracted features is highlighted in Table 4 . The superscripts above the features demonstrate the number of replicates in the studies.

Table 4. List of features extracted from articles.

Demographic features	Age, 17 Sex, 15 Weight (kg), 8 Height (cm), 6 Race, 2 Patient alive
Clinical features	Physical examination	Hypertension (HTN), 18 systolic blood pressure (SBP), 15 heart rate (HR), 14 electrocardiography (ECG) findings 13 (STEMI), 8 ST-segment depression, 3 NSTEMI, 2 T-wave inversions, transient ST-segment elevation, right bundle branch block (RBBB), 2 left bundle branch block (LBBB) 2 , current smoking, 13 diastolic blood pressure (DBP), 11 Killip Class 9 (Class I—II, Class III, Class IV), Echocardiographic finding 9 left ventricular ejection fraction (LVEF) 8 ), cardiogenic shock, 7 chest pain, 6 heart rhythm, 6 dyspnea, 4 body mass index (BMI), 3 sweat, 2 bleeding, 2 abdominal circumference, vertigo and systemic weakness, awareness, estimated glomerular filtration rate (EGFR), ischemia location, mitral regurgitation grade, waist-to-hip ratio (WHR)
	Medical history	post percutaneous coronary intervention (PCI), 12 history of congestive heart failure (CHF), 11 post coronary artery bypass graft (CABG), 10 history of stroke, 10 history of myocardial infarction (MI), 8 family history of heart disease, 8 history of smoking, 8 history of peripheral artery disease (PAD), 6 previous angina, 3 cardiac arrest, 3 history of bleeding, 2 history of dyslipidemia, 2 hyperlipidemia, 3 history of atrial fibrillation (AF), 3 history of ischemic heart disease (IHD), 2 history of chronic obstructive pulmonary disease (COPD), past regular medication
	Comorbid conditions	Diabetes mellitus, 18 chronic renal disease, 8 dyslipidemia, 6 cancer, 4 coronary heart disease (CHD), 4 chronic lung disease, 4 arteriosclerosis, 3 chronic liver disease, valvular heart disease
	Laboratory findings	Creatinine, 16 fasting blood sugar (FBS), 14 maximum troponin T, 9 hemoglobin, 8 low-density lipoprotein (LDL) cholestrol, 7 triglyceride, 7 total cholesterol, 7 maximum troponin i, 8 HDL (High-density lipoprotein) cholestrol, 7 maximum creatine kinase (CK), 8 creatine kinase myoglobin form (CK-MB), 7 white blood cell counts (WBC), 6 potassium, 5 C-reactive protein (CRP), 5 alanine aminotransferase (ALT), 5 aminotransferase aspartate (AST) 4 , N-terminal pro-brain natriuretic peptide (NT-probnp), 4 sodium, 3 cystatin, 2 blood urea nitrogen (BUN), 2 red blood cell counts (RBC), 2 platelet, 2 prothrombin time (PTT), 2 urinalysis (UA), 2 total serum bile acids (TSBA), calcium, thrombin time (TT), international normalized ratio (INR), albumin, urine color, ApoA-1, B-type natriuretic peptide, D-dimer, hematocrit, thyroid-stimulating hormone (TSH), thromboplastin time, uric acid, neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), monocyte-to-lymphocyte ratio (MLR), high systemic inflammation response index (SIRI), systemic immune-inflammation index (SII), gamma-glutamyl transferase (GGT), α-Hydroxybutyrate dehydrogenase (α-HBDH), carbamoyl phosphate synthetase (CPS), creatinine clearance, hemodynamics instability, dominance (right, left, or balanced)
Therapeutic features	Treatment: coronary angiographic finding 7 (three-vessel disease, left main disease, 3 stenosis in left anterior descending coronary artery (LAD), stenosis in left main coronary artery (LMCA), stenosis in right coronary artery (RCA), left circumflex coronary arteries (LCX)), angiographic variables 5 (PCI 4 PCI with drug-eluting stent 2 revascularization 3, vascular access, thrombus aspiration), CABG surgery, 3 intra-aortic balloon pump (IABP), 2 Complications, 2 infection during hospitalization, New York Heart Association (NYHA) functional class, stress test, post procedural TIMI, mitral regurgitation grade, initial therapeutic strategy, medical therapy in hospital, resuscitation Medication: Statin medication, 6 aspirin, 5 angiotensin-converting enzyme (ACE) inhibitors, 5 beta blocker medication, 5 anticoagulation, 5 angiotensin receptor blockers (ARBs), 4 antiplatelet 4 (glycoprotein IIB/IIIA Inhibitors [GP], 1 P2Y12 inhibitors), diuretic medication, 2 antagonist medication 2 (calcium antagonist medication), hypoglycemic medications, 2 thrombolysis medication, 2 length of stay, 2 spironolactone, lipid-lowering medication, oral insulin, warfarin, proton-pump inhibitors (PPI), clopidogrel, prasugrel, ezetimibe, calcium channel blockers (CCB), heparin (unfractionated heparin (UFH,LMWHS), medications at discharge

Note: The superscripts above the features demonstrate the number of replicates in the studies.

Table 5 presents the number of demographics, clinical, and therapeutic features in the prediction model. In a study by Raza et al, 45 1-year mortality was predicted with 24 features, such as History of MI, hyperlipidemia, HR, SBP, DBP, diabetes mellitus, Killip class type, and ECG finding. The AUC achieved by LR model in this study was 0.843. 45 Killip class is an important independent predictor of mortality and higher types are associated with increased mortality risk in ACS patients. 46 Killip classes are defined as class I-IV. Class I is defined as patients without any clinical sign of heart failure, class II refers to patients with crackles or rales in the lungs, class III is defined as patients with evident acute pulmonary edema, and class IV refers to patients with cardiogenic shock or hypotension. 47 Killip classes were used in nine (40.90%) studies.

Table 5. Important features of the articles.

ID	First author, year of publication Reference	Event	No. of predictors features	Important features
				Demographic features	Clinical features	Therapeutic features
1	Sherazi et al, 2021 10	MACEs	36	Age Sex Height Weight	Abdominal circumference, SBP, DBP, HR, Chest pain, dyspnea, current smoking, previous angina, history of MI, family history of heart disease, history of dyslipidemia, history of HTN, diabetes mellitus, post PCI, FBS, creatinine, maximum CK, maximum CK-MB, maximum troponin I, maximum troponin T, total cholesterol, LDL, HDL, triglyceride, CRP, NT-Probnp, echocardiographic finding (LVEF), ECG finding (STEMI, ST-Segment depression, RBBB, LBBB)	–
2	D'Ascenzo et al, 2021 4	One-year mortality	25	Age Sex	Diabetes mellitus, HTN, hyperlipidemia, history of (PAD), EGFR, chronic renal disease history of MI, post PCI, post CABG, history of stroke, history of bleeding, cancer, ECG findings (STEMI), hemoglobin, echocardiographic finding (LVEF)	Beta blocker medication, Statin medication, ACE inhibitors, anticoagulation, proton-pump inhibitors, angiographic variables (PCI with drug-eluting stent, revascularization, vascular access), coronary angiographic finding (three-vessel disease)
3	Borracci et al, 2021 43	In-hospital mortality	8	Age	Killip Class, SBP, ECG findings (STEMI, ST-segment depression), cardiac arrest, creatinine, maximum CK, maximum CK-MB, maximum troponin I, maximum troponin T, HR	–
4	Bai et al, 2021 29	One-year mortality	37	Age Sex	HTN, diabetes mellitus, current smoking, history of stroke, chronic renal disease, ECG findings, cardiogenic shock, WBC, BUN, creatinine, cystatin, FBS, ALT, AST, HDL, LDL NT-Probnp, RBC, PLT, Maximum CK, Maximum CK-MB, uric acid, HR, SBP, DBP, NLR, PLR, MLR, SIRI, SII, hemoglobin, GGT, Α-HBDH	–
5	Khera et al, 2021 28	In-hospital mortality	29	Age Weight Sex Race	Diabetes mellitus, history of HTN, history of dyslipidemia, current smoking, chronic lung disease, chronic renal disease, history of MI, history of CHF, post PCI, post CABG, history of AF, history of stroke, history of (PAD), cardiogenic shock, HR, SBP, ECG findings maximum troponin I, maximum troponin T, creatinine, creatinine clearance, hemoglobin	–
6	Aziz et al, 2021 49	In-hospital mortality 30 day mortality One-year mortality	50	Age Race Sex	Current smoking, history of HTN, diabetes mellitus, family history of heart disease, history of mi, history of CHF, chronic lung disease, chronic renal disease, history of stroke, HR, SBP, DBP, Killip Class Total cholesterol, HDL, LDL, triglyceride, FBS ECG finding (STEMI, ST-segment depression, T-wave inversion, RLBB, LLBB)	Aspirin, Beta blocker medication, Statin medication, ACE inhibitors, ARBS, angiographic variables (PCI), CABG antiplatelet (glycoprotein IIB/IIIA inhibitors (GP), diuretic medication, calcium antagonist medication, heparin (UFH, LMWHS), lipid-lowering medication, oral hypoglycemic medications, insulin, antiarrhythmic medication
7	Lee et al, 2021 40	In-hospital mortality 3-month One year mortality	55	Age Sex Height Weight	HTN, diabetes mellitus, dyslipidemia, history of MI, post PCI, history of stroke, current smoking, history of smoking, chest pain, dyspnea, awareness, sweat, vertigo and systemic weakness, SBP, DBP, HR, history Of CHF, cardiogenic shock, ECG finding, history of AF, Maximum troponin I, Maximum troponin T, creatinine, hemoglobin, echocardiographic finding (LVEF)	Aspirin, Statin medication, ACE inhibitors, ARBs, warfarin, clopidogrel, prasugrel, ticagrelor, beta blocker medication, CCB, ezetimibe, anticoagulation, oral hypoglycemic medications, coronary angiographic finding (three-vessel disease, left main disease)
8	Lee et al, 2020 65	One-year mortality	95	Age Sex	CHD, BMI, diabetes mellitus, HTN, dyslipidemia, current smoking, family history of heart disease, history of MI, previous angina, SBP, HR, Killip Class, ECG finding (STEMI), bleeding, heart rhythm, CHF, cardiogenic shock, FBS, Maximum troponin I, creatinine, CRP, LDL echocardiographic finding (LVEF)	Aspirin, statin medication, beta blocker medication, ACE inhibitors, ARBs, angiographic variables (PCI) coronary angiographic finding (three-vessel disease, stenosis in LAD), medications at discharge, post procedural TIMI, antiplatelet (P2Y12 inhibitors), spironolactone
9	Sherazi et al, 2020 9	One-year mortality	69	Age Sex	SBP, DBP, HR, WHR, chest pain, BMI, FBS, creatinine, maximum CK, maximum CK-MB, maximum troponin I, maximum troponin T, total cholesterol, triglyceride, HDL, LDL, CRP, NT-Probnp, echocardiographic finding (LVEF), dyspnea, previous angina, ECG finding, ischemia location, heart rhythm, history Of IHD, history Of HTN, diabetes mellitus, history of dyslipidemia, history of smoking, family history of heart disease, past regular medication, Killip class, post PCI, post CABG, Echocardiographic Finding	Angiographic variables (PCI, PCI with drug-eluting stent, revascularization), coronary angiographic finding, thrombolysis medication, medications at discharge, IABP, stress test, mitral regurgitation grade, complications, initial therapeutic strategy, medical therapy in hospital, resuscitation
10	Li et al, 2020 66	One-year mortality	59	Age Sex	HR, SBP, DBP Killip Class ≥2 chest pain, heart rhythm, history of smoking, history of HTN, diabetes mellitus, history of COPD, history of bleeding, history of CHF echocardiographic finding (LVEF), RBC, hemoglobin, platelet, WBC, TSBA, ALT, albumin, FBS, BUN, creatinine, cystatin, uric acid, triglyceride, total cholesterol, HDL, LDL, sodium, potassium, PTT, TT, D-dimer, B-type natriuretic peptide	Beta blocker medication, statin medication, ACE inhibitors, ARBs, anticoagulation coronary angiographic finding (three-vessel disease, left main disease), antiplatelet, angiographic variables (PCI, revascularization, thrombus aspiration), thrombolysis medication, diuretic, medication, IABP, NYHA ≥2 At discharge, infection during hospitalization
11	Kwon et al, 2019 6	In-hospital mortality one-year mortality	40	Age Sex	HTN, BMI, diabetes mellitus, dyslipidemia, current smoking, history of CHF, chronic renal disease, chronic lung disease, chronic liver disease, cancer, history of MI, history of stroke, post PCI, post CABG, family history of heart disease, chest pain, dyspnea, Killip class, SBP, DBP, HR, ECG finding (STEMI), cardiac arrest, FBS, creatinine, maximum CK-MB, maximum troponin I, total cholesterol	Aspirin, statin medication, anticoagulation antiplatelet
12	Duan et al, 2019 67	MACEs	22	Age Sex Height Weight	SBP, DBP, diabetes mellitus, HTN, history Of CHF, arteriosclerosis, history of smoking, current smoking, creatinine, maximum CK, ALT, AST, maximum troponin T, FBS, post PCI, post CABG, echocardiographic finding (LVEF)	CABG surgery, length of stay
13	Hu et al, 2019 35	MACEs	22	Age Sex Height Weight	SBP, DBP, diabetes mellitus, HTN, history of CHF, arteriosclerosis, history of smoking, current smoking, echocardiographic finding (LVEF) Creatinine, maximum CK, ALT, AST, maximum troponin T, FBS post PCI, post CABG	CABG surgery, length of stay
14	Payrovnaziri et al, 2019 44	One-year mortality	279	–	–	–
15	Kim et al, 2019 50	MACEs	51	Age Sex height weight	HTN, HR, Killip class, heart rhythm, diabetes mellitus, chest pain, dyslipidemia history of smoking, family history of heart disease, history of IHD, FBS, creatinine, maximum CK, maximum CK-MB, maximum troponin I, maximum troponin T, total cholesterol, triglyceride, HDL, LDL, CRP, NT-Probnp, hemoglobin	–
16	Raza et al, 2019 45	One-year mortality	24	_	Post angina, history of MI, history of CHF, post PCI, post CABG, history of stroke, post PAD, history of smoking, diabetes mellitus, HTN, hyperlipidemia, HR, SBP, DBP, Killip class, maximum CK, maximum CK-MB, ECG finding	__
17	Piros et al, 2019 42	30-day and 1-year mortality	23	alive Date/death Date /admission	History of MI, history of CHF, HTN, history of stroke, diabetes mellitus, post PAD, hyperlipidemia, current smoking, cardiogenic shock ECG finding (STEMI, NSTEMI), creatinine	PCI during hospital stay
18	Hernesniemi et al, 2019 41	Six-month mortality	76	Age Sex	Creatinine, WBC, CRP, maximum troponin T, hemoglobin, potassium, FBS, platelet, INR, history of CHF, sodium, hemodynamics instability, potassium, post PCI, cardiac arrest, hematocrit, history of stroke, history of PAD, cancer, post CABG, diabetes mellitus, dominance (right, left or balanced), chronic renal disease, valvular heart disease, post angina, history of AF, diabetes mellitus	Anticoagulation, angiographic finding stenosis (RCA, LAD, LCX, RCA, LMCA)
19	Pieszko et al, 2018 36	In-hospital mortality	29	Sex	Diabetes mellitus, FBS, HTN, history of smoking, current smoking, triglyceride, sodium, potassium, TSH, total cholesterol, UA, hemoglobin, AST, ALT, WBC, history of lung disease, history of stroke, chronic renal disease, thromboplastin time, history of CHF, post CABG, history of MI, CHD, family history of heart disease, post PCI, history of PAD	__
20	Li et al, 2017 64	In-hospital mortality	17	Age Weight	Post CABG, cardiogenic shock, Killip class, cancer, heart rhythm, ECG finding (STEMI), history of CHF, HR, potassium, WBC, FBS, creatinine, SBP	_
21	Mansoor et al, 2017 68	In-hospital mortality	11	Age	Current smoking, HTN, dyslipidemia, family history of heart disease, CHD, post PCI, chronic renal disease, cardiogenic shock	_
22	Hu et al, 2016 69	MACEs	RBMLP 286 CRFs 284	Weight Height	HR, SBP, DBP, HTN, heart rhythm, sweat, current smoking, CHD, arteriosclerosis, angina, bleeding, HDL, calcium, triglyceride, FBS, urine color, WBC, Apoa-1, PTT, UA, CPS	Anticoagulation, antagonist medication, angiographic finding

Abbreviations: ALT, alanine aminotransferase; AST, aminotransferase aspartate; CABG, coronary artery bypass graft; CHD, coronary heart disease; CPS, carbamoyl phosphate synthetase; DBP, diastolic blood pressure; FBS, fasting blood sugar; HDL, high density lipoprotein; HR, heart rate; HTN, hypertension; LAD, left anterior descending coronary artery; LBBB, left bundle branch block; LMCA, left main coronary artery; MLR, monocyte-to-lymphocyte ratio; NLR, neutrophil-to-lymphocyte ratio; PAD, peripheral artery disease; PCI, percutaneous coronary intervention; PLR, platelet-to-lymphocyte ratio; PTT, prothrombin time; RBBB, right bundle branch block; SBP, systolic blood pressure; SIRI, systemic inflammation response index; RCA, right coronary artery; TT, thrombin time; WBC, white blood cells.

Clinical features consisted of physical examination, medical history, comorbid conditions, and laboratory findings. The most frequent features were in the physical examination category, they included HTN (18 studies), SBP (15 studies), HR (14 studies), ECG findings (13 studies), current smoking (13 studies), and DBP (11 studies), which were used in 10 to 18 studies. A detailed description of the medical abbreviations is available ( Supplementary Material B , available in the online version).

Quality Assessment of the Included Studies

All studies were classified as intermediate–high quality (intermediate: 10 studies, high: 12 studies) in the quality assessment, meaning that the outcomes were less susceptible to bias.

As shown in Table 6 all studies highlighted the limits in non-ML approaches. However, during the model training process, it was found that two studies 35 45 lacked information about feature engineering methods, and one study 35 did not provide the program or the platform for model training. Moreover, hyper parameters, which are necessary for the training process, were not found in nine (4.9%) studies. In three (13.63%) studies, categorical features were transformed through one-hot encoding. According to the table, all studies suggested possible clinical application of the developed ML algorithm, 18 (81.81%) studies provided a valid method to combat over-fitting, and eight (36.36%) studies validated the models in an external database. However, only one (4.5%) study did not report how to interpret the predictors.

Table 6. Quality assessment of machine learning studies.

Study	Unmet need	Reproducibility			Robustness		Generalizability	Clinical significance		The no. of positive responses
	Limits In current non-machine-learning approach	Feature engineering methods	Platforms/Packages	Hyperparameters	Valid methods to overcome over-fit	The stability of results	External data validation	Predictors explanation	Suggested clinical use
MACEs
Sherazi et al, 2021 10	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
Duan et al, 2019 67	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
Kim et al, 2019 50	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
Hu et al, 2019 35	Yes	No	Yes	No	Yes	Yes	No	Yes	Yes	6
Hu et al, 2016 69	Yes	Yes	Yes	No	Yes	Yes	No	No	Yes	6
In-hospital mortality
Aziz et al, 2021 49	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
Khera et al, 2021 28	Yes	Yes	Yes	No	Yes	Yes	No	Yes	Yes	7
Borracci et al, 2021 43	Yes	Yes	Yes	No	No	Yes	No	Yes	Yes	6
Lee et al, 2021 40	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	9
Kwon et al, 2019 6	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	8
Li et al, 2017 64	Yes	Yes	No	No	Yes	Yes	No	Yes	Yes	6
Pieszko et al, 2018 36	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	7
Mansoor et al, 2017 68	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	8
One-year mortality
D'Ascenzo et al, 2021 4	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	9
Aziz et al, 2021 49	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
Lee et al, 2021 40	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	9
Lee et al, 2020 65	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	9
Bai et al, 2021 29	Yes	Yes	Yes	No	Yes	Yes	No	Yes	Yes	7
Sherazi et al, 2020 9	Yes	Yes	Yes	Yes	No	Yes	No	Yes	Yes	7
Li et al, 2020 67	Yes	Yes	Yes	Yes	No	Yes	No	Yes	Yes	7
Payrovnaziri et al, 2019 44	Yes	Yes	Yes	No	Yes	Yes	No	Yes	Yes	7
Raza et al, 2019 45	Yes	No	Yes	No	Yes	Yes	Yes	Yes	Yes	7
Piros et al, 2019 42	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
6-month mortality
Hernesniem et al, 2019 41	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	9
3-month mortality
Lee et al, 2021 40	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	9
30-day mortality
Aziz et al, 2021 49	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8
Piros et al, 2019 42	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	8

Discussion

The results of this scoping review highlighted a significant variation in ML techniques, data collection, and reporting of results that should be taken into consideration.

According to the results ML algorithms have been increasingly used in models for predicting adverse events and mortality 48 and obtained a high degree of AUC (between 0.8 and 0.99). These algorithms outperformed traditional regression models in predicting adverse event outcome, 49 specifically in-hospital mortality, 36 1-year mortality, 9 22 and MACEs. 50

Furthermore, ML techniques are non-invasive and low-cost tools that can be considered as favorable methods if they use obtainable variables. 14 The highest AUC (99.61%) was achieved by the SVE classifier in a study by Sherazi et al. 10 The current review has focused only on supervised learning techniques, and studies on unsupervised or semi-supervised learning techniques were not included. However, none of the models in this review has been integrated into practice. Only one study has introduced an online calculator for risk score of each outcome. 4

The analysis of indicators for evaluating the performance of ML algorithms indicated that their performance depends on the type and number of predictors used in the model. Therefore, the researchers compared the performance of the algorithms based on the number of predictors and the performance of each model.

As shown in Table 3 , Ensemble methods are the most frequently used techniques among the best performances. Ensemble learning is a potential approach used to increase performance without losing too much interpretability of ML models. These methods combine the outcomes of multiple training models and produce a unified general result for each data sample. 39 In fact, Ensemble methods include RF and Boosting Ensemble technique, specially XGBoost.

RF and DTs are highly capable of distinguishing final classification attributes and are used to indicate the relationship between variables. The output of the DTs is intuitive and interpretable. In prognostic studies, the DTs are employed to extract prognostic subgroups 51 52 ; nevertheless, in medical context, the interpretability of ML by its users is of great importance. 53 Based on Fig. 3 , RF is the most frequently used technique among the best performances. The performance range obtained by RF in predicting all outcome categories was between 0.75 and 0.99. In fact, RF best performance was 92% in predicting in-hospital mortality with 55 predictors and 99% in 1-year mortality with 37 predictors.

After RF, boosting types were the most popular and successful Ensemble methods. The Boosting technique usually provides very accurate models. 33 In fact, Boosting Ensemble technique sequentially combines multiple ML models with high bias models to correct the predictions of models, obtain better predictions, 34 and counterbalance overfitting. 54 XGBoost was the most frequently used technique and obtained the best performance among Boosting Ensemble techniques. The performance range achieved by XGBoost in predicting all outcomes categories was 0.89 to 0.942. In fact, XGBoost best performance was obtained in predicting 1-year mortality with 59 predictors. However, CatBoost with 37 predictors had the highest performance in this category (0.99).

As shown in Fig. 2 , LR is the second most used method. It is a powerful and efficient supervised learning method that is well understood; it performs very well and can be easily applied to smaller datasets. 55 According to Table 3 , the best LR performance in predicting in-hospital mortality with 24 predictors was 0.88 and in 1-year mortality with 50 predictors was 0.84. LR has comparable performance to complex techniques, such as ANN and DNN. 56

The performance of ANN algorithms in handling noisy data are better than other algorithms. They are strongly dependent on setting the input parameters 51 and require the adjustment of several parameters. However, these models, compared with other models, are difficult to interpret. It is also a challenge to detect their important predictors. 57

Therefore, they may not be the best choice in medical settings since clinicians want to be aware of the logic behind the outputs 58 and do not trust or adopt a system which is hard to understand and is considered as a black box. 59 The term black-box refers to lack of transparency of a mechanism which produces solutions. 60

Table 3 presents the best ANN performance for predicting in-hospital mortality as well as 1-year and 30-day mortality. Complexity and privacy concerns are the main barriers to access medical data and prevent training very complex models, such as DNNs. 33 Although DNN was used only in five studies, it performed exceptionally well and achieved a high AUC (90%) in three studies. Table 3 summarizes the best DNNs performance for predicting MACE, in-hospital mortality as well as 1-year and 30-day mortality. This model presents comparable prediction performance when used in large datasets.

The population, sample size, and the predictors used in each study were different which, consequently, resulted in different adverse event outcomes and the use of various predictor variables. The most frequent variables that appeared as strong predictors included physical examination (HTN, SBP, HR, DBP, current smoking, ECG findings), medical history (post PCI, history of CHF, post CABG, history of stroke), laboratory findings (creatinine, FBS, maximum troponin T), diabetes mellitus, and Killip class type. The higher type of Killip class is associated with increased mortality risk in ACS patients. 46 The results also indicated that the evaluation indices for articles with at least one therapeutic feature were above 85%. Thus, it can be concluded that the use of these features, regardless of the type of algorithms, can noticeably enhance the prediction of MACEs. However, these features are not cost-efficient and are dangerous due to the nature of interventions.

According to the findings, only in one paper the ML model was trained with unstructured data, 44 the use of which seemed to be a challenge. 61 Free-text notes or unstructured data should be transformed to numerical values through feature engineering process to be used in prediction models. 62

ML algorithms varied significantly from a model with eight variables from clinical features to models with comprehensive data category. However, complex models with more variables, compared with simple ones, did not achieve better performance, nor did they differ significantly.

As highlighted in the study, the number of databases on important complications of cardiovascular disease is limited, but most studies used the recordings of Korea Acute Myocardial Infarction Registry, 6 BleeMACS registry, RENAMI registry, 4 the global registry of acute coronary incidents, 63 and the Acute Myocardial Infarction of China. 66 Furthermore, 13 (59.09%) studies focused on the national registry of developed countries, and this highlights the importance of establishing a heart registry in developing countries.

According to the results, only few studies addressed calibration, an important component of predictive model development, and 18 (81.81%) studies did not provide a sufficient report on modeling steps. The enhancement of transparency and reproducibility necessitates a thorough report on modeling stages and analyses. 16 The results also showed heterogeneity in the studies using ML; however, none of them indicated confidence intervals or standard deviations for their performance measures. Finally, all reviewed studies were retrospective and had not been operationally implemented, and this was a major issue in clinical utility. In fact, a prospective approach would be needed to determine the utility of predictive models and compare their performances with those of clinicians. Further research is required to assess the impact of ML model on clinical decision making, patient orientated outcomes, and patient and physician acceptability. The heterogeneous nature of the studies highlighted various approaches to solve problems in applying models which predict MACE in ACS patients.

Limitations and Problems

Several important limitations need to be considered in this study. First, no comparison was made among different scenarios across the same dataset to avoid disruption. Some algorithms were rarely used in the literature; therefore, the results obtained through comparing the performance of these models and other ML models are inconclusive. In addition, some studies did not show a certain amount of performance metrics. Finally, only articles written in English were reviewed. Owing to the heterogeneity of reported performance and descriptive statistics, only a narrative synthesis was possible for this study.

Conclusion

This review was conducted by specialists in medical informatics and can be used by computer or data scientists, physicians, or multidisciplinary teams. The findings provided additional evidence to support and define ML approaches for predicting MACEs and preventing cardiovascular mortality. It seems that ML algorithms, if modeling process is correct, have a high potential for predicting MACEs and cardiovascular deaths in ACS patients. Additionally, the use of these algorithms in designing clinical decision support systems cannot only guarantee the therapeutic process but also assist the health care team, patients, and their families in the process of clinical decision making. Finally, the findings could lead to the development of intelligent, feasible, and effective prediction models and can have potentially important implications for optimizing the quality of care in ACS patients in future.

ML algorithms rendered acceptable results to predict MACEs and mortality outcomes in ASC patients. However, they have never been integrated into practice. Further research needs to be conducted to develop feasible and effective ML prediction models to optimize the quality of care in ACS patients.

Clinical Relevance Statement

Acute coronary syndrome is the topmost cause of death worldwide; therefore, it is very important to predict MACEs and cardiovascular deaths in ACS patients so that one can make correct and timely clinical decisions. This review synthesized the studies which used machine learning algorithms for predicting MACEs in ACS patients to highlight algorithms and important predictor variables. The result of this study can be useful for designing clinical decision support systems which help the health care team, patients, and their families make proper clinical decisions.

Multiple Choice Questions

1. What does MACEs stand for in this article?

   1. Mechanical Aerospace Civil Engineering.

   2. Modelling Autonomic Communications Environments.

   3. Major adverse cardiovascular events.

   4. Modified Antigen Capture ELISA.

   Correct Answer: The correct answer is option c. Major adverse cardiovascular events (c) refer to the major cause of mortality and morbidity in cardiovascular patients. Approximately 15% of patients with ACS, experience MACEs, such as death, heart failure, or revascularization (i.e., PCI, and CABG) 1 year after diagnosis.

2. Extracted variables in these studies were divided into:

   1. Demographic, clinical, and therapeutic features.

   2. Demographic and therapeutic features.

   3. Clinical testing and genetic.

   4. Clinical evaluation and testing.

   Correct Answer: The correct answer is option a. The extracted variables in the selected studies were divided into three categories, i.e., demographic, clinical, and therapeutic features (a). The important reported variables included: HTN, diabetes mellitus, age, creatinine, sex, SBP, FBS, HR, post PCI, history of CHF, ECG finding, current smoking, DBP, post CABG, history of stroke, maximum troponin T, and Killip class which were all mentioned in nine or more studies.

Funding Statement

Funding None.