Abstract
Background
The integration of AI in diagnosing and managing ACS shows increasing promise, yet challenges remain in translating AI-CDSS into clinical practice. This study evaluates the advancements and limitations of AI for ACS over the past three years, purpose of understanding the scope, limitations, and potential of AI-CDSS in ACS.
Materials and methods
We conducted a systematic review of recent literature, adhering to guidelines for systematic reviews. We applied QUADAS-2 and PROBAST tools for quality assessment, focusing on biases in study designs. Ten studies about AI-CDSS in ACS management underwent critical analysis, emphasizing the strength of their research methods and the thoroughness of their prospective validation to ensure theoretical integrity and practical reliability.
Results
Our research reveals that while discourse around AI-CDSS in ACS management intensifies, obstacles hinder efficacy in practical settings. These challenges include biases in tests and unrepresentative patient selection, pointing to the need for rigorous and inclusive samples. The lack of sufficient external and prospective validation in studies also raises concerns clinical utility of AI-CDSS. The result is the gap between the potential benefits of AI-CDSS and the actual impact of improving diagnostic accuracy and outcomes for ACS limitations identified.
Conclusions
While AI-CDSS shows promise for improving diagnostic accuracy, treatment efficacy, and workflows in ACS, this study highlights the imperative to enhance model validation, including prospective validation, and address lingering diagnostic gaps. Improving study design and mitigating biases remain crucial for the acceptance and effectiveness of AI-CDSS in acute cardiac care settings.
1. Introduction
1.1. Acute coronary syndrome (ACS) and the importance of timely intervention
Cardiovascular diseases (CVDs) are the primary cause of death worldwide, responsible for approximately 17 million fatalities annually [1]. Among these, ACS—a critical CVD subset—poses severe health risks, particularly ST-elevation myocardial infarction (STEMI), caused by acute coronary artery blockage. STMI has an in-hospital mortality rate ranging between 2.2 % and 9.4 %, signifying a substantial short-term mortality risk for hospitalized individuals [2]. Diagnosing ACS involves evaluating symptoms, ECG analysis, and blood tests. Chest pain radiating to the arm or jaw is a common symptom, often accompanied by shortness of breath. ECG detects cardiac abnormalities, while blood tests identify biomarkers of heart muscle damage. [3]. Given ACS's time-critical nature, clinical practice guidelines emphasize a golden 90-min window intervention window to improve outcomes [4]. Patients arriving at the emergency department require ECG assessment within 10 min for rapid identification. Delayed diagnosis jeopardizes survival, increasing permanent heart damage risk. Notably, 30 % of ACS patients present non-typical symptoms like fatigue, nausea, or abdominal pain, complicating early detection and increasing life-threatening risks. These challenges make ACS differentiation from other conditions (e.g., Takotsubo cardiomyopathy [5], fulminant myocarditis [6] difficult. Thus, rapid diagnosis isn't always feasible, reinforcing the urgent need for advanced diagnostic and predictive tools to ensure timely ACS identification.
1.2. The potential of AI in diagnosing and predicting ACS
The potential of artificial intelligence (AI) to address the diagnostic and predictive challenges of diseases such as ACS is increasingly recognized within the medical community. Researchers understand the urgency of prompt ACS diagnosis and treatment. They believe AI can stratify patient risk by identifying high-risk patients, enabling timely intervention and reducing adverse outcomes [7]. Several retrospective studies have explored using AI models for pre-diagnosis ACS prediction. One study found the AI model improved specificity from 1 % to 11 % over routine care in identifying ACS patients [8]. The increase in NPV from 94 % to 99 % suggests the model's enhanced capacity to rule out ACS correctly. Another study by Fernández-Ruiz [9] showcases using machine learning to predict 1-year ischemic and major bleeding risks in ACS patients. This aligns with the findings of Wang [10], who highlight the potential of AI techniques like Naive Bayes and Neural Networks for risk prediction and outcome classification among ACS patients, demonstrating high accuracy and performance. Iannattone [11] further supports this, reporting excellent sensitivity and NPV when employing ML algorithms for ACS diagnosis. Consequently, with its ability to enable more precise and timely interventions, the integration of AI into clinical settings holds promise for enhancing patient outcomes [12].
1.3. Integrating CDSS in ACS care: technology and clinical insights
The emergence of CDSS brought hope to the medical treatment of ACS. However, integrating CDSS into routine care faces unique challenges, including technical limitations (e.g., ECG interpretation interference) and system delays (concerns about emergency response) [13]. To overcome these challenges, CDSS has become a crucial AI application [14]. AI-enhanced CDSS aims to improve diagnostic accuracy, streamline treatments, and offer promising medical solutions [15]. Machine learning-based AI algorithms integrating various data sources have significantly enhanced CDSS predictive accuracy in ACS management [[16], [17], [18]].
Compared to other diseases, AI-CDSS for ACS poses distinct challenges due to its time-sensitive nature. Unlike chronic diseases, where AI assists in long-term monitoring, ACS requires rapid intervention within 10 min of patient arrival, as per AHA guidelines [3]. AI must distinguish ACS from conditions like myocarditis and stress cardiomyopathy, where ECG patterns may overlap [19]. Furthermore, AI-CDSS should integrate real-time physiological data, including continuous ECG monitoring, rather than relying on static test results alone [20]. To enhance the applicability, adherence to standardized diagnostic frameworks, such as the ESC Guidelines for ACS [21] and TRIPOD reporting framework [22], is essential to ensure consistent model development and validation. AI models should explicitly report patient selection criteria, data preprocessing methods, and model evaluation metrics in compliance with these guidelines to facilitate reproducibility across institutions.
This gap is particularly critical for ACS, as real-time decision-making demands not only accuracy but also clinician trust in AI interpretability and reliability. Multi-center validation studies, such as the CoDE-ACS [23], demonstrated the feasibility of external validation across geographically diverse datasets to strengthen AI-CDSS robustness. Moreover, federated learning (FL) approaches [24] enable AI model training across multiple healthcare institutions without requiring direct data sharing, addressing privacy concerns and improving model adaptability in clinical settings. Martin-Isla et al. [25] suggest overcoming the "black box" issue by enhancing model interpretability and transparency through simplification, feature selection, and visualization techniques. To ensure the successful AI-CDSS integration into clinical workflows, it is essential to mitigate biases and enhance clinicians' trust.
This study aims to address the trends and challenges associated with the use of CDSS in ACS management by exploring the integration of AI models. By focusing on the acceptability and clinical validity of AI-CDSS, our study seeks to shed light on its potential to be broadly accepted and effectively implemented in clinical practices, ensuring that AI-CDSS not only meets technical standards but also fulfills clinical requirements, thereby improving the quality of care and outcomes for ACS patients.
1.4. Research questions
Our research seeks to delve deep into the myriad ways AI is revolutionizing ACS care with the implementation of AI-CDSS. Our investigation is structured around a series of research questions designed to uncover the extent of AI applications within CDSS for ACS, the prevalence and effectiveness of AI algorithms in managing ACS, and how these technologies are adapted to mitigate biases and function across various clinical settings. The formulated research questions are as follows.
RQ1: How is AI algorithms applied in CDSS for ACS, and what is the scale of these applications? This question aims to delineate the specific areas within ACS management where AI applications are being implemented. We intend to provide an overview of AI technology's applications in the CDSS for ACS, offering a summary of its use in many fields.
RQ2: How are biases addressed in the AI algorithms implemented in AI-CDSS for the management of ACS? This aspect of our research will explore the identification and mitigation of biases within AI algorithms deployed in AI-CDSS. Understanding how biases are recognized and corrected is crucial for ensuring the reliability and equality of AI-supported decision-making in ACS management.
RQ3: How applicable are AI algorithms in AI-CDSS for the management of ACS across different clinical settings? The focus here is on evaluating the versatility and adaptability of AI algorithms within AI-CDSS for ACS management in diverse clinical scenarios. This includes assessing their generalizability to various patient demographics, compatibility with existing clinical workflows, and suitability for different healthcare infrastructures.
Through these inquiries, we aim to build a robust understanding of AI's role in enhancing CDSS for ACS, highlighting the challenges, opportunities, and future directions of AI in cardiovascular care. This exploration will provide valuable insights into the potential of AI to improve outcomes for ACS patients by enhancing the precision, efficiency, and personalized nature of their care.
2. Methodology
We conducted a scoping review to obtain a comprehensive overview of research findings pertinent to our research questions, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews Protocols (PRISMA-ScR) [26]. This review targeted literature published from January 2021 to October 2023, a period characterized by rapid advancements in AI within healthcare. Our methodology unfolded in three main steps: initially identifying literature on the integration of AI in CDSS for ACS within this timeframe; subsequently, we conducted a comprehensive search utilizing predetermined keywords and index terms, extracting data based on relevant criteria and meticulously recording our findings; finally, after screening titles and abstracts for relevance, we charted the findings, aligned the results with our research questions, and drew conclusions through a qualitative synthesis of the evidence.
2.1. Search strategy
In this study, a rigorous approach was taken to select eligibility criteria aligned with our research focus on investigating the application of AI-CDSS in diagnosing ACS. These criteria were meticulously designed to filter for articles that were not only pertinent but also highly specific to our research goals. To ensure a comprehensive collection of relevant literature, we targeted four leading academic databases: PubMed, EMBASE, Scopus, and Science Direct. Each database was chosen for its unique contribution to the breadth of scholarly research available, thereby maximizing our coverage of the literature. We conducted a systematic search across these databases using a carefully curated set of keywords: "Artificial Intelligence", "AI", "Deep Learning", "Machine Learning", "Clinical Decision Support Systems", "Clinical Decision Support", "Chest Pain", "ACS" and "Acute Coronary Syndrome". These terms were specifically chosen to narrow our search to studies that focus on the application of AI techniques in medical diagnostics, with a particular emphasis on ACS, ensuring our review was both targeted and relevant to our research objectives.
2.2. Eligibility criteria
To identify eligible studies for this review, we developed a set of inclusion and exclusion criteria based on the scope of our research aim. The inclusion criteria were: (1) studies related to AI-CDSS for ACS; (2) papers describing applications of AI models; and (3) articles published in English. Studies were excluded if they (1) were not original research such as literature reviews or editorials; (2) had no relevance to AI; (3) were unrelated to CDSS; or (4) had no association with ACS. Two independent reviewers (SHC &CCW) screened titles, abstracts, and full texts sequentially against these preset criteria. Only original peer-reviewed studies in English were included to ensure high academic quality. Certain publication types were excluded, including conference proceedings, letters to editors, commentaries, and case reports. By instituting rigorous inclusion/exclusion criteria and a systematic screening process, we aimed to identify the most high-quality and relevant research on AI techniques for enhancing ACS management in CDSS.
2.3. Data extraction
In our data extraction process, we thoroughly reviewed each of the selected research articles for accuracy and details. Two independent reviewers (SHC and CCW) used a structure form to extracted key information, including the authors, publication year, title, journal name, study population, AI models used, performance metrics, and limitations. To ensure the reliability and validity, we applied the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) [27] tool for quality assessment of diagnostic accuracy studies and the Prediction model Risk Of Bias Assessment Tool (PROBAST) [28] tool for quality and bias assessment. QUADAS-2 assesses risk of bias in diagnostic accuracy studies, while PROBAST evaluates bias and applicability in prediction model studies. The results of these quality assessments are summarized in [Table 2] and [Fig. 2], providing an overview of the evidence base's risk of bias and applicability. Despite the limited number of studies, the in-depth review provided essential data for further analysis and interpretation.
Table 2.
Risk of bias according to PROBAST domain.
| Risk of bias according to PROBAST Domain | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Author, Year |
Risk of Bias |
Applicability |
Overall |
||||||
| 1. Participants | 2. Predictors | 3. Outcome | 4. Analysis | 1. Participants | 2. Predictors | 3. Outcome | Risk of Bias | Applicability | |
| Valente et al., 2021 | Low | Low | Low | Low | Low | Low | Low | Low | Low |
| Lin et al., 2022 | Low | Low | Low | Low | Low | Low | Low | Low | Low |
| Wibring et al., 2022 | Low | Low | Low | Low | Low | Low | Low | Low | Low |
| Jokardarabi et al., 2022 | Low | Low | Low | Low | Hight | Low | Low | Low | Low |
| Sherazi et al., 2023 | Low | Low | Hight | Hight | Hight | Hight | Hight | Hight | Hight |
| Koning et al., 2023 | Low | Low | Low | Hight | Low | Low | Low | Hight | Hight |
Hight: Hight risk of bias.
Low: Low risk of bias.
Fig. 2.
QUADAS-2 summaries.
3. Findings and discussions
After a comprehensive literature search across multiple databases, 99 records were initially identified: PubMed (9 articles), EMBASE (10 articles), Scopus (31 articles), and Science Direct (49 articles) [Fig. 1]. Snowball searching added two more, totaling 101 articles. After removing 11 duplicates, 90 articles remained. Screening titles and abstracts based on our criteria yielded 22 relevant articles. Excluding non-CDSS studies and those with insufficient detail or unavailable texts, ten studies meeting our inclusion criteria were selected for analysis.
Figure: 1.
Flowchart of study selection for scoping review on AI/ML in medical diagnosis.
3.1. Scope of AI algorithms in CDSS for ACS
In addressing Research Question 1, "How are AI algorithms applied in CDSS for ACS, and what is their application scale?" this study aims to provide a systematic literature review. We have noted that certain studies may span several domains, a discovery that underscores the breadth and complexity of AI-CDSS applications in AC treatment. Our classification method primarily seeks to emphasize the principal contribution of each piece of literature while also acknowledging that the scope of research could exceed its classified domain. By this means, we strive to offer a structured overview of academic works, reflecting the interdisciplinary nature and rich diversity of scientific inquiry. In the process of analysis, we integrated data from 10 studies and summarized the applications of AI algorithms in CDSS for ACS in [Table 1].
Table 1.
Synthesis of research on AI applications.
| Authors/Year | Title | Source origin | Study Population | AI Models | Performance | Limitations |
|---|---|---|---|---|---|---|
| Doudesis et al. 2023 |
Machine learning for diagnosis of myocardial infarction using cardiac troponin concentrations | UK, Scotland | N = 10,038 | Logistic regression, naïve bayes, random forest and XGBoost | The CoDE-ACS system, AUC: 0.953; 95 % CI: 0.947–0.958 |
|
| Lin et al. 2022 |
Deep learning-enabled coronary CT angiography for plaque and stenosis quantification and cardiac risk prediction: An international multicenter study | USA, UK, Germany, Australia, and Japan | N = 1096 | Deep learning, ConvLSTM/CNNs/RNNs) | Deep learning total plaque volume (ICC 0·949) and minimal luminal area (ICC 0·904) |
|
| Bouzid et al., 2021 | In search of an optimal subset of ECG features to augment the diagnosis of acute coronary syndrome at the emergency department | USA | N = 1244 | Logistic regression, ANN,Fss (feature selection strategies), Random forest |
73 hybrid ECG features outperformed experts, reclassifying 10–29 % ACS cases. |
|
| Valente et al. 2021 |
A new approach for interpretability and reliability in clinical risk prediction: Acute coronary syndrome scenario | Portugal | N = 1111 | Logistic regression, neural network, Artificial neural network (ANN) |
The proposed approach 3 steps methodology + ANN(AUC = 81 %, GM = 74 %, PPV = 17 %, NPV = 99 %) |
|
| Gibson et al. 2022 |
Evolution of single-lead ECG for STEMI detection using a deep learning approach | (Latin) Brazil, Colombia, Mexico, and Argentina | N = 11053 | CNN, deep learning, machine learning | Lead V2 achieved the highest accuracy at 90.5 % for STEMI detection |
|
| Emakhu et al. 2023 |
A hybrid machine learning and natural language processing model for early detection of acute coronary syndrome | USA | N = 16,096 | Logistic regression, AdaBoost, and linear discriminant analysis | AdaBoost outperformed others with 94 % accuracy and an F1-score of 0.943 in ACS prediction |
|
| Jokardarabi et al. 2022 |
Development of a decision-maker system to predict in-hospital mortality of acute myocardial infarction patients; an artificial intelligent approach: Yazd cardiovascular disease registry data use | Iran | N = 2711 | KNN and GB, XGBoost, SVM, logistic regression, random forest | Random forest model after applying LASSO feature selection: 98.25 % precision, 98.27 % accuracy, 98.27 % F1-score, and 99.47 % recall | N/A |
| Emakhu et al. 2022 |
Acute coronary syndrome prediction in emergency care: A machine learning approach | USA | N = 61,893 | AdaBoost, gradient boosted trees, XGBoost | Gradient boosting model achieves 86.3 % sensitivity and 93.3 % AUROC. Accuracy (85.7 %), precision(86.3 %)、specificity (93 %), MCC (80 %), F1 (86.3 %) | Not yet implemented in clinical practice |
| Sherazi et al., 2023 | A machine learning-based applied prediction model for identification of acute coronary syndrome (ACS) outcomes and mortality in patients during the hospital Stay | Korea | N = 13,104 | Random forest, extra tree, gradient-boosting machine | Soft-voting ensemble classifier (SVEC), Accuracy: 99.07 % Precision: 99.07 % Recall: 99.07 % F1-score: 99.97 % AUC: 99.97 % |
|
| Koning et al., 2023 | AI algorithm to predict acute coronary syndrome in prehospital cardiac care: Retrospective cohort study | Netherlands | N = 7458 | Support vector machine (SVM), random forest (RF), KNN and logistic regression (LR). | AI model: Increase in specificity (from 1 % to 11 %) and NPV (from 94 % to 99 %) |
|
3.1.1. Diagnostic support
These studies offer tools and models to assist or enhance the process of diagnosing cardiac diseases.
Doudesis et al. [29]: This study focuses on developing and utilizing the CoDE-ACS score, leveraging an XGBoost model to significantly improve myocardial infarction diagnosis by incorporating clinical indicators such as cardiac troponin, patient demographics, and symptom details.
-
•
Bouzid et al. [30]: Their research developed a method to enhance acute coronary syndrome diagnosis by utilizing a novel subset of temporal ECG features and spatial ECG features. This approach integrates both domain-specific expertise and data-driven approaches to improve diagnostic accuracy and risk assessment in emergency department settings.
-
•
Gibson et al. [31]: Developing a deep learning-based AI tool to accurately detect and localize STEMI using both single-lead and 12-lead electrocardiograms (ECGs), aiming to enhance early diagnosis and efficient triage in pre-hospital settings.
3.1.2. Risk assessment
These studies are focused on predicting ACS, especially before the occurrence of ACS.
-
•
Lin et al. [32]: Employing deep learning for the analysis of coronary CT angiography data to quantify atherosclerotic plaque burden and stenosis, thereby assessing patients' cardiac risk for future adverse events such as myocardial infarction.
-
•
Koning et al. [8]: Developing an AI model to predict acute coronary syndrome in prehospital cardiac patients, aiming to enhance triage accuracy and reduce unnecessary emergency department admissions by identifying low-risk patients who can safely remain home.
3.1.3. Prognosis prediction
These studies offer predictions on disease progression outcomes, helping patients understand potential long-term results and assisting physicians in making informed decisions for optimal patient disposition.
-
•
Jokardarabi et al. [33]: This study utilized a range of machine learning models, such as KNN, Gradient Boosting, XGBoost, SVM, AdaBoost, LASSO, and Logistic Regression, combined with data on clinical indicators, medical history, laboratory results, and patient demographics, to enhance the precision of predicting in-hospital mortality rates for patients with acute myocardial infarction.
-
•
Sherazi et al. [34]: Developing a machine learning-based predictive model to forecast outcomes of ACS in hospitalized patients, including STEMI and NSTEMI (Non-ST-elevation myocardial infarction), for both cardiac and non-cardiac mortality rates.
-
•
Valente et al. [35]: Developing and validating a methodology to predict post- ACS mortality risk by combining traditional risk scores with machine learning models to enhance prediction accuracy and personalization.
3.2. Addressing bias in AI-CDSS implementation
In this section, we address Research Question 2. “How were biases addressed in the AI algorithms implemented in AI-CDSS for the management of ACS?” With the assistance of QUADAS-2 and PROABST, we summarized potential risks of bias identified across ten original articles. These biases, if not promptly identified and mitigated, could lead to unfairness in the model performance. To effectively address these issues, we recommended to approach from three main aspects.
3.2.1. Index testing bias indicators
3.2.1.1. Negative cases
Case 1 & 2: Emakhu et al. [36,37] utilized ICD-10 for cohort identification, which is widely used but varies significantly between institutions. The inconsistency in ICD-10 code application may impact the validity of index test comparisons across datasets.
3.2.1.2. Positive cases
Case 1: Doudesis et al. [29]: employed the fourth edition of the Universal Definition of Myocardial Infarction, incorporating high-sensitivity cardiac troponin results with clinical judgment. This standard, endorsed by the WHO, AHA, and ESC, enhances diagnostic reliability.
Case 2: Gibson et al. [31]: utilized the LATIN database, where each digital ECG was interpreted by expert cardiologists, audited by senior specialists, and cross-referenced with angiography reports, minimizing misclassification bias.
Recommendation: Given the practices in positive cases, it is advisable to use explicit standards and reviews by multiple clinicians in research to ensure consistency and accuracy in diagnoses.
3.2.2. Flow and timing bias
QUADAS-2 framework, "Flow and timing," addresses critical elements of specific aspects of diagnostic accuracy studies, focusing on patient flow and time-related factors associated with testing. Diagnostic studies often overlook the importance of timing in data collection and the interval between this collection and the eventual diagnosis. Ensuring temporal consistency in such studies is key to improving diagnostic model accuracy and reliability. Significant discrepancies between the times of feature collection and diagnosis can adversely affect the accuracy of diagnoses.
3.2.2.1. Negative case
Case 1: Gibson et al.'s study [31] employs deep learning algorithms to distinguish between STEMI and NSTEMI by analyzing the characteristics of individual heartbeats in ECG data. The absence of clear information on the timing of feature extraction compromises the study's ability to guarantee the accuracy and clinical relevance of the deep learning model.
3.2.2.2. Positive cases
Case 1: Doudesis et al.'s [29] study specifically highlights potential time and flow issues in AI models and thus addresses these by developing machine learning models that integrate cardiac troponin concentrations and clinical features to improve diagnostic accuracy. This approach aims to resolve biases traditionally introduced due to variations in measurement timing.
Case 2: Bouzid et al. [30] focus on using real-time or near-real-time ECG data, minimizing the difference between feature data collection and diagnosis time, thus less likely to significantly impact study outcomes.
3.2.2.3. Recommendation
Studies should detail the patient follow-up process, especially the time interval between STEMI diagnosis and treatment, to reduce time-related bias. Attention should be paid to potential biases introduced by measuring cardiac troponin at different times, recommending the standardization of measurement timing in study design.
3.2.3. Analysis bias in AI predictive models
The 'analysis' section within the PROBAST framework evaluates the methodologies of predictive AI models for clinical validity and reliability.
3.2.3.1. Negative cases
Case 1: Sherazi et al. [34].
Feature selection criteria and process are unclear: The paper mentions 125 features selected without detailing the criteria or process, leading to a lack of transparency and explainability.
Lack of parameter tuning details: The paper does not specify the hyperparameter tuning process for models such as random forests, extremely randomized trees, or gradient boosting machines. Without clear reporting of key hyperparameters, such as learning rate, dropout rate, or model depth, assessing model reproducibility remains challenging. Additionally, the study does not describe the tuning methodology used (e.g., Grid Search, Bayesian Optimization), making it difficult to evaluate optimization strategies.
Insufficient analysis of model predictive ability: The author should analyze predictive indicators for different cardiovascular events and explore model (STEMI and NSTEMI) performance variances across these subcategories.
Insufficient analysis of model predictive ability: The author should analyze predictive indicators for different cardiovascular events and explore model (STEMI and NSTEMI) performance variances across these subcategories.
Case 2: Koning et al. [8].
The AI model's focus on predicting ACS, without considering other potentially fatal causes of chest pain such as pulmonary embolism or aortic dissection, introduces spectrum bias. This exclusion of complex cases can lead to an over-optimistic overestimation of the model's performance.
3.2.3.2. Recommendation
Emphasize transparency and explainability in feature selection during model development, including a clear description of criteria and processes. Explicitly report the hyperparameter tuning process, including key parameters (e.g., learning rate, dropout rate, network depth) and tuning techniques (e.g., Grid Search, Bayesian Optimization).Assess model stability using K-fold cross-validation and external validation datasets to ensure robustness and reproducibility. Future research should ensure standardized reporting of AI-CDSS performance metrics to facilitate cross-study comparisons and clinical adoption.
3.3. Clinical application of AI-CDSS in ACS
In this discussion, we address Research Question 3: "What is the extent of applicability of AI algorithms in AI-CDSS for ACS across diverse clinical environments?" Utilizing QUADAS-2 and PROBAST frameworks, we identified the key challenges affecting the applicability of these AI algorithms in reviewed studies. Further understanding this can help recognize the potential of AI-CDSS in aiding clinical decision-making in ACS.
3.3.1. External validation
External validation involves testing the validity of research conclusions using independent datasets, a critical process for confirming the results' universality and robustness. Both PROBAST and QUADAS-2 consider external validation crucial for ensuring the generalizability and robustness of the study conclusions.
3.3.1.1. Negative cases
Case 1: Sherazi et al. [34].
They discuss a machine learning-based predictive model for ACS but lacks a process of testing the model on an independent dataset outside of the development phase. The absence of external validation raises questions about the model's applicability and robustness beyond the specific dataset used in the study and reduces confidence in its widespread applicability across different patient groups or clinical settings.
Case 2: de Koning et al. [8].
Their research is a retrospective cohort study that developed an AI model to predict ACS in prehospital settings. The study primarily analyzed retrospective data from emergency medical services. A critical point to note is the study's retrospective nature, which positions it as a proof-of-concept. The study lacks an external validation process, which is necessary to test the model's effectiveness and reliability in various clinical settings and with different patient populations. Furthermore, the study's reliance on structured prehospital data, which might be challenging to obtain in practice, presents another limitation to its practical deployment.
Case 3: Emakhu et al. [37].
The study aims to enhance the accuracy and efficiency of ACS diagnosis in emergency medical scenarios using AI technology. It focuses on identifying patterns and indicators in patient data for ACS prediction, potentially improving early diagnosis and treatment outcomes. However, integrating the model into actual medical environments may require consideration of doctors' usage habits, diagnostic processes, and integration with existing medical information systems. Due to the nature of the data being trained on a single center, the model's generalizability remains uncertain.
3.3.1.2. Positive case
Case 1: Lin et al. [32].
Firstly, by conducting external validation on different patient cohorts, specifically a group of 175 patients (1081 lesions), the study ensures the deep learning model's generalizability, demonstrating its applicability and reliability across various clinical settings and patient groups. Secondly, validation using actual measurement data from intravascular ultrasound in 50 patients (84 lesions) who had CCTA enhances the accuracy of the model's predictions. This dual approach to external validation enhances the model's value in clinical decision support.
Case 2: Doudesis et al. [29]:
This work demonstrates rigorous external validation by conducting it across six distinct cohorts—APACE, IMPACT, ADAPT, EDACS, SPACE, and UTROPIA—spanning Switzerland, Spain, Poland, the Czech Republic, Australia, New Zealand, and the United States. This comprehensive external validation, encompassing several countries and regions, showcases the adaptability and reliability of the suggested method, confirming that it is robust and widely applicable.
Recommendations: To ensure robust external validation of AI models, testing should be conducted using independent datasets not included in the training phase. Standardizing unstructured data is crucial for calculating performance metrics. Natural Language Processing (NLP) techniques, such as Named Entity Recognition (NER) and Clinical Concept Extraction, convert physician notes into structured data. Frameworks like UMLS and OMOP help align extracted clinical variables with standardized diagnostic codes, enabling consistent AI model evaluation. Example: The MIMIC-III dataset applies NLP to extract key predictive indicators (e.g., troponin levels, ECG findings) allowing AI-CDSS models to compute performance metrics more reliably, such as AUC-ROC, F1-score, and calibration curves. This ensures consistency in AI-CDSS validation and enhances comparability across studies.
3.3.2. Integrating AI models in clinical workflows
Our analysis through QUADAS-2 and PROBAST in relation to integrating AI models into clinical workflows is presented as follows.
3.3.2.1. Negative example
The study by Sherazi et al. [34] lacks detailed guidance on parameter adjustment and optimization parameter adjustment and optimization, creating uncertainty for clinicians and hindering the model's integration into clinical practice for decision-making and patient management.
-
•
Predictors: The study does not provide detailed guidance on integrating the model into existing clinical workflows, posing a challenge for medical professionals who may be uncertain about using the model's predictions and balancing them with their clinical experience.
-
•
Outcome: The study lacks a discussion on the technical requirements for deploying the model in clinical settings, such as integration with Electronic Health Record systems, which might necessitate additional IT support and affect the practical application of the model.
3.3.2.2. Recommendation
To successfully integrate AI models into clinical practice, studies should provide detailed guidance on model customization to support clinician adoption and evaluate performance across diverse patient populations and settings to ensure generalizability. Co-designing systems with clinician input can help align AI with existing workflows, while explainable AI and decision support tools increase appropriate use by building clinician trust. Establishing benchmarks and continuous monitoring across implementation sites is crucial for ensuring the ongoing reliability and safety of the AI model.
3.3.3. Generalize applicability
The generalized applicability of research refers to the extent to which the findings can be effectively implemented across various healthcare systems, highlighting their relevance and utility in diverse medical contexts and for different patient populations. Based on these two tools, QUADAS-2 and PROBAST, our analysis can be summarized as follows.
3.3.3.1. Negative example
Case 1: Doudesis et al. [29].
Their research primarily included Caucasian participants, potentially limiting the model's applicability to other ethnic groups.
Case 2: Emakhu et al. [37].
They acknowledged the limitations in their patient selection process, as they chose a cohort with relatively less severe conditions. This might limit the broad applicability of the research. By optimizing the patient selection criteria to cover a wider range of severity levels, they could ensure that the model is applicable to patients with both severe and milder symptoms.
Case 3: Emakhu et al. [36].
The research was conducted in secondary and tertiary hospitals in Scotland, which may have limited applicability due to different patient demographics and medical practices in various regions. Conducting more studies in different areas is recommended to better understand the model's applicability in diverse medical environments and consider regional differences.
Case 4:Sherazi et al. [34].
Their study focused only on Korean individuals, limiting its applicability to other ethnicities. The study mentioned selecting 125 features but did not detail the criteria and process, potentially leading to insufficient generalizability. It is suggested to clearly describe feature selection criteria and processes to improve generalizability across different patient groups.
3.3.3.2. Positive example
Case 1: Lin et al. [32].
Their research validation involved a substantial and varied group of 10,286 patients from seven prospective cohort studies across six countries. This extensive and diverse patient base enhances the AI model's applicability and reliability in real-world ACS management across a range of clinical settings.
Case 2: Gibson et al. [31].
They highlighted sampling bias due to data from specific Latin American countries, with the dataset including nearly 800,000 ECG records from hospitals in Brazil, Colombia, Mexico, and Argentina. The vast dataset from different Latin American countries contributes to understanding and improving AI models for more accurate and representative use in ACS management.
Recommendation: To enhance real-world applicability, future research should prioritize diverse, representative patient samples across demographics, ethnicities, regions, and healthcare settings. Rigorous validation of AI models in various medical contexts and populations is essential and should be reported transparently. International collaborations can enable access to heterogeneous datasets and testing environments to evaluate model generalizability. Continuous performance monitoring across cohorts and clear documentation of feature selection processes are key to ensuring fairness and adoption. Studies should acknowledge limitations in sample diversity and generalizability and discuss their impact.
3.4. Study limitations in ten selected papers
Within the ten selected studies, we have identified several key limitations, as discussed within the studies themselves, that may affect the effectiveness and reliability of AI models when applied in clinical practice [Table 3]. Firstly, two studies [34,35] highlighted the issue of insufficient sample size. This finding suggests that the datasets used were not large enough, limiting the generalization ability and prediction accuracy of the models. Secondly, five studies [[29], [30], [31], [32],34] pointed out the problem of insufficient population representation. This indicates a lack of diversity in race and geographical distribution within the datasets, which could lead to biases in the AI models' predictions across different populations.
Table 3.
Review insights on AI-CDSS bias and ACS application.
| Addressing Bias in AI-CDSS Implementation | Clinical Application of AI-CDSS in ACS | |||||
|---|---|---|---|---|---|---|
| Potential bias | Testing bias indicators | Time and flow bias | Analysis of bias in AI predictive models | Generalize applicability | Integrating AI models in clinical workflows | External validation |
| Negative | Emakhu et al. Emakhu et al. |
Gibson et al. Emakhu et al. |
Sherazi et al. Koning et al. |
Doudesis et al. Emakhu et al. Emakhu et al. Sherazi et al. |
Sherazi et al. | Sherazi et al. de koning et al. Emakhu et al. |
| Positive | Doudesis et al. Gibson et al. |
Doudesis et al. Bouzid et al. |
X | Lin et al. Gibson et al. |
X | Lin et al. |
| Recommendation |
|
|
|
|
|
|
Additionally, four studies [8,29,32,35] emphasized the lack of external validation, including prospective validation. External validation is a crucial step in assessing the generalization ability of AI models. The absence of this aspect could result in poor performance of the models in various clinical settings, suggesting that current research might be overly reliant on specific datasets or clinical environments. Furthermore, two studies [31,34] mentioned the issue of insufficient explainability of AI models. This is a significant concern for medical professionals, as they need to understand the decision-making process of the models to trust and effectively use their predictions. The "black box" nature of the models could hinder their acceptance and trust in clinical practice.
Lastly, two studies [8,37] specifically discussed the lack of application in clinical practice. This includes issues related to the applicability of AI models in real clinical settings, operational complexity, and the practical implementation of prediction results. This indicates that, despite the theoretical advantages of AI technology, it still faces challenges in application within the real-world clinical environment.
4. Limitations
This scoping review has the following key limitations: 1) we examined only ten articles from the past three years on AI-CDSS applications in ACS diagnosis and treatment, providing a narrow scope that might fail to capture the full extent of this rapidly evolving field; 2) we reviewed only peer-reviewed journal articles, excluding other forms of literature such as conference papers and editorials, although this may have missed some emerging research directions and perspectives; 3) the reviewed studies generally have small sample sizes and lack sufficient external validation, impacting the broad applicability of the conclusions. Nonetheless, future studies could expand the timeframe and adopt systematic review methods to examine the literature more comprehensively.
5. Conclusion
We examined the application, value, and challenges of AI clinical decision support systems (AI-CDSS) in the clinical management of ACS. AI-CDSS offers new hope for improving diagnostic accuracy, optimizing treatment regimens, and precisely predicting prognoses for ACS. However, it faces core obstacles such as insufficient sample sizes, imbalanced population representation, and lack of adequate external validation, especially prospective validation, which are major empirical risk issues in existing studies. Researchers should also emphasize the explicitness of feature selection criteria and the interpretability of the process in model development. Meanwhile, the tuning and optimization process of model parameters should be reported in detail to ensure model stability. Regarding clinical utility, the widespread acceptance of AI-CDSS depends on the interpretability of its diagnostic results and whether it can be easily integrated into complex clinical workflows. We look forward to relevant researchers addressing our suggestions to jointly promote AI-CDSS to better benefit ACS patients.
Author's contributions
SHC spearheaded the literature review and drafted the manuscript, including the development of figures and tables. CCW contributed to the literature review and offered revisions to the draft. KFC conducted a critical review, making substantial edits to ensure the manuscript met publication standards.
Funding
This work was supported by the NIH/NIAID Center of Excellence in Influenza Research and Surveillance contract HHS N2772201400007C, the National Science and Technology Council in Taiwan (111-2314-B-182 -017 -MY3) and Chang Gung Memorial Hospital in Taiwan (CMRPG2L0282, CORPG2P0071, CMRPG2P0121).
Declarations of interest
None.
Contributor Information
Shu-Hui Chen, Email: phdgloria@cgmh.org.tw.
Chin-Chieh Wu, Email: wujinja@gmail.com.
Kuan-Fu Chen, Email: kfchen@cgmh.org.tw.
References
- 1.Organization W.H. Cardiovascular diseases. 2023. https://www.who.int/health-topics/cardiovascular-diseases#tab=tab_1
- 2.Gong M, Liang D, Xu D, Jin Y, Wang G, Shan P. Analyzing predictors of in-hospital mortality in patients with acute ST-segment elevation myocardial infarction using an evolved machine learning approach. Comput Biol Med. 2024;170 doi: 10.1016/j.compbiomed.2024.107950. [DOI] [PubMed] [Google Scholar]
- 3.Bhatt DL, Lopes RD, Harrington RA. Diagnosis and treatment of acute coronary syndromes: a review. JAMA. 2022;327(7):662–75. doi: 10.1001/jama.2022.0358. [DOI] [PubMed] [Google Scholar]
- 4.Shiraishi Y, Kawana M, Nakata J, Sato N, Fukuda K, Kohsaka S. Time‐sensitive approach in the management of acute heart failure. ESC Heart Fail. 2021;8(1):204–21. doi: 10.1002/ehf2.13139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Templin C, Ghadri JR, Diekmann J, Napp LC, Bataiosu DR, Jaguszewski M, et al. Clinical features and outcomes of takotsubo (stress) cardiomyopathy. N Engl J Med. 2015;373(10):929–38. doi: 10.1056/NEJMoa1406761. [DOI] [PubMed] [Google Scholar]
- 6.Huang X, Gao Y, Hua F, Ying J. Differential diagnosis of fulminant myocarditis and acute coronary syndromes in the case of failure of coronary angiography: a case report. Front Cardiovasc Med. 2021;8 doi: 10.3389/fcvm.2021.690974. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Valeri de Freitas R, Sousa T, Fuziy Nogueira De Marchi M, Ruiz Uemoto V, Hortegal R, Nakayama Ohe LN, et al. Leveraging advanced AI algorithms for predicting outcomes in non-acute coronary syndrome chest pain patients. Circulation. 2023;148(Suppl_1):A15702–A. [Google Scholar]
- 8.de Koning E, van der Haas Y, Saguna S, Stoop E, Bosch J, Beeres S, et al. AI algorithm to predict acute coronary syndrome in prehospital cardiac care: retrospective cohort study. JMIR cardio. 2023;7 doi: 10.2196/51375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Fernández-Ruiz I. Machine learning predicts risk in ACS. Nat Rev Cardiol. 2021;18(4):230. doi: 10.1038/s41569-021-00521-8. [DOI] [PubMed] [Google Scholar]
- 10.Wang H, Zu Q, Chen J, Yang Z, Ahmed MA. Application of artificial intelligence in acute coronary syndrome: a brief literature review. Adv Ther. 2021;38(10):5078–86. doi: 10.1007/s12325-021-01908-2. [DOI] [PubMed] [Google Scholar]
- 11.Iannattone PA, Zhao X, VanHouten J, Garg A, Huynh T. Artificial intelligence for diagnosis of acute coronary syndromes: a meta-analysis of machine learning approaches. Can J Cardiol. 2020;36(4):577–83. doi: 10.1016/j.cjca.2019.09.013. [DOI] [PubMed] [Google Scholar]
- 12.Johnson KW, Torres Soto J, Glicksberg BS, Shameer K, Miotto R, Ali M, et al. Artificial intelligence in cardiology. J Am Coll Cardiol. 2018;71(23):2668–79. doi: 10.1016/j.jacc.2018.03.521. [DOI] [PubMed] [Google Scholar]
- 13.Knoery CR, Heaton J, Polson R, Bond R, Iftikhar A, Rjoob K, et al. Systematic review of clinical decision support systems for prehospital acute coronary syndrome identification. Crit Pathw Cardiol. 2020;19(3):119-25. doi: 10.1097/HPC.0000000000000217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gupta PK, Ramachandran AT, Keerthi AM, Dave PS, Giridhar S, Kallapur SS, et al. An overview of clinical decision support system (CDSS) as a computational tool and its applications in public health. Applications in ubiquitous computing. 2021:81–117. [Google Scholar]
- 15.Narindrarangkura P, Kim MS, Boren SA. A Scoping Review of artificial intelligence algorithms in clinical decision support systems for internal medicine subspecialties. ACI Open. 2021;5(2):e67–79. [Google Scholar]
- 16.Haick H, Tang N. Artificial intelligence in medical sensors for clinical decisions. ACS Nano. 2021;15(3):3557–67. doi: 10.1021/acsnano.1c00085. [DOI] [PubMed] [Google Scholar]
- 17.Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230–43. doi: 10.1136/svn-2017-000101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Skuban-Eiseler T, Orzechowski M, Denkinger M, Kocar TD, Leinert C, Steger F. Artificial intelligence-based clinical decision support systems in geriatrics: an ethical analysis. J Am Med Dir Assoc. 2023;24(9):1271–1276. doi: 10.1016/j.jamda.2023.06.008. [DOI] [PubMed] [Google Scholar]
- 19.Buttà C, Zappia L, Laterra G, Roberto M. Diagnostic and prognostic role of electrocardiogram in acute myocarditis: a comprehensive review. Ann Noninvasive Electrocardiol. 2020;25(3) doi: 10.1111/anec.12726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lancia L, Toccaceli A, Petrucci C, Romano S, Penco M. Continuous ECG monitoring in patients with acute coronary syndrome or heart failure: EASI versus gold standard. Clin Nurs Res. 2018;27(4):433–49. doi: 10.1177/1054773817704653. [DOI] [PubMed] [Google Scholar]
- 21.Byrne RA, Rossello X, Coughlan JJ, Barbato E, Berry C, Chieffo A, et al. 2023 ESC guidelines for the management of acute coronary syndromes. Eur Heart J Acute Cardiovasc Care. 2024;13(1):55–161. doi: 10.1093/ehjacc/zuad107. [DOI] [PubMed] [Google Scholar]
- 22.Patzer RE, Kaji AH, Fong Y. TRIPOD reporting guidelines for diagnostic and prognostic studies. JAMA Surg. 2021;156(7):675–6. doi: 10.1001/jamasurg.2021.0537. [DOI] [PubMed] [Google Scholar]
- 23.Boeddinghaus J, Doudesis D, Lopez-Ayala P, Lee KK, Koechlin L, Wildi K, et al. Machine learning for myocardial infarction compared with guideline-recommended diagnostic pathways. Circulation. 2024;149(14):1090–101. doi: 10.1161/CIRCULATIONAHA.123.066917. [DOI] [PubMed] [Google Scholar]
- 24.Ng D, Lan X, Yao MM, Chan WP, Feng M. Federated learning: a collaborative effort to achieve better medical imaging models for individual sites that have small labelled datasets. Quant Imag Med Surg. 2021;11(2):852-7. doi: 10.21037/qims-20-595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Martin-Isla C, Campello VM, Izquierdo C, Raisi-Estabragh Z, Baeßler B, Petersen SE, et al. Image-based cardiac diagnosis with machine learning: a review. Front Cardiovasc Med. 2020;7:1. doi: 10.3389/fcvm.2020.00001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]
- 27.Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36. doi: 10.7326/0003-4819-155-8-201110180-00009. [DOI] [PubMed] [Google Scholar]
- 28.Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med. 2019;170(1):W1–33. doi: 10.7326/M18-1377. [DOI] [PubMed] [Google Scholar]
- 29.Doudesis D, Lee KK, Boeddinghaus J, Bularga A, Ferry AV, Tuck C, et al. Machine learning for diagnosis of myocardial infarction using cardiac troponin concentrations. Nat Med. 2023;29(5):1201–10. doi: 10.1038/s41591-023-02325-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Bouzid Z, Faramand Z, Gregg RE, Frisch SO, Martin-Gill C, Saba S, et al. In search of an optimal subset of ECG features to augment the diagnosis of acute coronary syndrome at the emergency department. J Am Heart Assoc. 2021;10(3) doi: 10.1161/JAHA.120.017871. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Gibson CM, Mehta S, Ceschim MRS, Frauenfelder A, Vieira D, Botelho R, et al. Evolution of single-lead ECG for STEMI detection using a deep learning approach. Int J Cardiol. 2022;346:47–52. doi: 10.1016/j.ijcard.2021.11.039. [DOI] [PubMed] [Google Scholar]
- 32.Lin A, Manral N, McElhinney P, Killekar A, Matsumoto H, Kwiecinski J, et al. Deep learning-enabled coronary CT angiography for plaque and stenosis quantification and cardiac risk prediction: an international multicentre study. Lancet Digit Health. 2022;4(4):e256–65. doi: 10.1016/S2589-7500(22)00022-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Jokardarabi M, Zare HK, Namayandeh S, Owlia MS, mehdi Lotfi M. 2023. Development of a decision-maker system to predict in-hospital mortality of Acute Myocardial Infarction patients; an artificial intelligent approach: Yazd Cardiovascular Disease Registry data use. [Google Scholar]
- 34.Sherazi SWA, Zheng H, Lee JY. A machine learning-based applied prediction model for identification of acute coronary syndrome (ACS) outcomes and mortality in patients during the hospital stay. Sensors (Basel) 2023;23(3):1351. doi: 10.3390/s23031351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Valente F, Henriques J, Paredes S, Rocha T, de Carvalho P, Morais J. A new approach for interpretability and reliability in clinical risk prediction: acute coronary syndrome scenario. Artif Intell Med. 2021;117 doi: 10.1016/j.artmed.2021.102113. [DOI] [PubMed] [Google Scholar]
- 36.Emakhu J, Etu E-E, Monplaisir L, Aguwa C, Arslanturk S, Masoud S, et al. A hybrid machine learning and natural language processing model for early detection of acute coronary syndrome. Healthcare Analytics. 2023;4 [Google Scholar]
- 37.Emakhu J, Monplaisir L, Aguwa C, Arslanturk S, Masoud S, Nassereddine H, et al. Acute coronary syndrome prediction in emergency care: a machine learning approach. Comput Methods Progr Biomed. 2022;225 doi: 10.1016/j.cmpb.2022.107080. [DOI] [PubMed] [Google Scholar]


