Artificial Intelligence and Decision-Making in Healthcare: A Thematic Analysis of a Systematic Review of Reviews

Abstract

Introduction

The use of artificial intelligence (AI), which can emulate human intelligence and enhance clinical results, has grown in healthcare decision-making due to the digitalization effects and the COVID-19 pandemic. The purpose of this study was to determine the scope of applications of AI tools in the decision-making process in healthcare service delivery networks.

Materials and methods

This study used a qualitative method to conduct a systematic review of the existing reviews. Review articles published between 2000 and 2024 in English-language were searched in PubMed, Scopus, ProQuest, and Cochrane databases. The CASP (Critical Appraisal Skills Programme) Checklist for Systematic Reviews was used to evaluate the quality of the articles. Based on the eligibility criteria, the final articles were selected and the data extraction was done independently by 2 authors. Finally, the thematic analysis approach was used to analyze the data extracted from the selected articles.

Results

Of the 14 219 identified records, 18 review articles were eligible and included in the analysis, which covered the findings of 669 other articles. The quality assessment score of all reviewed articles was high. And, the thematic analysis of the data identified 3 main themes including clinical decision-making, organizational decision-making, and shared decision-making; which originated from 8 subthemes.

Conclusions

This study revealed that AI tools have been applied in various aspects of healthcare decision-making. The use of AI can improve the quality, efficiency, and effectiveness of healthcare services by providing accurate, timely, and personalized information to support decision-making. Further research is needed to explore the best practices and standards for implementing AI in healthcare decision-making.

Introduction

The literature suggests that the COVID-19 pandemic, by enforcing physical distancing measures, facilitated the digitalization and virtualization of health and social services. ¹ This phenomenon has prompted researchers to characterize the period following the COVID-19 pandemic as the era of digitalization, artificial intelligence (AI), and substantial increase in the adoption of AI digital platforms, particularly in the provision of healthcare services. ²

AI refers to the ability of a machine or system to perform tasks that require human intelligence, such as reasoning, learning, and decision making. ³ AI encompasses various technologies, such as machine learning, deep learning, natural language processing, robotics, speech processing, and other automation technologies. ⁴

Various AI-based digital platforms have attracted considerable attention from service providers and users, as they can offer the opportunity to implement patient-centered care within the healthcare systems, owing to a number of potential benefits that these technologies provide. For service users from diverse populations, settings, and conditions, potential benefits include convenience and enhanced accessibility, especially where factors such as physical mobility challenges and more importantly the opportunity to provide services based on the preferences of patients as distinct individuals with distinct differences.^5,6

In the field of clinical decision making, AI has demonstrated remarkable capabilities for predicting and classifying diagnoses, as well as providing recommendations and insights.^4,7 The increasing empirical evidence suggests that knowledge-based computerized decision support (CDS) and, in particular, knowledge-based clinical decision support systems, have the potential to enhance practitioner performance.^8,9 The literature indicates that AI systems have been successful in multiple medical imaging use cases, such as detecting mitosis in breast cancer histology images, ¹⁰ classifying skin cancer with dermatologist-level accuracy, ¹¹ diagnosing diabetic retinopathy from retinal fundus photographs, ¹² and predicting cardiovascular risk factors from retinal fundus photographs. ¹³ These studies demonstrate the potential of AI systems to assist healthcare providers in improving diagnosis and prognosis, as well as providing personalized and customized information. Moreover, they have been demonstrated to reliably predict the risk of imminent suicide attempts, ¹⁴ and they have been employed to assess the likelihood of patients developing serious conditions or being transferred to palliative care. ¹⁵

The decisions within the healthcare service delivery system are often based on the physicians’ own opinions. ¹⁶ Meanwhile, many patients lack an advance directive or a surrogate who can decide for them, so strangers have to make the choices ¹⁷ ; Patients also have difficulties with planning their future care and understanding resuscitation scenarios, so they may not communicate their wishes or may have unrealistic expectations (eg, overestimating the success of cardio-pulmonary resuscitation). ¹⁸ Moreover, relatives who act as surrogates may experience stress and may not respect the patient's preferences. ¹⁹ These are only a few examples that demonstrate the need for novel approaches to decision making within the healthcare service delivery system. And, one of the potential options is known to be the use of AI within the decision-making process. AI is changing the way clinical providers make decisions in healthcare. AI can assist healthcare professionals in diagnosing diseases, planning treatments, predicting outcomes, and managing population health. AI can also improve the quality and efficiency of healthcare decision making, as well as the satisfaction and involvement of users. ²⁰

AI is a rapidly evolving field that has the potential to transform various aspects of healthcare, such as diagnosis, treatment, prevention, and management. However, the literature on the application domains of AI in healthcare decision-making is scattered and fragmented, making it difficult to obtain a clear and comprehensive picture of the current state and future directions of this emerging field. This phenomenon motivated us to conduct a systematic review of the literature on the use of AI in the decision-making process of healthcare service delivery in the healthcare system. The purpose of this research was to accumulate and determine the application areas of AI in the decision-making process of providing health care through conducting a review of reviews.

Materials and Methods

This article was a qualitative content analysis that conducted a systematic review of the existing reviews in the literature from 2000 to 2024. The aim of this study was to review systematically the existing reviews on the utilization of AI in decision-making processes within the delivery of healthcare services. A review of reviews is a type of secondary article that synthesizes and evaluates the methods and quality of other review articles on a specific topic. Review articles are systematic summaries of the current state of knowledge and evidence on a topic. Review of reviews aims to compare and contrast the different types of review articles, assess their reliability and validity, and identify the best sources of evidence for researchers, practitioners, and policymakers. Review of reviews can also help to identify gaps and controversies in the literature, and suggest directions for future research. Review of reviews are also referred to as overviews of reviews, umbrella reviews, or systematic reviews of systematic reviews. ²¹

Data Gathering and Search Strategy

The present study adhered to the PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for systematic reviews to ensure the quality and validity of the research. ²² A systematic search was conducted to identify all published review articles related to the utilization of AI in decision-making processes within the delivery of healthcare services, from 2000 to 2024 and in English only. The databases searched were PubMed, Scopus, Cochrane, and ProQuest. We used MeSH terms to categorize all keywords into 3 groups: AI, delivery of healthcare, and decision-making. The synonymous keywords were combined using the logical operator “OR.” The first, second, and third groups of keywords were then merged using the logical operator “AND.” EndNote X7.1 software was used to organize the references. And, the search was conducted on February 3, 2024. The search strategy is presented in Table 1.

Table 1.

The Search Strategy Used for Conducting the Systematic Review.

No.	Domains	Keywords
#1	Artificial intelligence	“artificial intelligence” OR AI OR “machine learning” OR “deep learning”
#2	Decision-making	“decision making” OR decision-making* OR decision* OR choice* OR “choice behavior” OR judgment*
#3	Delivery of healthcare	health care system* OR healthcare system* OR health service* OR healthcare OR health OR “health policy”
Search strategy		#1 AND #2 AND #3 AND (time-span 2000–2024) AND [English]

Inclusion and Exclusion Criteria

We included articles published in English from 2000 to 2024 that focused on decision-making using AI in healthcare delivery. We excluded articles that: (1) did not cover a relevant AI topic in healthcare, (2) lacked a title or abstract describing an AI application in healthcare, or (3) lacked a title, abstract, or full-text providing any decision-making usage of AI in healthcare application in any healthcare system. Also, the articles that were recognized as low quality based on the quality assessment score using CASP Systematic Review Checklist, ²³ were excluded. Also, some types of articles including short communications, letters to the editor, and other irrelevant publications were removed.

Screening and Data Extraction

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart ²⁴ was followed to select the articles for the review. Any duplicates were removed and the remaining articles were screened by their title and abstract. Any articles that were not relevant to the research objective were discarded, and the full text of the rest was read. Only those that met the eligibility criteria were included in the final analysis. The whole process was carried out independently by 2 researchers. In case of any conflict regarding the results of the procedure, the other authors were consulted to complete the screening process.

The data from the final articles that aligned with the study objective were extracted independently by 2 authors. They used Microsoft Office Excel 2013 to create a data extraction form for data collection. The form consisted of sections such as authors, title, year, country, journal, indexed databases, study type, digital tool, disorder type, number of final articles, and summary of results.

Quality Assessment of Final Articles Using the CASP Checklist

Since this is a relatively new research field, we deemed it important to conduct a qualitative assessment of the presented articles. The quality of the selected studies was assessed using the Critical Appraisal Skills Program (CASP) for review articles (CASP Systematic Review Checklist). ²¹ The main CASP checklists (for randomized controlled trials and systematic reviews) were adapted from the Journal of American Medical Association (JAMA) User's Guides to the Medical Literature in 1994. The CASP checklist is a tool used to assess the quality of research studies. It comprises various categories and questions that help evaluate the validity, relevance, and applicability of a research study. The checklist is usually evidence-based in medicine for determining the strength of evidence for clinical decision-making. ²⁵

The checklist consists of 10 questions that cover 3 broad issues: are the results of the review valid, what are the results, and will the results help locally. Two questions are descriptive and 8 questions are scored, with 2 for yes, 1 for I can’t say, and 0 for no. The quality of the studies is measured by the percentage of the total score out of 16. The percentage range is between 0 and 100, where higher scores indicate higher quality. The studies are classified into 4 quality levels: very low (0-25), low (26-50), medium (51-75), and high (76-100). Only studies with medium or high quality are deemed eligible for research purposes.

Data Analysis

A qualitative thematic analysis and an inductive approach ²⁶ were used to analyze the data extracted from the previous steps. The authors repeatedly read all the included articles and extractions to gain a thorough understanding of the data and then assigned initial codes to each meaningful extraction. All initial codes were examined and finalized before grouping them into subthemes and main themes. The subthemes and main themes were then named, described, and presented in a table. Lincoln and Guba's ²⁷ criteria were used to evaluate the trustworthiness of the research. Trustworthiness refers to the extent to which the findings of a study are credible, transferable, dependable, and confirmable.

Results

Review

As depicted in Figure 1, we found 14 219 records in the database searching. After duplicate removal, we screened 6483 records. Finally, the quality of the articles was assessed using the CASP checklist, and 18 articles were included.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart for study selection.

Table 2 presents a summary of the characteristics of 18 studies that examined the application of AI-based systems in various domains of healthcare decision-making. The studies included in this research covered the results of 669 articles.

Table 2.

Summary of the characteristics of the18 included studies .

No	Author(s)/citation	Title	Year	Aim	Method	Number of included articles
1	Akay et al 28	Artificial Intelligence for Clinical Decision Support in Acute Ischemic Stroke: A Systematic Review	2023	Reviewed AI-based clinical decision support systems in the development stage, specifically regarding methodological robustness and constraints for clinical implementation.	Systematic review	121
2	Al-Namankany 29	Influence of Artificial Intelligence-Driven Diagnostic Tools on Treatment Decision-Making in Early Childhood Caries: A Systematic Review of Accuracy and Clinical Outcomes	2023	Aimed to assess the performance and applicability of machine learning algorithms in early childhood caries (ECC) prediction and detection. A comprehensive search was conducted to identify studies utilizing machine learning algorithms to predict or detect ECC.	Systematic review	6
3	Ali et al 30	Improving Diagnosis and Prognosis of Lung Cancer Using Vision Transformers: A Scoping Review	2023	Aimed to identify the recent developments in vision transformer-based AI methods for lung cancer imaging applications.	Scoping review	34
4	Amin et al 31	Artificial Intelligence to Improve Antibiotic Prescribing: A Systematic Review	2023	Explored whether the use of AI can improve antibiotic prescribing for human patients.	Systematic review	5
5	Benzinger et al 32	Should Artificial Intelligence be Used to Support Clinical Ethical Decision-Making? A Systematic Review of Reasons	2023	Aimed to provide a comprehensive overview of the reasons given in the academic literature for and against the use of artificial intelligence-based applications in clinical ethical decision making.	Systematic review	44
6	Cresswell et al 33	Investigating the Use of Data-Driven Artificial Intelligence in Computerized Decision Support Systems for Health and Social Care: A Systematic Review	2020	The objective of this study was to evaluate the effectiveness of artificial intelligence-based computerized decision support systems in health and social care settings.	Systematic review	5
7	Dang et al 34	Extent of Use of Artificial Intelligence and Machine Learning Protocols in Cancer Diagnosis: A Scoping Review	2023	Aimed to explore the extent of actual use of artificial intelligence/machine learning protocols for diagnosing cancer in prospective settings	Scoping review	18
8	Fernandes et al 35	Clinical Decision Support Systems for Triage in the Emergency Department Using Intelligent Systems: A Review	2020	The objective of this paper was to evaluate the impact of intelligent clinical decision support systems for triage on the enhancement of quality of care in the emergency department and to identify the challenges encountered regarding implementation.	Scoping review	62
9	Higgins et al 36	Artificial Intelligence (AI) and Machine Learning (Ml) Based Decision Support Systems in Mental Health: An Integrative Review	2023	Aimed to investigate the incorporation of artificial intelligence (AI) and machine learning (ML) based decision support systems in mental health care settings.	Scoping review	4
10	Khan et al 37	Adoption and Utilization of Medical Decision Support Systems in the Diagnosis of Febrile Diseases: A Systematic Literature Review	2023	Aimed to review the development, adoption, and utilization of medical decision-support systems in diagnosing Febrile Diseases	Systematic review	128
11	Liao et al 38	Recent Advancement in Integrating Artificial Intelligence and Information Technology With Real-World Data for Clinical Decision-Making in China: A Scoping Review	2023	Aimed to provide an overview of the current status of artificial intelligence-/information technology-based clinical decision support tools in China.	Scoping review	37
12	Michel et al 39	Clinical Decision Support System in Emergency Telephone Triage: A Scoping Review of Technical Design, Implementation and Evaluation	2024	Aimed to identify literature gaps for the future development and evaluation of clinical decision-support systems for emergency telephone triage.	Scoping review	19
13	Moazemi et al 40	Artificial Intelligence for Clinical Decision Support for Monitoring Patients in Cardiovascular ICUs: A Systematic Review	2023	Aimed to investigate studies and tools that address artificial intelligence/machine learning-based approaches towards clinical decision support (CDS) for monitoring cardiovascular patients in intensive care units (ICUs).	Systematic review	21
14	Nida et al 41	Applications of Artificial Neural Networks in Healthcare Organizational Decision-Making: A Scoping Review.	2019	The aim was to provide a seminal review of the applications of artificial neural networks to healthcare organizational decision-making.	Scoping review	80
15	Rahimi et al 42	Application of Artificial Intelligence in Shared Decision Making: Scoping Review.	2022	The paper aimed to identify and evaluate published studies that had tested or implemented artificial intelligence to facilitate shared decision-making.	Scoping review	6
16	Tiwari et al 43	Artificial Intelligence's Use in the Diagnosis of Mouth Ulcers: A Systematic Review	2023	Aimed to identify the application of AI in the diagnosis of various forms of mouth ulcers	Systematic review	23
17	Tricco et al 44	Implemented Machine Learning Tools to Inform Decision-Making for Patient Care in Hospital Settings: A Scoping Review	2023	The aim was to identify machine learning tools that were used in hospital settings and to examine how they were implemented to inform decision-making for patient care.	Scoping review	20
18	Uzun Ozashin et al 45	The Systematic Review of Artificial Intelligence Applications in Breast Cancer Diagnosis	2023	Aimed to provide detailed insights into AI applications in breast cancer diagnosis, categorize key specializations of interest to the academic community, and identify research themes in each category through citation and thematic issue analysis.	Systematic review	36

Study No. 1 was a systematic review published in 2023. The study aimed to review AI-based clinical decision support systems in the development stage, specifically regarding methodological robustness and constraints for clinical implementation. The review included 121 articles. ²⁸

Study No. 2 was a systematic review published in 2023. The study aimed to assess the performance and applicability of machine learning algorithms in early childhood caries (ECC) prediction and detection. The review included 6 articles. ²⁹

Study No. 3 was a scoping review published in 2023. The study aimed to identify the recent developments in vision transformer-based AI methods for lung cancer imaging applications. The review included 34 articles. ³⁰

Study No. 4 was a systematic review published in 2023. The study aimed to explore whether the use of AI can improve antibiotic prescribing for human patients. The review included 5 articles. ³¹

Study No. 5 was a systematic review published in 2023. The study aimed to provide a comprehensive overview of the reasons given in the academic literature for and against the use of AI-based applications in clinical ethical decision making. The review included 44 articles. ³²

Study No. 6 was a systematic review published in the United Kingdom in 2020. It reviewed 5 final articles that assessed the effectiveness of AI-based computerized decision support systems in health and social care settings. This study was the only one that employed a systematic review method, which is regarded as more rigorous and comprehensive than a scoping review method. However, the low number of final articles reviewed indicates a lack of evidence on the effectiveness of AI-based computerized decision support systems within health and social care settings. ³³

Study No. 7 was a scoping review published in 2023. The study aimed to explore the extent of the actual use of AI/machine learning protocols for diagnosing cancer in prospective settings. The review included 18 articles. ³⁴

Study No. 8 was a scoping review published in 2019. It reviewed 62 final articles that assessed the impact of AI-based decision support systems on various outcomes and populations in emergency department triage settings. This study was the only one that focused on emergency department (ED) triage, which is a process that involves prioritizing patients based on their urgency and severity of conditions. While the high number of final articles reviewed indicates a wide range of AI-based decision support systems in ED triage settings. ³⁵

Study No. 9 was a scoping review published in 2023. The study aimed to investigate the incorporation of AI and ML-based decision support systems in mental health care settings. The review included 4 articles. ³⁶

Study No. 10 was a systematic review published in 2023. The study aimed to review the development, adoption, and utilization of medical decision-support systems in diagnosing febrile diseases. The review included 128 articles. ³⁷

Study No. 11 was a scoping review published in 2023. The study aimed to provide an overview of the current status of AI-/information technology-based clinical decision support tools in China. The review included 37 articles. ³⁸

Study No. 12 was a scoping review published in 2024. The study aimed to identify literature gaps for the future development and evaluation of clinical decision-support systems for emergency telephone triage. The review included 19 articles. ³⁹

Study No. 13 was a systematic review published in 2023. The study aimed to investigate studies and tools that address AI/machine learning-based approaches towards clinical decision support (CDS) for monitoring cardiovascular patients in intensive care units (ICUs). The review included 21 articles. ⁴⁰

Study No. 14 was a scoping review published in 2019. It reviewed 80 final articles that explored the use of artificial neural networks (ANNs) across all levels of healthcare organizational decision-making. This study was the only one that focused on ANN, which is a type of AI system that can learn from data and tailor solutions to a given problem. Moreover, the high number of final articles reviewed indicates a growing interest and potential of ANN in healthcare organizational decision-making. ⁴¹

Study No. 15 was a scoping review published in 2022. It reviewed 6 final articles that investigated the use of AI systems to support shared decision-making in healthcare settings. This study was the only one that focused on shared decision-making (SDM), which is a process that involves patients and practitioners making decisions together based on evidence and preferences. However, the low number of final articles reviewed indicates a lack of evidence on AI systems to support SDM in healthcare settings. ⁴²

Study No. 16 was a systematic review published in 2023. The study aimed to identify the application of AI in the diagnosis of various forms of mouth ulcers. The review included 23 articles. ⁴³

Study No. 17 was a scoping review published in 2023. It reviewed 20 final articles that identified and assessed the strategies for the implementation of machine learning tools in hospital settings. This study was the only one that focused on implementation strategies, which are methods and approaches used for introducing and integrating machine learning tools into healthcare settings. Meanwhile, the moderate number of final articles reviewed indicates some evidence of implementation strategies for machine learning tools in hospital settings. ⁴⁴

Study No. 18 was a systematic review published in 2023. The study aimed to provide detailed insights into AI applications in breast cancer diagnosis, categorize key specializations of interest to the academic community, and identify research themes in each category through citation and thematic issue analysis. The review included 36 articles. ⁴⁵

Quality Assessment of Selected Review Articles

In Table 3, the result of the qualitative assessment of the selected review articles in this study is reported. Using the CASP checklist, we evaluated the articles and found that all 18 articles scored full marks in the “result validity” dimension (including Q1 and Q2), which indicates a clear focus on the research question and appropriate selection of papers. For the “worth continuing” dimension (including Q3, Q4, and Q5), we found that all articles obtained a full score in the “combination of review results” question (Q5). However, despite these positive findings, none of the articles fully met our criteria in the “inclusion of all the important, relevant studies” question (Q3). Only 6 studies received a full score in the “assessment of quality of included studies” question (Q4). Moreover, all articles received full marks in the “local impact of results” dimension (including Q8, Q9, and Q10). In general, the quality assessment score of all reviewed articles was high.

Table 3.

Quality Assessment of Review Articles Using the CASP Checklist.

No.	Author(s)/citation	Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8	Q9	Q10	Score
1	Akay et al 28	Yes	Yes	?	No	Yes	The findings of the study indicated substantial threats to validity, inconsistencies in reporting methodologies, and obstacles in the clinical application. This document delineates pragmatic guidelines for the efficacious execution of artificial intelligence research in the treatment and diagnosis of acute ischemic stroke.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
2	Al-Namankany 29	Yes	Yes	?	No	Yes	The utilization of machine learning algorithms has facilitated the enhancement of clinical decision-making processes, the implementation of targeted preventive measures, and the improvement of early childhood caries (ECC) management. The research underscored the significance of taking into account a multitude of factors—demographic, environmental, and genetic—in the construction of predictive models for dental caries.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
3	Ali et al 30	Yes	Yes	?	No	Yes	Models based on vision transformers are gaining prominence in the development of artificial intelligence methodologies for applications in lung cancer. Nonetheless, their computational intricacy and clinical pertinence are crucial considerations for forthcoming research endeavors.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
4	Amin et al 31	Yes	Yes	?	Yes	Yes	The employment of machine learning techniques has the potential to enhance the prescription of antibiotics in both primary and secondary healthcare environments. It is noteworthy that none of the studies assessed the process of integrating their models into clinical practices.	The results were deemed precise.	Yes	Yes	Yes	15/16 (93.7%)
5	Benzinger et al 32	Yes	Yes	?	No	Yes	The potential advantages of employing artificial intelligence in clinical ethical decision-making are numerous. However, its development and utilization must be approached with caution to circumvent ethical dilemmas. A number of issues that are fundamental to the discourse on clinical decision support systems, including justice, explicability, and human-machine interaction, have thus far been overlooked in the dialogue concerning the use of AI in clinical ethics.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
6	Cresswell et al 33	Yes	Yes	?	Yes	Yes	The final studies differed greatly in quality, settings, outcomes, and technologies. None was in social care settings, and 3 randomized controlled trials showed no difference in patient outcomes. These trials involved Bayesian triage algorithms, image pattern recognition, and the Kalman filter technique. The other 2 trials, involving computer vision and neural networks, and learning algorithms, showed significant and important differences to the control groups. However, these studies were of low quality with poor methods and only one double-blind design. The evidence of the effectiveness of data-driven artificial intelligence for decision-making in health and social care settings was limited. The effectiveness of interventions depended on context and needed various study designs to investigate mechanisms of action.	The paucity of included studies limits the precision of the results.	Yes	Yes	Yes	15/16 (93.7%)
7	Dang et al 34	Yes	Yes	?	Yes	Yes	The conclusions drawn from this review indicate a noticeable gap in the translation from the validation of artificial intelligence/machine learning protocols to their practical application in cancer diagnosis. The establishment of a regulatory framework specifically tailored for the use of AI/ML in healthcare is of paramount importance.	The results were deemed precise.	Yes	Yes	Yes	15/16 (93.7%)
8	Fernandes et al 35	Yes	Yes	?	No	Yes	In the papers where clinical decision support systems were validated in the emergency department, the authors found that there was an improvement in the health professionals’ decision-making thereby leading to better clinical management and patients’ outcomes. However, it was found that more than half of the studies lacked this implementation phase. The authors concluded that for these studies, it was necessary to validate the clinical decision support systems and to define key performance measures in order to demonstrate the extent to which incorporation of clinical decision support systems at triage can actually improve care.	Due to the number of reviewed articles and reporting credibility, the study has acceptable precision.	Yes	Yes	Yes	13/16 (81.2%)
9	Higgins et al 36	Yes	Yes	?	Yes	Yes	The review underscored the imperative of clinician involvement in all phases of AI research, development, and deployment in healthcare. Prioritizing clinician trust is crucial for the successful implementation of AI-based decision support systems. Encouraging clinicians to contribute to the development of new health technologies can help preempt missed care, enhancing public safety and ethical implementation. AI-based tools in mental health settings hold significant potential, contingent on clinician trust and confidence.	The results were deemed precise.	Yes	Yes	Yes	15/16 (93.7%)
10	Khan et al 37	Yes	Yes	?	No	Yes	The study found fuzzy logic as the dominant MDSS model, with many models discussed yet unimplemented. These models were proposed primarily to enhance accuracy and precision. Adoption by medical facilities was influenced by usefulness, relative advantage, and ease of use. However, half of the studies did not express the reasons for MDSS adoption. Those who did adopt MDSS primarily used it to boost effectiveness and guideline adherence.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
11	Liao et al 38	Yes	Yes	?	No	Yes	The study found that 32.0% of the models were designed for disease diagnosis, 54.0% for risk prediction and classification, and 14.0% for disease management. Chronic diseases, particularly cardiovascular and cerebrovascular, were the primary focus. Single-center electronic medical records were the primary data source, with internal validation predominantly used for model evaluation.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
12	Michel et al 39	Yes	Yes	?	No	Yes	This review underscores the potential of a hybrid system that is user-customized, flexible, and integrated with the electronic health record. This system can process oral, video, and digital data. It also emphasizes the necessity to assess clinical decision support systems based on their inherent characteristics and their impact on clinical practice, iteratively at each unique stage of the information technology lifecycle.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
13	Moazemi et al 40	Yes	Yes	?	Yes	Yes	Clinical time series and electronic health records (EHRs) data emerged as the predominant input modalities. Analytical methods such as gradient boosting, recurrent neural networks (RNNs), and reinforcement learning (RL) were frequently employed. It is noteworthy that 75% of the chosen papers did not validate against external datasets, thereby underscoring the issue of generalizability. Furthermore, the interpretability of AI decisions was pinpointed as a crucial factor for the successful incorporation of AI in healthcare.	The results were deemed precise.	Yes	Yes	Yes	15/16 (93.7%)
14	Nida et al 41	Yes	Yes	?	No	Yes	The final articles were published from 1997 to 2018 and originated from 24 countries, with most papers (26 articles) published by authors from the United States. Types of artificial neural networks used included artificial neural networks, feed-forward networks, or hybrid models; reported accuracy ranged from 50% to 100%. The majority of artificial neural networks informed decision-making at the micro level, between patients and health care providers. Fewer artificial neural networks were deployed for intraorganizational and system, policy, or interorganizational (10 articles) decision-making. The review identified key characteristics and drivers for market uptake of artificial neural networks for healthcare organizational decision-making to guide further adoption of this technique.	Due to the number of reviewed articles and reporting credibility, the study has acceptable precision.	Yes	Yes	Yes	13/16 (81.2%)
15	Rahimi et al 42	Yes	Yes	?	No	Yes	The paper concluded that the application of AI in SDM was still emerging. The authors` review revealed similar modes of AI support for SDM across the selected studies. However, they noticed a lack of attention to patients’ values and preferences, as well as inadequate reporting of AI interventions, leading to ambiguity about various aspects. The issues of understandability of AI interventions and end-user involvement in their design and development were scarcely addressed.	The paucity of included studies limits the precision of the results.	Yes	Yes	Yes	13/16 (81.2%)
16	Tiwari et al 43	Yes	Yes	?	No	Yes	AI can enhance diagnostics, improve patient outcomes, and reduce invasive procedures. It can also provide personalized treatment plans and streamline workflows for dentists. AI-powered tools can improve patient interactions. However, challenges such as data privacy, algorithm validation, ethical concerns, professional training, and the cost and accessibility of AI technology must be considered.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
17	Tricco et al 44	Yes	Yes	?	Yes	Yes	The most common implementation strategies for the tools were clinician reminders that incorporated machine learning predictions, followed by facilitated relay of clinical information and staff education. The main barriers to the successful implementation of machine learning tools were time and reliability, while the main facilitators were time/efficiency and perceived usefulness. Furthermore, the authors found scarce evidence regarding the implementation of machine learning tools to assist clinicians with patient healthcare decisions in hospital settings.	Due to the number of reviewed articles and reporting credibility, the study has acceptable precision.	Yes	Yes	Yes	15/16 (93.7%)
18	Uzun Ozashin et al 45	Yes	Yes	?	No	Yes	Most AI applications use classification models for breast cancer prediction, with accuracy (99%) being the primary performance metric, followed by specificity (98%) and area under the curve (0.95). The convolutional neural network (CNN) was often the preferred model.	The results were deemed precise.	Yes	Yes	Yes	13/16 (81.2%)
Critical Appraisal Skills Program (CASP) questions scoring: Yes = 2, Can't tell (?) = 1, No  =  0
Q1: Did the review address a clearly focused question? Q2: Did the authors look for the right type of papers? Q3: Do you think all the important, relevant studies were included? Q4: Did the review's authors do enough to assess the quality of the included studies? Q5: If the results of the review have been combined, was it reasonable to do so?							Q6: What are the overall results of the review? Q7: How precise are the results? Q8: Can the results be applied to the local population? Q9: Were all important outcomes considered? Q10: Are the benefits worth the harms and costs?

Thematic Analysis of the Findings

After thoroughly reviewing the articles included in the study, we have identified 3 main themes: “clinical decision-making,” “organizational decision-making,” and “shared decision-making.” Additionally, there are 8 subthemes related to these main themes: remote monitoring, computerized interpretation of graphs, facilitating diagnosis and prognosis, forecasting administrative and quality indicators, offering cost-effective solutions for time and resource management, providing personalized and customized information, the possibility of patient self-management, enhancing patient medication adherence. In Table 4, the main themes and subthemes, and brief definitions for each theme are presented.

Table 4.

Findings From the Thematic Analysis.

Main themes	Subthemes	Definition	Reference(s)
Clinical decision-making	Remote monitoring	Decisions regarding to the curative approach led by hospitals and clinics	Akay et al, 28 Al-Namankany, 29 Ali et al, 30 Amin et al, 31 Cresswell et al, 33 Dang et al, 34 Fernandes et al, 35 Higgins et al, 36 Khan et al, 37 Liao et al, 38 Michel et al, 39 Moazemi et al, 40 Tiwari et al, 43 and Uzun Ozsahin et al 45
	Computerized interpretation of graphs
	Facilitating diagnosis and prognosis
Organizational decision-making	Forecasting administrative and quality indicators	Decisions regarding to the management of resources within healthcare organizations and the healthcare system	Nida et al 41 and Tricco et al 44
	Offering cost-effective solutions for time and resource management
Shared decision-making	Provision of personalized and customized information	Decisions shared with users of services(patients)	Benzinger et al, 32 Rahimi et al, 42 and Tiwari et al 43
	Enabling patient self-management
	Enhancing patient medication adherence

Discussion

The aim of the study was to determine the application domains of various types of AI in the decision-making process in the healthcare system. Three main themes including clinical decision-making, organizational decision-making, and shared decision-making were identified, which originated from 8 subthemes. Each is discussed in detail below:

Clinical Decision-Making

As per the existing scholarly works, the theme of clinical decision-making encompasses various subthemes to where AI can be applied. These subthemes include remote monitoring, computerized interpretation of graphs, and facilitating diagnosis and prognosis.

One of the subthemes that refers to the application of AI in the clinical field is remote monitoring. AI can help classify physical activities and detect abnormal events or behaviors using wearable sensors and cameras. This can help remote-monitor the patients’ mobility, safety, and adherence to treatment plans. ⁴⁶ One study evaluated the performance of algorithms enabling automated triaging based on existing data sets. Despite the reported lack of statistical significance of the effect of AI on the outcomes, the increased availability of remote monitoring was reported to potentially enhance the outcomes and efficiency through the use of such AI platforms. ⁴⁷ Interestingly, a study has reported that AI has also the potential to perform in emergency circumstances, assisting in monitoring cardiovascular patients in ICUs. ⁴⁰

The next subtheme is a computerized interpretation of graphs. The task of identifying the discriminatory and latent attributes from large-scale heterogeneous data is challenging for both nodal graphs and graph images (also known as chart images). Therefore, interpretation-based methods using AI emerged as a promising solution. ⁴⁸ One possible application of AI in the computerized interpretation of graphs in the clinical field is to use machine learning algorithms to analyze electrocardiogram signals. ⁴⁹ A study evaluated the performance of AI facilitating the interpretation of cardiotocographs through image pattern recognition by applying computerized interpretation. Surprisingly, the study reported that the application of such AI tools does not enhance the clinical outcomes for maternal or neonatal health. ⁵⁰ Another paper declared that various tasks, such as analyzing medical images or associating symptoms and biomarkers from electronic medical records with the disease's characterization and prognosis, have shown that AI algorithms can perform equally or better than humans. ⁵¹ Moreover, vision transformers, a deep learning model, have become popular in medical imaging, especially for lung cancer. They analyze image classification as a sequence prediction task, capturing long-term dependencies within image patches. ³⁰ These results demonstrate the considerable potential of employing AI in the analysis and interpretation of medical graphs.

The third subtheme is facilitating diagnosis and prognosis. AI can help prevent and diagnose various diseases, such as mental disorders, cancer, diabetic retinopathy, Alzheimer's disease, etc, using deep learning models. This can help provide faster and more reliable diagnosis and reduce human errors.^{28,29,34,36-38,43,45,52} Besides that, AI has shown the capability to streamline the drug prescription process. ³¹ Moreover, one paper reported that AI can aid in patient triage by offering guidance and evaluating the severity and priority of cases. ³⁹ Another paper described the application of ANNs in the process of clinical decision-making in healthcare and reported the potential promise of their use in situations involving complex, unstructured, or limited information. ⁴¹ Similarly, a study evaluated the performance and benefit of ANNs as clinical decision-making tools in cancer. It indicated that ANNs had demonstrated promising outcomes in diagnosis and prognosis. ⁵³ These findings delineate the potentially significant contribution of AI in the process of diagnosis and prognosis of diseases.

Organizational Decision-Making

Evidence shows that AI can also be used in organizational decision-making. This main theme includes 2 subthemes which are forecasting administrative and quality indicators, and providing cost-effective solutions for time and resource management.

The first subtheme is forecasting administrative and quality indicators. AI can help forecast the performance and outcomes of healthcare processes, such as patient satisfaction, safety, quality of care, and efficiency, using machine learning algorithms and clinical data. This can help identify best practices, monitor progress, and provide feedback and recommendations for improvement. ⁵⁴ It is shown that ANNs have been effectively applied to forecast quality factors (responsiveness, security, and efficiency) that affect the adoption of e-government services. ⁵⁵ Moreover, it is reported that some forms of AI tools are used for distinct organizational levels of healthcare systems; in this regard, it was reported that ANNs were mostly used for microlevel decisions between patients and providers, and less for meso- and macrolevel decisions. ⁴¹

The next subtheme is providing cost-effective solutions for time and resource management. AI can help automate and streamline administrative tasks, such as scheduling, invoicing, reporting, and customer service, using natural language processing and machine learning. This can help save time and improve efficiency and productivity. ⁵⁶ It is shown that nonclinical applications have involved enhancing the management of healthcare organizations, as well as forecasting important indicators such as cost or resource utilization.^57,58 In such context, ANNs have been employed in decision support models that offer cost-effective solutions for time and resource management to healthcare providers and the healthcare system. ⁵⁹ AI can help predict the demand and supply of healthcare resources, such as beds, staff, equipment, and drugs, using historical data, current trends, and external factors. This can help optimize the allocation and utilization of resources and reduce costs and waste. ⁶⁰

Shared Decision-Making

Shared decision-making is the last main theme, and it includes 3 subthemes: providing personalized and customized information, enabling patient self-management, and enhancing patient medication adherence.

The first subtheme is providing personalized and customized information. One study delineated that AI has the potential to enhance patient autonomy by enabling patients to receive their treatment of choice. ³² Meanwhile, a research proposed a system that integrated AI estimations, physician's interpretation, and parents’ perspectives to support ethical treatment decisions for newborn patients; The results showed that the system had delivered personalized and customized information to different users and facilitated shared decision-making involving parents. ⁶¹ AI can help create adaptive and personalized educational software that can tailor the learning content, pace, feedback, and assessment to each individual learner. ⁶² This technology can help deliver customized education and guidance to patients and caregivers using natural language processing and chatbots. ⁶⁰ In such context, “Software as a Medical Device” (SaMD) is software intended for medical purposes, independent of hardware. It spans from computer applications to mobile apps, aiding in disease diagnosis, treatment, and prevention. ⁶³ Interestingly, through the provision of guidance and regulations for the development, evaluation, and approval of Software as a SaMD, the FDA and other global regulators facilitate stakeholders in promoting safe innovation while ensuring patient safety. ⁶³

The second subtheme is enabling patient self-management. One study investigated the use of learning algorithms to enhance patient self-management; The system reduced the time and visits required by clinicians, and patients expressed satisfaction with it. Moreover, no loss of monitoring was observed. ⁶⁴ AI can help patients with musculoskeletal pain, such as neck or lower back pain, to improve their physical activity, posture, and coping skills using a smartphone app that adapts to their individual needs and preferences. ⁶⁵ Similarly, AI can help patients with diabetes to manage their blood glucose levels, diet, and exercise using a chatbot that provides education and coaching based on natural language processing and machine learning. ⁶⁶

The last subtheme is enhancing patient medication adherence. A study evaluated the use of an AI platform on mobile devices in measuring and enhancing medication adherence in stroke patients. The study demonstrated that the platform was effective and improved adherence by 50%. ⁶⁷ Even though, contrary to such finding, a trial that tested an algorithm for warfarin dosing in self-managing patients; found no difference in warfarin treatment quality between the algorithm and standard care. ⁶⁸ AI can help patients with chronic diseases, such as diabetes, hypertension, or cancer, to adhere to their medication regimens by providing reminders, alerts, education, and coaching using mobile phone applications, chatbots, or wearable devices.^69,70 Furthermore, AI can prevent medication nonadherence using machine learning algorithms and clinical data, such as patient characteristics, medication history, disease severity, or psychosocial factors. ⁶⁹

Limitations and Implications

This study has several strengths, such as conducting a review of reviews and using a thematic analysis approach, applying a rigorous and transparent search strategy and quality assessment of selected articles, and providing a comprehensive and detailed synthesis of the themes and subthemes. However, this review has limitations that must be acknowledged. The first and most important limitation of the current research is that the selected studies were only in English. Second, it did not explore the perspectives or experiences of the end-users of the AI systems, such as patients, providers, managers, or policymakers. Therefore, the findings may not capture the acceptability, usability, feasibility, or satisfaction of the AI systems in healthcare decision-making domains.

Conclusions

This article showed that AI systems are useful in various areas of healthcare decision-making, including clinical, organizational, and shared decision-making. The findings indicated that AI has the potential to enhance the quality, efficiency, and effectiveness of healthcare decisions.

Author Biographies

Mohsen Khosravi is a researcher specializing in healthcare services. His areas of expertise include digital health, patientcentered care, sustainable development, and structural adjustment policies.

Zahra Zare is a researcher in the field of healthcare services, with a particular focus on oral health, regionalization, and public health.

Seyyed Morteza Mojtabaeian is a researcher in healthcare services. His research interests lie in healthcare economics, payment systems, and digital services.

Reyhane Izadi is a researcher in healthcare services. Her research primarily focuses on digital health, telemedicine, and pharmaceutical services.