Skip to main content
. 2024 Mar 5;11:23333928241234863. doi: 10.1177/23333928241234863

Table 3.

Quality Assessment of Review Articles Using the CASP Checklist.

No. Author(s)/citation Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Score
1 Akay et al 28 Yes Yes ? No Yes The findings of the study indicated substantial threats to validity, inconsistencies in reporting methodologies, and obstacles in the clinical application. This document delineates pragmatic guidelines for the efficacious execution of artificial intelligence research in the treatment and diagnosis of acute ischemic stroke. The results were deemed precise. Yes Yes Yes 13/16
(81.2%)
2 Al-Namankany 29 Yes Yes ? No Yes The utilization of machine learning algorithms has facilitated the enhancement of clinical decision-making processes, the implementation of targeted preventive measures, and the improvement of early childhood caries (ECC) management. The research underscored the significance of taking into account a multitude of factors—demographic, environmental, and genetic—in the construction of predictive models for dental caries. The results were deemed precise. Yes Yes Yes 13/16
(81.2%)
3 Ali et al 30 Yes Yes ? No Yes Models based on vision transformers are gaining prominence in the development of artificial intelligence methodologies for applications in lung cancer. Nonetheless, their computational intricacy and clinical pertinence are crucial considerations for forthcoming research endeavors. The results were deemed precise. Yes Yes Yes 13/16
(81.2%)
4 Amin et al 31 Yes Yes ? Yes Yes The employment of machine learning techniques has the potential to enhance the prescription of antibiotics in both primary and secondary healthcare environments. It is noteworthy that none of the studies assessed the process of integrating their models into clinical practices. The results were deemed precise. Yes Yes Yes 15/16
(93.7%)
5 Benzinger et al 32 Yes Yes ? No Yes The potential advantages of employing artificial intelligence in clinical ethical decision-making are numerous. However, its development and utilization must be approached with caution to circumvent ethical dilemmas. A number of issues that are fundamental to the discourse on clinical decision support systems, including justice, explicability, and human-machine interaction, have thus far been overlooked in the dialogue concerning the use of AI in clinical ethics. The results were deemed precise. Yes Yes Yes 13/16
(81.2%)
6 Cresswell et al 33 Yes Yes ? Yes Yes The final studies differed greatly in quality, settings, outcomes, and technologies. None was in social care settings, and 3 randomized controlled trials showed no difference in patient outcomes. These trials involved Bayesian triage algorithms, image pattern recognition, and the Kalman filter technique. The other 2 trials, involving computer vision and neural networks, and learning algorithms, showed significant and important differences to the control groups. However, these studies were of low quality with poor methods and only one double-blind design. The evidence of the effectiveness of data-driven artificial intelligence for decision-making in health and social care settings was limited. The effectiveness of interventions depended on context and needed various study designs to investigate mechanisms of action. The paucity of included studies limits the precision of the results. Yes Yes Yes 15/16
(93.7%)
7 Dang et al 34 Yes Yes ? Yes Yes The conclusions drawn from this review indicate a noticeable gap in the translation from the validation of artificial intelligence/machine learning protocols to their practical application in cancer diagnosis. The establishment of a regulatory framework specifically tailored for the use of AI/ML in healthcare is of paramount importance. The results were deemed precise. Yes Yes Yes 15/16
(93.7%)
8 Fernandes et al 35 Yes Yes ? No Yes In the papers where clinical decision support systems were validated in the emergency department, the authors found that there was an improvement in the health professionals’ decision-making thereby leading to better clinical management and patients’ outcomes. However, it was found that more than half of the studies lacked this implementation phase. The authors concluded that for these studies, it was necessary to validate the clinical decision support systems and to define key performance measures in order to demonstrate the extent to which incorporation of clinical decision support systems at triage can actually improve care. Due to the number of reviewed articles and reporting credibility, the study has acceptable precision. Yes Yes Yes 13/16 (81.2%)
9 Higgins et al 36 Yes Yes ? Yes Yes The review underscored the imperative of clinician involvement in all phases of AI research, development, and deployment in healthcare. Prioritizing clinician trust is crucial for the successful implementation of AI-based decision support systems. Encouraging clinicians to contribute to the development of new health technologies can help preempt missed care, enhancing public safety and ethical implementation. AI-based tools in mental health settings hold significant potential, contingent on clinician trust and confidence. The results were deemed precise. Yes Yes Yes 15/16 (93.7%)
10 Khan et al 37 Yes Yes ? No Yes The study found fuzzy logic as the dominant MDSS model, with many models discussed yet unimplemented. These models were proposed primarily to enhance accuracy and precision. Adoption by medical facilities was influenced by usefulness, relative advantage, and ease of use. However, half of the studies did not express the reasons for MDSS adoption. Those who did adopt MDSS primarily used it to boost effectiveness and guideline adherence. The results were deemed precise. Yes Yes Yes 13/16 (81.2%)
11 Liao et al 38 Yes Yes ? No Yes The study found that 32.0% of the models were designed for disease diagnosis, 54.0% for risk prediction and classification, and 14.0% for disease management. Chronic diseases, particularly cardiovascular and cerebrovascular, were the primary focus. Single-center electronic medical records were the primary data source, with internal validation predominantly used for model evaluation. The results were deemed precise. Yes Yes Yes 13/16 (81.2%)
12 Michel et al 39 Yes Yes ? No Yes This review underscores the potential of a hybrid system that is user-customized, flexible, and integrated with the electronic health record. This system can process oral, video, and digital data. It also emphasizes the necessity to assess clinical decision support systems based on their inherent characteristics and their impact on clinical practice, iteratively at each unique stage of the information technology lifecycle. The results were deemed precise. Yes Yes Yes 13/16 (81.2%)
13 Moazemi et al 40 Yes Yes ? Yes Yes Clinical time series and electronic health records (EHRs) data emerged as the predominant input modalities. Analytical methods such as gradient boosting, recurrent neural networks (RNNs), and reinforcement learning (RL) were frequently employed. It is noteworthy that 75% of the chosen papers did not validate against external datasets, thereby underscoring the issue of generalizability. Furthermore, the interpretability of AI decisions was pinpointed as a crucial factor for the successful incorporation of AI in healthcare. The results were deemed precise. Yes Yes Yes 15/16 (93.7%)
14 Nida et al 41 Yes Yes ? No Yes The final articles were published from 1997 to 2018 and originated from 24 countries, with most papers (26 articles) published by authors from the United States. Types of artificial neural networks used included artificial neural networks, feed-forward networks, or hybrid models; reported accuracy ranged from 50% to 100%. The majority of artificial neural networks informed decision-making at the micro level, between patients and health care providers. Fewer artificial neural networks were deployed for intraorganizational and system, policy, or interorganizational (10 articles) decision-making. The review identified key characteristics and drivers for market uptake of artificial neural networks for healthcare organizational decision-making to guide further adoption of this technique. Due to the number of reviewed articles and reporting credibility, the study has acceptable precision. Yes Yes Yes 13/16 (81.2%)
15 Rahimi et al 42 Yes Yes ? No Yes The paper concluded that the application of AI in SDM was still emerging. The authors` review revealed similar modes of AI support for SDM across the selected studies. However, they noticed a lack of attention to patients’ values and preferences, as well as inadequate reporting of AI interventions, leading to ambiguity about various aspects. The issues of understandability of AI interventions and end-user involvement in their design and development were scarcely addressed. The paucity of included studies limits the precision of the results. Yes Yes Yes 13/16 (81.2%)
16 Tiwari et al 43 Yes Yes ? No Yes AI can enhance diagnostics, improve patient outcomes, and reduce invasive procedures. It can also provide personalized treatment plans and streamline workflows for dentists. AI-powered tools can improve patient interactions. However, challenges such as data privacy, algorithm validation, ethical concerns, professional training, and the cost and accessibility of AI technology must be considered. The results were deemed precise. Yes Yes Yes 13/16 (81.2%)
17 Tricco et al 44 Yes Yes ? Yes Yes The most common implementation strategies for the tools were clinician reminders that incorporated machine learning predictions, followed by facilitated relay of clinical information and staff education. The main barriers to the successful implementation of machine learning tools were time and reliability, while the main facilitators were time/efficiency and perceived usefulness. Furthermore, the authors found scarce evidence regarding the implementation of machine learning tools to assist clinicians with patient healthcare decisions in hospital settings. Due to the number of reviewed articles and reporting credibility, the study has acceptable precision. Yes Yes Yes 15/16 (93.7%)
18 Uzun Ozashin et al 45 Yes Yes ? No Yes Most AI applications use classification models for breast cancer prediction, with accuracy (99%) being the primary performance metric, followed by specificity (98%) and area under the curve (0.95). The convolutional neural network (CNN) was often the preferred model. The results were deemed precise. Yes Yes Yes 13/16 (81.2%)
Critical Appraisal Skills Program (CASP) questions scoring: Yes = 2, Can't tell (?) = 1, No  =  0
Q1: Did the review address a clearly focused question?
Q2: Did the authors look for the right type of papers?
Q3: Do you think all the important, relevant studies were included?
Q4: Did the review's authors do enough to assess the quality of the included studies?
Q5: If the results of the review have been combined, was it reasonable to do so?
Q6: What are the overall results of the review?
Q7: How precise are the results?
Q8: Can the results be applied to the local population?
Q9: Were all important outcomes considered?
Q10: Are the benefits worth the harms and costs?