Clinical decision support systems (CDSS) in assistance to COVID‐19 diagnosis: A scoping review on types and evaluation methods

Abstract

Background and Aims

Due to the COVID‐19 pandemic, a precise and reliable diagnosis of this disease is critical. The use of clinical decision support systems (CDSS) can help facilitate the diagnosis of COVID‐19. This scoping review aimed to investigate the role of CDSS in diagnosing COVID‐19.

Methods

We searched four databases (Web of Science, PubMed, Scopus, and Embase) using three groups of keywords related to CDSS, COVID‐19, and diagnosis. To collect data from studies, we utilized a data extraction form that consisted of eight fields. Three researchers selected relevant articles and extracted data using a data collection form. To resolve any disagreements, we consulted with a fourth researcher.

Results

A search of the databases retrieved 2199 articles, of which 68 were included in this review after removing duplicates and irrelevant articles. The studies used nonknowledge‐based CDSS (n = 52) and knowledge‐based CDSS (n = 16). Convolutional Neural Networks (CNN) (n = 33) and Support Vector Machine (SVM) (n = 8) were employed to design the CDSS in most of the studies. Accuracy (n = 43) and sensitivity (n = 35) were the most common metrics for evaluating CDSS.

Conclusion

CDSS for COVID‐19 diagnosis have been developed mainly through machine learning (ML) methods. The greater use of these techniques can be due to their availability of public data sets about chest imaging. Although these studies indicate high accuracy for CDSS based on ML, their novelty and data set biases raise questions about replacing these systems as clinician assistants in decision‐making. Further studies are needed to improve and compare the robustness and reliability of nonknowledge‐based and knowledge‐based CDSS in COVID‐19 diagnosis.

1. INTRODUCTION

COVID‐19 is a respiratory disease developed by the SARS‐CoV‐2 virus. ¹ Fever, fatigue, sore throat, and dry cough are the most common manifestations of this disease. ² Other respiratory illnesses, such as influenza and respiratory syncytial virus, can also cause these symptoms and contribute to the difficulty in controlling the outbreak. ³ According to the World Health Organization (WHO), as of November 22, 2023, the coronavirus has infected 772,166,517 people worldwide and caused 6,981,263 deaths. ⁴ This outbreak has presented substantial challenges in delivering affordable and high‐quality healthcare services to a growing number of patients. ⁵ Strategies to prevent and control COVID‐19 include early diagnosis, patient isolation, contact monitoring, quarantine, and vaccination. ⁶

Several methods are used to diagnose COVID‐19, including clinical symptoms, epidemiological history, real‐time polymerase chain reaction (RT‐PCR) tests, chest computerized tomography (CT) scans, X‐ray imaging, enzyme‐linked immunosorbent assay (ELISA), biosensors, and point‐of‐care testing (POCT). ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² Currently, the RT‐PCR test for COVID‐19 confirmation is expensive, manual, and complex. ¹³ However, it has been shown to have high rates of false positives or negatives, which makes it unreliable for detection. ¹⁴ Also, the RT‐PCR test for COVID‐19 confirmation is an expensive, manual, and complex approach. ¹³ In some healthcare settings, commercial test kits, swabs, PCR machines, or their expertise may be less available. ¹³ , ¹⁵ Additionally, CT scans have been found to have high rates of false negatives. ¹⁴ This is why combining diagnostic methods could improve COVID‐19 detection accuracy.

Clinical decision support systems (CDSS) are helpful methods for healthcare providers in clinical decision‐making and the early screening of patients. ¹⁶ They integrate various information such as characteristics of individual patients, radiological images, clinical examination, and clinical guidelines and provide patient‐specific recommendations or suggestions. ¹⁶

CDSS may include artificial intelligence (AI) methodologies for assisting in quick and accurate medical diagnoses. ¹⁷ , ¹⁸ , ¹⁹ In this case, intelligent clinical decision support systems (ICDSS) are created. The CDSS based on AI are classified into two types: (1) the ICDSS based on expert system (ES) and (2) the ICDSS based on machine learning (ML). ²⁰ , ²¹

The ICDSS based on ES or knowledge‐based CDSS aim to automate diagnosing COVID‐19, typically performed by medical experts. ²⁰ , ²¹ These systems primarily consist of a knowledge base that contains medical expertise, an inference engine that uses the knowledge base to generate a diagnosis for the patient, and a way to communicate to the user (input and output). ²⁰ , ²¹ The ICDSS based on ML or nonknowledge‐based CDSS learn to solve human problems by simulating human learning on a computer. ²⁰ , ²² ML is the ability of machines to find patterns and learn hidden knowledge from large data sets using analytical techniques. ²⁰ , ²² , ²³ The goal is to teach automated techniques to classify and cluster data, learn behavior, generate patterns, and predict future actions using decision support systems. ²⁰ , ²² , ²³ Generally, ML can be divided into supervised and unsupervised learning. ²⁰ , ²² , ²³ The supervised methods are commonly used for disease prediction and include regression, Support Vector Machine (SVM), Random Forest (RF), Naive Bayes (NB), K‐Nearest Neighbor (KNN), Decision Tree (DT), and Artificial Neural Network (ANN). ²⁰ , ²² , ²³ A type of ML is deep learning (DL), which uses multiple‐layer artificial neural networks. ²⁰ , ²² , ²³ One of the most widely used subsets of DL is the Convolutional Neural Network (CNN), which allows the system to learn data representation. ²⁰ , ²² , ²³

Previous review studies have examined the impact of AI or ML models in screening, diagnostics, and prediction of COVID‐19, ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ ML models for diagnosing infectious diseases, ²³ CDSS based on AI for early recognition of respiratory infections, ²⁹ and CNN for the diagnosis and prognosis of COVID‐19. ³⁰ Moreover, there is little understanding of the corresponding techniques that explain the use of different types of CDSS to assist in diagnosing COVID‐19. To our knowledge, no comprehensive review exists on the methods for developing CDSS for COVID‐19 diagnosis. Given the importance of accurate COVID‐19 diagnosis, this scoping review aimed to (1) review the types of CDSS to assist in COVID‐19 diagnosis and (2) investigate metrics for evaluating the performance accuracy of CDSS in the diagnosis process.

2. METHODS

2.1. Search strategy

The current study was a scoping review study reported based on the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses guideline extension for scoping reviews (PRISMA‐ScR). ³¹ We conducted searches in the Scopus, PubMed, Web of Science, and Embase databases without time limitation up to September 2023 to identify relevant articles. The search strategy involved three sets of keywords. Synonymous words within each set were combined using the OR operator. Subsequently, the three sets of keywords were combined using the AND operator. The first set of keywords pertained to COVID‐19 (Group A), the second set was related to CDSS (Group B), and the third set was implied to diagnose (Group C) (Appendix 1). Several synonyms and spelling variations for each search term were used to capture as many articles as possible. ³² , ³³

2.2. Eligibility criteria

In this study, articles were included that (1) published in English, (2) were about any CDSS, (3) reported on the detection or diagnosis of COVID‐19, (4) used various types of CDSS to assist in the diagnosis of COVID‐19. Books, book chapters, letters to the editors, and conference article abstracts were excluded, as they may be less reliable without undergoing rigorous peer review. Also, we excluded articles that used only AI or ML to diagnose COVID‐19 and did not provide a role for CDSS to assist in diagnosing COVID‐19.

2.3. Study selection and data extraction

Abstracts of articles retrieved from four scientific databases were entered into EndNote X8.0. Duplicate articles were excluded, and the remaining articles were independently screened by three researchers (A. R. A., F. S., and A. T. A.) based on the title and abstract to select relevant studies. The lists of independently selected articles were reviewed jointly by A. R. A., F. S., and A. T. A. After final approval, the full texts were independently reviewed by three researchers to extract information using a data extraction form. Three medical informatics specialists confirmed the validity of this form. The form includes fields such as authors' names, publication year, study location, sample size or data sources, types of CDSS, computational methods, performance measures, and main results. All findings in the data extraction form were re‐examined and validated by A. R. A., F. S., and A. T. A. The fourth researcher (K. B.) consulted and resolved disagreements in each stage. The extracted data were analyzed in Microsoft Excel 2016 and presented in terms of percentage, frequency, and graphs.

3. RESULTS

3.1. Study selection

This study identified a total of 2199 articles from four databases, of which 701 were repeated studies, and 1263 were unrelated upon screening based on titles and abstracts. After careful examination of 1498 remaining articles based on inclusion and exclusion criteria, 235 studies were selected for full‐text reading. For several reasons, 167 articles were excluded from the final full‐text review. These reasons included narrative reviews, brief communications, letters, protocols, and reviews. Some studies only used ML to diagnose COVID‐19, while others required a better understanding of how CDSS can assist in diagnosing COVID‐19. However, some studies utilized CDSS for other purposes, such as monitoring, resource allocation, risk and severity assessment, and reviewing prognosis and outcomes in COVID‐19 patients. These studies also recommended the use of CDSS in future research. Finally, 68 articles met all the inclusion criteria. ⁵ , ¹⁷ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷⁰ , ⁷¹ , ⁷² , ⁷³ , ⁷⁴ , ⁷⁵ , ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁷⁹ , ⁸⁰ , ⁸¹ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ , ⁸⁶ , ⁸⁷ , ⁸⁸ , ⁸⁹ , ⁹⁰ , ⁹¹ , ⁹² , ⁹³ , ⁹⁴ , ⁹⁵ , ⁹⁶ , ⁹⁷ , ⁹⁸ , ⁹⁹ The flowchart of the selection process is shown in Figure 1. The authors did not assess study quality due to the review type, which was a scoping review. ³¹

Figure 1.

PRISMA‐ScR flowchart showing the search process. CDSS, clinical decision support systems; PRISMA‐ScR, Preferred Reporting Items for Systematic Reviews and Meta‐Analyses guideline extension for scoping reviews; WOS, Web of Sciences.

3.2. Characteristics of the included articles

The most of articles were published in 2021 (n = 23) (Figure 2). The first authors of most studies were affiliated in India (n = 12), Turkey (n = 7), the United States (n = 5), and Saudi Arabia (n = 5) (Figure 3). Chest X‐ray images and computed tomography (CT) scans of COVID‐19 patients and healthy individuals were used as data sources in most studies (n = 43) (Appendix 2). Details of the selected studies are shown in Appendix 2.

Figure 2.

Distribution of the studies in terms of publication year.

Figure 3.

Number of publications by country based on authors' countries.

3.3. Types of CDSS to assist in diagnosing COVID‐19

Types of CDSS to assist in diagnosing COVID‐19 are shown in Figure 4. Most of the studies used ICDSS based on ML (nonknowledge‐based CDSS) (n = 52 [76.5%]). ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷⁰ , ⁷¹ , ⁷² , ⁷³ , ⁷⁴ , ⁷⁵ , ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁷⁹ , ⁸⁰ , ⁸¹ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ In these studies, the most common methods for designing CDSS were CNN (n = 33), ³⁸ , ⁴⁰ , ⁴¹ , ⁴² , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵⁴ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷¹ , ⁷² , ⁷⁸ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ SVM (n = 8), ³⁵ , ³⁶ , ³⁹ , ⁴³ , ⁴⁴ , ⁵⁴ , ⁵⁶ , ⁵⁷ RF (n = 7), ³⁴ , ³⁵ , ³⁷ , ³⁹ , ⁴² , ⁴⁴ , ⁵⁵ and KNN (n = 7) ³⁶ , ³⁷ , ³⁹ , ⁴² , ⁴³ , ⁵⁵ , ⁵⁶ (Table 1 and Appendix 2). Some studies used a combination of ML methods to develop CDSS (n = 14) ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁹ , ⁴² , ⁴³ , ⁴⁴ , ⁵³ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁷⁰ , ⁸⁴ (Appendix 2). Rule sets (n = 8), ⁵ , ¹⁷ , ⁸⁸ , ⁸⁹ , ⁹³ , ⁹⁴ , ⁹⁶ , ⁹⁷ fuzzy mathematical models (n = 5), ⁸⁶ , ⁹⁰ , ⁹¹ , ⁹² , ⁹⁵ and ontology (n = 3) ⁸⁷ , ⁹⁸ , ⁹⁹ were used for developing ICDSS based on ES (knowledge‐based CDSS), respectively (Appendix 2).

Figure 4.

Types of CDSS in the selected studies. CDSS, clinical decision support systems; ES, expert systems; ICDSS, intelligent clinical decision support systems; ML, machine learning.

Table 1.

The computational methods of ICDSS based on ML in the selected studies.

Computational methods in CDSS	Frequency in the studies
AdaBoost	2
ANN	1
Association rules	1
BNs	2
CART	1
Catboost	1
CNN	33
DNN	4
DT	7
GB	1
HGB	1
KNN	7
LDA	1
Lightgbm	1
LR	5
MLP	2
NB	5
PNN	1
QLDA	1
RF	7
RFC	1
SVC	1
SVM	8
XGB	1
XGBboost	1
XGBC	1
XGBoost	2

Abbreviations: AdaBoost, Adaptive Boosting; ANN, Artificial Neural Network; BNs, Bayesian Networks; CART, Classification and Regression Tree; CDSS, clinical decision support systems; CNN, Convolutional Neural Networks; DNN, Deep Neural Networks; DT, Decision Tree; GB, Gradient Boosting; HGB, HistGradient Boosting; ICDSS, intelligent clinical decision support; KNN, K‐Nearest Neighbors; LDA, Linear Discriminant Analysis; LR, Logistic Regression; ML, machine learning; NB, naive Bayes; PNN, Probabilistic Neural Network; QLDA, Quadratic Linear Discriminant Analysis; RF, Random Forest; SVC, Support Vector Classification; SVM, Support Vector Machine; XGBC, XGBoost classification; XGBoost, eXtreme Gradient Boosting.

3.4. Evaluation methods of CDSS

The most common metrics for evaluating ICDSS based on ML are shown in Table 2. Most studies used a combination of performance metrics to evaluate CDSS (n = 40) (Appendix 2). In most of these studies, accuracy (n = 43), ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷¹ , ⁷² , ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁸⁰ , ⁸¹ , ⁸² , ⁸³ , ⁸⁵ sensitivity (recall) (n = 35) ³⁵ , ³⁶ , ³⁷ , ³⁹ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁶ , ⁴⁷ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁶ , ⁵⁷ , ⁶¹ , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷¹ , ⁷² , ⁷⁶ , ⁷⁸ , ⁸¹ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ and F1‐score (n = 26) ³⁴ , ³⁵ , ³⁶ , ³⁹ , ⁴¹ , ⁴² , ⁴³ , ⁴⁶ , ⁴⁷ , ⁵⁰ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁶ , ⁵⁷ , ⁶⁰ , ⁶¹ , ⁶³ , ⁶⁴ , ⁶⁶ , ⁶⁸ , ⁶⁹ , ⁸¹ , ⁸² , ⁸⁴ , ⁸⁵ were used (Table 2). Other metrics for evaluating ICDSS based on ML were the Matthews correlation coefficient (MCC), ⁵² , ⁵⁶ , ⁵⁸ receiver operating characteristic (ROC) curve, ³⁴ , ³⁶ , ⁷⁰ and so on (Appendix 2).

Table 2.

The most common metrics for evaluating intelligent clinical decision support based on machine learning in the selected studies.

Evaluation criteria	Description	Frequency
Accuracy	The overall effectiveness of a classifier	43
Sensitivity (recall)	Effectiveness of a classifier to identify positive labels	35
F1‐score	Relations between data positive labels and those given by a classifier	26
Precision	Class agreement of the data labels with the positive labels given by the classifier	20
Specificity	How effectively does a classifier identify negative labels	20
Area under the curve (AUC)	Classifier's ability to avoid false classification	12

The most common metrics for evaluating ICDSS based on ES were diagnosis rate (n = 3) ¹⁷ , ⁹² , ⁹³ and accuracy (n = 3) ⁸⁷ , ⁹⁰ , ⁹⁵ (Appendix 2). Other metrics (such as ease of use, precision, recall, and time) for evaluating these systems are shown in Appendix 2.

4. DISCUSSION

4.1. Principal findings

This scoping review examined the assistance of CDSS in COVID‐19 diagnosis. The most frequently used method for this purpose was the ICDSS based on ML (nonknowledge‐based CDSS), followed by ICDSS based on ES (knowledge‐based CDSS). Most studies have indicated that using CDSS have positively impacted the accurate diagnosis of COVID‐19. ⁵ , ¹⁷ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷⁰ , ⁷¹ , ⁷² , ⁷³ , ⁷⁴ , ⁷⁵ , ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁷⁹ , ⁸⁰ , ⁸¹ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁶ , ⁸⁷ , ⁸⁸ , ⁸⁹ , ⁹⁰ , ⁹² , ⁹³ , ⁹⁴ , ⁹⁵ , ⁹⁶ , ⁹⁷ , ⁹⁸ , ⁹⁹

4.2. Nonknowledge‐based CDSS

ML was the most frequently used method to develop CDSS for COVID‐19 diagnosis. The most common ML methods in the reviewed studies were CNN. ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷⁰ , ⁷¹ , ⁷² , ⁷³ , ⁷⁴ , ⁷⁵ , ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁷⁹ , ⁸⁰ , ⁸¹ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ In line with this result, other review studies also indicated CNN to be the most common method to develop a system for COVID‐19 diagnosis. ²³ , ²⁵ , ²⁷ , ²⁸ , ³⁰ , ¹⁰⁰ A CNN is a type of DL algorithm used for processing medical images, particularly for identifying specific features in chest radiographs of COVID‐19 patients. ³⁰ CNN is more valuable than other methods for developing CDSS due to its excellent performance accuracy and much lower preprocessing. ²⁵ , ³⁰ For example, in the reviewed studies, CDSS based on CNN achieved a performance accuracy ranging from 75% to 99.62%. ³⁸ , ⁴⁰ , ⁴¹ , ⁴² , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵⁴ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷¹ , ⁷² , ⁷⁸ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ Also, these studies indicated that CDSS based on CNN could be effective in detecting COVID‐19, assisting domain specialists, physicians, and radiologists in the decision‐making process, and enhancing radiologists' working productivity ²⁵ , ³⁰ , ⁴² , ⁶⁶ , ⁸⁷ It is challenging to create a CDSS using CNN due to the scarcity of big data and low quality of data sources. ²⁵ Developing these data sets in medicine is costly and necessitates specialized labor. In addition, ethical and privacy concerns must be assessed. ³⁰ Therefore, these findings do not mean that CDSS are a production‐ready solution because the diagnostic power of these systems relies on chest X‐ray images and CT scans of COVID‐19 patients. ⁵⁰ It is important to acknowledge the bias that is introduced in these studies because someone who has a chest X‐ray or a CT scan is more likely to have COVID‐19. Many of the symptoms of COVID‐19 are nonspecific and thus difficult to differentiate from overlapping symptoms of other diseases. Future research can study some of the more unique presenting symptoms of COVID‐19 (anosmia, thromboses, etc.) in light of how CDSS could be used to assist in diagnosing these cases.

4.3. Knowledge‐based CDSS

In the present scoping review, the results of 16 studies identified a positive impact on a diagnosis of COVID‐19 with the assistance of CDSS based on ES. ⁵ , ¹⁷ , ⁸⁶ , ⁸⁷ , ⁸⁸ , ⁸⁹ , ⁹⁰ , ⁹¹ , ⁹² , ⁹³ , ⁹⁴ , ⁹⁵ , ⁹⁶ , ⁹⁷ , ⁹⁸ , ⁹⁹ In the reviewed studies, rule sets were the most common methods in developing a knowledge base of CDSS. ⁵ , ¹⁷ , ⁸⁸ , ⁸⁹ , ⁹³ , ⁹⁴ , ⁹⁶ , ⁹⁷ Rule‐based systems acquire contextual knowledge from extracted data stored and manipulated in other approaches. ¹⁰¹ The reviewed studies extracted data from various sources, such as observed symptoms, specific medical measurements, pre‐existing medical conditions or any hospitalization history, recent PCR test results, clinical guidelines, websites (e.g., WHO), and knowledge of experts to develop rule sets. ⁵ , ¹⁷ , ⁸⁶ , ⁸⁷ , ⁸⁸ , ⁸⁹ , ⁹⁰ , ⁹¹ , ⁹² , ⁹³ , ⁹⁴ , ⁹⁵ , ⁹⁶ , ⁹⁷ , ⁹⁸ , ⁹⁹ Therefore, knowledge‐based CDSS in these studies can provide a comprehensive view of the patient's health information. This feature enables health practitioners to access treatment recommendations and risk classification, recommend test lists, quickly track test results and symptoms, and access clinical guidelines in the COVID‐19 pandemic. ⁵ Generally, these types of CDSS require knowledge bases and input variables (e.g., fever, cough, cell blood count, respiratory rate, CT chest/RT‐PCR, family history, and age). Hak et al. conducted a literature review and found that many CDSS are inadequate due to a lack of standardization and structure in their knowledge base. ¹⁰² Therefore, understanding the different ways of representing, maintaining, and updating knowledge in rule‐based systems is important.

The reviewed studies used different metrics, including diagnostic rate, accuracy, ease of use, and time, to evaluate the performance of knowledge‐based CDSS. ⁵ , ¹⁷ , ⁸⁶ , ⁸⁷ , ⁸⁸ , ⁸⁹ , ⁹⁰ , ⁹¹ , ⁹² , ⁹³ , ⁹⁴ , ⁹⁵ , ⁹⁶ , ⁹⁷ , ⁹⁸ , ⁹⁹ The reason for this could be the widespread COVID‐19 pandemic, leading to a lack of time to evaluate these systems' performance properly. It is essential for researchers and software developers to thoroughly assess the performance of these systems before implementing them in real clinical settings for COVID‐19 diagnosis.

In summary, the present scoping review studied the types of CDDS that assist in the COVID‐19 diagnosis. Previous studies used nonknowledge‐based CDSS and knowledge‐based CDSS for this purpose. The sample size in most studies was the clinical characteristics (e.g., radiological images) of public data sets of COVID‐19 patients and healthy individuals. The performance and accuracy of CDSS may depend on extracted information from data sets to predict the infection of COVID‐19. These assumptions about performance metrics may change with the emergence and availability of new data. Therefore, these systems should be used by healthcare providers and domain experts to validate and evaluate their clinical usefulness in real workflow. Also, focusing on the early stages of COVID‐19 detection is crucial, as most methods effectively identify the disease only in advanced stages. Future studies can assess the impact of CDSS to assist in predicting and detecting the COVID‐19 disease in the initial stage. Finally, stakeholders (e.g., researchers, healthcare professionals, and health policymakers) of CDSS should first consider the type of design methods for developing a CDSS. The type of design method assists in the accuracy and precision of the diagnoses in CDSS. These issues can also be considered in future studies.

4.4. Limitations

The present study has two limitations. First, the study's exclusion of non‐English articles may cause language bias. However, we employed a comprehensive search strategy. By using this method, the likelihood of missing relevant articles may have been reduced. Our search strategy was run in January 2021 and updated in September 2023. The number of included studies may be changed after a period of time because new studies will be conducted on various CDSS to assist in diagnosing COVID‐19 at different stages of severity.

Second, we searched four scientific databases: Scopus, PubMed, Embase, and Web of Science. While searching additional databases may uncover new articles, these four databases will likely retrieve the most relevant articles.

5. CONCLUSION

This scoping review studied the assistance and impact of CDSS on the detection and diagnosis of COVID‐19. The results showed that COVID‐19 can be diagnosed by assisting nonknowledge‐based CDSS and knowledge‐based CDSS. These studies used various techniques such as ML methods (e.g., CNN, SVM, RF) and ES methods (e.g., rule sets, fuzzy mathematical models, ontologies) to design CDSS. ML methods were the most common techniques in developing CDSS for COVID‐19 diagnosis. The greater use of these techniques can be due to the availability of public data sets about chest X‐ray images and CT‐scan scans of COVID‐19 patients. Novelty and data set biases raise questions about the performance accuracy of these systems. Most models still need to be deployed enough to show their real‐world functionality. However, they can help combat the pandemic. Future studies can evaluate the usefulness and performance accuracy of CDSS for COVID‐19 diagnosis in different healthcare environments from the perspective of healthcare providers. Also, future studies can examine the impact of CDSS in other care cycle stages of COVID‐19, such as prevention, screening, surveillance, treatment, and rehabilitation, and compare the accuracy, robustness, and reliability of different methods in developing CDSS. This paper provides insights into how CDSS can be used to detect and mitigate the COVID‐19 pandemic. It will be helpful for researchers, healthcare providers, government officials, and policymakers.

ETHICS STATEMENT

Kerman University of Medical Sciences' Research Ethics Committee approved this study (Code of Ethics: IR.KMU.REC.1399.686). Informed consent was not applicable for this scoping review.

AUTHOR CONTRIBUTIONS

Arefeh Ameri: Conceptualization; data curation; formal analysis; investigation; methodology; project administration; resources; software; supervision; validation; visualization; writing—original draft; writing—review and editing. Atefeh Ameri: Formal analysis; investigation; methodology; writing—original draft; writing—review and editing. Farzad Salmanizadeh: Conceptualization; data curation; formal analysis; investigation; methodology; project administration; resources; software; supervision; validation; visualization; writing—original draft; writing—review and editing. Kambiz Bahaadinbeigy: Funding acquisition; validation; writing—original draft; writing—review and editing.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest.

TRANSPARENCY STATEMENT

The lead author Farzad Salmanizadeh affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Supporting information

Supporting information.

Supporting information.

Ameri A, Ameri A, Salmanizadeh F, Bahaadinbeigy K. Clinical decision support systems (CDSS) in assistance to COVID‐19 diagnosis: a scoping review on types and evaluation methods. Health Sci Rep. 2024;7:e1919. 10.1002/hsr2.1919

DATA AVAILABILITY STATEMENT

All data generated and analyzed during this study are included in this published article. Upon a reasonable request, the corresponding author can provide more information on the data sets used and analyzed during the current study.