An overview of deep learning techniques for COVID-19 detection: methods, challenges, and future works

Abstract

The World Health Organization (WHO) declared a pandemic in response to the coronavirus COVID-19 in 2020, which resulted in numerous deaths worldwide. Although the disease appears to have lost its impact, millions of people have been affected by this virus, and new infections still occur. Identifying COVID-19 requires a reverse transcription-polymerase chain reaction test (RT-PCR) or analysis of medical data. Due to the high cost and time required to scan and analyze medical data, researchers are focusing on using automated computer-aided methods. This review examines the applications of deep learning (DL) and machine learning (ML) in detecting COVID-19 using medical data such as CT scans, X-rays, cough sounds, MRIs, ultrasound, and clinical markers. First, the data preprocessing, the features used, and the current COVID-19 detection methods are divided into two subsections, and the studies are discussed. Second, the reported publicly available datasets, their characteristics, and the potential comparison materials mentioned in the literature are presented. Third, a comprehensive comparison is made by contrasting the similar and different aspects of the studies. Finally, the results, gaps, and limitations are summarized to stimulate the improvement of COVID-19 detection methods, and the study concludes by listing some future research directions for COVID-19 classification.

Introduction

Coronavirus 2 (SARS-CoV-2) is an infectious virus that causes severe acute respiratory syndrome and is recognized as COVID-19 disease [1, 2]. Coronaviruses are large, positively spliced, single-stranded RNA viruses [3] that infect humans and a variety of other living organisms [4]. The disease first appeared in Wuhan, China, and eventually spread worldwide. The World Health Organization (WHO) has determined that COVID-19 is an outbreak that poses an exceptionally high risk to millions of lives worldwide, particularly in countries with less developed health systems [5] and is recognized as a pandemic. COVID-19 has infected over 640,395,651 people worldwide and caused over 6,618,579 deaths (as of December 3, 2022). Although the pandemic appears to be over these days, an average of 378,000 new cases occur daily, according to WHO [6]. A collective response was quickly formed to prevent the further spread of COVID-19. Most affected countries closed their borders and temporarily suspended transportation and travel. WHO and the Centers for Disease Control (CDS) published international guidelines to be followed by international and national businesses, governments, and citizens to fully contain the pandemic. CDC and WHO have identified and listed symptoms that indicate plausible COVID-19 infection, including dry cough, fever, diarrhea, myalgia, and vomiting. Public awareness has been raised in all countries to seek treatment as soon as possible to reduce morbidity rates. Governments have advocated and supported research and development of vaccines against COVID-19. Containment of the outbreak requires increased attention given the increase in COVID-19 cases worldwide.

Reverse transcription-polymerase chain reaction (RT-PCR) is a technique for detecting the COVID-19 virus [7]. Rapid and accurate detection is critical for disease control [8]; However, test results can be obtained within 2–48 h [9].

Due to the development of coronavirus ribonucleic acid (RNA), numerous vaccines have been developed. These vaccines use various biological agents such as attenuated live viruses, proteins or subunits, messenger RNA (mRNA), and deoxyribonucleic acid (DNA). Vaccines are still 95% effective, although they can slow the progression of infection and help people build immunity by triggering the production of exemplary antibodies [10]. Even when a vaccine is available, early detection of coronavirus is critical, because it facilitates the identification of individuals who have been directly or indirectly exposed to the virus. Identification of these individuals may prevent the further spread of the pandemic, as COVID-19 infections manifest as pulmonary infections. Therefore, computed tomography scans (CT) and chest X-rays can be used to detect COVID-19 disease [2].

Medical imaging systems have been used since the mid-1960s to aid decision-making in diagnosing chest disease [11]. In the mid-1980s, cancer diagnosis using chest X-rays was one of the most typical applications of computer-aided diagnosis systems (CAD) [12]. More recently, researchers have used deep learning (DL) architectures, particularly convolutional neural network (CNN) models, to classify diseases based on medical images [13]. The COVID-19 pandemic is the focus of other research and development initiatives. Machine learning (ML) and DL are now commonly used to detect COVID-19. DL methods for detecting COVID-19 infections from images have received a major boost from the recent pandemic. The originality of the current research is that it focuses on the difficulties associated with DL and image processing approaches for detecting COVID-19.

One of the major problems in studying COVID-19 is the lack of sufficient and reliable data in the first quarter of the pandemic. Because of the limited number of tests, many deaths and viral infections cannot be detected. No country in the world has been able to provide reliable datasets on the presence of the virus in a representative sample of the mass population. The accessibility of large and high-quality datasets in the ML and DL applications plays a crucial role in the robustness of the results. Therefore, researchers had to work with datasets in a limited range by combining available datasets or using data augmentation techniques. Figure 1 illustrates the general structure of a deep learning-based COVID-19 diagnostic system.

Fig. 1.

General illustration of the COVID-19 detection systems using deep learning

In Fig. 1, inpatients are considered participants in the data collection phase. Medical imaging techniques such as CT and X-rays are used for COVID-19 diagnosis. The following step is data preprocessing, which converts the data into a model-friendly format. Data preprocessing includes noise removal, image scaling, data augmentation, and data partitioning, where the data is divided into training, validation, and test groups. For data partitioning, the cross-validation method is usually used. A particular model is trained and evaluated using this partitioned data. The classification and feature extraction phase of a DL-based COVID-19 recognition system is crucial. In this phase, features are extracted through a series of procedures using numerous feature extraction techniques. One-hot encoding is another preprocessing method for processing categorical data in ML and DL applications. Categorical data must be modified in the preprocessing phase because the models require numerical input variables. In this method, a new column is created for each feature value in the original categorical column [14].

In the literature, there are some studies on COVID-19 detection [15–20]. Some of these studies were published during the early pandemic COVID-19. Thus, they do not cover more recent works. These works also focus on ML and artificial intelligence (AI) [16], mathematical, ML, and DL [18], ML and LD [19], and ML [20] methodologies. In this work, we focus on especially DL work that considers the identification of COVID-19 and it includes several papers using different datasets consisting of CT scans, X-rays, MRI, blood tests, cough sound, and ultrasound.

This study examines approaches from recent articles published by respected publishers to help the research community and regulatory agencies formulate recommendations to address the disease. In addition, it highlights the contribution of ML and, in particular, DL to the control of the COVID-19 pandemic. We provide an overview of the most advanced technologies developed.

The main contributions of this study can be summarized as follows:

1. The review includes information about COVID-19 and the prevalence of the disease, which motivates and emphasizes the need for faster prediction mechanisms to combat the disease.

2. Some of the key elements of current studies, such as the dataset, proposed architecture, data partitioning method, and evaluation criteria, are discussed, and the basics are explained clearly, concisely, and understandably.

3. The role of DL applications in the medical area to predict COVID-19 is supported by a detailed discussion.

4. We summarized future approaches to epidemic containment in countries, methodology, datasets, evaluation metrics, research barriers, and lessons learned.

The remainder of this paper is organized as follows. The reason for using DL is to analyze and process medical images from recent works, essential information on DL, and give information for CNN architectures presented in Sect. 3. Section 4 provides a detailed overview of the use of DL in COVID-19 and is divided into two subsections. Section 5 analyzes and discusses available datasets and implementations of DL for medical imaging. Recommendations for challenges and future directions of study are included in Sect. 7. Further discussion can be found in Sect. 8. Finally, conclusions are presented in Sect. 9.

Review methodology

This review used a comprehensive non-systematic review methodology. COVID-19 diagnosis articles using DL and published in the last 4 years (2020–2023) were included. Articles from several major databases, such as Scopus, Google Scholar, and Crossref, were included in this review. Important keywords were used in the search, such as: ‘COVID-19’, ‘Sars-CoV2’, ‘Coronavirus’, ‘Deep Learning’, ‘Diagnosis’, ‘CT scans’, ‘X-rays’, ‘Medical Imaging’, ‘Deep Learning for COVID-19’ and others were included. The different combinations of these keywords were also used. Full-text articles on COVID-19 diagnosis with DL and ML were identified as inclusion criteria. The following articles were excluded as exclusion criteria: pre-prints (such as arXiv and medRxiv), pure medical diagnostic studies that do not involve deep learning and AI, animal studies, letters to the editor, commentaries, case reports, and articles without full-text content. We also preferentially selected articles from reputable journals in the Q1–Q2 quartile.

The final list of articles selected for review was 86, including COVID-19 diagnosis based on various modalities such as CT scans, X-rays, MRIs, cough sounds, blood tests, and ultrasound analysis.

Backgrounds

Before discussing the details of the current methods for detecting COVID-19 disease, it is important to learn about ML, DL, and common CNN architectures. In this section, we describe common CNN models used in detecting COVID-19 and give a brief overview of their architecture.

Fundamentals of machine learning and deep learning

ML is the core of the field of artificial intelligence. In ML approaches, algorithms are developed to find patterns or make predictions from empirical data. ML is increasingly used in many fields, including manufacturing, retail, medicine, finance, robotics, telecommunications, and social media.

DL is a relatively new area in ML. DL has seen a significant upsurge in today’s scientific research in recent years due to its excellent learning capabilities [21–23]. DL is also used in various applications due to its ability to adapt to different data types in numerous domains, such as classification problems, image recognition, and object recognition [24].

The learning methods in ML–DL can be divided into three categories such as unsupervised, semi-supervised, and supervised. In supervised learning, the model is trained with the input–output pair. Each detected layer generates an input vector and a control signal corresponding to the expected value. Based on the available labels, the method predicts the output labels [25]. A strategy that lies between supervised and unsupervised learning is called semi-supervised learning. In semi-supervised learning, both unlabeled and labeled values form the training data. When used together with labeled data, unlabeled data can significantly increase learning accuracy [26]. Unsupervised learning analyzes and groups unlabeled data using ML methods. These find hidden patterns or datasets without human intervention [27]. Anomaly detection and clustering are two examples of algorithms used in conjunction with these methods. Clustering is about finding patterns or anomalies in a dataset that are similar to each other [28]. This unsupervised learning technique for anomaly detection is commonly used in security domains [29]. Figure 2 shows the different techniques used in DL.

Fig. 2.

Applications and techniques of deep learning

Machine learning methods for detecting COVID-19

ML techniques and medical data such as computed tomography (CT), X-rays, and clinical blood tests have been used to diagnose numerous diseases. ML assists medical personnel in the rapid, reliable, and accurate detection of COVID-19 [30–32].

In ML, supervised classification algorithms of different categories such as the linear logistic regression model (LR), the K-nearest neighbor classifier (KNN), tree-based ensemble models such as random forest (RF), and boosting methods (XGBoost and AdaBoost) are also used for COVID-19 detection [17, 32].

In addition, blood test results from COVID-19 patients can change rapidly, and these results can be used for prescreening in many medical studies. While the presence of this infection can be confirmed by diagnosis, a series of initial screening tests may provide a possible clue to the presence of the disease. It is difficult for a specialist to obtain complete information from the various laboratory blood tests [33]. However, ML models can easily distinguish the different patterns that emerge from the blood parameters. Therefore, many researchers have developed ML techniques to diagnose COVID-19.

Convolutional neural networks

DL methods successfully automate the task of learning feature representations and attempt to gradually eliminate the tedious task of manually processing features. DL and convolutional neural networks (CNNs) attempt to mimic the structure and operation of the human visual cortex system by using a hierarchical layer of feature representation. This layered feature representation approach enabled automatic learning of various image features, allowing CNNs to outperform hand-built feature detection methods. In reality, CNNs are simply multilayer neural networks. They combine a multilayer stack of neurons with mathematical operations that have several variable parameters to preprocess small amounts of data [34].

CNN models are designed to analyze multidimensional data such as time series or image data by computing weights and extracting features during the training phase. Since they use a convolution operator to simplify complicated tasks, they have been given the term convolution. Because CNNs can automatically generate features from images, they are most commonly used in health applications [35]. Through transfer learning (TL) and fine-tuning, CNNs can also learn from one task and apply it to another. The efficiency of this strategy has been demonstrated in categorization tasks [36].

In addition, CNN models have proven successful in image classification problems. CNN-based research has significantly improved the best performance for many image databases, including the MNIST database, the NORB database, and the CIFAR10 dataset [37]. CNN models are an excellent feature extractor used in several studies to classify COVID-19 from chest X-rays or CT [38].

CNN architectures for detecting COVID-19

CNN models are of great interest to researchers in detecting and classifying COVID-19 because they can recognize and distinguish the patterns in radiological images [39].

Several CNN architectures have been presented in recent years. In this section, we examine the most commonly used CNN architectures for detecting COVID-19, such as AlexNet [40, 41], Xception [42, 43], ResNet [44, 45], DenseNet [46, 47], VGG [48, 49], MobileNet [50, 51], and Inception [52, 53]. Examining the properties of these architectures, such as depth, robustness, and input size is critical to selecting the right architecture for the task at hand. Table 1 provides an overview of the CNN architectures.

Table 1.

Summary of the CNN models

Model	Finding	Depth	Source	Input size
Alexnet	ReLU and utilize dropout	8	ImageNet	227 $\times$ 227 $\times$ 3
CapsuleNet	Pays attention to special relationships between features	3	MNIST	28 $\times$ 28 $\times$ 1
DenseNet	Blocks of layers, layers connected to each other	201	CIFAR10, CIFAR100, ImageNet	224 $\times$ 224 $\times$ 3
HRNetV2	High resolution representations	–	ImageNet	224 $\times$ 224 $\times$ 3
InceptionV3	Utilizes small filtersize, better feature representation	48	ImageNet	229 $\times$ 229 $\times$ 3
InceptionV4	Divided transform and information concepts	70	ImageNet	229 $\times$ 229 $\times$ 3
MobileNetv2	Interved residual structure	53	ImageNet	224 $\times$ 224 $\times$ 3
ResNet	Robust against overfitting due to symmetry mapping based skip links	152	ImageNet	224 $\times$ 224 $\times$ 3
Xception	A depthwise convolution followed by pointwise convolutional	71	ImageNet	229 $\times$ 229 $\times$ 3
VGG16	Increased depth, small filter size	16	ImageNet	224 $\times$ 224 $\times$ 3
VGG19	Increased depth, small filter size	19	ImageNet	224 $\times$ 224 $\times$ 3
EfficientNetB0	Scales all dimensions of depth/width/resolution using a compound coefficient	237	ImageNet	224 $\times$ 224 $\times$ 3
EfficientNetV2	Scales all dimensions of depth/width/resolution using a compound coefficient	42	CIFAR10	300 $\times$ 300 $\times$ 3
InceptionResNetV2	Incorporates residual connections	164	ImageNet	299 $\times$ 299 $\times$ 3
ResNext50	An additional dimension on top of the width and depth of ResNet	128	ImageNet	224 $\times$ 224 $\times$ 3
ShuffleNetV2	Speed or memory access cost, to measure the network’s computational complexity	164	ImageNet	224 $\times$ 224 $\times$ 3
MnasNet	Directly measures real-world inference latency by executing the model on mobile phones	18	ImageNet	224 $\times$ 224 $\times$ 3

According to the first CNN-based model AlexNet, each subsequent architecture uses more layers in its network to reduce the error. With smaller layers, a significant DL problem known as a vanishing/exploding gradient occurs as more layers are added. This causes the gradient to become zero or too large, and the error rate during training and testing increases as the number of layers increases [54]. The ResNet architecture introduced the concept of residual blocks to solve the gradient problem. In this architecture, a technique called skip connections was proposed. Skip connections connect first layer activations to other layers by skipping some intervening layers. ResNets are formed by stacking residual blocks, such as ResNet18, ResNet34, ResNet50, ResNet101, ResNet110, ResNet152, ResNet164, and ResNet1202 [55]. The winner of ILSVRC 2015, one of the most widely used architectures, is ResNets. In the ResNet family architecture, a residual block, a net within a network, is used. The architecture is defined in five steps with identity and convolutional blocks, and the input size is 224 $\times$ 224 [56]. AlexNet is known for its success in image recognition and classification. AlexNet was first proposed by Krizhevsky et al., who also increased the learning capacity of CNN by extension and numerous parameter optimization techniques [57].

Like ResNet, the DenseNet architecture uses a residual identity block, but instead of summation, concatenation is used. Unlike the L(L+1)/2 connections used by DenseNet, conventional CNN models use L connections for L layers [58]. There is a feedforward connection between each layer. All preceding layers receive data from the feature map of the current layer, while all other layers receive the feature map of the current layer as input. The model has a size of 224 $\times$ 224. Xception is another CNN architecture that uses inception blocks with convolutional layers in depth. The size of the Inception image is 229 $\times$ 229 and the number of layers is 71 [59]. The HRNet model consists of advanced algorithms for facial feature recognition, semantic segmentation, and human posture estimation. Datasets such as Context, LIP, AFLW, PASCAL, 300W, Cityscapes, and COFW have shown significant results in semantic segmentation [60]. Another model is the VGG architecture, which has been presented for use in image recognition software. Layers 16 and 19 are used by weighting them into VGG16 and VGG19 with a 3 $\times$ 3 convolutional filter size. The input image is 224 $\times$ 224 [61].

ILSVR (ImageNet) participants in 2014 included the Inception model used for image classification. Participants use multiple filter sizes simultaneously for the input image, rather than adding more layers to the model to make it deeper. The next inception block gets the chain of the inception block [56]. There are several variants of the Inception model, including InceptionV1 [62], InceptionV2 [63] and InceptionV3 [64], InceptionV4 [65], and InceptionResNet [66]. Each version of Inception is an iterative upgrade of the previous version. Understanding the upgrades can help develop optimized classifiers for speed and accuracy [67]

MobileNets are small architectures that can be used on both embedded and mobile devices. They have separable convolutional layers, and 2D convolutional layers are used in this architecture. This is associated with reducing the number of parameters, computations, training time, and memory requirements [68]. In addition, MobileNets have 54 layers and the input image size is 224 $\times$ 224. Capsule Networks can retrieve spatial information and other important features to avoid data loss during pooling operations. The architecture consists of six layers. The first three layers are called encoders, whose task is to convert the input image into vector form, and the last three layers are called decoders, which recover the image [69].

Summary of the research methods

This section explores the potential of ML and DL in medical imaging to combat the COVID-19 pandemic by implementing strategies. In recent years, remarkable changes in healthcare have opened up new opportunities to improve people’s lives. DL has been used in research to detect various diseases by analyzing medical images. Previously, medical staff has spent a lot of time manually reviewing reports and analyzing images. ML and DL have shortened this process by providing better results with computer-aided diagnostic applications. ML and DL have become increasingly important due to their promising effects and revolutionizing healthcare.

Machine learning for medical image processing in COVID-19

Researchers have used various ML algorithms to diagnose COVID-19. Since the pandemic outbreak, there have been several areas where ML has been used. These involve determining the number of people infected with COVID-19, predicting the occurrence and severity of the next wave, predicting mortality rates, and detecting people who do not wear face masks or adhere to social distancing several use cases for ML. Early detection of COVID-19 patients is critical to preventing disease progression in an individual.

Kukar et al. developed the ML model to detect COVID-19 using the clinical markers of 160 COVID-19 and 5333 COVID-19-negative patients. The classifiers used were random forest (RF), DNN (deep neural network), and XGBoost (extreme gradient boosting). XGBoost provided the best results. The average sensitivity and AUC of the different classifiers were 88.9% and 97%, respectively [70]. Muhammed et al. employed supervised ML models to detect COVID-19. Feature extraction was used to train the ML models. In addition, 5 ML algorithms were used: naive Bayes, decision tree, artificial neural networks (ANN), logistic regression, and support vector machine. They attained an accuracy of 94.99%, 94.4%, 94.36%, 92.4%, and 89.2%, respectively [71]. Plante et al. utilized the ML model to recognize COVID-19. 2183 COVID-19 patients were included in this study. They used XGBoost and achieved a sensitivity and AUC score of 95.9% and 91%, respectively [72].

Quiroz-Juarez et al. used ML to identify COVID-19 patients. They used four ML algorithms, namely, neural networks, logistic regression, support vector machines, and KNN, and obtained accuracies of 93.5%, 92.1%, 92.5%, and 89.3%, respectively [73]. Arpaci et al. implemented 14 clinical features to distinguish COVID-19 infection, and the dataset included 114 COVID-19 cases. Six different classifiers were used, with logistic regression providing the best results with 84.21% accuracy [74]. Mukherjee et al. implemented a KNN algorithm to detect COVID-19 using an internet of things (IoT) system. They used an ant colony optimization (ACO) algorithm for feature selection and achieved 97% accuracy for the model [75].

Blood markers and epidemiological parameters can be used for the diagnosis of COVID-19. Unlike CT scans and X-rays, these facilities can be easily collected in hospitals and used with PT-PCR tests. Wu et al. presented an ML model that detected COVID-19 from blood parameters. They used 105 COVID-19 patients and attained accuracy, specificity, and sensitivity obtained were 95.95%, 95.13%, and 96%, respectively [76]. Fernandes et al. analyzed the blood markers of 235 COVID-19 patients. 15 blood parameters were used for feature extraction, and SVM was used for optimal prediction. The AUC, sensitivity, and specificity were 85%, 68%, and 85%, respectively [77]. Tschoellitsch et al. implemented an ML model using blood tests to predict COVID-19. They obtained a dataset of 1357 patients who underwent blood tests at local hospitals. A random forest ML model, consisting of 28 clinical markers, was trained to predict RT-PCR outcomes and attained 81% accuracy [78]. Chadaga et al. used ML algorithms to diagnose COVID-19 using blood tests. They used four machine learning algorithms, namely logistic regression, K Nearest Neighbors, Random Forest, and XGboost, which were employed. Random forest achieved the best result with an accuracy of 92% [33]. Sobrinho et al. used ML models to prioritize patients for testing based on blood markers. The dataset consists of 55,676 patients and 12 features. They obtained 89.12% accuracy using the decision tree algorithm [79]. The distribution of DL algorithms is given in Fig. 3

Fig. 3.

Distribution of classical ML algorithms based on reviewed works in this study

Deep learning for medical image processing in COVID-19

Deep learning algorithms and CNN models in image analysis and processing in the biomedical field have produced successful results for years [80]. Moreover, various CNN-based deep neural networks can achieve remarkable results in ImageNet competition [57]. X-rays, CTs, magnetic resonance imaging (MRI), and ultrasounds are widely used in biomedical systems. Pneumonia and early-stage cancer are both diagnosed with X-ray technology. However, scanning CT is a more advanced method based on X-ray technology that can detect organ changes. Soft tissue cannot be analyzed with X-rays and 2D imaging. CT uses 3D computer vision technology that takes multiple images of the organ from different angles. Although both X-rays and scans from CT can provide images of internal body structures, conventional X-rays tend to overlap. In contrast, a scan from CT eliminates this overlap, highlights internal anatomy, and provides a clear picture of health status.

In addition, in respiratory diseases, the voices of the patient and the lungs change significantly. Auscultation is a method of detecting abnormal lung sounds such as crackles, wheezes, and high-pitched sounds that can help determine the presence of lung disease [17]. With the development of ML and DL methods, it is possible to diagnose COVID-19 based on the patient’s voice, breath sounds, vibration, heart sounds, MRI, and ultrasound. MRI is a medical imaging technique that uses radio waves to examine organs. In this technique, a magnetic field is constantly applied, and radio signals pass through the surface of the body and are then reflected by water molecules in the body. These signals are received and converted into images that allow accurate diagnosis. These medical images are better than CT or X-rays because MRI can capture more detail.

This study provides a taxonomy to categorize the diagnostic system of COVID-19. As shown in Fig. 4, two different perspectives are used for DL techniques and imaging modalities, namely pre-trained models and customized DL techniques.

Fig. 4.

Overall workflow summary of the methods

In this study, we divided DL-based methods using medical images into four categories: CT scans, X-rays, ultrasounds, and cough sounds. 55 diagnostic models were studied for COVID-19. 24 studies included pre-trained models; the other 31 included adjusted DL models for COVID-19 diagnoses.

Pre-trained models

A stored network, previously trained using a large dataset, especially for an image classification task, is called a pre-trained model. This model could be adapted to a specific task using its pre-trained weights and changing top layers to adapt to new tasks, and it is named transfer learning (TL) [81]. Using a pre-trained model in deep TL has several advantages. Training a model from scratch takes a lot of time when dealing with large datasets and requires high computational power [82]. The pre-trained model allows faster convergence in TL [83]. We found that different models were successfully applied in the investigation of recent studies. ResNet and its derivatives were mainly determined as the main architecture in these studies.

A stored network previously trained with a large dataset is called a pre-trained model, especially for an image classification task. This model can be adapted to a specific task by using its pre-trained weights and changing the top layers to fit a new study called TL [81]. Using a pre-trained model in deep TL has several advantages. Training a model from scratch takes a long time for large datasets and requires high computational power [82]. The pre-trained model allows faster convergence in TL [83]. We found that different models have been successfully used in recent studies. ResNet and its derivatives were mainly determined to be the main architecture in these studies.

A COVID-19 detection system based on DL with multiview fusion was presented by Wu et al. The system used ResNet50, a pre-trained CNN model. Data were obtained from two local hospitals in China. For the experiment, 495 images were used, including 368 with COVID-19 cases and 127 with other disease infections. For training, testing, and validation, the dataset was divided into three segments of 80%, 10%, and 10%, respectively. The input image used by the system was resized to 256 $\times$ 256 before the model was created. The system achieved an accuracy, sensitivity, and specificity of 76%, 81.1%, and 61.5%, respectively [44]. In another study implementing a three-class detection model for COVID-19, the authors used CT scans to detect the disease. They used ResNet18 with TL and achieved a success rate of 86.7% using model-based segmentation methods [45]. In [84], they developed a ResNet101-based model to distinguish bacterial pneumonia, viral pneumonia, and normal images. They also applied their model to diagnose COVID-19, and the model’s success rate is 98.75%. Another COVID-19 diagnostic system was proposed by Jin et al. using ResNet152. They evaluated their model using 496 images with COVID-19 and 1385 healthy images and achieved an accuracy rate of 94.98% [85].

In another study using CT scans as input [48], the authors presented a system for diagnosing COVID-19 using pre-trained CNN models, including AlexNet, MobileNetV2, ResNet18, ResNet101, VGG-16, VGG-19, SqueezeNet, GoogleNet, ResNet50, and Xception. They used 1020 CT healthy and COVID-19 CT scans in the proposed approach, and the experimental results showed that the ResNet101 model achieved 99.51% accuracy using a 20–80% hold-out validation method. In addition, Narin et al. presented an approach to detect COVID-19 using X-ray images implementing pre-trained CNN models. The dataset contains 100 X-rays that consist of 50 COVID-19 and 50 healthy images. They used the cross-validation method to evaluate the model and achieved 98% accuracy with the ResNet50-based model [86].

Bukhari et al. proposed another diagnostic model for COVID-19 using a pre-trained ResNet50 model. They evaluated their model using 278 chest X-rays from three different classes: 89 COVID-19, 93 healthy, and 96 pneumonia images. They used the hold-out method, splitting the dataset into 80% for training and 20% for testing. Their model achieved an accuracy rate of 98.18%, precision of 98.14%, sensitivity of 98.24%, and F1 score of 98.19% [87]. Xing et al. used ultrasound to detect COVID-19. The ultrasound device was built with a DL classifier. They used pre-trained ResNet50, GoogleNet, and VGG19 models for training and obtained 12,949 images for model training. The final model achieved accuracy and an F1 score of 96.1% and 96.1%, respectively[88]. In another work, Born et al. proposed a NasNetMobile model for detecting COVID-19 from lung ultrasound images. The model achieved a sensitivity and specificity of 81% and 96%, respectively [89].

Another common model used in COVID-19 detection is VGG and its derivatives. In [52], the authors implemented VGG-19 and InceptionV2 pre-trained models and applied fine-tuning to these models. They also used decision trees to evaluate the data. Using VGG-19, InceptionV2, and decision trees, they achieved success rates of 91%, 78%, and 60%, respectively. Moutounet–Cartan presented an approach for detecting COVID-19 and lung infections from X-ray images. This approach uses VGG-16, VGG-19, InceptionResNetV2, Xception, and InceptionV3 pre-trained CNN architectures. To evaluate the model, they collected 327 images, including 152 healthy, 125 COVID-19, and 50 images of various lung diseases. They determined VGG-16 as the primary model with an accuracy of 84.1% [90]. Horry et al. presented a model for diagnosing COVID-19 in X-rays using deep TL models, including Inception, Xception, Resnet, and VGG. The dataset for testing included 200 healthy, 100 pneumonia, and 100 COVID-19 images. The precision, sensitivity, and F1 score were 83%, 80%, and 80%, respectively [49].

Another study using the VGG model, COVIDX-Net, was developed by Hemdan et al. to identify COVID-19 using X-ray images. They used 50 images consisting of 25 healthy and 25 COVID-19. According to their numerical results, DenseNet and VGG-19 achieved a successful classification rate of 90% and an F1 score of 91% [51].

AlexNet is another popular architecture used in medical image analysis. Cifci presented an approach to detect COVID-19 from CT based on deep TL approaches. He used 5800 CT scans from a public repository, of which 80% were used for training and 20% for testing. In the experimental results, the AlexNet model performed better. The overall accuracy of AlexNet was 94.74%, and its specificity and sensitivity were 87.37% and 87.45%, respectively [40]. In another study, the authors presented a system combining deep TL methods such as GoogleNet, ResNet18, AlexNet, and generative adversarial network. The authors used 307 images, including COVID-19, normal, bacterial, and viral images. Googlenet had the highest accuracy rate of 80.6% for four-class classification. For three- and two-class classification, Alexnet achieved 85.2% accuracy [41].

Sethy and Behra proposed a hybrid model that uses both SVM and pre-trained CNN models to identify COVID-19 cases. They used pre-trained CNN models for feature extraction and SVM for classification using these features. They employed two datasets, the first of which included 25 COVID-19 X-ray images and 25 normal images and the second of which included 133 X-rays images that were both healthy and diseases such as acute respiratory distress syndrome, SARS, and Middle East respiratory syndrome. The proposed hybrid model achieved an accuracy rate of 95.38% [91]. Abbas et al. proposed another model that distinguished between COVID-19 and healthy images and used ResNet18 architecture as the main design. This model was trained using 196 images, including 80 from healthy individuals, 105 from COVID-19, and 11 from SARS patients. They used the 30–70% hold-out validation technique and achieved 95.12% accuracy, 97.9% sensitivity, 91.87% specificity, and 93.36% precision [92].

In addition to these popular pre-trained architectures, some studies were conducted with models such as DenseNet, Inception, MobileNet, UNet, and NasNetLarge. Apostolopoulos et al. proposed a model to distinguish between normal, viral, and COVID-19 cases. They used three different datasets, implemented a pre-trained MobileNetV2 as deep TL, and achieved success rates of 98.66%, 96.46%, and 96.78% for dataset1, dataset2, and dataset3, respectively. Kaya and Gursoy implemented a TL model using a pre-trained MobileNetV2 model. They used 3616 COVID-19 and 1576 normal (healthy) and 4265 pneumonia X-ray images. Their models achieved an average accuracy rate of 97.61% for 3-class cases with fivefold cross-validation [2].

In [84], they developed a CNN model to distinguish viral pneumonia, bacterial pneumonia, and healthy images. They also applied their model to diagnose COVID-19, and the model’s success rate was 98.75%.

Yousefzadeh et al. presented a model using CT images, the COVID-AI system implementing ResNet, EfficientNetB0, DenseNet, and Xception. They used a dataset that contained 2124 CT scans, including 1418 normal and 706 COVID-19 images. They employed a 20–80 hold-out validation method to evaluate their model. The proposed model achieved an overall accuracy of 96.4% [46]. Gupta et al. used several pre-trained CNN models to classify COVID-19 and pneumonia by extracting features from X-rays. Their model had a success rate of 99.08% [93]

Luz et al. also implemented a TL model using a pre-trained EfficientNet to analyze CT scans. Their experiments achieved an accuracy rate of 93.9 [94]. In addition, Jin et al. proposed a COVID-19 identification model using DL methods, including 3D U-Net++ with a set of pre-trained CNN models containing DPN92, Inception-v3, ResNet50, and Attention ResNet50. The system was trained with 541 healthy and 850 COVID-19 samples. They achieved an accuracy of 97.1% [53].

Chen et al. used the UNet++-based method to identify COVID-19 using CT scans. This work used 46,096 images from a hospital in China and achieved 98.85% accuracy. In another study, Javaheri et al. proposed a DL model, CovidCTNet, for diagnosing COVID-19 from CT scans. The system was developed using the U-Net architecture. The model used 89,145 CT scans, of which 32,230 were COVID-19 samples, 25,699 were with community-acquired pneumonia (CAP), and 31,216 were healthy scans. They achieved accuracy, sensitivity, specificity, and AUC rates of 91.66%, 87.5%, 94.0%, and 95.0%, respectively [95].

Minaee et al. suggested a model for detecting COVID-19 in X-rays using deep TL approaches such as ResNet18, DenseNet-121, ResNet50, and SqueezeNet. A total of 5071 images were obtained from various publicly available datasets, and they used 3100 images (100 COVID-19 and 3000 healthy images) to test their model. They reported that SqueezeNet obtained the best results, achieving a sensitivity of 100% and an accuracy of 98.00% [47]. In another study using ResNet, NASNetLarge, DenseNet169, Inception ResNetv2, and Inceptionv3 as TL architecture, Punn and Agarwal presented a diagnostic model for COVID-19 detection. They used 1076 chest X-ray images to evaluate their model. The training, testing, and validation sets were assigned 80%, 10%, and 10% of the total dataset, respectively. Experimental results show that NASNetLarge performed significantly better than other methods, achieving 98% accuracy, 88% precision, 91% sensitivity, 99% specificity, and 89% F1 score [96].

Table 2 provides an overview of the COVID-19 diagnostic methods using pre-trained models. The data sources, collection of images and class types, data partitioning methodology, diagnostic approach, and performance metrics of the reported works are summarized.

Table 2.

Summary of DL-based COVID-19 recognition models utilizing pre-trained transfer learning approaches

References	Data source	Data type	Number of images	Class	Partition	Model	Acc.(%)
[50]	[97–99]	X-ray	2870 (448 COVID-19, 1414 pneumonia, 1008 normal)	2	Random	MobileNetV2	96.78
[44]	Local hospitals in China	CT	495 (368 COVID-19, 127 pneumonia)	2	Train= 80% Validation=10% Test= 10%	ResNet50	76.0
[45]	Local hospitals in China	CT	618 (219 COVID-19, 224 IAVP, 175 normal)	3	Random	ResNet18	86.70
[84]	[97]	X-ray	400 (200 COVID-19, 200 normal)	2	Train = 80% Validation = 20%	ResNet101	98.75
[46]	Local hospitals	CT	2124 (706 COVID-19, 1418 normal)	2	Train = 80% Validation = 20%	DenseNet	96.40
[93]	[98, 100]	CT+X-ray	1088 (361 COVID-19, 362 pneumonia, 365 normal)	3	Random	Inceptionv3	99.08
[52]	[101]	CT+X-ray	394 (360 COVID-19, 16 SARS, 18 streptococus)	3	Random	VGG19, Inceptionv2	91.00
[85]	Local hospitals from China	CT	1881 ( 496 COVID-19, 1385 normal)	2	Random	ResNet152	94.98
[94]	[97, 102], Local hospital	X-ray	13,969 (183 COVID-19, 8066 normal, 5521 pneumonia)	3	Train = 90% validation = 10%	EfficientNet	93.90
[53]	Local hospital from China, [100]	CT	1391 (850 COVID-19, 541 normal)	2	Random	Inceptionv3, ResNet50	97.10
[95]	Local hospitals from Iran	CT	89,145 (32,230 COVID-19, 25,699 CAP, 31,216 other diseases)	3	Train = 90% validation = 10%	U-Net	91.66
[48]	Local hospitals	CT	1020 (510 COVID-19, 510 normal)	2	Train = 80% validation 20%	VGG16,VGG19	99.51
[40]	Public datasets from Internet	CT	5800	2	Train = 80% Validation = 20%	AlexNet, Inceptionv4	94.74
[41]	[102]	X-ray	307 (69 COVID-19, 79 bacterial pneumonia, 79 viral pneumonia, 79 normal)	4	Train = 80% validation =10% test = 10%	AlexNet, GoogleNet, ResNet18	85.20
[49]	[97, 102, 103]	X-ray	400 (100 COVID-19, 100 pneumonia, 200 normal)	3	Train =80% test = 20%	VGG16, VGG19, ResNet50	80
[91]	[104, 97, 99]	X-ray	183 (25 COVID-19 25 normal 133 pneumonia	3	Train = 60% test = 20% validation = 20%	AlexNet, VGG16, VGG19, GoogleNet, ResNet18, ResNet50	95.38
[47]	[104]	X-ray	3100 (100 COVID-19, 3000 normal)	2	Train = 40% test = 60%	ResNet18, ResNet50, SqueezeNet, DenseNet	98.0
[86]	[104], public datasets from Internet	X-ray	100 (50 COVID-19, 50 normal)	2	Cross validation	ResNet50, Inceptionv3, Inception-ResNetv2	98.0
[96]	[101], public datasets from Internet	X-ray	1076 (108 COVID-19, 515 pneumonia, 453 normal)	3	Train = 80% test = 10% validation = 10%	ResNet, Inceptionv3, Inception-ResNetv2, DenseNet169, NASNet	98.0
[87]	[101, 105]	X-ray	278 ( 89 COVID-19, 93 normal, 96 pneumonia)	3	Train = 80% test = 20%	ResNet50	98.18
[90]	[104, 99]	X-ray	327 (125 COVID-19, 152 normal, 50 pneumonia)	3	Cross validation	VGG16, VGG19, Inception-ResNetv2, Inception, Xception	84.1
[2]	Public datasets from Internet	X-ray	9457 (3616 COVID-19, 1576 normal, 4265 pneumonia)	3	Cross-validation	MobileNetv2	97.61
[88]	Public datasets from Internet	Ultrasound	12,949	3	Random	ResNet50, GoogleNet, VGG19	96.10
[89]	Public datasets from Internet	Ultrasound	202	4	Random	NasNetMobile	63.00

Customized deep learning techniques

Customized deep learning techniques enable architectural development with more accurate performance because they are tailored to the specific application. Combining a particular DL technique or algorithms with other AI areas such as data mining, ML, and nature-inspired algorithms [106, 107], customized models have been developed [108]. Unlike pre-trained models, the network does not use prior biases and weights. Therefore, it requires a lot of computational power and execution time. This subsection summarizes trained from scratch and other hybrid models proposed for COVID-19 diagnosis.

In [109], the authors presented a novel approach, COVNet, based on ResNet50 to identify COVID-19 on CT scans. The dataset included 4536 CT scans, consisting of 1296 COVID-19, 1735 CAP, and 1325 other lung diseases. They divided the dataset into training and testing datasets, which comprised 90% and 10%, respectively. They achieved a specificity, accuracy, and AUC of 96%, 90%, and 96%, respectively.

In another study, Ozturk et al. proposed a DNN to present a specific version of DarkNet, DarkCovidNet, for diagnosing COVID-19 from chest X-ray images. They obtained F1 score, precision, specificity, and accuracy values of 96.51%, 98.03%, 95.13%, and 98.08%, respectively [110]

To identify COVID-19 from CT scans, He et al. proposed a hybrid DL approach that reveals a new supervised learning technique combined with transfer learning called CRNet. This approach was evaluated using 746 CT scans, including 349 COVID-19 and 397 healthy images. The training, testing, and validation sets had weights of 60%, 25%, and 15%, respectively. They achieved an accuracy of 86%, an F1 score of 85%, and an AUC of 94% [111]

Song et al. [112] proposed a DL fuzzy model for diagnosing COVID-19 using CT images. In their model, they first implicated DL to calculate low-level features of CT images and then analyzed these features to reduce the feature size and obtain more relative features. Using a fuzzy classifier, they achieved accuracy and F1 scores of 94.2% and 93.8%, respectively.

Another hybrid method combining CNNs and the whale optimization algorithm (WOA) was proposed by Elghamrawy and Hassanien to identify COVID-19 from CT scans. In this method, CNNs were used for segmentation, and predictions were made about the probability of the patient’s response considering some factors. The dataset, which included 583 CT scans consisting of 432 COVID-19 images and 151 pneumonia images, was obtained from publicly available databases. They used a 65–35% hold-out validation method and achieved a detection rate of 96.40% [113]. In another study, Khan et al. customized the Xception model to identify COVID-19 images automatically. Their study attained an accuracy rate of 87.5% [42].

In [114], the authors implemented an inception-based architecture, called modified-inception, to detect COVID-19. They evaluated the proposed model using 1040 CT scans, of which 740 were classified as COVID-19 and 325 as healthy. Experimental results show that their model has an accuracy of 79.3%, a sensitivity of 83%, a specificity of 67%, and a precision of 55%. In another study that used CT scans to diagnose COVID-19, Chen et al. recommended the Unet++ model, based on the UNet model. The dataset used for this model included 51 COVID-19 and 55 other diseases. They achieved a success rate of 95.25% and an F1 score of 93.55% [115].

Another diagnostic system using a CNN model on CT scans was proposed by Liu et al. The proposed model, called COVIDNet, was based on DenseNet-264 and consisted of 4 dense blocks. Each block contained numerous units, each of which had two connected stacks and received feature maps from the previous layers via dense block connections. These blocks included a normalization layer, a convolutional layer, and a ReLU activation layer. The dataset includes 1073 healthy and 920 COVID-19 CT scans. The dataset is divided into training, testing, and validation groups of 60%, 20%, and 20%, respectively. The accuracy of the model was 94.3% [116].

Song et al. proposed a model, DeepPneumonia, based on a DRE network. Image features were extracted by combining the feature pyramid network and the ResNet50 model, which was used to develop the proposed system. A total of 1990 CT scans were included in the dataset, consisting of 708 healthy, 505 bacterial pneumonia, and 777 COVID-19 images. The proposed model achieved an accuracy of 94%, precision of 96%, F1 score of 94%, sensitivity of 93%, and AUC of 99% [117].

Similarly, Zheng et al. developed a CNN model called DeCoVNet. They used three particular layers: the subsampling layer, the 3D convolutional layer, and the batch norm layer. They evaluated their model using 630 CT scans, applying an 80–20% hold-out validation method. They achieved an accuracy of 90.1%, specificity of 91.1%, AUC of 95.9%, NPV of 84%, and sensitivity of 90.7% [118].

In [119], the authors propose an approach that uses two decoders for segmentation, an MLP for classification, and an encoder for reconstruction. For each of the three tasks, a single encoder was used to input the sample data with CT scans, two decoders were used for reconstruction and segmentation, and the multilayer perceptron was used to classify the images based on the presence of COVID-19 cases. 1044 CT scans were used, including 449 COVID-19, 100 healthy, 98 lung cancer, and 397 other lung disease images. The proposed model achieved a sensitivity of 94%, a specificity of 79%, an accuracy of 86%, and an AUC of 93%.

In another study, Hasan et al. developed a Q-deformed entropy technique (QDE-DF) using DL features to identify COVID-19 from CT scans. Deep features were extracted using CNN and Q-deformed entropy, and then LSTM was used to categorize cases based on the deep features. They used 321 CT scans, including 118 COVID-19, 96 pneumonia, and 107 healthy images. The dataset was split into training and testing ratios of 70% and 30%. This model achieved a classification accuracy of 99.68% [120].

Singh et al. presented a hybrid model for detecting COVID-19 using CT scans. In this work, multicenter differential evolution was used to determine the initial parameters of the model. They implemented an artificial neural network (ANN) and an ANN with a fuzzy inference system (ANNFIS). 150 CT scans were included in the evaluation dataset, 75 COVID-19 and 75 healthy subjects. They tested different validation ratios of training and test for the experiment, including 20—80%, 30–70%, 40–60%, 50–50%, 60–40%, 70–30%, 80–20%, and 90–10%. The model’s sensitivity was 90.70%, the accuracy was 93.25% and F1 score was 92.20% [121].

Ucar and Korkmaz proposed a Bayes SqueezeNet-based model, COVIDiagnostic, to diagnose COVID-19. They used 1591 pneumonia, 45 COVID-19, and 1203 healthy chest X-rays. The experimental results showed that their model achieved an accuracy rate of 98.26% and a specificity of 98.26%. Farid et al. presented an innovative method for identifying COVID-19 infections from CT images. They used a hybrid composite feature extraction technique to calculate the features, and the stack hybrid classification (SHC) method was used to classify these extracted features. SHC combines multiple models, such as ensemble learning, to improve model performance. To validate the model, 10-fold cross-validation was performed, and they achieved an accuracy of 94.11%, a precision of 99.4%, an F1 score of 94%, and an AUC value of 99.4% [122].

Rahimzadeh and Attar proposed a modified CNN model for detecting COVID-19 on chest X-ray images. They used ResNet50V2 and Xception CNN architectures as the main design and evaluated their model using 180 COVID-19, 6054 pneumonia images, and 8851 healthy images. A fivefold cross-validation scheme was implemented, and a success rate of 99.50% was achieved [43].

In [123], the authors developed a novel system using the CoroNet model and AutoEncoder to diagnose COVID-19 individuals on X-rays. They use semi-supervised learning and classify them using a user-defined network to extract the features. The semi-supervised learning approach of the designed architecture solved the problem of insufficient data. A total of 18,529 images related to different diseases were used. Specifically, 9579 images were pneumonia, 99 were COVID-19, and 8851 were healthy people. 90% of the data were used for training and 10% for test sets. They achieved accuracy, precision, recall, and F1 score values of 93.50%, 93.63%, 93.50%, and 93.51%, respectively.

Farooq and Hafeez proposed a DL architecture, COVID -ResNet, based on the ResNet50 model to identify COVID-19. They trained it from scratch by optimizing the parameters and discriminating the learning rate. They used 648 X-ray images, consisting of 239 bacterial pneumonia, 8 COVID-19, 234 healthy pneumonia, and 149 viral pneumonia, and achieved 96.23% accuracy [124]

Islam et al. developed a hybrid model for the automatic detection of COVID-19 based on the hybridization of CNN with LSTM. The dataset of 4575 X-rays, including 1525 COVID-19, 1525 normal, and 1525 viral pneumonia images, was used to evaluate the effectiveness of the hybrid model. According to the numerical results, the model achieved 99.20% accuracy, 99.30% sensitivity, and 98.90% specificity [125].

In [126], the authors employed two DL approaches to detect COVID-19 cases from chest X-ray images. These proposed architectures use artificial neural networks for automatic lung segmentation (preprocessing) on CT scans and X-rays. The AlexNet architecture is implied as TL in both architectures. The second model presented has a hybrid structure that includes a BiLSTM (bidirectional long short-term memory) layer that also considers temporal aspects. They used 2905 images, including 219 COVID-19, 1345 pneumonia, and 1341 healthy images. The second hybrid architecture has a classification accuracy of 98.70%, while the first design achieved an accuracy of 98.14%.

Turkoglu presented a hybrid model called COVIDetectioNet, which combines DL and classical ML methods such as AlexNet and SVM to identify COVID-19 from X-ray images. The researchers collected a dataset from GitHub and Kaggle that included 219 COVID-19, 4290 pneumonia, and 1583 normal images. COVIDetectioNet achieved 99.18% accuracy [127].

In [128], the authors proposed a DL approach, namely CovidSORT, to detect COVID-19 using 5910 chest X-ray images from local hospitals in China to evaluate the performance of the proposed method. According to the numerical results, the accuracy of CovidSORT model was 96.83%, precision was 98.75%, sensitivity was 96.57%, and F1 score was 97.65%.

Mahmud et al. developed a unique CNN method, CovXNeT, based on depth-stretched folding to identify COVID-19. CovXNeT was applied to a combined dataset containing 305 images in each class from different sources. Experimental results show that CovXNeT achieved an accuracy of 97.4% for the classification of two classes and an accuracy of 90.3% for the classification of four classes [129].

Awasthi et al. proposed a DL model for COVID-19 using lung ultrasound images. The new method, namely Mini-CovidNet, modified MobileNet with focal loss. Mini-CovidNet achieved an accuracy of 83.2% [130]. Chen et al. implemented a novel CNN model to detect COVID-19 by analyzing lung ultrasounds. It was found that the reliability of lung ultrasound results was highly dependent on the physician’s experience. They collected 1527 images of lung ultrasounds from 31 patients diagnosed with COVID-19, and the proposed model attained 87% accuracy [131]. In addition, Dastier et al. proposed another ultrasound-based model to identify COVID-19. They proposed a CNN architecture implementing an auto-encoder network and separable convolutional branches fused with a modified DenseNet201-based model to create a noise-free classification model and achieved a fivefold cross-validation accuracy of 79.1% [132].

Some studies diagnose COVID-19 by analyzing the patient’s voice, breath, vibration, heart sounds, and MRI. In [133], the authors proposed a model called CR19 using genetic ML classifiers to diagnose COVID-19 from cough sounds. For this study, public datasets were used, and various ML models such as logistic regression, KNN, support vector machine, and decision tree algorithms were employed, with KNN achieving the best results with precision and F1 score of 97 and 98%, respectively. Pahar et al. used a deep-transfer learning algorithm to diagnose COVID-19 based on breath, cough, and speech. In this study, three modified CNN models were implemented. The modified ResNet50 model outperformed the other models with an AUC of 98%, 94%, and 92% for the cough, breath, and speech classes, respectively [134]. In [135], the authors propose a novel CNN model for detecting COVID-19 from acoustic sounds. The Modified-Mel-frequency cepstral coefficient technique was used for disease classification. The model achieved accuracy and F1 score of 92.32% and 93.48%, respectively.

In addition, Laguarta et al. developed a CNN model to identify COVID-19 from cough recordings collected via cell phones. In this model, the cough recordings were converted into a CNN-based architecture consisting of a Poisson Biomarker layer and three ResNet50 layers connected in parallel to generate a binary prior diagnosis. The model achieved 97.1% accuracy in predicting COVID-19 based on 5320 cough sounds [136]. Pahar et al. proposed a classifier to detect COVID-19 from cough sounds. They used two publicly available datasets that included a total of 1291 samples. This study implemented different classifiers, including MLP (Multilayer Perceptron), LR, LSTM, SVM, CNN and RNN. The modified ResNet50 classifier achieved the best results among the classifiers with an accuracy of 95.3% [137].

In [138], the authors proposed a diagnostic tool for detecting COVID-19 by MRI. In 32 patients diagnosed with COVID-19, scans of chest CT and MRIs were performed within 24 h. They found that MRIs achieved better results with a sensitivity and specificity of 91.67% and 100% for the diagnosis of COVID-19. In addition, studies such as [139–141] use MRI images.

The DL-based COVID-19 diagnostic systems using the customized CNN models are summarized in Table 3. We provide an overview of the key elements, including data sources, data types, number of images, data partitioning method, diagnostic strategy, classes, and performance metrics of the reported work.

Table 3.

Summary of deep learning based COVID-19 diagnosis using the customized models

References	Data source	Data type	Number of images	Class	Partition	Model	Acc.(%)
[109]	Local hospitals	CT	4536 (1296 COVID-19, 1735 CAP, 1325 pneumonia	3	Train = 90% test = 10%	COVNet	90.0
[110]	[97]	X-ray	1127 (127 COVID-19, 500 normal, 500 pneumonia)	3	Cross validation	DarkNet	98.08
[111]	Public databases from internet	CT	746 ( 349 COVID-19, 397 normal)	2	Train = 60% test = 25% Validation = 15%	CRNet	86.0
[113]	Public databases from internet	CT	583 (432 COVID-19, 151 pneumonia)	2	Train = 65% test = 35%	WOA-CNN	96.40
[42]	[97], public databases from Internet	X-ray	1251 (284 COVID-19, 310 normal, 330 bacterial pneumonia, 327 viral pneumonia)	4	Train = 80% test = 20%	CoroNet	89.50
[114]	Local hospitals from China	CT	1065 (740 COVID-19, 325 normal)	2	Random	Modified-Inception	79.30
[115]	Local hospitals from China	CT	106 (51 COVID-19, 55 other diseases)	2	Train = 80% Test = 20%	UNet++	95.24
[116]	Local hospitals from China	CT	1993 (19 COVID-19, 777 bacterial pneumonia, 708 normal)	2	Train = 60% test = 20% Validation = 20%	COVIDNet	94.30
[117]	Local hospitals	CT	1990 (777 COVID-19, 505 pneumonia, 708 normal)	3	Train= 60% Test = 10% Validation = 30%	DRE-Net	94.30
[118]	Local hospitals	CT	630	2	Train = 80% test = 20%	DeCOVNet	90.1
[119]	[143]	CT	1044 (449 COVID-19, 595 normal)	2	Train = 80% test = 10% Validation = 10%	Encoder, decoder with multilayer perceptron	86.0
[120]	Public databases from internet	CT	321 (118 COVID-19, 96 pneumonia, 595 normal)	3	Train = 70% validation = 30%	QDE-DF	99.68
[121]	Public databases from Internet	CT	150 (75 COVID-19, 75 pneumonia)	2	Random	Mode-CNN	93.25
[144]	[97, 102]	X-ray	2839 (45 COVID-19, 1203 normal, 1591 pneumonia)	3	Train = 80% test = 10% Validation = 10%	Bayes-SqueezeNet	98.26
[122]	Public databases from internet,[145]	CT	102 (51 COVID-19, 51 SARS)	2	Cross-validation	CNN	94.11
[43]	[97], public databases from Internet	X-ray	15,085 (180 COVID-19, 6054 pneumonia, 8851 normal)	3	Cross-validation	Concatated CNN	99.50
[123]	[97], public databases from Internet	X-ray	18,529 ( 99 COVID-19, 9579 pneumonia, 8851 normal)	3	Train = 90% validation = 10%	CoroNet	93.50
[124]	[102]	X-ray	637	4	Train = 90% Test = 10%	Covid-ResNet	96.23
[125]	[97, 105, 146, 147] [148]	X-ray	4575 (1525 COVID-19, 1525 pneumonia, 1525 normal)	3	Random	Cross validation	99.20
[126]	[97]	X-ray	2905 (219 COVID-19, 1345 pneumonia, 1341 normal)	3	Random	CNN+BILSTM AlexNet	98.70
[127]	Public databases from Internet	X-ray	6092 (219 COVID-19, 4290 pneumonia, 1583 normal)	3	Random	COVIDetectioNet	99.18
[128]	Local hospitals	X-ray	5910	3	Cross-Validation	COVIDSort	96.83
[129]	Public databases from Internet	X-ray	7076	2	Cross-validation	ConvxNet	97.4
[130]	Public databases from Internet	Ultrasound	1103	3	Random	Mini-CovidNet	83.2
[131]	Local hospitals	Ultrasound	1521	2	Random	Novel CNN	87.0
[132]	Public databases from Internet	Ultrasound	14,311	4	Cross-validation	Modified DenseNet201	79.10
[133]	Public databases from Internet	Cough sounds	469	2	Random	CR19	94.32
[134]	Public databases from Internet	Cough sounds	11,202	2	Random	Modified ResNet	AUC = 98.00
[135]	Public databases from Internet	Cough sounds	13,000	3	Random	Modified ResNet	AUC = 98.00
[136]	Public databases from Internet	Cough sounds	5320	2	Random	Modified ResNet	97.10
[137]	Public databases from Internet	Cough sounds	1291	2	Random	Modified ResNet50	95.30

Comparison of machine learning and deep learning techniques

We investigated two approaches for detecting COVID-19 using pre-trained or customized DL models. Although the performance of the proposed approaches is promising, these studies have certain similarities and differences that should be considered. The sunburst plot of both approaches described is given in Fig. 5.

Fig. 5.

Sunburst plot for pre-trained and customized DL models

There are studies using the same dataset for pre-trained [49, 50, 84, 91] and customized CNN models [42, 110, 125, 126, 142]. Although the models’ architectures differed, they obtained similar results using the same datasets. A direct comparison of the results would not be fair because they were not trained with the same training parameters on the same dataset. In summary, however, the authors in [48] and [91] applied the VGG16 model to different datasets and the results differed by 14.74%. Another reason the studies get various results because they use different types of images in the same model. In [40] and [41], the authors used the AlexNet model, which uses CT scans and X-ray images, and their success rates differed by about 5%.

Insufficient data and class imbalance also led to different results. It is difficult to train the system with small datasets, and the visual classification task becomes susceptible to overfitting. Some studies used small datasets, which does not guarantee that the proposed model is fully trained. In [51, 115, 121, 122], the authors could not guarantee the accuracy of their models because they trained on datasets with less than 150 images.

Datasets

Although there are a variety of diagnostic techniques, RT-PCR is the leading standard for identifying COVID-19 pathogen laboratories. However, test results are not available for several hours, and this relatively late response is the main obstacle to early intervention. Researchers have focused on alternative methods that use X-rays or CT scans to shorten the time to diagnose the disease.

CT is a technique that takes X-ray images from different angles and combines them into a single image. To illustrate COVID-19, pneumonia, and normal (healthy) CT scans, see Fig. 6.

Fig. 6.

Sample CT images

A CT scan is faster and easier to demonstrate but also more costly. Therefore, researchers have developed a COVID-19 detection model that uses X-rays rather than CT images. The use of X-ray images is readily available and inexpensive because most underdeveloped countries have X-ray imaging technologies. Figure 7 shows X-ray images of COVID-19, pneumonia, and a healthy sample.

Fig. 7.

Sample X-ray images

In the first quarter of the pandemic, medical imaging began to be stored and made publicly available during COVID-19. At that time, there were not enough records freely available to the public. Therefore, the first publications suffered from the small size of the training and test images [44, 45, 49, 51, 84, 87, 90, 120–122], and these works cannot be considered reliable. This is because the methods of DL require sufficient data to consistently train the model [149].

Toward the end of the pandemic, researchers have sufficient data available thanks to publicly available datasets [94, 117, 126–128].

Researchers also used clinical and laboratory data such as ultrasound [88, 89, 130–132], MRI [133, 138–141], cough sounds [133–137], and blood tests [33, 76, 77, 79] instead of X-rays and CT.

The results of the work during this period are more consistent. We summarize general information about the datasets in the literature and their resources in Table 4. As can see from Table 4, the authors have been very successful in providing a variety of datasets and types that can be very helpful in diagnosing COVID-19.

Table 4.

Datasets

References	Data type	Size	Classes
[100]	CT	829	829 COVID-19
[143]	CT	679	216 COVID-19, 463 healthy
[145]	CT	20	20 COVID-19
[150]	CT	620	310 COVID-19, 310 healthy
[151]	CT	3520	3520 COVID-19
[152]	CT	384	384 COVID-19
[97]	X-ray	646	468 COVID-19, 5 varicella, 16 SARS, 4 influenza, 3 herpes, 13 Streptococcus spp., 9 Klebsiella spp., 7 Legionella spp., 4 Escherichia coli, 1 aspiration disease, 10 MERS-CoV, 24 Pneumocystis spp., 4 Nocardia spp., 5 Mycoplasma spp., 2 unknown diseases, 1 Chlamydophia spp., 1 Staphylococcus spp., 8 lipoid diseases, 2 Aspergillosis spp., 59 unknown diseases
[98]	X-ray	3551	219 COVID-19, 1341 healthy, 1345 pneumonia
[99]	X-ray	109,322	37,456 CNV, 11,599 DME, 8867 Drusen, 51,390 healthy
[102]	X-ray	13,975	COVID-19 and 14 different lung diseases
[148]	X-ray-CT	17,099	9471 COVID-19, 8128 healthy
[104]	X-ray	79	79 COVID-19
[105]	X-ray	112,120	112,120 COVID-19
[153]	X-ray	20,226	4558 COVID-19, 5403 healthy, 4497 viral pneumonia, 5768 bacterial pneumonia
[154]	PCR and blood test	475	475 COVID-19
[154]	PCR and blood test	475	475 COVID-19
[155]	Ultrasound	173	173 COVID-19
[155]	Ultrasound	150	150 COVID-19
[156]	Ultrasound	242	242 COVID-19
[157]	Cough sound	734	697 COVID-19 37 normal
[158]	Cough sound	25,000	25,000 COVID-19

In this study, we determine that 23 works used X-ray images [41–43, 47, 49–51, 84, 86, 87, 90–92, 94, 96, 110, 123–129], 20 works use CT scans [40, 44–46, 48, 53, 85, 95, 109, 111, 113–122], 2 works used both [52, 93], 5 works ultrasound [88, 89, 130–132], 5 works MRI [133, 138–141], 5 works cough sounds [133–137], and 4 works blood tests [33, 76, 77, 79] as data sources.

In some studies, because of a lack of data size, researchers comprised their test dataset using more datasets to increase the data size [49, 50, 87, 90–92, 125, 144]. Thus, they achieved a more robust result for the diagnosis of COVID-19.

From a different perspective, it is noted that due to the lack of data in the early days of the pandemic, a two-class classification (COVID-19 and healthy) was applied [40, 44, 48, 51, 85]. As more relevant data became available over time and COVID-19 was evaluated with different lung diseases, studies were conducted to classify them into three and four classes [41, 42, 45, 52, 91, 130, 132, 135]. In addition, some researchers use blood testing techniques [33, 75, 77–79] and have used more than four classes for classification.

CT scans are more detailed than X-ray images and are therefore better suited for detecting COVID-19 and developing detection models [159]. 18 CT Image-based work used the COVID-19 CT scans to develop their architecture. The first dataset [100] contains 100 CT images of 40 individuals. The experts classified the images into three categories: Pleural effusion, consolidation, and actual class.

In [143], the authors presented the COVID-CT dataset consisting of 216 CT scans with COVID-19 and 463 images of healthy individuals. An experienced radiologist who has identified and treated COVID-19 patients since the onset of the epidemic confirmed the importance of this dataset [143].

Another COVID-19 CT scan dataset was published at Kaggle [145]. This dataset contains 20 CT scans of patients diagnosed with COVID-19 and an expert-generated segmentation of lungs and infections. Finally, some researchers use the COVID-CT database on GitHub. This dataset includes 620 individual CT scans, of which 310 are COVID-19, and 310 are healthy [150]. Researchers working for local hospitals had the opportunity to obtain local images from their hospitals and collected their test datasets from these hospitals [45, 115, 117, 118]

Thanks to the cheaper cost of X-ray images, they are more accessible than CT scans. Therefore, most authors use chest X-rays in their experiments. The most commonly used COVID-19 X-ray dataset by researchers is COVID-19 Image Data Collection [97]. The database also includes lung segment annotations and metadata and contains a limited number of images but is constantly updated with the addition of new images.

Another common database is the COVID-19 Radiography Database. When it was first published, it consisted of 219 COVID-19, 1341 healthy, and 1345 viral pneumonia images. The first update included 1200 images of COVID-19 in the database. The second update included 1345 viral pneumonia images, 6012 pulmonary opacities (not COVID pulmonary infection), 10,192 healthy and 3616 COVID-19 images. This database is continually updated as new X-ray images with COVID-19 arrive [98].

Some authors used the dataset of Kermany et al. [99], which includes 741 X-ray images comprising four classes: COVID-19, viral pneumonia, bacterial pneumonia, and healthy. In [102], the authors presented an open-access benchmark dataset, the COVIDx, which consists of 13,975 X-ray images from 13,870 patients, one of the highest numbers of images. The comprehensive COVID-19 X-ray Images Dataset consisted of X-rays and CT scans with a total of 17,099 images, of which 9544 are X-rays [148]. The COVID-19 X-ray images dataset is publicly available on Kaggle and contains 76 COVID-19 X-ray images [104].

In another public Kaggle dataset, the NIH Chest X-rays database was used by the authors to expand their dataset with 5863 images in two classes: pneumonia and normal [105]. Finally, the curated dataset for COVID-19 Posterior–Anterior Chest Radiography Images (X-rays) is a public dataset available on Mendeley Data. This dataset contains a total of 20,226 x-ray images [153].

It is important to use enough data to train a DL model and avoid overfitting consistently. Therefore, most researchers have extended the dataset using various image preprocessing techniques such as data augmentation [42]. In addition, some recent works are trained on a combined dataset that integrates the COVID-19 dataset with normal or pneumonia chest X-ray images from another repository. In addition, lung diseases such as COVID-19 influenza, pneumonia, Middle East respiratory syndrome (MERS), severe acute respiratory syndrome (SARS), and tuberculosis can cause CT and X-ray images to be misclassified [126]

To get around this problem, the researchers modified their dataset to include images of diseases similar to COVID-19. When accurately distinguishing COVID-19 patients from healthy individuals is critical, the authors also augmented their dataset with images of healthy individuals [92]. Some authors combined COVID-19 images with other lung diseases such as SARS, MERS, influenza, bacterial pneumonia, fungal pneumonia, and tuberculosis [125].

Thread to validation

There are some validity cases for each DL and ML work. The results obtained in the study should be balanced and not tend to be under or overfitting. In addition, the models and methods developed should work efficiently in real time. The distribution of the dataset should be balanced and appropriate for the metrics used in model training. If the dataset has an imbalance of classes, the training does not prove that the model is reliable. Evaluation criteria such as precision, accuracy, AUC, F1 score, Cohen Kappa, and other criteria should also be used [17].

The COVID-19 symptoms and treatments may differ between the new context and the study. Data scaling may also be a critical factor. It is critical to the success of any research to increase validity.

Challenges and future directives

From a technological perspective, the RT-PCR technique is the most common, safest, and effective method for identifying COVID-19 cases. The application of DL in healthcare is a new and rapidly growing area of research. DL-PCR-based methods cannot be fully replaced in their current form because they are readily available. Therefore, healthcare professionals could use DL-based detection systems as an initial screening method or a tool to support their decision-making. DL-Methods for detecting COVID-19 present numerous challenges. Despite encouraging results, DL-based detection of COVID-19 from X-ray images and CT scans still faces several societal and technical obstacles [160].

DL-based approaches are well suited for automation, but creating a reliable diagnostic system requires a large amount of data. Because COVID-19 is a relatively new area of research, the lack of standardized data complicates diagnosis, and image data from COVID-19 patients can be mislabeled, noisy, incomplete, and unclear. With such extensive and diverse datasets, training DL models is challenging due to many issues, such as data redundancy, missing values, and sparsity. Various datasets obtained from online sources were used for the experiment. The data were processed according to the system specifications and evaluated using various metrics.

In addition, medical images are usually low-contrast images. Therefore, an attempt is made to increase the contrast of these images so that the images can be better transformed into feature space when run through a DL model. A large heterogeneity in the quality of images acquired at different locations using different imaging devices introduces a potential bias in image analysis. This challenge highlights the need to improve image quality in a preprocessing step. Preprocessing techniques are commonly used in the literature to improve the quality of images and make them more visually appealing.

Lack of data is another problem for COVID-19 detection systems based on DL. Because the COVID-19 pandemic is still relatively new, clinical information is still sparse and tightly controlled, leading to problems with the under or overfitting of data and affecting system performance. Another critical problem for COVID-19-based diagnostic methods is the imbalance of class. COVID-19 Data from X-rays and CT scans are far less available than for other common lung diseases. During the training phase of DL models, bias often occurs due to an imbalance of data. The lower the proportion of positive samples, the more difficult it becomes for the model to perform the intended classification.

Many studies have shown that most datasets used for binary and multiclass classification to diagnose COVID-19 are category imbalanced or insufficient [41, 44, 51, 52, 66, 80, 83, 87, 90, 91, 115, 120–122]. This may cause bias in the performance of the model. It is of great importance that researchers use appropriate datasets, especially since it is not possible to collect enough high-quality images in the first quarter of the pandemic.

In addition, the entire COVID-19 genome was sequenced using information from millions of patients suffering from the disease worldwide. The higher mutation rate of the COVID-19 virus has proven to be very beneficial for this genome sequence. The accuracy of current diagnostic methods for identifying individual viral genes depends on the targeted regions of the genome [161]. Alarmingly, the impact of mutations on diagnostic tests increases the likelihood that a patient with the disease will receive a false negative result. These diagnostic tests analyze the genetic mutations of coronavirus, which often change as the infection spreads from person to person. As the number of infected people increased and the pandemic spread, numerous containment strategies were considered, including lockdowns and social isolation [162].

More publicly available datasets could be collected and created in the future. There could be new research opportunities in creating, annotating, and providing metadata for data. Researchers could validate models with large datasets in the future, and integrating features from different transfer learning models could create new hybrid models that perform better. For multiclass classification, which achieves lower classification scores than binary classification, there are very few proposed models to date. Future research can focus on multiclass classification models to increase their effectiveness. Since the COVID-19 virus is believed to be in a severe stage of infection in the lung region, researchers could also study other organs affected by the virus. Researchers can apply DL to develop an end-to-end system that can be used in widespread hot spots to identify COVID-19 cases.

Discussion

For this work, 86 COVID-19 papers were consulted, of which 55 were examined in detail. 24 papers used pre-trained models, while the remaining 31 used customized DL architectures. The results of each approach are analyzed for clarity. Six common modalities were used, such as CT scans, X-rays, MRI, ultrasound, cough sound, and blood tests.

The problems discussed in the literature are examined in detail, as some issues will be resolved in future studies. First, most COVID-19-based diagnostic systems are described despite DL methods and without detailed explanations of background information and mathematical representations.

Second, the reader is urged to consult the relevant references rather than discuss the specifics of the CNN, especially for custom architectures. These modifications include batch size, dropout layer, optimization parameters, learning rate, layer specifications, loss function, and the number of epochs. Third, most of the work achieved over 90% accuracy, whether it was a pre-trained model or customized model for CT and X-ray images with few or large datasets. The need for a large dataset annotated by an experienced physician or radiologist is one of the main problems in using DL for the diagnosis of COVID-19. More publicly available datasets are needed to use large medical images for model training. Despite these problems, CNN models have shown promising results.

Finally, the main advantage of DL models is that they can be easily used for feature selection [163, 164]. However, in the case of medical images, feature selection was more important than any other task. Medical personnel cannot comprehensively analyze the models from DL. Therefore, reliability is not guaranteed, and these models can only be used as an early detection system [165]. We have discussed the challenges associated with current DL approaches to COVID-19 diagnosis. It is important to note that every study in the literature has shown that the potential for automated detection of COVID-19 simultaneously faces challenges or lack of analysis. The proposed approaches should be evaluated from different perspectives. We believe that consolidating key observations can serve as a benchmark and help researchers develop DL-based automatic and efficient COVID-19 detection solutions.

Conclusion

The COVID-19 pandemic has a serious impact on the well-being of people around the world. Technology is advancing rapidly, particularly in the areas of ML and DL. ML and DL have already made a significant contribution to the human response to the pandemic.

Because of their fascinating perspectives, ML and DL have been hailed as one of the best strategies to combat the COVID-19 epidemic. Numerous DL applications of medical imaging have inspired the current COVID-19 pandemic activities described in this article over the years. This comprehensive review presents various diagnostic approaches used by ML and DL to detect COVID-19. It also describes the methods of ML and DL, available datasets, and performance. Despite the encouraging achievements, ML and DL, the successful processing of COVID-19 in medical images still requires significant time and effort, as well as close coordination among numerous government and industry partners. With the help of ML, DL, and other technologies, including biomedicine, data science, and mobile communications, we expect the COVID-19 outbreak to end soon.

This review provides a detailed state-of-the-art overview of existing methods and applications for ML and DL researchers and the broader health community, with descriptions of how ML and DL will advance the status of COVID-19 and further studies to prevent the COVID-19 outbreak. To stop the spread of COVID-19, we also hope that our work will serve as a helpful resource and inspire several new studies on digital radiography and medical imaging. While this paper does not claim to be an in-depth analysis of these studies, it does provide a helpful perspective and a fair comparison of the research conducted in recent years that researchers can use in determining their future work.

Author contributions

The authors contributed equally to this work. EG and YK contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript.

Funding

No funds, grants, or other support was received.

Availability of data and materials

None.

Code availability

None.

Declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.