Medical image processing and COVID-19: A literature review and bibliometric analysis

Abstract

COVID-19 crisis has placed medical systems over the world under unprecedented and growing pressure. Medical imaging processing can help in the diagnosis, treatment, and early detection of diseases. It has been considered as one of the modern technologies applied to fight against the COVID-19 crisis. Although several artificial intelligence, machine learning, and deep learning techniques have been deployed in medical image processing in the context of COVID-19 disease, there is a lack of research considering systematic literature review and categorization of published studies in this field. A systematic review locates, assesses, and interprets research outcomes to address a predetermined research goal to present evidence-based practical and theoretical insights. The main goal of this study is to present a literature review of the deployed methods of medical image processing in the context of the COVID-19 crisis. With this in mind, the studies available in reliable databases were retrieved, studied, evaluated, and synthesized. Based on the in-depth review of literature, this study structured a conceptual map that outlined three multi-layered folds: data gathering and description, main steps of image processing, and evaluation metrics. The main research themes were elaborated in each fold, allowing the authors to recommend upcoming research paths for scholars. The outcomes of this review highlighted that several methods have been adopted to classify the images related to the diagnosis and detection of COVID-19. The adopted methods have presented promising outcomes in terms of accuracy, cost, and detection speed.

Introduction

As an international health crisis, COVID-19 gained the global attention of researchers, health organizations, and governments. As indicated by WHO [1], the mortality rate of COVID-19 is around 2.06, with around 224,511,226 confirmed cases, entailing 4,627,540 deaths by September 2021. A small ratio of patients can suffer from breathing difficulties, septic shock, or organ malfunctioning [2]. Several factors have been linked to the mortality rate of COVID-19; such as the availability of healthcare resources [3], environmental factors [4], and socioeconomic factors [5]. In general, elderly people and patients who suffer from severe signs constitute the majority of COVID-19 deaths. Mortality rate increases by 3.5% with every ratio point increase in the proportion of the overweight among people in the high-income regions [6]. Accordingly, COVID-19 has been regarded as a serious national and international health crisis in the twenty-one decade [[7], [8], [9], [10], [11], [12]].

Refereeing to specialists’ points of view, the coronavirus primarily impacts the respiratory system causing severe pneumonia with various signs of high temperature, breathlessness, dry cough, and exhaustion [[13], [14], [15]]. The available regular confirmed clinical examination, reverse transcription-polymerase chain reaction (RT-PCR), for identifying the virus, is manually actuated, complicated, and consumes a considerable time [16]. The restricted supply of test kits, the lack of field specialists in the health care organizations, and the fast increase in the number of diseased people indicate the need for an automated screening approach, which can be utilized as an additional aid for specialists to rapidly detect the disease, particularly among people who might need fast medical treatment.

Given the quick ratio of scientific development and the utilization of medical data gathered by a huge number of individuals infected by COVID-19, researchers can analyze the available data to present potential solutions that can help in treating and detecting the disease [17]. Still, the demand for high-efficiency computation has commonly been linked with pricy software systems. On the other hand, many research fields have grown considerably without the necessity for costly investment in computational systems such as modeling and simulation [18], social sciences [19], statistical analysis [19], data-intensive programs [19], online design automation [19], and image processing [[20], [21], [22], [23], [24], [25]]. These research fields have presented promising directions for researchers in several contexts and domains.

Nowadays, researchers in data analysis and data science are considerably inspecting health-related data to aid in medical research development. Regarding this, deep learning, data mining, pattern recognition, and machine learning approaches have been adopted to extract related features from image data sets and categorize them for appropriate disease detection and prediction [[26], [27], [28], [29], [30]]. Medical imaging is an ongoing and developing field in which advanced computer-aided algorithms can be utilized in the recognition, diagnosis, and surgical planning of a particular treatment [31]. Severe inflammation of the cavities and bronchioles can be detected in COVID-19 infected people; this is caused by the harm caused to their lungs. The indications obtained using X-ray, Lung Ultrasound (US), and Computed Tomography (CT) images are essential in reaching an appropriate medical diagnosis. With the utilization of a suitable approach, researchers can present considerable aid in the inspection of the COVID-19 available image datasets and demonstrate logical results that can help in fighting the disease.

Although image processing has been utilized in medical researches and gets the attention of researchers in numerous fields [32,33], in the context of COVID-19 diagnosis and detection research, image processing is still in its earlier phases. Besides, although the preliminary studies have indicated hopeful outcomes by utilizing imaging modalities for the diagnosis of COVID-19, there has been no previous work to systematically revise and synthesize this area of research, to present a plain perception of image processing for scholars and medical experts.

In order to fill this gap, this study conducted a systematic literature review (SLR) approach to define several articles in previous literature related to the detection and diagnosis of COVID-19 using modalities of medical imaging. SLR presents a means for the assessment and synthesis of the current research which is related to a particular field, addressing a particular question, or exploring a phenomenon that attracts researchers’ interest [34]. Additionally, this paper investigates the adopted methods used for the detection of COVID-19 based on image processing techniques. This study also presents a conceptual map to describe the research themes related to data gathering, processing, and evaluation in the surveyed studies. The rest of this research is systematized as follows. Section “Methodology of research” describes the research methodology applied in this SLR. Section “Data synthesis” presents the synthesis of the studies and reveals a conceptual map to elaborate on research themes in the surveyed studies. Section “Discussion” discusses the outcomes. Section “Limitation and future research direction”, presents the limitations and future work. Finally, the conclusion of this study is presented in Section 6. To simplify, a list of the abbreviations used in this study is presented in Table 1 .

Table 1.

List of abbreviations.

Abbreviation	Term	Specification
AI	Artificial intelligence	–
A-lines	Arterial line	–
APLE	Automatic pleural line extraction	Feature extraction tecnique
AR	Association rule	Learning tecnique
BE	Bagging ensemble	Optimization technique
BGWO	Binary grey wolf optimization	Optimization technique
CNN	Convolutional neural network	–
CSDB	Channel-shuffled dual-branched	–
CSDB-DFL	Channel-shuffled dual-branched with distinctive filter learning	–
CSSA	Chaotic salp swarm algorithm	Optimization technique
CT	Computed tomography	Imaging modalities
DBSN	Deep Bayes-SqueezeNet	Deep learning model
DL	Deep learning	–
DNN	Deep neural network	–
DT	Decision tree	Classification technique
DTL	Deep transfer learning	Deep learning model
ELM	Extreme learning machine	Classification technique
FE	Feature extraction	–
FO-MPA	Fractional-order and marine predators algorithm	Feature SelectionTecnique
GLCM	Gray level co-occurrence matrix	–
GLRLM	Gray level run length matrix	–
KNN	K-nearest neighbour	Classification technique
LBGLCM	Local binary gray level co-occurrence matrix	–
LD	Linear discriminant	Classification technique
LSTM	Long short term memory	Classification technique
ML	Machine learning	–
MPA	Marine predators algorithm	Feature SelectionTecnique
MRFO	Manta ray foraging optimization	Feature SelectionTecnique
MVBCE	Majority voting based classifier ensemble	Classification technique
NB	Naïve bayes	Classification technique
NCR	Non-convex regularization	Optimization technique
NN	Neural-networks	–
OS-ELM	Online sequential extreme learning machine	Learning tecnique
PARL	Prior-attention residual learning	Classification technique
PR	Pattern recognition	–
QA	Quality assessment	–
RBDR	Ranking-based diversity reduction	Optimization technique
RELBP	Residual exemplar local binary pattern	Feature extraction tecnique
ResNet	Relative traditional residual network	Deep learning model
RT-PCR	Reverse transcriptase-polymerase chain reaction	–
SF	Shrunken features	Classification technique
SFTA	Segmentation-based fractal texture analysis	Feature extraction tecnique
SLR	Systematic literature review	–
SSN	Semi-supervised network	–
SVM	Support vector machine	Classification technique
TL	Transfer learning	Classification technique
TRT	Training radiology technologists	–
TVUS	Transvaginal ultrasound	Imaging modalities
US	Ultrasound	Imaging modalities

Methodology of research

Research protocol

Systematic reviews start by defining a review protocol that considers the research objective being addressed and the methods to perform the study [34]. In this research, we conducted a SLR to investigate the published studies in the context of COVID-19 and image processing, as a potential approach to disease diagnosis. The research aims particularly to explore the adopted techniques and the utilized imaging modalities in the surveyed studies. To achieve the research objective, we employed a multi-disciplinary analysis of the surveyed studies by assessing and including the articles relevant to the study context. The review protocol of this study is presented in Fig. 1 .

Fig. 1.

The review protocol.

Search keywords and search process

We conducted a review of eight electronic databases: Elsevier, IEEE, PubMed, Wiley Online Library, Springer, Summon, Google Scholar, and Taylor and Francis to locate the researches that meet the research objective using predetermined keywords. In previous studies of systematic literature review, several and various electronic databases were used to retrieve the studies. The chosen electronic resources have been used in conducting systematic literature reviews in several disciplines and contexts [[35], [36], [37], [38], [39]]. The following search keywords were used to download the relevant articles: “COVID-19” OR “Coronavirus” OR “SARS-CoV-2” AND “Pandemic” OR “Crisis” OR “Disease” AND “Medical Image Analysis” OR “Medical image processing”. These studies are related to the application of image processing for the SARS-CoV-2 diagnosis. Considering the novelty and freshness of the research topic, no previous SLR was conducted in this area. To meet a certain quality level in the surveyed studies, the authors decided not to include case reports, case series, and preprints. The search was enlarged by utilizing a snowballing approach by inspecting the available references in the surveyed studies.

Identification of the inclusion and exclusion criteria

Research abstracts were read carefully to specify the studies that will be included in the SLR. Each study was checked in terms of the inclusion and exclusion conditions to decide to keep it for the next stages or not. The inclusion criteria are (1) area of the study: the study should fall within the image processing area, particularly, related to the COVID-19 crisis; (2) method: the method applied in image processing related to the diagnosis of the COVID-19; (3) complete studies and (4) English-language studies. The exclusion criteria are: (1) another domain rather than the research topic, (2) duplicated studies: we found many studies in which the study can be found in more than one electronic database; (3) preprint studies, commentary, or letter to the editor were excluded from the SLR; (4) uncompleted studies, and (5) non-English studies.

Quality assessment (QA)

This study referred to Kitchenham’s guidelines for conducting a systematic literature review. Hence, each included study should be evaluated, referring to some predefined quality criteria. Thus, we created a checklist to enable the evaluation of the research studies after fully reading the included studies. The quality assessment criteria were proposed to help the researcher to meet the research objective [34]. The following conditions were explored in the quality evaluation procedure:

QA1. Is the subject considered in the research related directly to image processing and COVID-19 diagnosis?

QA2. Is the research approach elaborated in the research?

QA3. Are the data gathering methods and the adopted dataset clarified in the research?

QA4. Are the analysis methods of the data elaborated in the research?

The four QA criteria presented above were applied to the 73 studies to strengthen our trust in the surveyed studies’ reliability. The authors utilized three degrees of quality to assess the articles as follows: high, medium, and low [40,41], in which the overall quality of each study depends on the sum of the calculated outcomes for all QA criteria. Studies that fulfill the criteria will be granted 2; studies that moderately fulfill the criteria will be granted 1; studies that do not outline the criteria will be given 0. Studies that have a score of 5 or more will be considered high, if they have a score of 4 they will be considered average, and if they get less than 4, they will be regarded as low quality and removed from the SLR. Following the adoption of the QA, 63 studies were kept because they fulfill the QA criteria.

Data extraction

The authors performed manual data extraction focusing on specific items to understand the research approach, research goal, research outcomes, and synthesize the outcomes in the surveyed studies. The gathering and classification of the research allowed us to get various, critical, and accurate findings. Table 2 presents extracted items in the final studies.

Table 2.

Data extraction Items in the final studies.

No.	Extracted data	Description
1	Authors	A list of the authors of the study
2	Title	The title that is presented in the study
3	Electronic source	The electronic database that is used to retrieve the study
4	Imaging modality	The imaging modality that is explored in the study.
5	Method	The basic method that is used in the study.
6	Dataset	The dataset that is used in the study.
7	Result	The findings of the study.

Distribution of studies by electronic database

This study included eight electronic databases to retrieve the studies. The distribution of the surveyed studies by electronic resources is presented in Fig. 2 . The majority of studies were published in Springer and Elsevier, with a percentage of 25.40%, each. Followed by IEEE and Summon with 19.05% of the surveyed databases, each. PubMed was ranked next with around 4.8% of the surveyed databases. Whereas 2 studies (3.17%) were published in Willey Online Library. Finally, each of Summon, Tylor & Francis has only one published study that is included in the surveyed studies.

Fig. 2.

Distribution of studies by electronic database.

Data synthesis

Keyword co-occurrence network

In bibliometric studies, two map classes can be utilized [42]; distance-based and graph-based diagrams. A distance-based keyword diagram sets up the distance between two keywords. The distance identifies the intensity of the link between the keywords. In this study, we utilized the VOSviewer program to measure distance-based keyword segmentation. A visualization of the keyword co-occurrence network is presented in Fig. 3 . The distance among the chosen items is developed by calculating how many studies that entail both items (which in our study indicate the keywords presented in the study). A huge number of co-occurrences is reflected by a short link among the chosen items. That distance is indicated in the co-occurrence graph and utilized to present the segments. Besides, bigger circles indicate more occurrences of the studies, hence, as the figure presents, “COVID-19” and “deep learning” are the most frequent keywords in the selected studies.

Fig. 3.

Keyword co-occurrence network visualization.

Term co-occurrence network

Additionally, based on the text data of the abstracts and titles of the studies, a term-co occurrence map was presented. The program was set up to ignore both structured abstract labels and copyright statements. Binary counting was used and the minimum number of occurrences among the selected studies was 3, hence, 236 terms met the pre-determined condition. For each of the 236 terms, a relevance score was measured. Finally, based on the relevance score, the most relevant items were chosen, hence, 142 items were kept. A visualization of the term co-occurrence network based on text data of the surveyed studies is presented in Fig. 4 . In the figure, three clusters of items were generated, each color represents a particular cluster of items. The red cluster focused on the applied technique and the evaluation measures of these techniques. This cluster includes terms like “X-ray image”, “sensitivity”, and “specificity”. The green cluster concentrated on the approaches used in image processing. As it can be seen in the figure, terms like: “screening”, “extraction”, and “ground-glass opacity” are included in this cluster. The blue cluster focused on the virus and the pandemic in general. Terms like: “world health organization”, “contribution”, “review”, “application”, and “survey” are included in this cluster. The relevance scores of the items are presented in Appendix C.

Fig. 4.

A visualization of term co-occurrence network based on text data.

Co-authorship network

Additionally, co-authorship-countries associations between authors are presented in Fig. 5 . The paths exhibit the number of co-authorship associations between a particular author with other authors. The path strength indicates the strength of the co-authorship association of a particular author with other authors. We set the program to the full counting method. In this method, if the strength of any path in the diagram is represented by A, this means that the two have co-authored A of researches. The total link strength ranges from 0 to 13, in which 14 countries have a total link strength more than three. This reflects the firm cooperation among several researchers in several countries in the study topic.

Fig. 5.

A visualization of co-authorship-countries network.

A conceptual framework of the surveyed studies

This study is based on an in-depth literature review, giving an overview of the research area to explore the steps involved in the image processing starting from data gathering until COVID-19 diagnosis. Hence, we examine these steps and highlight the basic themes and notions involved in each step. Besides, this study presents a conceptual map that can guide scholars for the future and possible research directions. The three basic folds that arise are presented in Fig. 6 : (1) data gathering and description, (2) the main steps of image processing, and (3) the evaluation metrics. We will outline each fold and its basic themes in which we see possible future research directions and address potential research gaps. In Fig. 7 we present a mind map that includes: (1) a description of research themes related to resources in the surveyed studies, (2) a description of research themes and notions related to the steps involved in COVID-19 diagnosis, and (3) a brief description of each of the main evaluation indicators related to the surveyed studies. We will elaborate more on these figures in the “Discussion” Section.

Fig. 6.

Conceptual map for research themes and notions.

Fig. 7.

Mind map for research themes and notions.

Discussion

COVID-19 tests reverse transcriptase-polymerase chain reaction (RT-PCR) are utilized broadly to detect the disease in regions where the crisis is spreading [43]. RT-PCR can detect viral nucleotides from specimens gathered by nasopharyngeal swab, oropharyngeal swab, tracheal aspirate, or bronchoalveolar lavage. Still, these tests are restricted in terms of accuracy, speed, cost, and supply [44,45]. Several COVID-19 cases could not be specified because of the low accuracy of RT-PCR tests. Hence, the RT-PCR test might require to be replicated many times in some cases to confirm the results [45]. Another obstacle faced by health organizations is the speed of obtaining the outcomes of the test, which usually ranges from hours to days [46,47]. The lack of accurate and fast disease detection approach can cause patients to spread the disease to the community without knowing that they are infected [48]. Furthermore, patients who require urgent medical care might not get suitable therapy at the right time. Additionally, RT-PCR exposes healthcare employees to a greater risk of being infected through the test. Another restriction is the cost of the supply of substances utilized in the testing kits. Although the cost differs from region to region over the world, the test kit price might reach around 60$ [49]. The cost depends also on the availability of the tests, which relies on the resources and population of the country. All these aspects indicate that other detection approaches need to be adopted to substitute RT- PCR test. Hence, there is a demand to explore other symptomatic approaches for detecting and diagnosis COVID-19. Other methods, which utilize imaging modalities for detecting the disease, can be applied as a replacement to the RT-PCR test for the detection of COVID-19 [[50], [51], [52], [53], [54], [55], [56]].

There are three main folds to present regarding the topic of the study: image processing modalities, COVID-19 diagnosis approaches, and steps of COVID-19 diagnosis.

Image processing modalities

Previous literature indicated that COVID-19 induces irregularity in human lungs which appears in the chest X-rays and CT images. This malformation can be in the form of ground-glass opacities. Besides, the general indication in all medical symptoms related to lung ultrasonography is the existence of various line artifacts. This entails the presence of A-lines and B-lines, in which the diagnosis of these lines is highly important. B-lines usually exist in the lung with interstitial edema. The existence or lack of these lines, the kind, and the number of B-lines in chest US can be utilized as an indication of COVID-19 disease. A dense and abnormality-shaped pleural line, along with the presence of focal, varifocal, and confluent vertical B-lines are vital signs of the COVID-19. On the other hand, the presence of A-lines is important evidence in the recovery stage [57].

People’s lives could be protected, the increase of the illness might be restrained, and large data could be obtained from artificial intelligence patterns with the fast and accurate detection of COVID-19 [44]. To achieve this goal, health care specialists can utilize X-rays, CT scans, and US imaging modalities in their clinical studies. COVID-19 traditional examinations have possible disadvantages represented by the lack of supply and high expenses of tests [46,47]. On the other hand, all emergency health centers have X-Rays and CT devices. Digital medical images have several kinds that were utilized in the surveyed studies. These types differ from each other in terms of how they are made and the task they perform. Datasets and imaging modalities in the surveyed studies are presented in Appendix B. The main types of imaging modalities explored in the surveyed studies are:

• i.X-ray: is a kind of radiation called electromagnetic waves. X-ray radiology contains radiant X-ray photons that move through body organs [58]. Based on the kind of body tissue, it allows or reduces the amount of the passed photons, leading to the generation of an image with a range of black and white shadows. This occurs because various tissues consume various quantities of radiation. X-rays penetrate body organs to generate a 2-D photo of the internal human body. Bones consume X-rays the highest, so they appear in the image in white, while other tissues appear in gray, and finally, the lung appears in black because the air consumes the least radiation. In the surveyed studies, the X-ray modality was explored in 39 studies.
• ii.Computed Tomography (CT) utilizes X-rays to generate specific pictures of the internal organs. These pictures can be used to produce 3-D photos. CT can generate pictures of the internal structure of the body; entailing organs, blood vessels, and bones. CT has been explored in many health studies in many contexts as it can aid in disease diagnosis. While X-ray radiology is utilized to inspect the bone structure and assess lung diseases, CT adopts the concept of X-ray by taking X-ray photos from various angles to generate cross-sectional images without dissecting the body. These photos are also named slices and entail more information than the traditional X-rays radiography. CT photos are adopted to indicate malformation in the organs, particularly in the lungs. This makes CT photos suitable for the diagnosis of COVID-19, as it attacks the lungs and the respiratory system. In the surveyed studies, CT modality was explored in 27 studies, as presented in Fig. 8 .
• iii.The ultrasound image is a kind of sonography imaging that can be generated using ultrasonic devices. They can be used in health applications because of their reasonable cost and their ability to generate real-time photos with high quality. US images have been utilized in medical applications because they capture body organs’ images using high-frequency sound waves. In the surveyed studies, US images were explored in three studies.

Fig. 8.

Distribution of studies by imaging modality.

COVID-19 diagnosis approaches using image processing

Artificial intelligence (AI) has been utilized in many aspects such as image classification, big data analysis, disease diagnosis [59]. AI approaches have been broadly adopted for speeding up the advancement of medical and biological researches [60]. ML, DL, and PR approaches mainly concentrate on finding solutions and reaching suitable decisions regarding current and emergent problems. In general, in such systems, datasets are pre-processed, segmented, the system is trained, tested, and then new data can be classified. Particularly, in a training step, the input data is pre-processed. Following that, significant features are extracted. The preprocessing stage is required to keep the original space of the data and entails procedures to reduce the noise and to rectify the image. As individuals infected by COVID-19 may experience lung inflammation as the virus attacks the lungs, the diagnosis of the COVID-19 through imaging modalities analysis can be effectively applied using AI techniques [61].

DL can be adopted to obtain medical image data to present indicators on the molecular condition, disease diagnosis, disease progress, or medicine sensitivity [62]. The development in image processing research has been utilized by the advancement of DL-underlying methods, allowing the involvement of many images and dealing with image differences. DL is well recognized these days, referring to its performance, especially in image segmentation and classification models [63]. Several kinds of DL approaches were utilized to serve several aims, e.g., object segmenting, classification, disease diagnosis, and speech recognition. In the surveyed studies, the most adopted DL technique is the convolution neural networks (CNN), as it was applied in 40 studies, as presented in Appendix A.

CNN is a significant approach in image processing, which allows accurate classification of pneumonia affected and normal samples when medical images are provided [64]. CNN has been broadly utilized as a promising imaging approach since its original launch [65]. As a kind of deep neural network, CNN has achieved high performance in categorizing and classifying images through the extraction of images’ topological features [66,67]. CNN also adopts a layered perceptron-driven structure that consists of fully connected networks, in which each neuron in one layer is tied to all neurons in other layers. CNN has three kinds of layers, with each type performing a particular function; (1) convolutional, (2) pooling, and (3) fully connected. In this structure, the convolutional layer works to extract the features. Following that, the fully connected layer utilizes the extracted features to determine which class belongs to the existing input. A pooling layer is responsible for minimizing the dimensions of feature maps and network parameters. Although CNN’s presents the best outcomes on huge input data, they need large data and computational resources to train. In the case of limited input data, which might not be adequate to train a CNN from scrape, the performance of CNNs can be leveraged and the computational costs can be reduced by using transfer learning approaches [68].

In the surveyed studies, several architectures of CNN were deployed. Karthik et al. [69] used two architectures of CNN; CSDB and CSDB-DFL. These two architectures provide double residual connectivity among blocks, connected network links, the relation of variably sized receptive fields, and channel-shuffling. Heidari et al. [70] utilized the VGG16 model in a transfer learning methodology regarding its effectiveness in image classification tasks. Turkoglu [71] utilized CNN-based AlexNet architecture in a transfer learning methodology. Compared to VGG16, the proposed AlexNet architecture presented a better performance in terms of classification accuracy. Goel et al. [72] proposed a new architecture named OptCoNet, which was used for feature extraction and classification tasks. The proposed system presented a high performance in terms of accuracy (97.78%). Sahlol et al. [73] proposed a new approach (FO-MPA), in which a CNN was used to perform the feature extraction task, while the Marine Predators Algorithm (MPA) was used for feature selection. Panwar et al. [49] proposed a new model for COVID-19 detection (nCOVnet), in which VGG16 was used for extracting the features. The proposed model was based on transfer learning in the training stage. Pathak et al. [74] deployed the ResNet-50 network for the feature extraction task. The transfer learning approach was used in the COVID-19 detection and presented effective outcomes compared to other related models. Duran-Lopez et al. [57] proposed a new model, entitled COVID-XNet, based on a deep learning approach. The proposed model presented an overall accuracy of 94.43%. Xu et al. [75] proposed a new 3D deep learning approach. In the proposed approach, two classification schemes were used based on the ResNet-18 model and presented an overall accuracy of 86.7%. Bahadur Chandra et al. [76] proposed a new system for COVID-19 diagnosis using radiomic texture descriptors to classify CXR images. Toğaçar et al. [60] deployed two deep learning models for the training process; MobileNetV2 and SqueezeNet. The classification process was performed using SVM tecnique. The overall accuracy of the proposed system was 99.27%, which outperform other state of art approaches. Ismael and Şengür [77] used the ResNet50 model for feature extraction and fine-tuning tasks while SVM was used for the classification process, in which an overall accuracy of 99.29% was acheived. A new model was proposed by Ucar and Korkmaz [78], entitled as COVIDiagnosis-Net. The new system was based on Bayes-SqueezeNet. Loey et al. [79] deployed DTL tecnique through using two models; Googlenet and Alexnet. The deployed tecnique was evaluated in different contexts and presnted promising outcomes. Several deep learning models; VGG19, VGG16, Inception-ResNet-V2, InceptionV3, DenseNet121, Xception, and Resnet50, were used in a comparative study by Shazia et al. [80]. Among the deployed models, DenseNet121 presented the best performance in terms of classification accuracy.

The deep transfer learning (DTL) approach could be used to overcome overfitting and under-fitting issues in small training input data, by taking benefits of a primary trained CNN using large-scale input data [70]. DTL methods can train the weights of networks on big input data and hyper-tuning the weights of pre-trained networks on small input data [81]. Furthermore, by utilizing DTL, the training time can be minimized, mathematical measurements can be reduced, and the intake of the obtainable hardware resources can be minimized [79]. The transfer learning approach was used in several models in the surveyed literature [49,59,[69], [70], [71],73,74,79,[81], [82], [83]].

Usually, the TL method is adopted when the available number of samples for the training step, in models entailing CNN architecture, is inadequate. Still, the transfer of trained features with general images can influence the performance of the deployed model. To overcome this limitation, a robust FE method is required. It is not often suitable to deploy traditional supervised approaches to inspect newly emerged input data entailing an inadequate number of observations. Furthermore, unbalanced input data might be unsuitable for the training process. When it is not feasible to locate additional labeled data, these input data can be efficiently delineated by features. Unbalanced input data issues can be addressed by conducting double-step data replication instead of one-sided data replication to present acceptable outcomes [84].

Steps of COVID-19 diagnosis

In the following subsections, we will present the main procedures that were utilized in the surveyed studies starting from data pre-processing, feature extraction, feature selection, to classification.

Image pre-processing and augmentation

Image pre-processing basically aims to enhance the quality of images included in the dataset; which accordingly enhances the display of particular pathologists and enhances the outcomes of FE and segmentation approaches [85]. The use of image processing approaches entails the management of digital photos by adopting several stages. Pre-processing step improves the quality and size of the obtained dataset. Several procedures can be conducted to achieve the following goals: reducing the noise in the initial images, enhancing the quality through raising the contrast, and removing the high/low frequencies [32]. This entails rifting the insufficient dataset into a substantial one. Several transformations can be applied to the dataset, including scaling, cropping, flipping, filtering, and rotation. Hence, each image can be transformed into a new one, by including required information to promote the following steps. Grayscaling can also be applied to initial datasets because they are usually generated by different types of machines. Hence, a histogram matching procedure can be performed on all samples by considering one sample as a reference to unify the histogram distribution of all samples [57].

The lack of data or unreliable data can influence the effectiveness of ML and DL due to a shortage of sufficient features [86]. The overfitting problem happens when a network learns a task with very high variance such as to perfectly model the training data [70,87]. These models of variation usually entail noise, changes in contrast, rotations, and translations. In biased input, data augmentation can be utilized to enhance the counts of infrequent samples. In data augmentation, general causes of variation are specifically added to training data. Particularly medical image processing researchers face limited accessibility to big data, hence, the success of the system can be linked to the deployed augmentation approach. Data Augmentation comprises a set of approaches that improve the quality and size of training datasets such that better DL systems can be utilized. An example, of that, is the application of non-rigid deformations of input samples and the required segmentation. Data augmentation is usually addressed using generative adversarial networks (GAN) [88]. Several studies considered GAN as a genuine tool to be applied in implementations that need data augmentation [88]. Still, several kinds of GANs can suffer from instability during the training step and are subject to gradient saturation. In the surveyed literature, several studies utilized image augmentation approaches to increase the number of images in the dataset [49,69,75,84,88]. Studies that used augmented datasets indicated the significant improvement of the system’s performance against systems that used initial datasets [76].

Feature extraction approaches

COVID-19 patients present several radiological patterns including patchy ground-glass opacities, pneumonic consolidations, reticulonodular opaqueness [76]. These elusive optical features can be effectively extracted with the aid of a suitable approach. Choosing the best algorithm that enables distinctive and full feature extraction is of great importance. This function is complex to infer, hence, it is essential to plan carefully for every new system. Feature extraction approaches were used to minimize the required time for computation and the complexity of the system, which accordingly enhances the performance of decision-making systems [89]. Effective FE approaches are needed to get better ML models. On the other hand, DL models are broadly applied in medical imaging applications as they can extract features automatically or by utilizing some pre-trained systems such as ResNet [74]. It is a primitive stage to extract features when utilizing medical images before the classification and diagnosis functions. Several methods were used to extract the features in the surveyed studies. We will explain the two main deployed approaches in the surveyed studies:

• •The texture feature approach was used to inspect features and to indicate similarities and patterns of images [90]. In literature this procedure is often called “hand-crafting” and it is utilized to anticipate the right category, which is usually predicted features. It is broadly employed in many medical domains [91]. In this approach, features can be manually designed, or “by hand” aiming to handle particular problems such as variations and occlusions in scale and illumination. The deployment of handcrafted approaches entails deciding the best trade-off between accuracy and computational performance. GLCM, LBGLCM, GLRLM, and SFTA can be mined to categorize pandemic diseases [84]. GLCM is the most utilized approach to extract features regarding its good performance in many disciplines particularly in oncology imaging when the texture features of the images are easily detachable [91,92].
• •DL approaches for FE are getting more attention regarding their high capability of extracting features [93]. Many researchers have adopted DL approaches and achieved outstanding performance in data extraction from images, regarding its capability to discover features on a new dataset on its own, contrasting to previous machine learning algorithms [61,94]. Several techniques were used in the surveyed studies for FE: AlexNet [71], Resnet50 [77,95,96], VGG19 [82,95], VGG16 [49,95], ResExLBP [97], DenseNet20, DenseNet201, Inception_ResNet_V2, Inception_V3, Resnet50, and MobileNet_V2 [95].

Feature selection approaches

Feature Selection (FS) entails choosing a group of related features of an input image and deleting the less fitting ones. This step presents many benefits in terms of minimizing the complexity of the proposed approach, minimizing overfitting, and enhancing accuracy. FS approaches can be categorized into three classes: embedded, filter, and wrapper [98]. The main goal of the FS stage is to establish the feature vector and extract the features with the least knowledge before the final categorization. Hence, a large number of discriminative features can be chosen using the deployed approach.

In the surveyed studies, only a few studies applied FS techniques. A study by Elaziz et al. [68] presented a new FS technique based on enhancing the behavior of MRFO using differential evolution. Another study by Bahadur Chandra et al. [76] used the BGWO approach, which mimics the leadership, hunting, and encircling procedure of wolves. This approach can overcome the problem of getting trapped in local minima. The relief algorithm, which is basically based on the KNN algorithm, was used as a feature selection approach that can enhance the diagnosis performance [71]. The main rule of the relief algorithm is to choose features by measuring the proxy statistical value. The relief algorithm was used in a study by Turkoglu [71] and Novitasari et al. [99] to extract the features effectively. Tuncer et al. [97] adopted an improved version of the Relief algorithm, which used Manhattan distance to produce weights instead of Euclidean distance.

Classification

Classification, which is also called Computer-Aided Diagnosis (CAD), has a vital part in medical image processing. The classification process takes sample images as an input and provides the diagnosis variable as an output [100]. DL in Computer-Aided Diagnosis was applied for the first time by Lo et al. [101] to categorize lung X-ray images and has been adopted in several studies since then. As the sample images are provided to the CNN, the classification process is performed by disclosing the content of the image. Following that, image localization is performed, in which bounding boxes are placed around the output position. This process aims to locate the disease in the image. The localization process is essential to locate some basic disease features [86].

Limitation and future research directions

This section presents potential routes for future studies that can address COVID-19 diagnosis and detection using medical image modalities. Despite the encouraging outcomes of the revised studies, there are some restrictions and obstacles that should be addressed about the utilized approaches for COVID-19 diagnosis. The most important obstacle we noticed in the surveyed studies was the need for an extensive training dataset. This is a general problem in training deep learning models for images in a medical-related context [63]. DL needs sufficient training data as the DL model’s achievement depends on the volume and quality of the input. It is unavoidable that the preliminary open accessible medical images for new medical cases such as COVID-19 will be limited in quantity and insufficient in quality aspects. Still, the shortage of data is a fundamental restriction to DL’s achievement in medical image processing generally. Additionally, building sufficient medical imaging data is complex because data annotation needs time, work, and cooperation from several professionals to avoid mistakes. Many studies augmented the original training to lower the selection bias and increase the diversity of the dataset [102]. Augmentation of data can be utilized to enhance the success of the adopted models [103]. Future studies are thus required to improve the augmentation approaches for developing advanced models and tolerating the lack of data. The second obstacle we faced while conducting this SLR was that each study has utilized a different resource to construct the dataset. Hence, the evaluation of the performance of the adopted models, among several studies, is pretty complex to check and compare. One approach to handle this issue is by presenting comparative studies, in which several models can be evaluated using the same data set and evaluation metrics.

Although the inspected studies have utilized several techniques for COVID-19 diagnosis, these approaches did not consider the incremental updates of the data for the classification process. Incremental learning has become significant with the provision of huge volumes of data that are generated from emerging technologies [[104], [105], [106]]. As the embedded patterns in big data could be dynamic, the proposed classification method should have the ability to learn incrementally in order to update the current patterns with the accumulation of new data [107]. Particularly, incremental learning differs from conventional methodologies in data accumulation and ensemble learning [108]. In the conventional methodologies, ML does not enable the model to update the produced classifier [109]. Hence, the model will be restricted and can not recognize new divergent images. In general, traditional supervised approaches cannot be utilized for incremental learning and need to recompute all the training data to build prediction models. As computation time and accuracy are two important criteria for the assessment of the medical diagnosis systems, incremental learning can enhance the predictive accuracy and reduce the computation time of COVID-19 diagnosis. Hence, future research direction can be followed by researchers to utilize incremental approaches to improve the performance of COVID-19 diagnosis.

Several other approaches can be used in the classification process, such as ensemble learning. Ensemble learning can aid to enhance the machine learning outcomes. As it can present significant generalization performance, it has gained researchers’ attention [110]. To improve the efficiency of the generalization of previous single classifiers, ensemble learning integrates two or more ML methods. Ensemble learning has been used in previous literature in image processing studies in various contexts [[111], [112], [113]].

Considering the collection of studies to be included in this literature review, the databases we choose have been used in several systematic literature review studies and presented robust outcomes. However, referring to the limitation of the study, it is important to mention that although we tried to cover several resources of data, utilizing other databases can provide broader coverage of the research area. Other databases could be used in future work.

Conclusion

COVID-19 is a worldwide issue that should be faced by researchers’ efforts in all scientific means. The COVID-19 crisis has joined researchers from several disciplines together to fight against the spread and the impact of the disease. AI applications depend on the quality of the input data to achieve high performance. The generalizability of the outcomes is also a significant aspect when deploying AI applications, which can be achieved if more data are available for researchers.

Considering the current advent of the COVID-19 crisis, which has put healthcare services under critical conditions over the world, there is a huge need for adequate large image datasets. Hence, to address the data scarcity issue, researchers have adopted data augmentation techniques, which have shown huge possibilities in several disciplines, entailing medical imaging [114]. Particularly, DL models need a huge volume of training data with clear labels. Still, most images are manually annotated, which indicates a time-consuming procedure that requires expert knowledge [115]. One way to address this issue is by enabling medical image sharing between different research and health centers over the world, which holds great promises for COVID-19 diagnosis. With this in mind, the post-COVID-19 stage may face more data-sharing among various national and international organizations. This will enable AI researchers to present applications that can provide accurate outcomes.

Medical image processing is a well-recognized method that could be useful in the identification of COVID-19. The results of this SLR indicated that researchers have adopted several approaches to classify the images related to COVID-19 diagnosis and detection, and these methods have presented promising outcomes in terms of the accuracy, cost, and speed of the detection. This review focused on imaging modalities, approaches, and procedures, that were utilized in the surveyed studies aiming to present upcoming directions for future research.

Funding

No funding sources.

Competing interests

None declared.

Ethical approval

Not required.

Appendix A. The related studies

No.	Author	Method
		DNN	CNN	MPA	RELBP	DTL	APLE	SF	DL	DBSN	SSN	NCR	AR	NN	PARL	SVM	MVBCE	RBDR	DT	KNN	NB	LD	TRT	LSTM	CSSA	ELM	BE	OS-ELM
1.	Ni et al. [61]	√							√
2.	Xu et al. [75]		√						√
3.	Abdel-Basset et al. [118]			√														√
4.	Abdani et al. [117]		√						√
5.	Moustafa and El-Seddek [96]		√						√
6.	Tuncer et al. [97]				√											√			√	√		√
7.	Panwar et al. [49]		√			√
8.	Das et al. [81]					√
9.	Carrer et al. [124]						√
10.	Karar et al. [135]		√						√
11.	Öztürk et al. [84]							√
12.	Ismael and Şengür [77]		√						√
13.	Elasnaoui and Chawki [95]		√
14.	Hu et al. [132]		√					√
15.	Loey et al. [79]		√			√		√
16.	Toğaçar et al. [60]		√						√
17.	Che Azemin et al. [125]		√
18.	Horry et al. [82]		√			√
19.	Pereira et al. [83]		√			√
20.	Sahlol et al. [73]		√	√		√
21.	Turkoglu [71]		√			√
22.	Ucar and Korkmaz [78]		√							√
23.	Oh et al. [59]		√			√
24.	Duran-Lopez et al. [57]		√
25.	Civit-Masot et al. [94]		√
26.	Yoo et al. [103]		√
27.	Pathak et al. [74]		√			√
28.	Sedik et al. [88]		√
29.	Novitasari et al. [99]		√													√
30.	Karakuş et al. [134]											√
31.	Anastasopoulos et al. [102]		√
32.	Suman et al. [148]																						√
33.	Hassantabar et al. [131]	√	√
34.	Kang et al. [92]													√
35.	Heidari et al. [70]		√
36.	Karthik et al. [69]		√			√
37.	Elaziz et al. [68]		√
38.	Goel et al. [72]		√
39.	Wang et al. [149]														√
40.	Altan and Karasu [119]								√																√
41.	Mohammed et al. [85]										√													√
42.	Singh et al. [146]		√						√
43.	Bahadur Chandra et al. [76]													√		√	√		√		√
44.	Das et al. [126]		√
45.	Dey et al. [127]																			√
46.	Singh et al. [147]		√						√							√										√	√	√
47.	Fung et al. [129]								√
48.	Shazia et al. [80]					√			√
49.	Kugunavar and Prabhakar [139]		√
50.	Garain et al. [130]													√
51.	Niu et al. [144]					√
52.	Xie et al. [151]								√
53.	Madan et al. [141]		√
54.	Manav et al. [142]					√			√
55.	Rahimzadeh et al. [145]								√
56.	Khan [137]		√													√
57.	Junior et al. [133]		√
58.	Arora et al. [122]		√
59.	Kaushik et al. [136]					√			√
60.	Zhang et al. [153]								√
61.	Khanna et al. [138]		√			√			√															√
62.	Baz et al. [123]		√
63.	Müller et al. [143]		√						√

Appendix B. Datasets and imaging modalities in the surveyed studies

No.	Ref.	Imaging modalities	COVID-19 Dataset
1.	Ni et al. [61]	CT images	•Gathered by the authors from hospitals
2.	Xu et al. [75]	CT images	•Gathered by the authors from three hospitals
3.	Abdel-Basset et al. [118]	X-ray	•Github
4.	Abdani et al. [117]	X-ray	•Radiology Society of North America (RSNA)
			•SIRM COVID-19 database
			•Novel corona virus 2019 dataset
			•COVID-19 positive chest X-ray images from different articles
			•COVID-19 chest imaging at thread reader articles
			•Chest X-ray images
5.	Moustafa and El-Seddek [96]	X-ray	•Cohen’s dataset
6.	Tuncer et al. [97]	X-ray	•Github
7.	Panwar et al. [49]	X-ray	•ImageNet dataset
8.	Das et al. [81]	X-ray	•Three class chest X-ray datasets
9.	Carrer et al. [124]	US	•A subset of the Italian COVID-19 lung ultrasound database
10.	Karar et al. [135]	X-ray	•Cohen’s dataset
11.	Öztürk et al. [84]	X-ray, CT	•Gathered by the authors
12.	Ismael and Şengür [77]	X-ray	•Github
			•Kaggle
13.	Elasnaoui and Chawki [95]	X-ray	•Previous literature
			•Covid chest X-ray
14.	Hu et al. [132]	CT images	•Cancer imaging archive (TCIA) public access
15.	Loey et al. [79]	X-ray	•Cohen dataset
			•The pneumonia dataset chest X-ray images, normal
16.	Bahadur Chandra et al. [76]	X-ray	•COVID-ChestXray set
			•Montgomery set
			•NIH ChestX-ray14 set
17.	Che Azemin et al. [125]	X-ray	•ChestX-ray14
			•COVID-19 X-ray data set
18.	Horry et al. [82]	X-ray, CT, US	•X-rays were obtained from the publicly accessible COVID-19 image
19.	Toğaçar et al. [60]	X-ray	•Cohen dataset
			•Kaggle website
			•Collected from patients
20.	Pereira et al. [83]	X-ray	•Cohen dataset
			•Radiopedia encyclopedia
			•NIH dataset
21.	Sahlol et al. [73]	X-ray	•Cohen and Paul Morrison and Lan Dao
			•Italian cardiothoracic radiologist
			•Kaggle website
			•Italian Society of Medical and Interventional Radiology (SIRM)
22.	Turkoglu [71]	X-ray	•Github website
			•Kaggle website
23.	Ucar and Korkmaz [78]	X-ray	•Cohen dataset
			•Kaggle chest
24.	Duran-Lopez et al. [57]	X-ray	•BIMCV-COVID19+ dataset
			•Cohen dataset
			•PadChest dataset by BIMCV
25.	Singh et al. [146]	X-ray	•Data set based on previous literature
26.	Oh et al. [59]	X-ray	•JSRT/SCR dataset
			•NLM(MC)
			•Cohen dataset
			•CoronaHack
27.	Civit-Masot et al. [94]	X-ray	•X-ray images from GitHub
28.	Yoo et al. [103]	X-ray	•Shenzhen set
			•NIH (National Institutes of Health, US) Clinical Center
			•Hospital dataset
			•COVID-chest Xray-dataset
29.	Pathak et al. [74]	CT images	•Data set based on previous literature
30.	Sedik et al. [88]	X-ray, CT images	•Hitherto dataset
31.	Novitasari et al. [99]	X-ray	•Github
			•Kaggle
32	Karakuş et al. [134]	US	•Nine COVID-19 patients
33.	Anastasopoulos et al. [102]	CT images	•COVID-19 patient dataset
			•CT datasets of patients with different medical histories other than COVID-19
34.	Suman et al. [148]	CT images	•Radiologists’ review [128]
35.	Hassantabar et al. [131]	X-ray	•COVID-CT-dataset [155]
			•COVID-SemiSeg dataset [116,139]
36.	Kang et al. [92]	CT images	•Datasets by three hospitals and collaborators:
37.	Heidari et al. [70]	X-ray	•Allen Institute for AI in partnership
			•Chan Zuckerberg initiative
			•Georgetown University’s Center for Security and Emerging Technology
			•Microsoft research
			•National library of medicine
			•National Institutes of Health
			•The White House Office of Science and Technology Policy
38.	Karthik et al. [69]	X-ray	•Joseph Cohen dataset
			•Radiopaedia
			•AG Chung dataset
			•ActualMed dataset SIRM
			•RSNA challenge
			•Paul Mooney dataset
39.	Elaziz et al. [68]	X-ray	•Cohen dataset
			•Team of researchers from Qatar University, Doha, Qatar, and the University of Dhaka, Bangladesh, along with their collaborators
40.	Goel et al. [72]	X-ray	•Provincial peoples hospital
			•Kaggle
41.	Wang et al. [149]	CT images	•CT scans of 4657 patients were collected from several hospitals
			•LUNA-16 dataset
42.	Altan and Karasu [119]	X-ray	•Gathered by the authors
43.	Mohammed et al. [85]	CT images	•Cohen dataset
44.	Das et al. [126]	X-ray	•COVID-19 collection
			•Kaggle CXR collection
			•Two publicly available Tuberculosis (TB) collections
45.	Dey et al. [127]	CT images	•COVID-19 CT segmentation dataset
46.	Singh et al. [147]	CT images	•COVID-19 collection
			•Italian society of medical and interventional research (60 COVID-19 CT scans)
			•A CT scan dataset of 617 COVID and non-COVID images
47.	Fung et al. [129]	CT images	•Integrative resource of lung CT images and clinical features
			•Med-Seg (medical segmentation) COVID19 dataset
48.	Shazia et al. [80]	X-ray	•7165 chest X-ray images of COVID-19 (1536) and pneumonia (5629) patients
49.	Kugunavar and Prabhakar [139]	CT images	•COVID-19 CT images
50.	Garain et al. [130]	CT images	•COVID-19 CT images
51.	Niu et al. [144]	CT images	•Office-31
		X-ray	•Caltech-256
			•Chest X-ray image
			•Lung CT
			•Covid19-CT
52.	Xie et al. [151]	CT images	•A public dataset contains CT images of more than 40 COVID-19 patients
53.	Madan et al. [141]	CT images	•2482 CT images
54.	Manav et al. [142]	X-ray	•Kaggle database
55.	Rahimzadeh et al. [145]	CT images	•48,260 CT scan images (normal)
			•15,589 images (infected)
56.	Khan [137]	X-ray	•340 X-ray images
57.	Junior et al. [133]	X-ray	•227 X-ray images
58.	Arora et al. [122]	X-ray	•5840 X-ray images
		CT images	•Kaggle database
59.	Kaushik et al. [136]	CT images	•SARS-CoV-2 CT-scan dataset [121]
60.	Zhang et al. [153]	CT images	•148 COVID-19 patients and 148 healthy control subjects [154]
61.	Khanna et al. [138]	X-ray	•Three open access datasets
		CT images	•One private customized dataset
62.	Baz et al. [123]	CT images	•Kaggle datasets [150,152]
		CXR scans
63.	Müller et al. [143]	CT images	•Two public datasets [120,140]

Appendix C. The relevance score of the items

The term	Occurrences	Relevance score
Development	7	0.4982
Identification	6	0.3322
Infected patient	6	0.6354
CT scan	6	0.9406
Application	5	0.5307
Chest	5	0.6295
Availability	5	0.7282
Challenge	4	0.359
Effectiveness	4	0.4265
Clinician	4	0.5798
Hospital	4	0.6364
China	4	0.7859
Computed tomography	4	0.8326
F1 score	4	0.9414
Country	4	0.966
CNN model	4	1.0171
CXR	4	1.2212
Experiment	4	4.0463
Average accuracy	3	0.5206
Auc	3	0.6694
Combination	3	0.7014
Artificial intelligence	3	0.779
Curve	3	1.2174
CXR image	3	1.3426
Clinical trial	3	1.8487
Coronavirus pneumonia	3	2.8663
COVID-19	3	5.3131
Abnormality	3	0.7001
36 large number	3	1.1211
37 layer	5	1.0512
38 lung	7	0.2469
39 novel coronavirus disease	4	1.0644
40 order	5	0.5125
41 outbreak	6	0.4311
42 pandemic	11	0.1988
43 person	5	0.8363
44 pneumonia patient	3	1.3492
45 pre	5	0.7848
46 precision	3	1.7955
47 problem	5	1.0945
48 proposed approach	3	1.239
49 research	6	0.7289
50 researcher	3	0.8186
51 resource	4	0.7872
52 rt pcr	3	1.114
53 score	4	0.963
54 severity	3	0.9426
55 specificity	6	0.3328
56 spread	7	0.4935
57 step	5	1.4488
58 strength	3	0.8021
59 support vector machine	5	1.8943
60 svm	4	1.6393
61 testing	6	0.706
62 tomography	3	1.1936
63 tool	10	0.5454
64 transfer learning	5	0.8498
65 treatment	7	0.2921
66 validation	4	0.9968
67 virus	9	0.4709
68 world	6	0.5198
69 X ray dataset	4	1.3707