COVID-19 classification using deep feature concatenation technique

Abstract

Detecting COVID-19 from medical images is a challenging task that has excited scientists around the world. COVID-19 started in China in 2019, and it is still spreading even now. Chest X-ray and Computed Tomography (CT) scan are the most important imaging techniques for diagnosing COVID-19. All researchers are looking for effective solutions and fast treatment methods for this epidemic. To reduce the need for medical experts, fast and accurate automated detection techniques are introduced. Deep learning convolution neural network (DL-CNN) technologies are showing remarkable results for detecting cases of COVID-19. In this paper, deep feature concatenation (DFC) mechanism is utilized in two different ways. In the first one, DFC links deep features extracted from X-ray and CT scan using a simple proposed CNN. The other way depends on DFC to combine features extracted from either X-ray or CT scan using the proposed CNN architecture and two modern pre-trained CNNs: ResNet and GoogleNet. The DFC mechanism is applied to form a definitive classification descriptor. The proposed CNN architecture consists of three deep layers to overcome the problem of large time consumption. For each image type, the proposed CNN performance is studied using different optimization algorithms and different values for the maximum number of epochs, the learning rate (LR), and mini-batch (M-B) size. Experiments have demonstrated the superiority of the proposed approach compared to other modern and state-of-the-art methodologies in terms of accuracy, precision, recall and f_score.

Introduction

The novel COVID-19 disease has appeared in China in Wuhan city (Zhu et al. 2020). Then, it has become a pandemic, and it has rapidly spread through the world. It has had destructive effects on public health, daily life, and global economy (Singhal 2020). The symptoms of this disease include cough, sore throat, runny nose, fever, headache, and lung infection (Huang et al. 2020). Pneumonia is one of the main effects of COVID-19, which must be promptly diagnosed. Thus, rapid diagnostic testing of COVID-19 has become a major challenge. The laboratory-testing Real Time Polymerase Chain Reaction (RT-PCR) is the most famous current diagnostic tool. However, it is expensive (in some countries), and time-consuming (Xie et al. 2020a). Nowadays, medical image processing has a vital and influential role in diagnosing and detecting the disease (Xie et al. 2020b; Narin et al. 2020). Convolutional Neural Networks (CNNs) can be beneficial for accurate classification of this disease, and they can also help to overcome the shortage of specialists, especially in villages. In contrary to traditional learning paradigms, transfer learning seeks to solve the related problems by using the pre-gained knowledge (Jin et al. 2020; Najafabadi et al. 2015; Chen and Lin 2014; Singh et al. 2017).

Concatenation of deep features extracted from multiple CNNs produces a discriminant and appropriate descriptor for the classification process. Noreen et al. (2020) designed a feature concatenation algorithm for brain tumor classification.  In this algorithm, two pre-trained models (Inception-v3 and DensNet201) have been used for deep feature extraction. DU. Chen et al. (2020) proposed a Selective Feature Connection Mechanism (SFCM), by which low-level and high-level features are linked together with a feature selector. Their mechanism can effectively demonstrate the effectiveness in multiple computer vision tasks including object detection, image classification, scene text detection, and image-to-image translation. Furthermore, Chenhui et al. Ma et al. (2019) presented a CNN-based fusion technique to integrate feature maps from various layers.

In this paper, we propose a DFC-based approach for automatic identification of COVID-19. This approach is applied to form a definitive classification descriptor. It is adopted in two different methods. The first method depends on the concatenation of deep features that are extracted separately from X-ray and CT images. Deep features are obtained using a proposed simple CNN that includes three deep layers, only. The second method aims to reach the optimal performance from either X-ray or CT scan by performing concatenation between the features extracted using three different CNNs, which are the proposed CNN architecture and two modern pre-trained CNNs (ResNet and GoogleNet). The main contributions of this paper can be summarized as follows:

• Providing a CNN-based framework for diagnosis and detection of COVID-19 disease.

• Introducing a simple proposed CNN architecture to solve the time consumption problem.

• Adjusting the CNN parameters such as optimization algorithm, maximum number of epochs, LR, and M-B size.

• Performing a DFC mechanism to take the advantage of combining deep features extracted from different image types.

• Executing extensive experiments to compare the performance of the proposed approach with related ones.

The rest of the paper is organized as following. Section  2 describes the related work. Section  3 provides the proposed DFC mechanism. Section  4 shows the experimental results and discussions. Finally, the conclusion is summarized in Sect. 5.

Related work

The CNN is a class of deep learning tools. There are CNN models trained on instance ImageNet and used for image classification (Krizhevsky et al. 2012). The AlexNet consists of eight layers. It was proposed in 2012 (Jmour et al. 2018a). Furthermore, in 2014, VGGNet was developed to have different numbers of layers (11, 16 and 19 layers) Sengupta et al. (2019). In addition, LeNet was announced with a structure of two convolution layers, two max-pooling layers, a fully-connected layer and a classification layer (LeCun et al. 2015). Furthermore, GoogleNet, developed by Christian in 2014, had 22 layers with the insertion of inception layers (Szegedy et al. 2015a). Besides, ResNet, proposed by Kaiming in 2015, consisted of different layers (18, 50, 101, 152, and 1202 layers) He et al. (2016). Furthermore, SqueezeNet, presented in 2016, provided less parameters with faster performance than that of the AlexNet Iandola et al. (2016). In 2017, DenseNet introduced dense connection between layers (Huang et al. 2017). In 2018, MobileNetV2 was presented using the idea of inverted residual structures, and it performed well on mobile devices (Sandler et al. 2018). In addition, an efficient network was introduced in 2019 that includes 5.3 million parameters (Tan and Le 2019).

Nowadays, CNNs accomplished a great success in the areas of medical image analysis such as brain tumor detection (Sajjad et al. 2019), classification of skin diseases (Esteva et al. 2017), diagnosis of pulmonary modules (Cheng et al. 2016), diagnosis of breast cancer (Braman et al. 2019) and recognition of immune cells (Chen and ChefdHotel 2014). Recently, additional medical applications appeared such as improving the storage and exchange of DICOM medical data on the cloud (Dorgham et al. 2018; Zheng et al. 2017), transformation of secret information by using virtue of quantum-watermarked images Qu et al. (2019), Nuclei segmentation of brain tumors Guo et al. (2018), and early detection of COVID-19 disease. Furthermore, a healthcare framework was developed through a medical image forgery detection system to ensure that images are related to the healthcare system or not (Ghoneim et al. 2018; Amato et al. 2018).

A series of CNN-based approaches were designed for COVID-19 detection. Ying et al. and Song et al. (2020) used ResNet-50 for COVID-19 detection from CT images. Images were divided into 101 bacterial pneumonia, 86 normal cases and 88 COVID-19 patients. Their methodology achieved an accuracy of 86.0% for classification of bacterial or COVID-19 pneumonia. Besides, Farooq and Hafeez (2020) adopted ResNet-50 and applied various augmentation techniques to achieve an accuracy of 96.23%. Moreover, Xu et al. (2020) used V-Net for infected region segmentation. Also, ResNet-18 was used for diagnosis of COVID-19 cases with an accuracy of 86.7%. Hall et al. (2020) utilized the VGG16 model with 10-folds for training with the entire dataset to reach an accuracy of 96.1% and an Area Under the Curve (AUC) of 99.7%.

Nguyen et al. (2018) used ResNet-152, Inception-v3 and Inception-ResNet-v2 for extracting deep features from input images. These features are combined to represent all details of the input PAP-Smear and 2D-Hela datasets. Their mechanism provided an accuracy of 92.6%. Liang et al. (2019) studied the fusion of both CT and MR images for training of their Multi-layer Concatenation Fusion Network (MCFNet). They achieved good classification results. Rahimzadeh and Attar (2020) proposed a concatenation-based approach to link the features extracted using ResNet-50V2 and Xception CNNs. Their approach was applied on an X-ray dataset that was divided into three categories: pneumonia, normal and COVID-19 images. An accuracy of 91.4% was achieved.

The proposed DFC methodology

As mentioned before, the proposed methodology aims at accurate diagnosis of COVID-19 disease through performing concatenation between deep features extracted from X-ray and CT images. Initially, the input images are prepared appropriately through performing the image segmentation process to extract the Region Of Interest (ROI). Then, CNNs are used for the deep feature extraction process. Deep features are extracted in two different methods. For the first one, a simple proposed CNN extracts features from X-ray and CT images as presented in Fig.  1. In the other method, three CNNs (the proposed CNN, ResNet-18, and GoogleNet) extract features from either X-ray or  CT images as shown in Fig.  2. Finally, the DFC methodology is applied to combine the extracted features into a single descriptor for classification. Some confirmed COVID-19 and non-COVID-19 images are given in Figs. 3 and  4 for CT and X-ray images, respectively.

Fig. 1.

The main architecture of the proposed approach, first method

Fig. 2.

The main architecture of the proposed approach, second method

Fig. 3.

CT images a COVID-19, and b Non-COVID-19

Fig. 4.

X-ray images a COVID-19, and b Non-COVID-19

Image segmentation

The image segmentation process is utilized to separate the pixels of interest and to locate the active contours of the lungs in input images (Michailovich et al. 2007). It can be performed with a snake in its intuitive manner, as it can be utilized to track objects. The contour is like a parametric curve $V(s)=\left\{X,(,s,),,,Y,(,s,)\right\}$, where $X\left(s\right)$, and  $Y\left(s\right)$ are coordinate functions in a parametric domain “s”. Active contour could be represented using an energy function as follows (Leymarie and Levine 1993):

$$\begin{array}{c}\hfill E_{\mathit{snake}}^i=E_{\mathit{internal}}^i-E_{\mathit{external}}^i\end{array}$$ 1

where $E_{\mathit{internal}}^i$ and $E_{\mathit{external}}^i$ are the internal and external energies at the $i^{\mathit{th}}$ position of the contour. The internal energy represents the contour strength as:

$$\begin{array}{cc}& E_{\mathit{internal}}=\frac{1}{2}{\int }_0^1[\alpha ((\frac{\mathit{dx}}{\mathit{ds}}{)}^2+(\frac{\mathit{dy}}{\mathit{ds}}{)}^2)+\beta ((\frac{d^{2}x}{d{s}^2}{)}^2+(\frac{d^{2}y}{d{s}^2}{)}^2)]\hfill \end{array}$$ 2

where $\alpha$ is the parameter that controls the snake tension and $\beta$ is the rigidity parameter. The external energy is responsible for moving the snake to the borders or ROI in an image. The external energy could be computed as:

$$\begin{array}{c}\hfill E_{\mathit{external}}={\int }_0^1(f[X(s),Y(s)])ds\end{array}$$ 3

Figure  5 depicts examples of CT and X-ray images before and after performing the segmentation process.

Fig. 5.

CT and X-ray images before and after performing the segmentation process for COVID-19 and Non-COVID-19 cases

Deep feature extraction

The process of feature extraction from images after segmentation is performed using CNNs. The CNNs are deep learning algorithms that contain multiple layers that are used for feature detection. These layers are convolutional, pooling, batch normalization, Rectified Linear Unit (ReLU), SoftMax, and fully-connected layers (Jmour et al. 2018b; Szegedy et al. 2015b). The convolutional layer depends on a number of kernels of weights ${\mathbf{w}}^l$ for each layer l to extract the local features from the input ${\mathbf{x}}^{l-1}$ as described in the following equation:

$$\begin{array}{c}\hfill {\mathbf{C}}^l={{\mathbf{w}}^l}^T{\mathbf{x}}^{l-1}+b^l\end{array}$$ 4

where ${\mathbf{C}}^l$ is the output feature map obtained by computing the dot products between the kernels and the input with added bias $b^l$.

The pooling layer has two essential types: maximum and average pooling. The output of the pooling process $P^l$ is the down-sampled version of the entire feature map ${\mathbf{C}}^l$ depending on the window size (m, n) as:

$$\begin{array}{c}\hfill {\mathbf{P}}^l=ma{x}_{(m,n)}{\mathbf{C}}^l\end{array}$$ 5

The fully-connected layer is the last essential layer. Let the layer l-1 be the fully-connected layer. Layer l expects $m_1^{(l-1)}$ feature maps having a size of $m_2^{(l-1)}\ast m_3^{(l-1)}$ as input. The output of a fully-connected layer ${\mathbf{Y}}_i^{(l)}$ could be described as:

$$\begin{array}{cc}\hfill {\mathbf{Y}}_i^{(l)}=& f({\mathbf{Z}}_i^{(l)})\hfill \end{array}$$ 6

$$\begin{array}{cc}\hfill {\mathbf{Z}}_i^{(l)}=& \sum _{j=1}^{m_1^{(l-1)}}\sum _{r=1}^{m_2^{(l-1)}}\sum _{s=1}^{m_3^{(l-1)}}{\mathbf{w}}_{i,j,r,s}^{(l)}{({\mathbf{Y}}_j^{(l-1)})}_{r,s}\hfill \end{array}$$ 7

where ${\mathbf{w}}_{i,j,r,s}^{(l)}$ represents the weights used to connect the position (r, s) in the $i^{\mathit{th}}$ unit in layer l and the jth feature map of layer $l-1$. Here, we propose a new CNN model that includes three deep layers only to avoid issues such as complexity and large time consumption. Table 1 provides the architecture of the proposed model in detail.

Table 1.

Description of the proposed CNN model

Name	#Filters	Filter size	Stride	Padding	Weighs	Output
Input layer of size [224 224 3]
Conv_1	32	$3\times 3$	$2\times 2$	Same	$3\times 3\times 3\times 32$	$112\times 112\times 32$
Batch Normalization + ReLU
Pool_1	–	$3\times 3$	$3\times 3$	[0 0 0 0]	–	$37\times 37\times 32$
Conv_2	128	$5\times 5$	$2\times 2$	Same	$5\times 5\times 32\times 128$	$19\times 19\times 128$
Batch Normalization + ReLU
Pool_2	–	$3\times 3$	$3\times 3$	[0 0 0 0]	–	$6\times 6\times 128$
Conv_3	256	$5\times 5$	$1\times 1$	Same	$5\times 5\times 128\times 256$	$3\times 3\times 256$
Batch normalization + ReLU
GAP	–	–	$6\times 6$	[0 0 0 0]	–	$1\times 1\times 256$
Two fully-connected layer					$2\times 256$
Softmax layer
Classification output layer						2

Deep feature concatenation (DFC)

Feature concatenation is an effective way to add different features together to enhance the classification process. In this work, the DFC is applied in two different ways. In the first one, CT and X-ray features are extracted using the proposed CNN. Then, these features are linked to form the classification descriptor as:

$$\begin{array}{c}\hfill FinalFeatureDescriptor={\mathbf{F}}^{(CT)}\cup {\mathbf{F}}^{(X-ray)}\end{array}$$ 8

On the other hand, the second way aims to obtain remarkable results from each dataset (CT or X-ray), separately. This can be accomplished through extracting features from images using three different CNNs: the proposed, ResNet-18, and GoogleNet. These features are combined together using the DFC as follows:

$$\begin{array}{cc}& FinalFeatureDescriptor={\mathbf{F}}^{(ProposedCNN)}\cup {\mathbf{F}}^{(ResNet-18)}\cup {\mathbf{F}}^{(GoogleNet)}\hfill \end{array}$$ 9

Classification

The classification process depends on the final feature descriptor to determine whether the input image is positive (COVID-19) or negative (Non-COVID-19). The fully-connected layer converts the input data into 1D vector. Then, the softmax layer calculates the scores of the classes as:

$$\begin{array}{c}\hfill {\mathbf{L}}_{\mathit{softmax}}=-\sum _{i=1}^{N}log\frac{e^{{\mathbf{w}}_{y_i}^T{\mathbf{f}}_i+b_{y_i}}}{{\sum }_{j=1}^k{e}^{{\mathbf{w}}_j^T{\mathbf{f}}_i+b_j}}\end{array}$$ 10

where ${\mathbf{f}}_i$ denotes features and $y_i$ is the true class label of the $i^{\mathit{th}}$ image. ${\mathbf{w}}_j$ and $b_j$ are the weights and bias of the $j^{\mathit{th}}$ class, respectively. N is the number of training samples and K is the number of classes.

Results and discussions

In this section, we discuss the results of the proposed DFC approach that could be implemented in two different methods. X-ray and CT image datasets are chosen to evaluate the proposed methodology (El-Shafai and El-Samie 2020). Each dataset includes positive (COVID) cases (2628 images) and negative (Non-COVID) cases (1620 images). For each class, images are divided into 70% for training and 30% for testing. Furthermore, the system performance is measured in terms of accuracy, precision, recall, and fscore. As discussed before, a new CNN model is proposed. In order to achieve its optimal performance, its hyper-parameters are adjusted including the optimization algorithm (Adam, RMS Prop, or SGDM), maximum number of epochs (30, 40, or 50), learning rate (LR) (0.001 or 0.0001), and Mini-Batch (MB) size (32, 64, or 128). Tables  2,  3, and  4 show the performance of the proposed model for the CT image dataset using a maximum number of epochs of 30, 40, and 50, respectively. For X-ray images, Tables  5,  6, and  7 present the results of the proposed CNN using maximum number of epochs of 30, 40, and 50, respectively. For each case, Figs.  6,  8,  10,  12,  14, and  16 provide accuracy versus iterations and loss versus iterations. Besides, for each case, Figs.  7,  9,  11,  13,  15, and  17 depict a graphical representation of the performance of the proposed model. In addition, the mean, variance, and standard deviation are also calculated.

Table 2.

Performance of the proposed CNN using CT dataset, max, Epochs = 30 (%)

OA	MB Size	LR	Accuracy	Precision	Recall	F_score
Adam	32	0.001	91.39	90.88	89.33	90.09
		0.0001	90.55	86.71	92.39	89.45
	64	0.001	89.52	88.89	90.32	89.59
		0.0001	92.16	93.67	90.55	92.08
	128	0.001	88.71	90	87.1	88.52
		0.0001	90.32	93.1	87.1	90
RMS Prop	32	0.001	92.04	92.92	90.32	91.6
		0.0001	91.94	94.83	88.71	91.66
	64	0.001	87.1	84.85	90.32	87.49
		0.0001	90.74	88.77	91.16	89.94
	128	0.001	91.13	91.8	90.32	91.05
		0.0001	91.94	89.39	91.18	90.27
SGDM	32	0.001	90.55	87.19	90.4	88.76
		0.0001	82.26	78.57	88.71	83.33
	64	0.001	86.1	88.86	84.55	86.65
		0.0001	83.06	85.96	79.03	82.34
	128	0.001	87.1	88.33	85.48	86.88
		0.0001	84.68	86.44	82.26	84.29
Mean Value			88.96	88.95	88.29	88.55
Variance			9.54	13.9	11.54	7.8
Standard deviation			3.09	3.73	3.4	2.79

Table 3.

Performance of the Proposed CNN on the CT Dataset, max, Epochs = 40

OA	MB Size	LR	Accuracy (%)	Precision (%)	Recall	F_score
Adam	32	0.001	91.94	88.24	96.77	92.3
		0.0001	93.07	91.75	94.77	93.23
	64	0.001	90.32	90.32	90.32	90.32
		0.0001	91.94	90.32	90.33	90.32
	128	0.001	92.74	90.77	95.16	92.91
		0.0001	87.1	83.82	91.94	87.69
RMS Prop	32	0.001	90.32	86.76	95.16	90.76
		0.0001	75.81	79.63	69.35	74.13
	64	0.001	91.13	94.74	87.1	90.75
		0.0001	92.55	90.91	93.77	92.31
	128	0.001	89.52	87.69	91.94	89.76
		0.0001	91.94	89.39	95.16	92.18
SGDM	32	0.001	86.29	83.58	90.33	86.82
		0.0001	89.52	86.57	93.55	89.92
	64	0.001	84.68	81.16	90.32	85.49
		0.0001	80.65	79.69	82.26	80.95
	128	0.001	90.32	86.76	95.15	90.76
		0.0001	82.26	80.3	85.48	82.8
Mean			88.45	86.8	90.49	88.52
Variance			21.78	19.44	40.03	23.24
Standard deviation			4.67	4.41	6.33	4.82

Table 4.

Performance of the proposed CNN using CT dataset, max. epochs = 50. (%)

OA	M-B Size	LR	Accuracy	Precision	Recall	F_score
Adam	32	0.001	90.32	87.88	93.55	90.62
		0.0001	93.61	92.31	93.77	93.03
	64	0.001	89.52	86.57	93.55	89.92
		0.0001	87.9	86.15	90.32	88.18
	128	0.001	90.32	93.1	87.1	90
		0.0001	84.68	87.72	80.65	84.03
RMS Prop	32	0.001	92.74	90.77	94.16	92.43
		0.0001	93.97	94.72	93.16	93.93
	64	0.001	93.77	93.14	92.72	92.92
		0.0001	94.35	96.61	91.94	94.21
	128	0.001	89.52	90.16	88.71	89.42
		0.0001	90.32	89.06	91.94	90.47
SGDM	32	0.001	93.55	90.91	96.77	93.74
		0.0001	85.48	82.35	90.32	86.15
	64	0.001	88.71	82.43	98.39	89.7
		0.0001	80.65	81.67	79.03	80.32
	128	0.001	91.94	91.94	91.94	91.94
		0.0001	86.28	86.89	85.48	86.17
Mean			89.87	89.13	90.75	89.84
Variance			13.46	17.47	23.9	13.2
Standard deviation			3.67	4.18	4.9	3.6

Table 5.

Performance of the proposed CNN on X-ray dataset, max. Epochs = 30 (%)

OA	M-B Size	LR	Accuracy	Precision	Recall	F_score
Adam	32	0.001	89.13	87.49	88.51	87.99
		0.0001	85.46	85.42	90.47	87.87
	64	0.001	88.43	86.5	89.5	87.97
		0.0001	90.22	92.28	88.73	90.47
	128	0.001	86.62	88.44	86.28	87.34
		0.0001	88.23	92.71	85.28	88.83
RMS Prop	32	0.001	90.95	91.53	88.45	89.96
		0.0001	89.85	93.64	84.89	89.05
	64	0.001	84.01	83.43	88.5	85.89
		0.0001	88.35	87.38	88.34	87.85
	128	0.001	89.04	90.41	88.5	89.4
		0.0001	89.85	88.5	89.26	88.87
SGDM	32	0.001	88.56	85.8	88.58	87.16
		0.0001	80.17	79.18	86.29	82.58
	64	0.001	84.31	87.47	82.73	85.03
		0.0001	85.97	84.77	80.21	82.42
	128	0.001	85.01	84.94	83.26	84.09
		0.0001	82.54	85.35	84.44	84.89
Mean Value			87.04	87.5	86.8	87.09
Variance			8.3	12.6	7.3	5.5
Standard deviation			2.88	3.5	2.7	2.3

Table 6.

Performance of the Proposed CNN using X-ray Dataset, max. Epochs = 40. (%)

OA	MB Size	LR	Accuracy	Precision	Recall	F_score
Adam	32	0.001	90.35	88.16	92.31	90.18
		0.0001	90.9	90.37	91.31	90.83
	64	0.001	88.15	89.24	88.36	88.79
		0.0001	90.73	89.54	88.87	89.2
	128	0.001	91.57	88.69	92.7	90.65
		0.0001	86.93	83.74	90.38	86.93
RMS Prop	32	0.001	89.17	85.63	90.7	88.09
		0.0001	88.64	84.55	87.89	86.18
	64	0.001	89.66	92.66	85.64	89.01
		0.0001	91.38	89.83	92.31	91.05
	128	0.001	89.35	86.41	90.48	88.39
		0.0001	90.77	88.31	91.7	89.97
SGDM	32	0.001	85.52	84.5	88.47	86.43
		0.0001	88.35	85.49	92.09	88.66
	64	0.001	84.51	80.08	88.86	84.24
		0.0001	83.44	80.61	80.08	80.34
	128	0.001	89.15	85.38	92.69	88.88
		0.0001	85.09	81.42	84.12	82.74
Mean			88.54	86.4	89.4	87.8
Variance			5.8	11.75	10.68	8.01
Standard deviation			2.4	3.43	3.27	2.83

Table 7.

Performance of the Proposed CNN using X-ray Dataset, max. Epochs = 50. (%)

OA	MB size	LR	Accuracy	Precision	Recall	F_score
Adam	32	0.001	88.91	86.31	90.67	88.43
		0.0001	91.22	91.94	91.39	91.66
	64	0.001	87.11	85.12	90.67	87.8
		0.0001	86.49	84.75	89.44	87.03
	128	0.001	86.91	90.73	86.41	88.51
		0.0001	82.87	86.35	78.37	82.16
RMS prop	32	0.001	92.36	89.34	92.89	91.08
		0.0001	91.56	92.35	91.28	91.81
	64	0.001	91.36	91.47	89.84	90.64
		0.0001	90.94	92.44	90.26	91.33
	128	0.001	87.17	89.79	85.83	87.76
		0.0001	87.51	88.69	90.3	89.48
SGDM	32	0.001	91.14	89.14	92.23	90.65
		0.0001	87.47	80.28	88.8	84.32
	64	0.001	88.23	86.06	90.51	88.22
		0.0001	82.24	83.3	85.15	84.21
	128	0.001	88.53	90.52	91.06	90.78
		0.0001	83.86	85.56	85.11	85.33
Mean			88.1	88	88.9	88.4
Variance			8.6	11.26	11.79	7.8
Standard Deviation			2.9	3.36	3.4	2.79

Fig. 6.

Accuracy vs. iterations and loss vs. iterations for the proposed CNN on the CT dataset using Adam algorithm for optimization, max number of epochs = 30, MB size = 64, and LR = 0.0001

Fig. 8.

Accuracy vs. iterations and loss vs. iterations of the proposed CNN on the CT dataset using: Adam algorithm for optimization, max. Epochs = 40, MB size = 32, and LR = 0.0001

Fig. 10.

Accuracy vs. iterations and loss vs. iterations of the proposed CNN on CT dataset using: Rmsprop algorithm for optimization, max. Epochs = 50, M-B size = 64, and LR = 0.0001

Fig. 12.

Accuracy vs. iterations and loss vs. iterations of the proposed CNN for X-ray dataset using: Rms prop algorithm for optimization, max. Epochs = 30, MB size = 32, and LR = 0.001

Fig. 14.

Accuracy vs. iterations and loss vs. iterations of the proposed CNN for X-ray dataset using: Adam algorithm for optimization, max. Epochs = 40, MB size = 128, and LR = 0.001

Fig. 16.

Accuracy vs. iterations and loss vs. iterations of the proposed CNN for X-ray dataset using: Rmsprop algorithm for optimization, max. Epochs = 50, MB size = 32, and LR = 0.001

Fig. 7.

Performance of the proposed CNN on the CT dataset, max. number of epochs = 30

Fig. 9.

Performance of the proposed CNN on CT dataset, max. Epochs = 40

Fig. 11.

Performance of the proposed CNN on CT dataset, max. Epochs = 50

Fig. 13.

Performance of the Proposed CNN using X-ray Dataset, max. Epochs = 30

Fig. 15.

Performance of the Proposed CNN using X-ray Dataset, max. Epochs = 40

Fig. 17.

Performance of the Proposed CNN using X-ray Dataset, max. Epochs = 50

From the previous results, it can be deduced that to reach the optimal performance of the proposed model, the proposed CNN hyper-parameters are adjusted as follows:

• For CT images: Rmsprop algorithm for optimization, maximum number of epochs = 50, MB size = 64 and LR = 0.0001.

• For X-ray images: Rmsprop algorithm for optimization, maximum number of epochs = 50, MB size = 32 and LR = 0.001.

These variables are adjusted with the mentioned values for each dataset during the implementation of the proposed approach. Now, the experimental results of the proposed approach (first method) will be discussed. Table  8 illustrates a comparison between the utilization of the proposed CNN to classify each dataset separately and the application of the proposed approach (first method) that combines both X-ray and CT features using DFC technique. The results in Table  8 confirm the superiority of the proposed approach and the improvement of the classification process as a result of the utilization of the DFC methodology. Additionally, Figure  18 shows a graphical comparison between the proposed CNN (CT Dataset), the proposed CNN (X-ray dataset), and the proposed approach (first method). In addition, Figures  19 and  20 present the ROC curves that demonstrate the superiority of the proposed DFC approach.

Table 8.

Performance of the proposed approach (first method). (%)

Methodology	Accuracy	Precision	Recall	F_score
Proposed CNN (CT Dataset)	94.35	96.61	91.94	94.21
Proposed CNN (X-ray dataset)	92.36	89.34	92.89	91.08
Proposed Approach (first method)	96.13	94.37	97.04	95.69

Fig. 18.

Graphical comparison between the proposed CNN (CT Dataset), the proposed CNN (X-ray Dataset), and the proposed approach (first method)

Fig. 19.

ROC curves of the Proposed CNN (CT Dataset) and the Proposed DFC methodologies

Fig. 20.

ROC curves of the Proposed CNN (X-ray Dataset) and the Proposed DFC methodologies

The main purpose of the second method of the proposed DFC approach is to obtain distinctive results from each image dataset X-ray or CT by performing concatenation between the features extracted from the dataset. Features are extracted using three different CNNs, which are the proposed CNN, ResNet-18, and GoogleNet. The results of the proposed approach (second method) are compared with the results of single-stage methods (single stage (ResNet-18), single-stage (GoogleNet), and single-stage (Proposed CNN)). Tables  9 and  10 display these results for both X-ray and CT  datasets, respectively. By analyzing the results in Tables  9 and  10, it is clear that the proposed approach (second method) is significantly superior to the single-stage methods for both datasets. This is also confirmed by: (a) Figs. 21 and  22 that show (accuracy vs. iterations)and (loss vs. iterations) for CT dataset; (b) Figs.  23 and  24 that show (accuracy vs. iterations) and (loss vs. iterations) for X-ray dataset; and (c) the ROC curves shown in Figs.  25 and  26 for both CT and X-ray datasets, respectively.

Table 9.

Performance of the proposed approach (second method) for CT dataset. (%)

Method	Accuracy	Precision	Recall	F_score
Single stage (ResNet-18)	95.81	96.64	94.9	95.77
Single stage (GoogleNet)	92.07	86.8	99.24	92.6
Single Stage (Proposed CNN)	94.35	96.61	91.94	94.21
The proposed approach (DFC) (second method)	98.9	99.36	98.5	98.92

Table 10.

Performance of the proposed approach (second method) for X-ray dataset (%)

Method	Accuracy	Precision	Recall	F_score
Single Stage (ResNet-18)	98.97	98.2	99.79	98.98
Single stage (GoogleNet)	94.2	89.67	100	94.55
Single stage (proposed CNN)	92.36	89.34	92.89	91.08
The proposed approach (DFC) (second method)	99.3	99.79	98.8	99.3

Fig. 21.

Accuracy vs. iterations of the Proposed Approach DFC (second method) for CT dataset

Fig. 22.

Loss vs. iterations of the Proposed Approach DFC (second method) for CT dataset

Fig. 23.

Accuracy vs. iterations of the Proposed Approach DFC (second method) for X-ray dataset

Fig. 24.

Loss vs. iterations of the Proposed Approach DFC (second method) for X-ray dataset

Fig. 25.

ROC curves of the Proposed Approach DFC (second method) for CT dataset

Fig. 26.

ROC curves of the Proposed Approach DFC (second method) for X-ray dataset

Finally, Table  11 provides a comparison between the proposed DFC approach and the state-of-the-art methodologies. These results also prove the superiority of the proposed DFC approach in accurately diagnosing the COVID-19 disease.

Table 11.

Comparison between the proposed DFC and state-of-the-art methodologies (%)

Methodology	Dataset	Accuracy	Precision	Recall	F_score
MobileNet	CT	93.14	92.43	95.78	94.07
	X-ray	83	89	89	89
DenseNet201	CT	91.72	90.61	92.73	91.65
	X-ray	84	94	94	94
Xception	CT	94.77	93.18	90.72	91.93
	X-ray	95	92	90	90
InceptionV3	CT	91.28	90.47	90.3	90.38
	X-ray	82	85	85	85
InceptionResNetV2	CT	93.04	91.53	94.95	93.2
	X-ray	84	88	88	88
ResNet152	CT	93.74	92.93	93.9	93.41
	X-ray	93	90	89	89
VGG16	CT	91.27	88.07	95.47	91.62
	X-ray	90	94	94	94
VGG19	CT	91.53	91.11	90.72	90.91
	X-ray	90	94	94	94
GoogleNet	CT	92.07	86.8	99.24	92.6
	X-ray	94.2	89.67	100	94.55
ResNet-18	CT	95.81	96.6	94.9	95.77
	X-ray	98.97	98.2	99.79	98.98
ResNet50V2	CT	94.3	96.75	96.93	96.83
	X-ray	87	92	91	91
ResNet101V2	CT	88.92	91.03	91.19	91.1
	X-ray	79	87	87	87
ResNet152V2	CT	92.25	93.29	94.17	93.72
	X-ray	90	91	90	90
Rahimzadeh (2020)	X-ray	91.4	N/A	80.5	N/A
C. Zheng (2020)	CT	83.5	N/A	90.7	N/A
F. Shi (2020)	CT	87.9	N/A	90.7	N/A
L. Li (2020)	CT	N/A	N/A	90	N/A
Proposed CNN	CT	94.35	96.61	91.94	94.21
	X-ray	92.36	89.34	92.89	91.08
Proposed DFC (second method)	CT	98.9	99.36	98.5	98.92
	X-ray	99.3	99.79	98.8	99.3
Proposed DFC (first method)		96.13	94.37	97.04	95.69

Conclusions

The COVID-19 has been considered as a life-threatening disease that affects lungs. Hence, the rapid diagnosis of COVID-19 has become a vital necessity. In this paper, a CNN-based DFC approach has been presented. It adopts the concatenation process of the extracted deep features of X-ray and CT images. The proposed approach has been implemented in two different ways. In the first way, X-ray and CT image features are extracted by the proposed CNN. Then, these features are combined to form the classification descriptor. This way has achieved an accuracy of $96.13\%$, a precision of $94.37\%$, a recall of $97.04\%$, and an f_score of $95.69\%$. The second way has been mainly intended to enhance the results for either X-ray or CT dataset. This could be accomplished by using three CNNs (the proposed CNN, GoogleNet, and ResNet-18) to extract features. Then, the DFC has been applied on these features to obtain the classification descriptor. For CT images, this way has achieved an accuracy of $98.9\%$, a precision of $99.36\%$, a recall of $98.5\%$, and an f_score of $98.92\%$. For X-ray images, this way has attained an accuracy of $99.3\%$, a precision of $99.79\%$, a recall of $98.8\%$, and an f_score of $99.3\%$. Hence, the proposed approach can be used efficiently for the diagnosis of COVID-19 cases from medical images.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.