COVID-RDNet: A novel coronavirus pneumonia classification model using the mixed dataset by CT and X-rays images

Abstract

Corona virus disease 2019 (COVID-19) testing relies on traditional screening methods, which require a lot of manpower and material resources. Recently, to effectively reduce the damage caused by radiation and enhance effectiveness, deep learning of classifying COVID-19 negative and positive using the mixed dataset by CT and X-rays images have achieved remarkable research results. However, the details presented on CT and X-ray images have pathological diversity and similarity features, thus increasing the difficulty for physicians to judge specific cases. On this basis, this paper proposes a novel coronavirus pneumonia classification model using the mixed dataset by CT and X-rays images. To solve the problem of feature similarity between lung diseases and COVID-19, the extracted features are enhanced by an adaptive region enhancement algorithm. Besides, the depth network based on the residual blocks and the dense blocks is trained and tested. On the one hand, the residual blocks effectively improve the accuracy of the model and the non-linear COVID-19 features are obtained by cross-layer link. On the other hand, the dense blocks effectively improve the robustness of the model by connecting local and abstract information. On mixed X-ray and CT datasets, the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), area under curve (AUC), and accuracy can all reach 0.99. On the basis of respecting patient privacy and ethics, the proposed algorithm using the mixed dataset from real cases can effectively assist doctors in performing the accurate COVID-19 negative and positive classification to determine the infection status of patients.

1. Introduction

The director-general of the World Health Organization announces [1] that the 2019 infectious coronavirus is named corona virus disease 2019 (COVID-19) on February 11, 2020. As of August 31, Beijing time, China, 219 million people worldwide have novel coronavirus pneumonia and 4 million 480 thousand deaths have been reported [2]. Medical imaging [3] has become an indispensable part of disease diagnosis and treatment. The latest research shows that with the development of medical imaging [4], [5], computed tomography (CT) [6], [7], [8], [9], and X-ray images [10], [11], [12] can effectively assist doctors to judge the COVID-19 infection status. Here, the CT images are the sensitive positive COVID-19 diagnostic method, which has good timeliness. Conversely, X-ray images with fast imaging speed and low radiation damage have become more and more important in the process of the COVID-19 diagnosis [13], [14].

Recently, because deep learning [15] can be used to assist doctors in medical image processing, significant research results have been achieved in the classification of negative and positive COVID-19 [16]. At present, the existing deep learning models [17] for negative and positive COVID-19 classification include Alexnet [18], Vgg [22], Resnet [26], and Densenet [27]. The traditional deep learning model mainly includes the following issues:

• •As the network deepens, the gain in model performance is not visible, and major network degradation problems may develop as a result of the overzealous pursuit of network depth and the expression capabilities of the non-linear model.
• •The feature maps in the convolution kernel with different scales are coupled to improve the feature dimension of the single layer, resulting in a considerable amount of duplicated information. The computational complexity has increased exponentially as the number of network levels has grown.
• •To increase the model's fitting speed, a large-scale kernel is utilized to retrieve the lesion's local feature. Meanwhile, the feature maps cannot be separated precisely during the convolution phase, resulting in a loss of coherence in the retrieved feature.
• •Due to the means of the sudden drop in the feature dimension, information loss may occur in some traditional networks, which reduces the accuracy of the model.

Here, Maghdid et al. [19] combine the Alexnet model with transfer learning, which can effectively classify negative and positive COVID-19 on mixed X-ray and CT images. Then, Turkoglu et al. [20] can effectively improve the classification accuracy through mitigation feature selection algorithms and support vector machine (SVM). However, this structure contains more layers and increases memory consumption. To solve this problem, Loey et al. [21] combine the Generic Advantageous Network (GAN) with the Alexnet model. The above networks [19], [20], [21] improve the Alexnet model through preprocessing and transfer learning, which use dropout randomly inactivated neurons in the full connection layer to prevent the over-fitting phenomenon. However, due to the non-differentiable points in the Alexnet model, some neurons are not activated after updating the parameters. On this basis, the Vgg model replaces the receptive domain with the same size by reusing the convolution kernel with the same size. Sitaula et al. [23] propose a novel Vgg model using the attention module, which can effectively fine-tune the parameters in the classification process by capturing the spatial relationship. Shibly et al. [24] can effectively detect COVID-19 patients from the chest X-ray images by fast regions with a revolutionary neural networks framework. Lee et al. [25] use the deep convolution neural network to optimize the parameters of the Vgg network and fine-tune each backbone network. However, because most of the training parameters come from the first full connection layer, the storage cost of the Vgg network training may be large.

Resnet model realizes feature connection across layers through bottleneck convolution, which can directly transfer the weight without any intermediate transformation to the next layer. To reduce the impact of weight noise on the classification accuracy, Zhou et al. [28] input the regrouped images into the residual blocks for feature extraction and then input the extracted features into the support vector machine for recognition. Sakib et al. [29] classify negative and positive COVID-19 by using Gan and general data enhancement methods. Hira et al. [30] effectively obtain the low-level features of COVID-19 by using the pre-trained network weights and fine-tuning the whole model at an appropriate learning rate. The Resnet model obtains more features based on the residual blocks to deepen the network and avoids network degradation through the cross-layer link. However, because the convolution kernel of the Resnet model can not be divided by the corresponding upper feature map, the problem of feature loss will occur.

To solve this problem, the Densenet model based on the dense blocks connects the non-linear extracted information by comprehensively using the shallow features. Tabrizchi et al. [32] and Zhang YuDong et al. [33] improve the Densenet model and transfer learning to improve the classification accuracy of COVID-19. Chowdhury et al. [34] use the pre-trained deep learning model to automatically detect COVID-19 from chest X-ray images. It can overcome the problem of feature loss by dense blocks and fine-tuning the pre-trained network. The Densenet model connects the lesion features to obtain a decision function with good generalization performance. However, a large amount of redundant information will appear in the connection process, which can change feature dimensions through bottleneck convolution.

Thus, a novel coronavirus pneumonia classification model based on the mixed dataset by X-ray and CT images is proposed in this paper. The adaptive region enhancement algorithm then enriches the COVID-19 features. By using a cross-layer link, the residual blocks can yield non-linear COVID-19 properties. Besides, the robustness can be improved by the dense blocks by connecting abstract and local properties. The mixed COVID-19 dataset can be classified using the proposed COVID-RDNet model. Its main contributions are as follows:

• •The proposed COVID-RDNet model may successfully tackle the problem of network degradation in conventional deep learning by obtaining a network with the ability to represent local and global features.
• •Based on a mixed dataset of public X-ray and CT images, the proposed model can identify negative and positive COVID-19.
• •The proposed method can effectively assist doctors in the diagnosis and treatment of COVID-19 patients.

The paper is organized as follows. In Section II, the structure and research significance of the proposed model are introduced. Section III presents the adaptive region enhancement algorithm and the proposed COVID-RDNet model. Section IV analyzes the experimental results. The proposed model is compared with state-of-the-art classification algorithms. Finally, the conclusion is given in Section V.

2. Methodology and theory

2.1. Related work

2.1.1. The Resnet model

To effectively solve the network degradation in traditional deep learning, the Resnet model [26] obtains local and global feature expression by adjusting the number of channels and stacking layers in the residual block. The Resnet model [27], [28], [29], [30] connects an across the layer to the next layer directly without any change of intermediate weight matrix. Besides, it can make the connection and propagation of local and global features of lesions smoother. Because the local feature of COVID-19 can be directly transmitted to the corresponding abstract information without any intermediate weight matrix transformation, the Resnet model has strong depth adaptive ability. The corresponding structure is shown in Fig. 1 .

Fig. 1.

The structure of the Resnet model.

By connecting the extracted abstract features with the corresponding upper features maps, the gradient vanishing problem can be effectively overcome. For the layer $\mathit{Lay}$ $X_{\mathit{IL}}=X_{\mathit{Il}}+{\sum }_{c=l}^{\mathit{Lay}-1}F(X_{\mathit{Ic}},W_c)$ of the Resnet model, its partial derivative is as follows:

$$\frac{\partial Loss}{\partial X_{\mathit{Il}}}=\frac{\partial Loss}{\partial X_{\mathit{ILay}}}\frac{\partial X_{\mathit{ILay}}}{\partial X_{\mathit{Il}}}=\frac{\partial Loss}{\partial X_{\mathit{ILay}}}\left(1+\frac{\partial }{\partial X_{\mathit{Il}}}\sum _{c=l}^{\mathit{Lay}-1}F\left(X_{\mathit{Ic}},W_c\right)\right)$$ (1)

When the $\mathit{Lay}$-th layer and the corresponding $X_{\mathit{Il}}$ contain ${\lambda }_c$ coefficient, the $\mathit{Lay}$-th layer can be expressed as:

$$\frac{\partial Loss}{\partial X_{\mathit{Il}}}=\frac{\partial Loss}{\partial X_{\mathit{ILay}}}\left(\left(\prod _{c=l}^{\mathit{Lay}-1}{\lambda }_c\right)+\frac{\partial }{\partial X_{\mathit{Il}}}\widehat{F}\left(x_c,W_c\right)\right)$$ (2)

However, the Resnet model uses convolution kernels with odd sizes for feature extraction. When the kernel size set by the model cannot be completely divided by the step size, it is very easy to increase the redundancy information of feature maps and result in the uneven overlap. Especially for the COVID-19 images, due to the small gray difference between the COVID-19 region and the normal region, this overlapping redundancy can be enlarged. Therefore, the proposed algorithm uses the residual blocks to extract COVID-19 local information, which can not only effectively obtain the non-linear features of lesions, but also overcome the problem of feature loss caused by a convolution operation.

2.1.2. The Densenet model

With the increasing number of deep network layers, batch normalization can alleviate the problems of gradient vanishing. However, because the model calculation increases with the increase of network depth, the training efficiency is low and there is still the problem of network degradation. The Resnet model superimposes the output of identity mapping and non-linear transformation by numerical addition, which destroys the information flow in the network to a certain extent. To solve the above problems, the Densenet model [31] proposed in 2017 deepens the number of network layers to improve performance, as shown in Fig. 2 . It alleviates the network degradation problem extent by l reusing local features and adding dimension. The Densenet model [32], [33], [34] can effectively enhance the combination of COVID-19 information by connecting local and abstract features. Besides, the network can extract more features from relatively fewer data.

Fig. 2.

The structure of Densenet model.

Different from the Resnet model, the Densenet model limits the realization of convolution operation by combining all relevant outputs before layer $\mathit{Lay}$ (from layer $0$ to layer $\mathit{Lay}-1$):

$$X_{\mathit{ILay}}=H_{\mathit{Lay}}([X_{I0},X_{I1},X_{I2},...,X_{\mathit{ILay}-1}])$$ (3)

In the Densenet model, the dense blocks contain two convolution operations and the kernel size is 1*1@128 and 1*1@32. Through 1*1 convolution operation, it can effectively realize the fusion of local features, which can enhance the classification accuracy of the model. In each dense block, assuming that the feature dimension of each non-linear transform $H$ is$M$, the layer $\mathit{Lay}$ is$M_0+(Lay-1)\times M$. Because the local features extracted from each layer are connected with the previous ones, the training weights are learned through the current global information, which ensures the amount of information updated by the global state. Therefore, compared with the traditional depth model, the Densenet model can adopt fewer feature maps as the output of the network layer through the extreme utilization of extracted features.

However, because the Densenet model reduces the size of the current feature map to the next stage by average pooling, the feature dimension is half of the size of the previously extracted one. With the increase in model depth, the dimension of the small-size feature map is larger and the model adopts 1*1@128 and 1*1@32. The sudden drop of feature dimension and the sudden rise of connection dimension, it may lead to the lack of robustness of the COVID-19 features. Therefore, to enhance the ability of feature extraction, the proposed algorithm obtains COVID-19 local information through the dense blocks, in which the minimum amount of data is used through continuous bottleneck convolution operation.

2.2. The proposed network

This paper proposes a COVID-19 classification model COVID-RDNet based on the mixed X-ray and CT images. The experimental process contains three steps: acquisition of the mixed dataset, enhancement of the adaptive COVID-19 feature, and establishment of the proposed COVID-RDNet model, as shown in Fig. 3 . Firstly, a mixed COVID-19 dataset with the X-ray images and the CT images is built through multiple image sources, which can ensure the authenticity and effectiveness of the proposed model in detecting novel coronavirus pneumonia cases. Then, the region containing the COVID-19 feature is enhanced and separated from the background by an adaptive threshold algorithm. Next, the COVID-RDNet model proposed in this paper is constructed. By combining the advantages of residual blocks and dense blocks, the non-linear connection between local features and global ones is realized.

Fig. 3.

The flow chart of the proposed algorithm.

2.2.1. Acquisition of the mixed dataset

The imaging features of X-ray and CT images will change with the severity and duration of infection [35], [36], as shown in Fig. 4 . In general, the lesions of early COVID-19 patients mostly occur at the bottom of the lung and below the pleura, mainly in the peripheral zone of the lung. It may present as a single or double lung multiple patchy ground-glass shadows, sometimes accompanied by a slight bronchial inflation sign, as shown in Fig. 4(a) [37], [38]. With the further progress of the lesion, consolidation changes may occur. The scope and types of inflammatory lesions increase, which can begin to spread to multiple lung lobes. At this time, the patient's images show a mixture of ground glass shadow and solid change shadow. Besides, some patients will also have mild cord shadow and a small amount of pleural effusion, as shown in Fig. 4(b) [39], [40]. Severe novel coronavirus pneumonia patients have larger lesion regions and diffuse interstitial changes will appear. The symptoms of the white lung will appear at the later stage, often accompanied by a solid change shadow, which will always endanger the patient's life and health, as shown in Fig. 4(c) [41].

Fig. 4.

Imaging features of COVID-19 in different periods. (The first row is the X-ray images and the second one is the CT images. The red region indicates the position of the COVID-19 signs.) (a) early stage; (b) middle stage; (c) later stage.

To effectively improve the detection of the novel coronavirus pneumonia, the mixed dataset is established by X-ray and CT images. Here, the X-ray images [42] are from the public dataset provided by Guangzhou Women's and children's medical center [43], as shown in Fig. 5 (a). The images are from the routine clinical examination of children aged 1 to 5 years. They are screened by removing all low-quality or unreadable X-ray images for quality control. To ensure the authenticity and effectiveness of the results, all the negative and positive are classified by two experts. The evaluation set is also checked by a third expert. There are 8447 images in the X-ray image dataset, which is composed of 1583 negative images and 6864 positive images. It is divided into three parts for training, verification, and testing. The CT images are from an open-source COVID-CT dataset [44], as shown in Fig. 5(b). Some researchers from the University of California at San Diego select images of clinical symptoms by artificial screening. The CT images of 333 cases with the positive detection of COVID-19 and 397 cases with the positive detection are provided.

Fig. 5.

The negative and positive COVID-19 of the X-ray images and the CT images (Red arrows indicate the location of the COVID-19).

The dataset used in this paper is a mixed public X-ray and CT images. The established database contains 5216 images, which consist of 1341 non-novel coronavirus pneumonia and 3875 novel coronavirus pneumonia. Here, 70 % is used for training, 20 % for testing, and 10 % for validation, as shown in Table 1 . To facilitate the training of the proposed COVID-RDNet model, the image size in the dataset is set to 224*224.

Table 1.

The dataset structure distribution.

	Training(70 %)	Testing(20 %)	Validation(10 %)	Total(100 %)
Negative COVID-19	939	268	134	1341
Positive COVID-19	2713	775	387	3875
Total	3652	1043	521	5216

Due to the complex and diverse features of COVID-19, this paper uniformly preprocessed CT and X-ray images through matrix normalization. It can not only increase the ratio of the brightest to the darkest location while maintaining the original gray-scale distribution features of the COVID-19 images. Meanwhile, it can also effectively overcome the problem of too slow running speed caused by the same increase and subtraction of the connected weights, so that the operation efficiency of the algorithm can be accelerated when the mean of all signals is infinitely close to zero.

2.2.2. The adaptive enhancement algorithm of the COVID-19 region

Fig. 6 shows the image features of different lung diseases taking the X-ray images as an example [45]. From this figure, one can see that the novel coronavirus has very similar image features to other diseases. Besides, there is no clear boundary between the lesion region and normal lung tissue, where it is in a gradual transition state. To improve the negative and positive COVID-19 classification accuracy, the proposed algorithm enhances the contrast between the COVID-19 region and the normal region by an adaptive enhancement algorithm [46], [47], [48], [49].

Fig. 6.

Imaging features of different lung diseases (The red region indicates the position of the lesion signs).

To effectively enhance the COVID-19 region, the proposed algorithm firstly divides the image $I$ into the COVID-19 region $r_1$ and normal region $r_2$ through threshold$T$. Here, the obtained region is represented by$R(x,y)$:

$$R(x,y)=\left\{\begin{array}{c}{r}_1,I(x,y)⩾T\\ r_2,I(x,y)<T\end{array}\right)$$ (4)

Algorithm 1 Adaptive threshold algorithm
Input: Image$I$, $(x,y)$ is used to represent the coordinates of the pixel. $w_{t_0}$ and $w_{t_1}$ represent the number of pixels in the COVID-19 region and the normal region, respectively. $u_{t_0}$ and $u_{t_1}$ represent the average gray levels of the two regions, respectively.
Output: Threshold $T$
Initialization: $t$=0
1 while $t$<=255
2 if $\max \{w_{t_0}\times w_{t_1}\times {(u_{t_0}-u_{t_1})}^2\}$
3 $T$=$t$;
4 bleak;
5 else $t$=$t$ + 1;
6 end

To obtain the threshold$T$, the weighted average value of the adjacent region around each pixel is calculated by the adaptive threshold algorithm. It can better process the images with large light and dark differences, which can retain more lesion features. The basic idea is to calculate the average value of each pixel by traversing the images under the current cyclic variable$t$. When the variance between the COVID-19 region and the normal region is the largest, the current $t$ value is the threshold$T$. The calculation process is shown in Algorithm 1. Here, $w_{t_0}$ and $w_{t_1}$ represent the number of pixels in the COVID-19 region and the normal region, respectively. Similarly, $u_{t_0}$ and $u_{t_1}$ represent the average gray levels of the corresponding regions.

Then, the COVID-19 images are enhanced by enlarging the feature difference between the COVID-19 region and the normal region [50], [51], [52]. The proposed algorithm segments the COVID-19 region $r_1$ into$[r_{\min },m_0],[m_0,m_1],...[m_i,m_{i+1}],...,[m_n,r_{\max }]$. Here, $m_0,m_1,...,m_n$ represent the pixel value, $r_{\min }$ and $r_{\max }$ are the minimum and the maximum pixel value of the image, respectively. For$[m_i,m_{i+1}]$, the $i$-th histogram is mapped to the new dynamic range$[R{s}_i,R{e}_i]$:

$$R{s}_i=\sum _{k=1}^{i-1}{R}_k+1$$ (5)

$$R{e}_i=\sum _{k=1}^{i-1}{R}_k$$ (6)

Here,

$$R_k=(L-1)\times \frac{F_k}{{\sum }_{c=1}^{n+1}{F}_c}=(L-1)\times \frac{\mathit{Hig}{h}_k-Lo{w}_k-\lg M_k}{{\sum }_{c=1}^{n+1}{F}_c}$$ (7)

where $\mathit{Hig}{h}_k$ and $\mathit{Lo}{w}_k$ represent the maximum and the minimum pixel value in each histogram, respectively. $M$ is the corresponding number of pixels and $L$ represents the gray level. For$[R{s}_i,R{e}_i]$, the corresponding equalized histogram is obtained:

$$r_{1i}=R{s}_i+(R{e}_i-R{s}_i)\sum _{c=R{s}_i}^i\frac{n_c}{M}$$ (8)

where $n_c$ represents the number of current pixels to$R{s}_i$. Thus, the final output $r_1^{\prime }$ is obtained:

$$r_1^{\prime }=\frac{M_i}{M_0}{r}_1=\frac{M_i}{M_0}\sum _{c=1}^i{r}_{1i}$$ (9)

where $M_i$ and $M_0$ represent the average brightness of the COVID-19 region $r_1$ before and after equalization, respectively. The corresponding enhancement results are shown in Fig. 7 .

Fig. 7.

Results of adaptive local COVID-19 region $r_1$ enhancement. (Each row corresponds to a group of images. The first and second columns represent the images before and after enhancement, respectively).

The proposed adaptive enhancement algorithm converts the COVID-19 region $r_1$ into an output $r_1^{\prime }$ with the same number of pixels at each gray level, which can effectively improve the dynamic range of pixel gray value by balancing the gray level distribution in the COVID-19 region$r_1$. Besides, it can make the difference between the COVID-19 region and the normal region larger, as shown in Fig. 8 .

Fig. 8.

Results of adaptive enhancement algorithm (Red regions indicate the lesion before and after the enhancement). (a) Before enhancement; (b) After enhancement.

2.2.3. Establishment of the proposed COVID-RDNet model

To overcome the problems in traditional deep learning, the proposed COVID-RDNet model is shown in Fig. 9 and its innovations are as follows:

• •Model construction: this paper constructs a depth COVID-19 classification model COVID-RDNet. The proposed model uses residual blocks to adjust the dimension of the feature maps, which can effectively broaden the depth of local and global feature extraction. To obtain a network with strong generalization ability, the proposed model connects the non-linear transformation extracted by the upper layer with the extracted features of the lower layer. It can overcome the network degradation problem caused by overfitting and improves the robustness of the algorithm.
• •Model performance: To enhance the non-linear extended ability of the proposed model, this paper uses continuous 3*3 expansion convolution to achieve the depth extraction operation. It can obtain more abstract local features through the superposition of convolution layers. Meanwhile, to effectively utilize the local features with relatively low shallow complexity, the feature reuse in the proposed model is realized by continuously using 1*1 bottleneck convolution, which greatly reduces the calculation parameters and improves the generalization performance.
• •Data training: the proposed network can effectively train and test a mixed dataset containing both public X-ray and CT images, which can meet the actual needs of novel coronavirus detection in terms of diagnosis cost and determination speed.

Fig. 9.

The structure of the proposed COVID-RDNet model.

In the proposed algorithm, each convolution block contains three 3*3 expansion convolutions. After each convolution layer, a batch normalization layer and a correction linear unit layer are added to correct the weight of single-layer input. Thus, it alleviates the decline of model accuracy caused by the extraction of lesion features in the convolution process. To speed up the training and reduce the sensitivity to network initialization, the proposed model uses a batch normalization layer between the convolution layer and non-linear features:

$$Y_{\mathit{Ii}}\leftarrow \gamma {\widehat{X}}_{\mathit{Ii}}+\beta$$ (10)

$${\widehat{X}}_{\mathit{Ii}}\leftarrow \frac{X_{\mathit{Ii}}-{\mu }_B}{\sqrt{{\sigma }_B^2+\epsilon }}$$ (11)

$${\mu }_B\leftarrow \frac{1}{m}\sum _{i=1}^m{X}_{\mathit{Ii}}$$ (12)

$${\sigma }_B^2\leftarrow \frac{1}{m}\sum _{i=1}^m{(X_{\mathit{Ii}}-{\mu }_B)}^2$$ (13)

where $m$ represents the size of the batch. ${\mu }_B$ and ${\sigma }_B^2$ represent the average and variance of batch data, respectively. $\gamma$ and $\beta$ are training parameters, initially, $\gamma \text{=}1$ and$\beta \text{=}0$. Through batch normalization, the original dense data distribution is more uniform and the non-linear transformation function falls into the region sensitive to input, which can avoid the problem of gradient disappearance. Then, the threshold operation is performed on each element through the non-linear activation layer, which effectively improves the accuracy of the COVID-19 negative and positive classification network.

The proposed COVID-RDNet model has the following advantages:

First, the proposed model obtains the local lesion feature by continuously reusing 3*3 expansion convolution of the same size, which effectively enhances the generalization ability of the proposed model. The novel coronavirus pneumonia feature reuse is achieved through the bottleneck convolution of 1*1 in four dense blocks and the computation amount of the training process is reduced to a great extent by means of dimension reduction in the proposed COVID-RDNet model.

Second, because the difference between the COVID-19 region and the normal region is not very significant, the traditional Resnet model adjusts the depth and width by adding cross-layer values. However, in the convolution process, the input of the upper layer cannot divide the convolution kernel, which results in the loss of the feature details. The proposed model overcomes the loss of details by cross-layer links.

Next, because it is used every time 1*1@128 and 1*1@32, the convolution kernel for feature extraction leads to the sudden drop and rise of the convolution layer scale with large dimensions of the subsequent feature maps, which are not conducive to the stability of the model. The proposed model uses the cross-layer link to gradually improve the extracted feature dimension, realizes the non-linear connection between local features and global features, and overcomes the problem of network degradation caused by the increase of model depth.

Finally, the 1*1 bottleneck convolution operation is used to map multiple feature components into two categories of labels. Thus, the mixed dataset can be classified, which makes the COVID-19 detection of the proposed COVID-RDNet model more accurate. Besides, the initial epochs, learning rate, and corresponding decay factors of the proposed model are set as 30,$1\times {10}^{-4}$, and 0.1, respectively. The gradient moment estimation is calculated by the adaptive moment estimation optimizer to stabilize the parameters in a dynamic range. The final output is obtained by $\mathit{softmax}(y^{\prime })=\frac{e^y}{{\sum }_{i=1}^N{\sum }_{j=1}^H{\sum }_{k=1}^W{e}^{y_{\mathit{ijk}}}}$.

3. Experiment and analysis

3.1. Dataset and evaluation indexes

To verify the effectiveness of the proposed model, two open datasets are used to train and test. Here, the X-ray images are from the public dataset provided by Guangzhou Women's and children's medical center. There are normal regions in the X-ray images, while the images of patients with COVID-19 usually show focal consolidation. Besides, the bilateral lung regions with severe viral pneumonia show diffuse interstitial. The CT images are from the open-source COVID-CT dataset provided by some researchers at the University of California, San Diego. The chest CT images of the novel coronavirus pneumonia usually show patchy or segmental ground glass density. Most of the lesions are gridded in shape and a few cases are associated with a small amount of pleural effusion or pericardial effusion. Because the image sizes in the two datasets are different, for the sake of fairness, this paper uniformly sets the size of all X-ray and CT images to 224*224. For fairness, this experiment employs Intel(R)Core(TM) i5-7200UCPU@2.50 GHz, 4 GB of memory, Windows 10 Professional, and 64-bit operating systems. Besides, to evaluate the performance of the proposed COVID-RDNet model to classify negative and positive COVID-19 [53], [54], sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy [55], [56] are used as follows:

$$\mathit{sensitivity}=\frac{\mathit{TP}}{\mathit{TP}+FN}\times 100\text{\%}$$ (14)

$$\mathit{specificity}=\frac{\mathit{TN}}{\mathit{TN}+FP}\times 100\text{\%}$$ (15)

$$\mathit{PPV}=\frac{\mathit{TP}}{\mathit{TP}+FP}\times 100\text{\%}$$ (16)

$$\mathit{NPV}=\frac{\mathit{TN}}{\mathit{TN}+FN}\times 100\text{\%}$$ (17)

$$\mathit{accuracy}=\frac{\mathit{TP}+TN}{\mathit{TP}+TN+FP+FN}\times 100\text{\%}$$ (18)

$\mathit{TP}=P_1\cap G_1$, $\mathit{FP}=P_1\cap G_2$, $\mathit{FN}=P_2\cap G_1$, and $\mathit{TN}=P_2\cap G_2$. $P_j$ and $G_j$ represent the classification results of the proposed model and ground truth, respectively ($j\in \{1,2\}$ represents normal people and COVID-19, respectively). The relevant code of the proposed algorithm is available on the web: https://pan.baidu.com/s/1bgwT-4k_5bmmBNYIaBKkww.

3.2. The results of the proposed model

3.2.1. The feature maps extracted by the residual block

The proposed algorithm contains two residual blocks, which may successfully extract different lesion features by extending the depth and width, as shown in Fig. 10 . The larger the weight of the retrieved negative and positive COVID-19 features, the lighter the hue in the feature maps. The residual blocks can significantly reduce feature error and the likelihood of network degradation by connecting feature maps from different convolution layers. As a result of gradually developing the network model, residual blocks can acquire more and more negative and positive COVID-19 properties, effectively improving non-linear expression ability.

Fig. 10.

The COVID-19 feature maps extracted from the residual blocks. (Columns 1–3 represent the convolution results of layers 1–3 of the residual blocksfrom left to right). (a) The 1st residual block; (b) The 2nd residual block.

3.2.2. The extraction feature by the dense block

Due to the particularity of medical image sampling, the amount of medical images is limited and public datasets are difficult to match the deep learning model's feature extraction requirements. The proposed model connects the non-linear information gathered from each layer with dense blocks to effectively handle this challenge. Thus, the proposed model does not depend on the lesion features extracted from the last layer in a small dataset. Meanwhile, the local features with low shallow complexity can be comprehensively used, which can better obtain the feature maps with strong generalization performance and then improve the classification accuracy. Besides, to alleviate the feature loss caused by the sudden drop of the feature dimension, the proposed algorithm uses the way of slowly rising the dimension to ensure abstract information. The dimension is reduced by a 1*1 convolution kernel simultaneously, which effectively reduces calculation. Fig. 11 shows the results of COVID-19 feature maps extracted from different dense blocks (blue points). Through the reuse and connection of feature maps, the coupling of the obtained lesion information is reduced.

Fig. 11.

The COVID-19 feature maps extracted from the dense blocks.

3.2.3. The classification results of the proposed model

The proposed network can effectively combine local information with abstract features through the residual blocks and dense blocks, which can ensure the classification accuracy of negative and positive COVID-19. Meanwhile, the COVID-19 features are reused through continuous bottleneck convolution, which greatly reduces the calculation in network training. Fig. 12 shows the COVID-19 visual features extracted at each layer of the residual blocks and the dense blocks, respectively. As can be seen from Fig. 12, the proposed network obtains color, edge, and other information in the residual blocks of layers 1–2. From the third layer of the dense blocks, the proposed model starts to obtain advanced texture features. With the increase of network depth, it can obtain useful features to distinguish between negative and positive COVID-19.

Fig. 12.

The COVID-19 visualized features extracted from the proposed model: (a) The residual block (Columns 1–3 represent the results of layers 1–2 of the residual blocks from left to right.); (b) The dense block.

The proposed COVID-RDNet model can classify negative and positive COVID-19, the average value of the accuracy, AUC, sensitivity, specificity, PPV, and NPV can reach 0.99, 0.9998, 0.999, 0.995, 0.998, and 0.998, respectively, as shown in Fig. 13 . Here, Fig. 13(a) shows the loss rate and accuracy of the proposed model. It can be seen that the accuracy and loss rate with the number of iterations increasing can improve the optimized classification weight, which has a good convergence speed. Fig. 13(b) shows the testing results. The ROC curve on the left can effectively express the influence of different learning rates on the generalization performance. In general, the closer the ROC curve is to the upper left corner, the less the total number of false positives and negatives. The confusion matrix on the right can directly get the data under different parameters and effectively assess the COVID-19 classification.

Fig. 13.

The results of the proposed COVID-RDNet model: (a) The accuracy and loss rate; (b) The testing results. (The left is the ROC curve and the right is the confusion matrix).

3.3. Analysis and comparison

For fairness, all experiments are repeated 10 times in the same environment. Besides, the average values are taken while removing the maximum and minimum ones, thus effectively measuring the effectiveness of the proposed algorithm.

3.3.1. Analysis of the mixed datasets

The proposed COVID-RDNet model can classify the negative and positive COVID-19 of individual X-ray images, CT images, and mixed datasets, as shown in Fig. 14 . It can be seen that the proposed COVID-RDNet model has the best classification performance in sensitivity, specificity, PPV, NPV, area under curve (AUC), and accuracy on the mixed X-ray and CT datasets. Besides, the classification performance of the proposed model using the X-ray images is slightly better than that of the CT images and the corresponding AUC results can reach 0.99 and 0.94, respectively. Therefore, the proposed algorithm can operate on the individual or the mixed dataset, which has good robustness and can obtain an assisted depth network that meets the actual medical diagnosis of doctors.

Fig. 14.

The performance comparison of the proposed model under different datasets.

3.3.2. Analysis of the learning rate

In general, the size of the learning rate setting will directly affect the final classification accuracy of the proposed model. If the learning rate is too large, it will lead to serious network oscillation and the model can not get the global optimal value. On the contrary, if the learning rate is too small, it will easily lead to the slow decline of the loss rate, which will not only reduce the classification accuracy, but also obtain the local optimal weight. Therefore, the proposed model obtains the COVID-19 local features through the residual blocks and connects it with the abstract information of lesions through the dense blocks, which can effectively reduce the impact of learning rate on model training. Fig. 15 shows sensitivity, specificity, PPV, NPV, AUC, and accuracy of the proposed COVID-RDNet model under different learning rates. It can be seen that when the learning rate is in the range of 0.00004 to 0.00018, the accuracy of the proposed model can be maintained at 0.99 and the AUC can reach 0.9998. The average results of sensitivity, specificity, PPV, and NPV are also relatively stable. While maintaining a high classification accuracy, this model also has strong robustness and fault tolerance. When the learning rate is greater than 0.00018 or less than 0.00004, the performance of the proposed network will decrease. Therefore, this paper selects the value in the range of 0.00004 to 0.00018 as the initial learning rate. After five rounds of model training, the learning rate [57], [58], [59] decreases to 1/10 of the initial value.

Fig. 15.

The performance of the proposed model under different learning rates. (From the upper left corner to the lower right corner are sensitivity, specificity, PPV, NPV, AUC, and accuracy, respectively).

3.3.3. Performance comparison of residual blocks and dense blocks in different models

To further verify the performance of the proposed algorithm, this paper measures the network by comparing the sensitivity, specificity, PPV, NPV, AUV, and accuracy of the mixed X-ray and CT images by different models, as shown in Fig. 16 . There are no residual blocks or dense blocks in the Alexnet [19] and the Vgg [22], [23] models. The Resnet network contains residual blocks and the values of adjacent feature maps are added through the cross-layer link, which effectively enhances the COVID-19 features and has high sensitivity and PPV. However, the classification accuracy of the Resnet [26] network is related to the initial learning rate and does not have good stability, especially in the specificity and the NPV. The Densenet model contains dense blocks, which effectively enhance the stability of the network by connecting local and global information. It performs smoothly in sensitivity, PPV, and NPV. However, the Densenet [31] network reduces the feature dimension in a sudden way, which results in the loss of COVID-19 features. The Resnet and Densenet models contain residual blocks and dense blocks, respectively, which improve the performance compared with the Alexnet and the Vgg models. To overcome the above problems, the proposed COVID-RDNet model effectively improves the accuracy and accuracy through the linkage reuse of the residual blocks and the dense blocks, which can realize the efficient classification of the negative and positive COVID-19.

Fig. 16.

The performance of different models. (From the upper left corner to the lower right corner are sensitivity, specificity, PPV, NPV, AUC, and accuracy, respectively).

3.3.4. Comparison with the results of the state-of-the-art algorithms

This paper compares the results of the proposed model with the state-of-the-art networks under the same dataset, as shown in Table 2 . Here, Maghdid et al. [19] enhance the classification accuracy by combining the Alexnet model with the CNN model of transfer learning. Besides, Turkoglu et al. [20] and Loey et al. [21] enhance the ability to extract COVID-19 features by improving the Alexnet model. However, the step size in the above models is larger than the pooling size, which may result in redundancy. To solve this problem, the Vgg model replaces the large convolution kernel in the Alexnet model by reusing a small kernel with equal size, which can effectively enhance computational efficiency. Sitaula et al. [23] obtain spatial features of lesions by adding an attention mechanism to the Vgg model. On this basis, Shibly et al. [24] and Lee et al. [25] enhance the efficiency of the Vgg network by integrating a fast CNN network. However, most of the parameters come from the first fully connected layer, which may result in the need for large storage space. To effectively improve the utilization of COVID-19 features, the Resnet model can save space costs through cross-layer links. Hira et al. [30] improved the classification accuracy through transfer learning, but the stability of model training is slightly insufficient. On the contrary, Sakib et al. [29] have good stability, but at the expense of accuracy. Based on refs. [29], [30], Zhou et al. [28] input the regrouped images into the residual blocks for lesion feature extraction, which can effectively improve both robustness and accuracy. However, due to the improper size of the convolution kernel in the Resnet model, the problem of feature loss occurs. To overcome the problem, Tabrizchi et al. [32] and Zhang YuDong et al. [33] connect abstract features with local features by combining the Densenet model and transfer learning. Chowdhury et al. [34] improve the classification accuracy of the Densenet model through image enhancement, but too many layers may lead to redundancy. The proposed model can effectively solve the problem of feature loss through residual blocks and dense blocks. The accuracy of negative and positive COVID-19 classification can reach 0.99.

Table 2.

The proposed model results are compared with the state-of-the-art network.

Method	Reference	Evaluating indexes
		Sensitivity	Specificity	PPV	NPV	Accuracy
Alexnet	Alexnet model [18]	0.86	0.92	0.91	0.94	0.91
	Maghdid et al. [19]	0.92	0.96	0.95	0.97	0.95
	Turkoglu et al. [20]	0.92	0.97	0.97	0.91	0.94
	Loey et al. [21]	0.92	0.96	0.93	0.93	0.93
Vgg	Vgg16 model [22]	0.92	0.94	0.91	0.97	0.93
	Vgg19 model [22]	0.96	0.96	0.95	0.97	0.96
	Sitaula et et al. [23]	0.96	0.96	0.94	0.97	0.96
	Shibly et al. [24]	0.97	0.95	0.93	0.98	0.96
	Lee et al. [25]	0.87	0.89	0.89	0.87	0.89
Resnet	Resnet18 model [26]	0.98	0.94	0.93	0.98	0.96
	Resnet50 model [26]	0.97	0.98	0.98	0.98	0.97
	Resnet101 model [26]	0.97	0.94	0.98	0.99	0.97
	Zhou et al. [28]	0.88	0.97	0.936	0.93	0.93
	Sakib et al. [29]	0.89	0.91	0.78	0.98	0.90
	Hira et al. [30]	0.96	0.94	0.97	0.87	0.90
Densenet	Densenet121 model [31]	0.97	0.97	0.97	0.97	0.97
	Densenet201 model [31]	0.98	0.91	0.95	0.98	0.97
	Tabrizchi et al. [32]	0.87	0.95	0.97	0.91	0.95
	Mangal et al. [33]	0.98	0.87	0.74	0.99	0.92
	Chowdhury et al. [34]	0.99	0.97	0.97	0.98	0.98
Proposed		0.99	0.98	0.99	0.99	0.99

4. Conclusion

In this paper, a deep learning network is proposed to effectively improve the classification accuracy of negative and positive COVID-19. Firstly, to ensure the authenticity and effectiveness of COVID-19 detection, a mixed X-ray and CT dataset are established through the published COVID-19 images. Then, the COVID-19 region features are effectively enhanced by the adaptive region enhancement algorithm. The non-linear COVID-19 features are obtained through the cross-layer link of the residual blocks. The local and abstract information is connected through the dense blocks. Compared with the state-of-the-art COVID-19 classification models, the proposed COVID-RDNet model has good negative and positive COVID-19 classification performance through sensitivity, specificity, PPV, NPV, AUC, and accuracy on the mixed X-ray and CT dataset. When the learning rate is in the range of 0.00004 to 0.00018, the accuracy of the proposed model can be maintained at 0.99 and the AUC can reach 0.9998.

Based on respect for patient privacy and ethics, the proposed model obtains high accurate classification results by manipulating the mixed dataset containing real cases, which can effectively assist doctors to determine the infection status of COVID-19 patients and has good application prospects. However, the proposed algorithm can not guarantee the computational complexity while improving the classification accuracy, which is the next research focus. Meanwhile, this paper also aims to overcome the problem that the number of medical annotation images is insufficient and does not meet the conditions of deep learning through unsupervised or self-supervised models, which can further meet the actual medical needs.

Ethical approval

This paper avoid distorting the research results and refuse to do anything that harms the trust of journals, the professionalism of scientific authors and the ultimate scientific efforts. Ensuring that the rules of good scientific practice are followed is conducive to maintaining the integrity of research and its statements.

Funding details

This work was supported by Natural Science Foundation of Liaoning Province under Grant 2021-MS-272 and Educational Committee project of Liaoning Province under Grant LJKQZ2021088.

Informed consent

All the authors of this paper, Lingling Fang and Xin Wang, agreed to submit the “COVID-RDNet: A novel coronavirus pneumonia classification model using the mixed dataset by CT and X-rays images” to the “Biocybernetics and Biomedical Engineering” journal.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.