A Transfer Learning-Based Approach with Deep CNN for COVID-19- and Pneumonia-Affected Chest X-ray Image Classification

Abstract

The COVID-19 pandemic creates a significant impact on everyone’s life. One of the fundamental movements to cope with this challenge is identifying the COVID-19-affected patients as early as possible. In this paper, we classified COVID-19, Pneumonia, and Healthy cases from the chest X-ray images by applying the transfer learning approach on the pre-trained VGG-19 architecture. We use MongoDB as a database to store the original image and corresponding category. The analysis is performed on a public dataset of 3797 X-ray images, among them COVID-19 affected (1184 images), Pneumonia affected (1294 images), and Healthy (1319 images) (https://www.kaggle.com/tawsifurrahman/covid19-radiography-database/version/3). This research gained an accuracy of 97.11%, average precision of 97%, and average Recall of 97% on the test dataset.

Introduction

Medical Imaging plays a significant role in the modern healthcare system. Advancements of medical imaging devices, like Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and X-Ray Imaging have become extremely necessary for disease diagnosis. Deep learning plays an essential factor in the medical industry as it can achieve human-level accuracy for image classification and segmentation tasks [1]. Deep learning can perform great on tasks in the fields like computer-aided diagnostics [2], recognition of lesions and tumors [3], and brain–computer interface [4]. Deep learning can easily execute time-consuming tasks, like detect symptoms of COVID-19 and Pneumonia from the chest X-ray images as the lungs of affected patients carry some visual marks of that distinct disease. Newly identified coronavirus causes an infectious disease namely COVID-19 [5]. Many patients who are suffering from COVID-19 can have minor to moderate effects and recover without additional care. An infected person’s cough and sneeze primarily spread COVID-19. Pneumonia is usually caused by viruses, fungi, and bacteria and creates infection in the lungs [6]. Both COVID-19 and pneumonia become life-threatening for those who have chronic diseases. These diseases decrease the body’s oxygen level and create complications for a person to breathe. Patients with serious conditions have to be hospitalized and need to be on ventilator support.

DBMS plays an important role in the medical field to save data for further analysis. Traditional Database Management Systems are inadequate in semi-structured and unstructured data because of not maintaining Big data features, namely velocity, value, variety, validity, veracity, volume, and volatility [7, 8]. Several high-tech companies like LinkedIn, Google, Facebook, and Amazon have customized storage systems to operate with huge data volumes. Not Only SQL (NoSQL) is a distributed database management system, and its demand is increasing to handle big data efficiently. As working with vast amounts of data, big companies develop their NoSQL databases, i.e., Ebay’s MongoDB [9], Google’s Bigtable [10], Facebook’s Cassandra [7].

In this paper, we developed a system using the transfer learning method to classify chest X-ray images into COVID-19, Pneumonia, and Healthy. At first, we applied several image processing techniques on the chest X-ray images, and used pre-trained VGG-19 architecture, and transferred weights for classification purposes. Finally, MongoDB [9] is used as a database to store the original image and corresponding category images as MongoDB needs less data load time and query time than traditional RDMS in the case of BLOB files. Here, our goal is to make a flexible system that can modify in the future. Our system has achieved a higher accuracy using a pre-trained model’s most of the features. Moreover, this work describes the background database and the advantage of using this kind of database to store classified data for further study. By connecting the deep learning model with the database makes this research complete and formulates a new dimension.

The rest of the paper is arranged as follows: “Related Works” describes some related and important recent works. “X‑ray Image Classification” outlines the methodology of this work. The experimental outcome is analyzed in “Performance Analysis”. Finally, a summary of our findings is described in the “Conclusions” section.

Related Works

In recent times, several good works are done utilizing Convolutional Neural Network to solve medical-related problems such as detecting brain tumors from MRI images, breast cancer recognization, classification of diseases using X-ray images [11]. Fukushima originally introduced CNN with the help of the model developed by Wiesel and Hubel [12]. By utilizing CNN, segmentation of neuron membranes was proposed by Ciresan et al. [13]. Wang et al. [14] developed a unique dataset with 108,948 X-ray images of 32,717 patients, which accomplished excellent outcomes utilizing a deep CNN. Rajpurkar et al. [15] developed a deep CNN architecture to identify 14 distinct pathologies from chest X-ray images, and the model contains 121 layers. Zhou et al. [16] applied the transfer learning approach for distinguishing between malignant and benign tumors by utilizing InceptionV3 architecture. Deniz et al. [17] introduced a classifier by utilizing a pre-trained VGG-16 architecture for breast cancer. During the feature extracting procedure, the proposed system used AlexNet architecture. Furthermore, transfer learning was applied to smaller datasets of MRI images, and the time of feature extraction portion utilize pre-trained networks [18]. Yang et al. [19] utilized GoogLeNet and AlexNet for their experiment on glioma grading, and GoogLeNet displayed excellent results for the analysis compare to AlexNet.

Here, we successfully applied the transfer learning approach on the pre-trained VGG-19 architecture. We measured the system’s performance for the classification of COVID-19-affected, Pneumonia-affected, and Healthy people from the chest X-ray images. Furthermore, MongoDB is used as a storage for the original images and classified labels.

X-ray Image Classification

This section includes the following steps: image pre-processing, classification of X-ray images using transfer learning, and finally, description of data storing process in MongoDB database. The entire working process of the system is shown in Fig. 1.

Fig. 1.

Overview of the whole system

Image Pre-processing

We applied several image processing steps to make the image ready for the experiment. The primary image was 512$\times 512\times$3 sized. We resize the images into 256$\times 256\times$3 size. Furthermore, apply a Gaussian filter on the resized image and produce a filtered image.

$$\begin{array}{c}\hfill G(X,Y)=\frac{1}{2\pi {\sigma }^2}{e}^{-\frac{X^2+Y^2}{2\pi {\sigma }^2}}.\end{array}$$ 1

Equation 1 represents the 2D Gaussian function with a kernel of size 5$\times$5 is applied. Here, X and Y indicate the length between origin and the horizontal axis and the length between origin and the vertical axis, respectively, and $\sigma$ indicates the standard deviation. It removes noise from a resized image. The histogram equalization is applied on every filtered image. This technique enhances the contrast of an image and also spread out values that appear most frequently. These X-ray images are multichannel, and each image has three different (RGB) channels. To utilize the full power of VGG and achieve higher accuracy multi-channel X-Ray images are used, as the VGG-19 [20] model is trained using three channels images. Therefore, we have to separate each channel and apply histogram equalization individually. After that, we merge back every histogram equalized channel and get an equalized image.

Figure 2b shows the resized image of Fig. 2a. Figure 2c represents the filtered image as it helps to reduce noise and make the image smoother. Also, it enhances the probability for detecting features. Furthermore, Fig. 2d shows the histogram equalized image. Histogram equalization is applied for balancing the contrast of an image by adjusting the intensity distribution. The goal of this method is to provide a linear relationship to the cumulative likelihood function. Figure 3 represents histogram on various pre-processing level outcomes. Figure 3a–c represents the histogram of primary, resized and filtered image, respectively, for an X-ray image. Figure 3d represents the histogram of an equalized image where all the contents are balanced.

Fig. 2.

Image pre-processing result for the system

Fig. 3.

Histogram of various pre-processing level outcomes

Transfer Learning for X-ray Image Classification

Transfer learning is a process to reuse parameters that have been trained before. It implies transferring the pre-trained parameters to a different model. Moreover, pre-trained layers can be applied as a feature extractor.

VGG-19 is a 19-layer deep architecture [20]. It is a CNN model trained on the ImageNet dataset that includes millions of images of 1000 categories. The input layer’s size of VGG-19 is 224$\times 224\times$3, followed by several filters, each with a kernel size of 3$\times$3. Therefore, originally it takes three-channel images. It contains multiple convolutional layers. The convolution layer applies several convolution kernels for the extraction of various features.

Transfer Parameters from VGG-19

Figure 4 shows the transfer parameters from VGG-19 for the classification task. From the 19 layers of the VGG-19 architecture, we used the first 16 layers. The architecture of the first 16 layers of VGG-19 is not the same as VGG-16 [20]. VGG-16 has fewer convolutional layers than VGG-19. All of these convolutional layers of VGG-19 are also sequentially organized in five blocks. These parameters are considered frozen or non-trainable.

Fig. 4.

Transfer parameters from VGG-19 for the classification task

The Classification Model

We replace the original input layer of the VGG-19 model with a layer that can take 256$\times 256\times$3 sized images as input. Instead of the last three layers, added a dense layer with three neurons and used softmax as an activation function. This layer is trainable. Figure 5 represents the entire architecture of the classification model. The overview of the trainable and non-trainable parameters is shown in Table 1.

Fig. 5.

Architecture of the classification model

Table 1.

Parameters for the classification model

Layers	Parameters
Block1 (Non-trainable)	38,720
Block2 (Non-trainable)	221,440
Block3 (Non-trainable)	2,065,408
Block4 (Non-trainable)	8,259,584
Block5 (Non-trainable)	9,439,232
Dense (Trainable)	98,307
Total	20,122,691
Total trainable	98,307
Total non-trainable	20,024,384

Mongodb as a Data Storage

In the medical domain, it is essential to store data in a database for further analysis. Figure 6 illustrates the workflow of data storage in the medical domain.

Fig. 6.

Workflow of data storage in the medical field

BLOB file is a collection of binary data. Images, audios, and other types of records are saved as blob files. If an image is converted into a BLOB file, that image becomes compressed and uses less space. Further, we can do several operations as it already been converted into a binary file. Moreover, at the moment of movement of the database not have to bother with the distributing files. For saving an X-ray image in the database, we have to convert it into a BLOB file. The relational database management systems are popular technology in the case of data storage. However, it is difficult to work with BLOB files in the traditional SQL database. Storing BLOB files into an SQL database requires a huge data query process. If the size of an image is small, it can easily store as a BLOB in the SQL DBMS. However, for a large file storing or retrieving BLOB on RDBMS is a costly process and needs large memory. Furthermore, the SQL databases face challenges in the situation like the scalability of the system, and fast data access during complicated operations [21]. It is easy to work with a BLOB file in the MongoDB database as it stores a BLOB file by splitting it into several numbers of chunks. At the time of data retrieval, all the chunks merge into one single file and produce the result. For this reason, MongoDB needs lesser data load time and query time than traditional RDMS. Therefore, we used MongoDB with GridFS interface as a data storage for the system.

After the classification step, the original images and the corresponding label have to store in the MongoDB database. However, MongoDB cannot retain a document file that crosses 16 MB in size. To tackle this problem, the GridFS file storage system is introduced. A combination of MongoDB and GridFS can store image files that are massive in size. Data are separated into two collections in the MongoDB server. These are fs.files and fs.chunks. In fs.files, there are different fields like id, file name, encoding type, MD5, chunk size, length, and upload time. The id field contains a unique key for a file. For this experiment, all the labels are stored in the file name field. The chunk size field determines the volume of a chunk in bytes. The upload time field stands for the time of uploading the file. The server automatically generates MD5, and it represents a check value.

In the case of fs.chunks collection, there are several fields like id, chunk number, file id, and data package. Here id field contains a unique key for a chunk. File id is the same as id in the fs.files for a particular file, and it’s fixed for all the chunks of the same file. The chunk number field represents the chunk’s position, and the data package contains a binary representation of the chunk for an image. Figure 7 represents the data storage for an X-ray image using MongoDB and GridFS interface.

Fig. 7.

Store images in the MongoDB server

Performance Analysis

The experimental outcomes are illustrated in this Section. We analyze system performance using different performance metrics.

Dataset

The chest X-ray images obtained from different publicly available sources [22, 23]. We used 3797 images, among these COVID-19 (1,184 images), Pneumonia (1294 images), and Healthy (1319 images). We separated them into three sets from the original dataset and 2733 images, 72% of the dataset, were used for training purposes. 304 images, 8% of the dataset, were used for validation purposes, and the rest of the 760 images, 20% of the dataset, were used for testing purposes. Table 2 represents the summary of the used dataset.

Table 2.

Description of dataset

Label	Train set	Validation set	Test set	Total
COVID-19	852	94	238	1184
Pneumonia	932	108	254	1294
Healthy	949	102	268	1319

Performance Metric

The performance of the classification model is tested using precision, recall, and F1-score. We also use the confusion matrix and ROC curve. We average all the metrics of different classes for validation purposes and give priority base on precision, recall, and F1 score, respectively. During measuring, the following equations are used.

$$\begin{array}{cc}\hfill \mathrm{Precision}=& \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}},\hfill \end{array}$$ 2

$$\begin{array}{cc}\hfill \mathrm{Recall}=& \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},\hfill \end{array}$$ 3

$$\begin{array}{cc}\hfill F1\text{-}\mathrm{score}=& 2\times \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}.\hfill \end{array}$$ 4

Experimental Setup

We executed all the image pre-processing steps analysis on a computer having an operating system of Windows 10, with NVIDIA GeForce GPU and 8 GB RAM CPU. The rest of the operations, including training and testing of the classification model, were executed on Colab.

Adam optimizer [24] is applied to compile the classification model. Adam optimizer is a powerful optimization algorithm. Moreover, it combines different optimization techniques, including Root Mean Square Propagation, and sparse categorical cross-entropy is used as a loss function. Here, learning rate $\alpha$ = 0.001 and momentum parameters ${\beta }_1$ = 0.9 & ${\beta }_2$ = 0.999. A batch size of 32 and 30 epochs were applied while fitting the training data on the model.

Experimental Results

Figures 8 and 9 represent the accuracy and the loss of the training and validation set, respectively. We got an accuracy of 97.11% while testing the model on the test dataset.

Fig. 8.

Accuracy curve for the classification process

Fig. 9.

Loss curve for the classification process

Figure 10 shows the confusion matrix for the X-ray image classification model. For testing purposes, 238 COVID-19-affected chest X-ray images as Class 0, 268 images of healthy chest X-ray as Class 1, and 254 images of pneumonia-affected chest X-ray as Class 2 were used. Table 3 represents performance measure indices. The model achieved a precision of 99%, 95%, 97% for Class 0 (COVID-19), Class 1 (Healthy), and Class 2 (Pneumonia), respectively. The average precision is 97%. Figure 11 represents the ROC curve. The overall accuracy grows higher as the curve moves nearer to the top left corner.

Fig. 10.

Confusion matrix. Class 0 (COVID-19), Class 1 (Healthy), and Class 2 (Pneumonia)

Table 3.

Performance measure indices

Class	Precision	Recall	F1-score	Support
Class 0 (COVID)	0.99	0.97	0.98	238
Class 1 (Healthy)	0.95	0.98	0.96	268
Class 2 (Pneumonia)	0.97	0.96	0.96	254
Avg./Total	0.97	0.97	0.97	760

Fig. 11.

ROC curve of the classification task

Moreover, the recommended system performance is analyzed in Table 4 with related works. This table also describes the methodology behind all the tasks. The proposed method has gained 97.11% accuracy, which is best in all the referenced studies.

Table 4.

Comparison of recently used approaches for COVID-19 detection

References	Dataset	Methodology	Accuracy (%)
This work	X-ray image	Transfer learning	97.11
Sethy et al. [25]	X-ray image	Deep learning and SVM	95.33
Heidari et al. [26]	X-ray image (Same dataset)	Transfer learning	94.50
Pathak et al. [27]	X-ray image (Same dataset)	Transfer learning	93.02
Oh et al. [28]	X-ray image	Statistical approach	88.90
Wang et al. [29]	CT scan	Transfer learning	85.20
Azemin et al. [30]	X-ray image	Deep learning	71.90

Conclusions

In this paper, our purpose is to develop a deep learning-based method to classify COVID-19- and Pneumonia-affected chest X-ray images by applying transfer learning. Here, we use the pre-trained VGG-19 architecture and transfer its weights and get an excellent accuracy. Moreover, we use MongoDB as a database to store the original image and corresponding category.

Our findings confirm that deep learning can make the diagnostic process more comfortable as it can achieve human-level accuracy for image classification tasks. Deep learning can efficiently execute time-consuming tasks, like detect symptoms of COVID-19 and Pneumonia from the chest X-ray images, and it will be an excellent asset for doctors. Therefore, Deep learning will play an impactful role in saving valuable lives.

Declarations

Conflict of interest

On behalf of all the authors, the corresponding author states that there is no conflict of interest.