Diagnosing COVID-19 from CT Image of Lung Segmentation & Classification with Deep Learning Based on Convolutional Neural Networks

Abstract

Early-stage exposure and analysis of diseases are life-threatening causes for controlling the spread of COVID-19. Recently, Deep Learning (DL) centered approaches have projected intended for COVID-19 during the initial stage through the Computed Tomography (CT) mechanism is to simplify and aid with the analysis. However, these methodologiesundergocommencing one of the following issues: each CT scan slice treated separately and train and evaluate from the same dataset the strategies for image collections. Independent slice therapy is the identical patient involved in the preparation and set the tests at the same time, which can yield inaccurate outcomes. It also poses the issue of whether or not an individual should compare the scans of the same patient. This paper aims to establish image classifiers to determine whether a patient tested positive or negative for COVID-19 centered on lung CT scan imageries. In doing so, a Visual Geometry Group-16 (VGG-16) and a Convolutional Neural Network (CNN) 3-layer model used for marking. The images are first segmented using K-means Clustering before the classification to increase classification efficiency. Then, the VGG-16 model and the 3-layer CNN model implemented on the raw and segmented data. The impact of the segmentation of the image and two versions are explored and compared, respectively. Various tuning techniques were performed and tested to improve the VGG-16 model's performance, including increasing epochs, optimizer adjustment, and decreasing the learning rate. Moreover, pre-trained weights of the VGG-16 the model added to enhance the algorithm.

Introduction

World Health Organization (WHO) formally announced the outburst of a virus COVID-19, the infection affected through Severe Acute Respiratory Syndrome (SARS) CoV-2 in March 2020. COVID-19 is extremely contagious and may become a lethal syndrome of Acute Respiratory Distress (ARDS). Initial exposure and analysis are the key elements in controlling the spread of virus disease. The most common screening process to detect and test the virus is Reverse-Transcription Polymerase Chain Reaction. Though it is a strenuous operation and several analyses have stated its less sensitivity in the primary stage [1]. CT screening dataset to diagnose the latest COVID-19 corona, SARS-COV-2, Virus ailment, which first appeared in Wuhan, China in December 2019 [2] and has caused 64,384 deaths to date, nearly one million confirmed cases worldwide, as of 5 April 2019, with many more to be tested [3]. However, for a large number of suspected and asymptomatic compromised patients, all affected patients will not be inspected and healed. Owing to the high rate of infection, a rapid diagnosis using Artificial Intelligence (AI) is becoming predictable. An accurate and quick analysis of COVID-19 can distinguish infectious patients to postpone disease progression [4].

An extensive selection of developers armed with a cloud subscription can now access knowledge that was once the domain of hardcore AI professionals. Machine Learning (ML) was once the domain of data scientists. Since 2016, we have seen the development of simpler models, granting independence to developers and the ability to create complex machine learning models with minimal effort and expertise. This wizard-like environment for graphical development allowed non-data scientists to play with their hands with ML models. With the Rapid Application Development (RAD) process, people with no experience in data science, equipped with a limited amount of data and transfer learning technology, build advanced, industry-standard production-ready AI models.

The availability of machine learning algorithms made available as a service has Microsoft Cognitive Services included in the Microsoft Azure cloud solution. This ML framework allows developers to upload their images and create computer vision for Microsoft Azure Custom Vision. It is based on a pre-trained CNN and offers a technique of transfer learning for users. By simply importing the training images and marking them into a model generator, it enables AI models to create using the web application Custom Vision. Because of the transfer learning technology, which starts with a pre-trained model and uses this model as a feature extractor, Custom Vision does not require as many images for training and testing as the regular CNN.A minimum requirement of 50 images per tag suggested.

In Custom Vision, create templates and use a REST API as a web application to run them. On the Microsoft Custom Vision documentation site for Curl, C #, Java, Javascript, ObjC, PHP, Python, Ruby, sample code documentation is available. Custom Vision is the state-of-the-art machine learning technology that enables its trained model export to Tensorflow, Tensorflowlite, Tensorflowjs, CoreML, ONNX, and Dockerfile formats by multiple platforms easily. Custom Vision currently enables image recognition and target detection. In Image classification, it has two subclasses, 1. Multilabel (Multiple tags, per image), 2. Multiclass (single tag per image) in four distinct domains. Such domains are pre-trained so that few sample images can train effectively. The lightweight domain custom Vision models can be passed to smartphones and edge computers to achieve real-time on-device inference.

A CT scan produces accurate organ images, muscles, fleshy tissue, and lifeblood vessels. Doctors can discern interior structures, shape, scale, texture, and density through CT images [5]. Thus, CT scans have far further accurate image conditions of the patients than standard X-Rays [6]. It is necessary to use this basic information to decide when a medical condition occurs as well as the degree and specific position of the issue. Aimed at these purposes, various DL methodologies for COVID-19 screening [7–9] have recently suggested in CT scans.

The primary bottleneck of a study is the absence of excellent quantitative datasets, such as those cited above. The COVID-CT dataset contains images excavated after academic journals were possibly the main effort to construct such a dataset [10]. For the most revised version, the precision, Area Under the Curve, and F1-score were 86%, 94%, and 85%, respectively [11, 12]. Another collection of CT scans has been made widely accessible and comprises 2482 CT images taken from clinics in Sao Paulo, Brazil [13]. For precision, sensitivity, and predicatively, reported 97.38 percent, 95.53% and 99.16% respectively.

Related Work & Datasets

To derive any sort of inference datasets are very crucial. If the use-case belongs to the healthcare domain, then proper use of DL Technique data points will be useful for building applications in the real world. In different studies, the COVID dataset is coming out of organizations and labs 19 [14]. On Kaggle, datasetobtainablePlatform: https:/www.kaggle.com/novelcorona/sudalairajkumar-Dataset-2019-virus. This dataset has various characteristics such as patient placement, age, symptoms, etc. Johns Researchers, the dataset was prepared by Hopkins University and made available to https:/github.com/ieee8023/covid-chestxraydataset for research:README.md/blob/master?Fbclid = IwAR30yTGBr55Br55WXdCngCoICDENHycmdL2bGwlvl1ckdZMucjGH10Uakz7MucjGH10Uakz7 Uh, khk. The following are other recent important repositories: https://github.com/mattroconnor/deep_learning_coronavirus_cure [15].

DL based segmentation method is for identifying the region of interest in CT lung images for COVID-19 quantification. 2D and 3D DL techniques proposed, and accuracy of 99.6 percent in COVID-19 background [16]. Gainesville Regional Utilities (GRU) used bidirectional mechanisms to scale the suggested solution from large COVID-19 datasets [17]. VGG-16 was used to identify the CT images on COVID-19 produces 86.7% accuracy on benchmark datasets using the DL context. Different attempts made in the research community to expose the negative effects of the corona virus mitigated.

The diagnosis of COVID-19 is accomplished by the multimodal and deep learning approach. In this work X-Ray and CT scans are utilized in the disease identification and the augmentation based identification is also accomplished by DNN [18, 19]. The information transmitted over the internet or stored in the cloud can be secured with Blockchain technology and other cryptography mechanism [20, 21]. The light weigh protocol and other cryptography mechanism are developed for securing the COVID and patient information over the internet [22, 23].

Proposed Methodology

In this pandemic, the COVID identification and the infection level of lung infection is a necessary process. The lack of medical experts for huge population, medical resource and early detection of disease has motivated this scheme. The suggested technique aimed at COVID-19 screening centered on CT scans discussed in this section. Stretched the CNN family's architecture and fitted the models with safe and SARS-COV-2 contaminated patient's CT images at the end. The CT scan images taken from the Kaggle stated above and are subject to the pre-processing defined below [24–26].

Deep Learning

DL is an extension of machine learning and consists of several high-level characteristics layers such as CNN, Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Restricted Boltzmann Machine, and Deep Belief Network. From data inputs, Fig. 1 demonstrates the deep neural architecture. The actual influence of profound learning on several industries is yet to arrive at different common ones exist architects of DL. The position of DL is well-known in the healthcare field, and it has been used to examine several diseases. The CNN architecture is inspired by the visual cortex's organization and functionality designed to replicate neurons' interaction patterns within the human brain. Within a CNN, the neurons separated into a three-dimensional structure, with each group of neurons evaluating a tiny area or image element. RNN is ideal, also called sequential data, for temporal data. It seems that CNN is stronger than RNN. As compared to CNN, RNN includes less function compatibility. This network takes inputs of a fixed size and produces outputs. A CNN is a subset of deep neural networks in deep learning, most widely applied to visual imagery processing. Biological mechanisms influenced CNN in that the communication pattern between neurons parallels the organization of the visual cortex of the animal.

Fig. 1.

DL architecture with CNN

Import Libraries

Import the library files of COVID-19 datasets using the TensorFlow backend, as shown in Fig. 2.For quick numerical computation, TensorFlow is a Google-created and published Python library. It is a simple library used to create DL models directly or by using wrapper libraries to simplify the process built on top of TensorFlow.

Fig. 2.

Importing library files

Pre-Processing of CT Scan Images

It is the most common method in Machine learning techniques. Its strategies can help to remove unnecessary noise, highlights the image parts can support by the identification challenge, or assist in the training stage of DL. The unwanted information and noise values are removed with the pre-processing technique.A pixel force normalization is applied in the range of [0, 1] and requires the model for convergence during the training phase.

The response images resized to ensure compatibility with the system's structural design for CNN models. In terms of memory and latency, efficient nets have a low dispensation cost, making it easy to exploit higher eminence response images shown in Fig. 3. Therefore, the effect of varying the response determination on the model accuracy. This stage sense a new network factor.

Fig. 3.

Image preprocessing

Image Segmentation of K-Means

This process is the division of an image into various categories. In the field of image segmentation, several forms of research have carried out using clustering. The K-Means clustering algorithm is to recite an image then cluster the different image regions shown in Fig. 4. The segmentation process enhances and simplifies further classification process. The segmentation that is partitioning of image is accomplished by the k-means.The COVID positive and negative original image segmentation is on the left, and the segmented image is on the right, as shown in Figs. 5 and 6, respectively. Hundreds of CT images are analyzed in seconds in the segmentation phase to speed up COVID-19 diagnosis and lead to its containment. It display a sample collected from the segmentation process.

Fig. 4.

Image Segmentation with K-means clustering

Fig. 5.

COVID Positive (left) original image and (right) segmented image

Fig. 6.

COVID Negative (left) original image and (right) segmented image

In Figs. 5 and 6, the segmented lung images are given. In Fig. 5, COVID infected patient image is shown and normal lung image is shown in Fig. 6.

Classification using VGG-16 Model

In some computer vision tasks, such as classifying images, CNN is now capable of outperforming humans. The weights are pre-trained by VGG-16, and then a novel yield layer introduced by the required number of modules is shown in Fig. 7. To use the pre-trained weights, new dataset weights need to add. Go to the Data tab and then press "Add Data Source" to search for the 'Keras Pre-trained Model' dataset that contains weights from different architectures, such as VGG-16, Inception, Resnet50, and Xception.

Fig. 7.

A training set with the validation dataset

The dataset has 3 directories: Preparation, Validation, and Research, and each sub-directory with different images with it. Classify the data with segmented images irrespective of loss and accuracy using VGG-16 is shown in Fig. 8.

Fig. 8.

Segmented image classification using VGG-16

Classification using Simple 3-Layer CNN Model

To become the formal art contrivance vision technique, CNNs have broken the mold and ascended the throne. CNN's image data space is widespread easily, and common among the various types of neural networks. Computer vision tasks, such as image processing and recognition, object detection, etc., phenomenally work well. Classify the data with segmented images irrespective of loss and accuracy using a 3-layer CNN model is shown in Fig. 9.This method partitions into patches all CT images, and then the trained CNN inputs these patches. At the same time, a mark to indicate each detected element allocated for each patch.The classification is effective with minimal error and proficient in training.

Fig. 9.

Segmented image classification using 3-layer CNN model

Comparison between Deep and Simple CNN Models

For both models, the image segmentation allows better learning. Minimize the variation by looking at the results on both segmented images and raw images of the two models. And for the 3-layer CNN model, it has achieved over fitting faster than the segmented images. However, the average accuracy of the segmented images is higher than that of the raw images. It is because of the decline of image features during the segmentation. Furthermore, for the validation accuracy of the VGG-16 model, a straight horizontal line on the segmented images shows that the model predicts the same classes for all iterations. It may be due to a lack of initialization of weight or preparation for optimization or incorrect settings.

Congestion is achieved on both types of images by the 3-layer CNN model during testing. Overfitting during testing, however, was not accomplished by the VGG-16 model. It is presumably due to the gradients disappearance is caused by its architecture. 20 epochs are not enough to induce the first convolutions of the model to the gradients. Thus, additional epochs and different hyper-parameters were provided to the VGG-16 model to enhance its learning. A comparison between these two models is shown in Fig. 10.

Fig. 10.

Comparison of deep and simple CNN models

Experimental Results

The VGG-16 model tuned by increasing epochs, optimizing the channels, and reducing the rate of learning. No influence on the learning of the model was seen by strictly increasing epochs to 200. The oscillation in both loss and precision indicates that the model is still unable to learn the features of the image. Changing the optimizer from Adaptive Moment Estimation (Adam) to Stochastic Gradient Descent (SGD) and decreasing the learning rate indicated a change in the learning model suggested by the continuously decreasing loss of training and testing shown in Fig. 11. The reduction rate, however, is marginal, so the precision is still fluctuating. The training of a deep architectural neural network such as VGG-16 may be limited, given the size of the available dataset of about 600 images. To benefit from the pre-trained weights and to test its ability to learn the image characteristics, the pre-trained VGG-16 model is thus loaded as shown in Fig. 12.

Fig. 11.

VGG-16 model set

Fig. 12.

Pre-trained VGG-16

Implementing the pre-trained model VGG-16 greatly enhanced the learning ability of the model. It achieves over-fitting at about 25 epochs and its precision ranges from 0.7 to 0.8. It is because there are weights that are already trained on large sets of images in the pre-trained model. Only the last two convolutional layers were trained to learn the characteristics of lung CT scan images. By using the pre-trained weights, the model resolved the inconveniences of its deep architecture and the tiny datasets.

Inception is a deep neural network in computer vision that reaches architecture. For embedded or handheld computing machines, the start-up algorithm does a lot of good. Increasing accuracy in deep CNN intended to enhance the excellence of effort and the number of units at each step, in average sizes. The average increase in this model to produce an average of 81.14%.

However, 91.29% and 96.51% respectively are the COVID and Non-COVID sensitivity, indicating that the COVID prediction is less reliable than that of the Non-COVID. The ROC curve of the three classifiers used in the predictive phase of non-responding situations shown in Fig. 13. A new Fully-Connected layer is introduced to adjust the grouping task to a novel field. It highlights the following operations that make up the blocks: dropout, Batch Normalization (BN), and swish activation functions.

Fig. 13.

Pre-trained VGG-16 and trained VGG-16 with CNN

The BN operation bounds the performance layer in a defined range, compelling standard deviation, and zero means. This mechanism is a means of normalization, enhancing the NN reliability, and speed up. By preventing insufficient neurons, imitating a trapped together of numerous NN for every mini-batch during the preparation of dropout operation also serves as regularization. The dropout parameter specifies the number of neurons that are inhibited (0–100% of one-layer neurons).

The DL architecture is used to detect early-stage COVID-19. It considered that the DL is better than the conventional one. Approaches to image classification system classification and with high accuracy effectively reduced the false-negative rate. Nevertheless, relative to inception, it is an ancient process to test with 94.74% with a precision of 81. Fourteen in all, which is significantly smaller than the previous state-of-the-art result. There is a high level of accuracy in the results of the proposed model.

From the observation of the simulation analysis of lung images and the categorization of COVID patient lung as well as the normal lung. The accuracy is comparatively high that shows the proposed technique is effective.

Conclusion

For the finding of COVID-19 designs in CT images using VGG-16 and 3-layer CNN models, a model suggested. The methods proposed to give comparable results on both datasets with the utmost precision. A cross-dataset analysis tested using three setups for two major public databases. The primary significance of the approaches used to identify COVID-19 using CT images is that the technique looks like an actual situation and exposes system weaknesses (for example, in the COVID-CT test collection scenario, the accuracy decreases from 87.68% to 56.16%). The analysis shows that the systems aimed at detecting COVID-19 in CT images need to be expressively improved to be considered a therapeutic alternative. The training of a deep architectural neural network such as VGG-16 may be limited, given the size of the available dataset of about 600 images. To benefit from the pre-trained weights and to test its ability to learn the image characteristics, the pre-trained VGG-16 model is thus loaded. Additional epochs of different hyper-parameters provided to the VGG-16 model to enhance its learning and 3-layer CNN model, it has achieved over fitting faster than the segmented images to improve accuracy.

Biographies

Dr. K. SitaKumari

working as an Associate Professor in the Information Technology Department of Velagapudi Ramakrishna Siddhartha Engineering College, Vijayawada since 2007. Awarded Doctorate from Acharya Nagarjuna University during 2017 for the thesis work done in Software Reliability. Published around 25 journal papers and 10 conference papers. Guided UG ad PG students projects in the areas of Software Engineering, Machine Learning, IoT , Networks, Image Processing and Testing. Data Structures, Computer Organization, Software Engineering, Software Testing,IoT, DAA, OOPs and Operating Systems are few of the courses taught.

Dr. SaritaSamal

obtained her B.Tech (Hons) in electrical engineering from Berhampur University in the year 2003 and M.Tech. in power system engineering from VSSUT, Burla in the year 2006. She has done her Ph.D.in electrical engineering from VSSUT, Burla under the guidance of Dr. P.K.Hota, Professor, VSSUT, Burla. Her current research interests include power quality improvement and application of power electronics with non-conventional energy sources. Currently she is working as Asst. Professor-II in the Department of Electrical Engineering at KIIT Deemed to be University, Bhubaneswar.

Ruby Mishra

received B.E (Mechanical Engineering) from Utkal University in 1999, MTech (Machine Design) from Jadavpur University, in 2007, and Ph.D. (Machine Design) from Jadavpur University in 2014. She has a teaching and research experience of 21 years. Her research interests include Kinematics of Machine and Mechanism, Biomedical Engineering, Biomedical signal and image processing, robotics, medical image analysis. She is a member of The Institute of Engineers and Indian Science Congress Association. She has published 3 national and international patents. She has published many articles that are indexed in Scopus, Web of Science, and Google Scholar. Currently, she is doing research on Biomedical Instruments like Robotic arm for C.T. Guided Biopsy and Single Lumen Microcatheter for Angiography and Embolization. At present, she is working as an Associate Professor in the School of Mechanical Engineering Department of KIIT Deemed to be University.

Gunashekhar Madiraju

received his Master’s Degree in Pedodontics & Preventive Dentistry from SRIHER (DU) in India. He has over 16 years’ of experience working with children to promote oral health. Dr. Shekhar actively involves in research projects related to the specialty and continuously seeks out training and professional development opportunities to remain knowledgeable about the latest technology in dental practice.

M. Nazargi Mahabob

BDS & MDS - - TAMILNADU GOVERNMENT DENTAL COLLEGE, The Tamilnadu Dr. MGR MEDICAL UNIVERSITY, CHENNAI. Phd - - Vinayaka Mission University. Specialist in Oral Medicine and radiology. Experience 17 years of Academic and Clinical experience.

Anil Bangalore Shivappa

Joined the College of Dentistry, King Faisal University in December 2016 as aLecturer, teaching Dental Anatomy and Occlusion and Oral Pathology. Also worked as coordinator of various Introduction to Patient Care (IPC) Coursesand as Committee member in Development and Quality Assurance Committee and Academic affairs Committee. Research interests include Salivary gland tumors, Odontogenic cysts and tumors. Obtained bachelor’s degree (Bachelor of Dental Surgery) from JSS Dental College and Hospital, Mysore Under RajivGandhi University of Health Sciences, Bangalore, India. Received Master of Dental Surgery (MDS) from Bapuji Dental College, Davangere, under Rajiv Gandhi University of Health Sciences, Bangalore, India. Has worked as Senior Lecturer in Anil Neerukonda Institute of Dental Sciences, Visakhapatnam, India. Used various methods of teaching including lectures, practical classes, tutorials, problem-based learning (PBL) and e-learning for dental students. Has done various publications and is an Author of the textbook – “An illustrated Atlas of Tooth Carving and Wax Up Technique”, UK and US edition.

Declarations

Conflict of interest

There is no conflict of interest.