Prognosticating various acute covid lung disorders from COVID-19 patient using chest CT Images

Abstract

The global spread of coronavirus illness has surged dramatically, resulting in a catastrophic pandemic situation. Despite this, accurate screening remains a significant challenge due to difficulties in categorizing infection regions and the minuscule difference between typical pneumonia and COVID (Coronavirus Disease) pneumonia. Diagnosing COVID-19 using the Mask Regional-Convolutional Neural Network (Mask R-CNN) is proposed to classify the chest computerized tomographic (CT) images into COVID-positive and COVID-negative. Covid-19 has a direct effect on the lungs, causing damage to the alveoli, which leads to various lung complications. By fusing multi-class data, the severity level of the patients can be classified using the meta-learning few-shot learning technique with the residual network with 50 layers deep (ResNet-50) as the base classifier. It has been tested with the outcome of COVID positive chest CT image data. From these various classes, it is possible to predict the onset possibilities of acute COVID lung disorders such as sepsis, acute respiratory distress syndrome (ARDS), COVID pneumonia, COVID bronchitis, etc. The first method of classification is proposed to diagnose whether the patient is affected by COVID-19 or not; it achieves a mean Average Precision (mAP) of 91.52% and G-mean of 97.69% with 98.60% of classification accuracy. The second method of classification is proposed for the detection of various acute lung disorders based on severity provide better performance in all the four stages, the average accuracy is of 95.4%, the G-mean for multiclass achieves 94.02%, and the AUC is 93.27% compared with the cutting-edge techniques. It enables healthcare professionals to correctly detect severity for potential treatments.

1. Introduction

Many respiratory viral diseases have spread over the last two decades, such as Severe-Acute-Respiratory-Syndrome (SARS), Middle-East-Respiratory-Syndrome (MERS), and other respiratory diseases. In 2019, major pathogens mainly targeting the respiratory system for humans include novel coronaviruses (CoV) like SARS-CoV, SARS-CoV-2, MERS, MERS-CoV, and influenza-A viruses. Their infections can cause anything from minor respiratory infections to a serious case of pneumonia, even leading to respiratory failure. The first beta coronavirus (SARS-CoV) outbreak in November 2002 led to 774 deaths. COVID-19, a new coronavirus disease that gives rise to a new beta coronavirus (SARS-CoV-2) in 2019, rapidly spreads around the world. Almost 613 million people were affected globally, which equates to about 6.5 million deaths as of September 2022. SAR-CoV-2 is considered more transmissible and virulent than the other two beta-coronaviruses, and it has immediately turned into a pandemic that has become a global medical crisis. Corona virus can be identified using several methods such as antigen testing, antibody testing, Reverse Transcription Polymerase Chain Reaction (RT-PCR) etc. But the challenge is to find the appropriate disease and its severity, which leads to death. Imaging modalities are essential in exposing censorious findings related to the continuation of serious illness. A diverse committee of medical professionals and radiologists from various countries has issued a directive to use chest X-ray and computerized tomography on Covid patients beginning in April 2020. If a person’s breathing is getting worse or they are at risk for a serious breathing illness, they should get chest imaging to find out how bad it is Aswathy et al. (2021). Several acute respiratory syndromes can be identified and classified based on the severity of the corona virus positive patients. Fig. 1 shows the statistical report of covid-19 severity using a CT score categorized by different age groups.

Fig. 1.

Age category of Covid-19 using CT Severity score.

Finding the severity of chronic obstructive pulmonary disease (COPE) at an early stage is still a big challenge. This work helps to categorize the severity of disease and save lives. Fig. 2 explains the general workflow of this research paper, where it initially does a binary classification. It tells whether the victim is affected by covid-19 or not affected by covid-19 using a pre-trained model of the Mask R-CNN algorithm. Madani et al. (2022) reviewed the importance of artificial intelligence to detect breast cancer with different imaging modalities and explained how imaging modalities are efficient in advanced diagnosis. Several AI-based networks have been used for various diagnoses; Mask R-CNN plays an efficient role in image segmentation and classification. Johnson JW proposed automatic nucleus segmentation using Mask R-CNN (Johnson, 2020). Segmentation and detection of pulmonary nodules using 3D CT scan images were discussed by Kopelowitz and Engelhard (2019) and Liu et al. (2018). Mask R-CNN also provides better results in liver segmentation from multi-model images, as proposed by Mulay et al. (2019). Dhieb et al. (2019) suggest using mask r-CNN deep learning models to create a framework for automatically classifying and counting blood cells. Anantharaman et al. (2018) talk about how an improved mask (R-CNN) can be used to find and divide up oral diseases. Hand images can be segmented at different viewpoints and tracked using the mask R-CNN combination proposed by Nguyen et al. (2018). Endoscopic images are used to diagnose gastric cancer at an early stage, as discussed by Shibata et al. (2020), and the identification of breast cancer and classifying its severity are evaluated using Mask R-CNN by Chiao et al. (2019).

Fig. 2.

General workflow of Research.

Meta-based few-shot learning: Previous works used meta-learning to construct a model or optimizer that can rapidly modify and update the values for adaptation to the unseen tasks outlined by Podder et al. (2021) in order to address the data insufficiency in few-shot learning. Ye et al. (2020), for instance, talked about the Meta Network, which gathered meta-level information for quick generalization. The Long Short Term Memory (LSTM) meta-learner model trains the optimization technique, as discussed by Munkhdalai and Yu (2017). By learning only the initial learner parameters, the model-agnostic meta-learning model (Finn et al., 2017, Antoniou et al., 2018) simplified the aforementioned meta-network and achieved rapid adaptation with respect to those initial values and good generalizability to the new tasks. The similarities between two images are also learned using meta-learning methods. Matching-Net (Vinyals et al., 2016) proposed to map a limited labeled​ support set to its label and find the closest labeled example, in addition to figuring out the class of each instance in the query set. With the use of the class-wise mean and the Euclidean distance, Proto-Net was developed by Snell et al. (2017), who generalized Matching Net from one-shot learning to few-shot learning. Relation-Net was made by Sung et al. (2018) to learn useful metrics. It uses few-shot graph neural network models and CNN-based relation modules.

Non-Meta-based Few-shot learning: In addition to those non-meta methods based on meta-learning, there are other non-meta methods that use high-density characterizations of image regions to calculate the distances, as in Zhang et al., 2020a, Zhang et al., 2020b or use cosine similarity measures to predict the novel classifier with a weight generator and instantly set the weights based on the activations of the embedding layer (Qi et al., 2018). In order to attain competitive performance, Chen et al. recommended limiting intra-class variation as well as constraining similarity. Both the meta and non-meta techniques used a fixed feature extractor trained on the based classes, which is insufficient to account for the domain difference between the base and novel classes (Chen et al., 2020). Approaching the few-shot problem by finding a meta-learning scheme through adaptive search that is compatible with both meta and non-meta approaches as opposed to learning more sophisticated optimizers or classification metrics

Here the severity of covid-19 affected person can be classified using non-meta based few shot leaning with the help of cosine similarity. The proposed model not only tells the severity level of a patient but also tells the infections which cause risk. The four main infections which help to differentiate the severity of covid positive patients are discussed below.

• •Sepsis: COVID-19 has the potential to cause sepsis. When an infection causes the immune system to overreact, which can lead to tissue damage, organ failure, or even death, it is a dangerous complication that can be caused by an infection.
• •Acute Respiratory Distress Syndrome (ARDS): Bacteria can be broken down more easily because of their longer lifespan. The core purpose of the monocyte cell is to provide a piece of microbe to the T cell so that it can recognize it again in potential attacks.
• •Covid-Pneumonia: It causes immune responses such as hives and hay fever, as well as diseases of the nervous and collagenous systems, the spleen, parasitic inflammation, and other types of allergies; however, their primary prey is parasites such as tapeworms and hook worms.
• •Covid-Bronchitis: Basophils help with allergic and antigen reactions by releasing histamine, which is a sign of an allergy. This causes blood vessels to widen and helps the immune system.

Train the model by fusing multi-model data, which includes the CT images of sepsis, ARDS, Covid Pneumonia, Covid Bronchitis etc. It is possible to forecast the exactingness level of Corona virus positive individuals by combining these multi-model data sets, allowing for better treatment. The novelty of this work is detecting not only the severity of the particular patient but also identifying the disease that is causing that severity. If the patient is affected by COVID-Sepsis, then the individual is in the critical stage, if the patient is affected by ARDS, they are in the severe stage. If the appropriate treatment is started quickly, it can prevent the loss. Patients with COVID pneumonia are in a moderate stage of illness, so proper medication will help them recover quickly. Covid-Bronchitis is in the early or mild stage, many people affected during this stage of the pandemic will be cured easily.

The main contributions of the proposed work are as follows:

• •Classification-1: The mask R-CNN is used to detect COVID-19 using chest CT scan images; the CNN used here is ResNet-50, which classifies whether the patient is affected by the coronavirus or not.
• •Classification-2: Prognosticate the severity of covid-19 as four classes using the four main infections (Sepsis, ARDS, Covid-Pneumonia and COVID-Bronchitis), which help to classify the covid-positive patients’ severity as critical, severe, moderate, and mild using the few-shot learning method using the cosine similarity of base classifier ResNet-50.
• •The output of classification-1 (Covid-19 positive) is used as input in classification-2 to find its severity. It not only classifies the severity but also classify the disease (infections) which causes the severity. It empowers healthcare professionals to correctly identify the detection of severity for potential treatments.

The remainder of the paper is systemized in this way. Section 2 discuss about the recent works related to the proposed models and literature survey. Section 3 explains about the working of proposed approach. The experimentation findings, implementations and evaluation metrics were discussed in Section 4. The final section ended with conclusion and future enhancement.

2. Related work and literature survey

Many research studies have been conducted for prediction or detection of corona virus as wells as for prognosing the covid-19 patient’s severity in orders to provide appropriate treatment. RT-PCR is the recommended, fast and majorly used diagnosing method for diagnosis covid-19. Easy to use, quick and accurate solutions for covid-19 diagnosis are Chest X-ray and Chest CT-images. Rapid growth in the research field of artificial intelligence, machine learning and deep neural learning algorithms, Covid-19 diagnosis and its severity have been proposed.

2.1. Deep learning methodology for Covid-19 detection

Deep learning is an authentic diagnosis method for classifying COVID cases (Bai et al., 2020). Due to the busy schedules of medical practitioners and the involvement of high-risk patients, the DL serves as an adjunct method for detecting the coronavirus and its severity. Bai et al. explain the efficiency of computational resources in terms of time.  The importance of imaging modalities like chest computed tomography images for covid-19 detection is discussed by Tenda et al. (2020). Ozsahin et al., 2020a, Ozsahin et al., 2020b discuss the ensemble of various Deep Neural Networks models for diagnosing covid19 using CT scan images. It explains the comparison of various pre-trained models like VGG16, ImageNet, ResNet, CovidNet, InceptionNet, etc. Deep learning provides a great solution for the classification of medical images for clinical treatment. Raghu et al. (2019) explain in detail how transfer learning techniques can be used in the analysis of medical images. The dataset of ImageNet is being trained; this pre-trained technique provides better speedup and scaling. Polsinelli et al. (2020) use light CT to detect covid19 using a chest CT scan with a very short detection time. The chest CT scan yields both conventional and non-conventional results. Yadav and Jadhav (2019) discuss how CNN provides a deep solution for diagnosing various diseases by classifying medical images. It uses CXR images to classify the different types of pneumonia and comes to the conclusion that Deep CNN is better than Convolutional Neural Networks at classifying images with small datasets.

A pre-trained model excelled in the prognosis of various diseases using medical images. Sajja et al. (2019) discusses the prediction of cancer in the lungs using chest CT scan images. It provides better accuracy when compared to CNN and other machine learning models. The transfer learning model works well with the availability of a small covid-19 dataset. The availability of lung CT scan images for using a pre-trained neural network to diagnose COVID-19 patients is extremely limited. The data augmentation technique increases the images by creating high resolution synthetic images to provide better performance. Di et al. (2021) discusses the information collected from the CT images with the help of hypergraph learning. It contains radiometric and geographic information that a weighted vertex hypergraph learning technique can analyze and detect whether the new individual has suffered from covid19 or not. Suganya and Kalpana (2022) explain the automated detection of the Corona virus using deep multiple-instance learning (DMIL). Due to the limited datasets, this method uses the technique of CycleGAN to balance the unbalanced dataset by creating several high resolution synthetic images, which helps to obtain good accuracy comparatively. The extreme learning machine used for detecting covid-19 consequently produces better accuracy. Non-iterative and fast deep learning: multilayer extreme learning machines was discussed by Zhang et al. (2020b), which clearly explains the importance of predicting the error rate rather than finding the accuracy. According to Turkoglu (2021), the COVID-19 detection using Extreme Learning Machine (ELM) with multiple kernels based on deep neural networks achieves an accuracy of 98.36 and an F1 score of 98.25. Rajpal et al. (2022) proposed COV-ELM with a local interpretable model agnostic explanation (LIME) for detecting corona coronavirus, which produces a sensitivity of 94% and an F1 score of 95%. Inam et al. (2021) discussed COVID detection using real-time statistical deep extreme learning machine (RTS-DEL), which achieves an accuracy of 98.18%. Table 1 shows the performance and datasets used in various networks to detect covid-19.

Table 1.

Various methodologies for covid-19 detection.

Authors	Dataset used	Methodology	Performance metrics
He et al. (2020)	350 COVID-19 397 WITHOUT-COVID-19	DenseNet – 201	Accuracy is $\sim$86.57% F1 Score is $\sim$85.32%
Yang et al. (2020)	350 COVID-19 465 WITHOUT-COVID-19	Multiple tasking learning	Accuracy is $\sim$89.10% F1 Score is $\sim$90.03%
Loey et al. (2020)	742 CT IMAGES 345 COVID-19 397 NON-C0VID	VGGNet-16, VGGNet-19 AlexNet, ResNet-50, GoogleNet	Accuracy is $\sim$83.74% Sensitivity is $\sim$78.54% Specificity is 88.92%
Liu et al. (2020)	560 COVID-19 659 WITHOUT-COVID-19	VGG-16 based DNN	Accuracy is $\sim$88.61% F1-Score is 88.38% Sensitivity is $\sim$89.91%
Wang et al. (2020)	315 COVID-19 229 WITHOUT-COVID-19	U-Net	Accuracy is $\sim$90.72% Specificity is $\sim$91.87% Sensitivity is $\sim$90.70%
Pathak et al. (2022)	415 COVID-19 440 WITHOUT-COVID-19	ResNet50 with 2-Dimensional CNN	Accuracy is $\sim$93.79% Specificity is $\sim$95.23% Sensitivity is $\sim$91.81%
Han et al. (2020)	230 COVID-19 130 NORMAL	Attenuation based Deep 3D-Multiple Instance Learning	Accuracy is $\sim$98.12% Sensitivity is $\sim$97.93% F1 Score is $\sim$97.30%
Harmon et al. (2020)	1030 COVID-19 1695 WITHOUT-COVID-19	AH-Net DenseNet-201	Accuracy is $\sim$91.41% Specificity is $\sim$93.04% Sensitivity is $\sim$84.67%
Jin et al. (2020)	495 COVID-19 1385 WITHOUT-COVID-19	Convolutional Neural Network	Accuracy is $\sim$95.54% Specificity is $\sim$96.97% Sensitivity is $\sim$94.81%
Ahuja et al. (2021)	350 COVID-19 397 NON- COVID-19	ResNet18	Accuracy is $\sim$99.00% Specificity is $\sim$98.51% Sensitivity is $\sim$99.03% F1 Score is 99.02%
Suganya and Kalpana (2022)	230 COVID-19 130 NON- COVID-19	DMIL with CycleGAN	Accuracy is $\sim$98.60% Specificity is $\sim$96.51% F1 Score is 96.32%
Turkoglu (2021).	1745 COVID-19 1985 NON-COVID-19	MK-CLM-DNN	Accuracy is $\sim$98.36% Specificity is $\sim$98.44% Sensitivity is $\sim$98.28% F1 Score is 98.25% AUC is 98.36%
Rajpal et al. (2022)	520 COVID-19 520 NORMAL	COV-ELM with LIME	Sensitivity is $\sim$94.42% F1 Score is 95.40%
Inam et al. (2021)	547 EMR FROM SERVICE HOSPITAL, PAKISTAN	D2C-RTS-DELM	Accuracy is $\sim$98.18% Selectivity is $\sim$98.44% Error rate is 1.12%

Various deep transfers learning technique is used for the image classification. In deep learning, several convolutions filter helps to extract the feature by itself. It automatically generates various feature maps to get solution to the problem by varying the size of the filters, activation function, convolutional layers, pooling layers etc. Several pre-trained deep learning architecture were used to predict the corona virus.

2.2. Deep learning methodology for COVID-19 severity detection

The use of multi-class classification to determine the severity of COVID-19 is discussed. A.L. Aswathy et al. discussed a technique for predicting the covid-19 patient’s severity with the help of back propagation neural networks and transfer learning techniques (Aswathy et al., 2021). Lung infections can be quantified using a VB-Net neural network, whose process of instance segmentation was discussed by Shan et al. (2021). The infected region is segmented and quantified to produce an accurate classification of severity. It uses the dice similarity coefficient as a metric to label the infections. Tang et al. (2021) explore the possibility of detecting the severity of COVID-19 using a random forest algorithm where quantitative features are extracted from chest CT scan images of the patients. It classifies and labels the output as severe or non-severe. Xiao et al. (2020) discussed the DL model of ResNet-34 that is used to classify the severity. In order to avoid over-fitting, it uses 5-fold cross-validation to train the datasets and uses the binary-classification method to label them as severe or non-severe. A bi-directionally elastic-registered algorithm is used to find the severity and progression of covid-19 as discussed by Pu et al. (2020). Initially, it segmented the boundary of the lung and vessels using U-Net architecture to identify the region affected by common and covid pneumonia and found the result by calculating the threshold value using the average lung density. For corona virus severity detection, it calculates the Pearson correlation between ground truth and prediction values. Shen et al. (2020) explored the adaptive region model to segment the lungs’ volume using thresholding, and then the lung region is cut out by segmenting pulmonary vessels. Finally, pneumonia detection occurs, and the authors evaluate the Pearson correlation for lung lesions and lung volumes using the proposed method (Ozsahin et al., 2020a).

Several existing systems compare the severity of covid-19 in Table 2. The main challenges in the existing system are to predict covid-19 with high accuracy by handling unbalanced datasets and predicting the severity accurately. Obtaining better accuracy with imbalanced classification is the most important challenge. Li et al. (2022) discussed an imbalanced matrix to handle the unbalanced class data using extreme learning machine. Multi-objective optimization-based adaptive class-specific cost-extreme learning machine for imbalanced classification use an imbalanced matrix and penalty adjustment matrix to make a cost-effective decision boundary. Previous articles have mostly classified severity into two or three classes or labels, namely severe/non-severe and severe/mild/non-severe. But the proposed approach categorizes the severity into four classes based on the positive chest CT samples. The difficulties in detecting COVID in patients and its severity are being investigated as a result of these studies. By overcoming these challenges and using the proposed methods, we can achieve better results for severity detection.

Table 2.

Various model used to predict the severity of Covid-19.

Authors	Dataset used	Classes	Method	Performance metrics
Aswathy et al. (2021)	Covid-CT public dataset with 349 images	Covid-19 severity (High, Moderate, Low)	ResNet-50, DenseNet-201	Accuracy is $\sim$97.81% Sensitivity is $\sim$95.12% Specificity is $\sim$97.31% Precision is $\sim$94.12%
Shan et al. (2021)	550 CT volumes	Infected region (segmentation and quantification)	VBNet	Dice similarity co-efficient value 92.41% 10.0
Tang et al. (2021)	176 patients CT images	Severe Non-Severe	Random forest Algorithm	Accuracy is $\sim$87.52%
Xiao et al. (2020)	23 810 Covid-19 images	Severe Non-Severe	ResNet-34	Precision is $\sim$81.52% AUC is 98.32%
Pu et al. (2020)	72 covid-19 images and 120 other volume images	Severity of covid-19 and its progression	UNet-BER Algorithm	Sensitivity is $\sim$95.76% Specificity is $\sim$85.92%
Shen et al. (2020)	CT images from $\sim$50 patients	Severe Non-Severe	Thresholding and AGR algorithm	Pearson correlation coefficient r ranges from 0.77 to 0.84, P$\mathrm{<}$0.05

3. Proposed approach

This research presents a classification of Covid-19 detection and its severity in two stages using novel methods. Initially, Covid-19 detection process occurs using a pre-trained model Mask R-CNN which classify whether the image is affected by covid-19 $+$ve or $-$ve. In second stage, it detects the severity of the corona virus positive patients. It uses a few shot learning method with a base classifier of ResNet-50 to detect the severity by using softmax as an activation function to achieve better accuracy.

3.1. COVID-19 detection

In first stage, COVID-19 is detected by inputting the chest Computerized Tomographic scan images to the architecture to identify whether the patient is affected by covid-19 or not. The pre-trained network, Mask R-CNN is used to predict the covid-19 $+$ve and $-$ve.

3.1.1. Mask R-CNN

Mask R-CNN generates segmentation mask with high quality for each instances from the object. Initially Chest CT images were taken as input image, the Feature-Pyramid-Network helps to extract the essential features in the chest image. In general Mask R-CNN performs pixel level segmentation, here the Chest CT image is segmented into pixel and classifies the images accurately (Podder et al., 2021). The feature maps directly send the extracted feature to ROI Align as well as to the Region Proposal Network (RPN). RPN process the featured image with the help of 3 $\ast$ 3 Convolution layers. The CNN used here is ResNet-50 which consists of 48 Convolution (Conv) layer along with 1 Max-Pool and 1 average (avg)-pool layer. It splits into two 1 $\ast$ 1 Conv layer, one with Boundary Box Regression and other using Softmax function. The proposal network is created from the output of Bounding-Box (bbox) Regression and Softmax activation function. The proposal output is send to ROI Align for further classification. Then the image from ROI align sent to fully connected layer, it classify the bbox regression as well as the classification which is covid or not covid from the chest image with the mask. Fig. 3 depicts the framework of Mask R-CNN for classifying the image into covid-19 and Not-covid-19.

Fig. 3.

Framework for Mask R-CNN for image classification.

The Mask R-CNN not only creates bounding-boxes on the targeting images but also create masks which classify the boxes based on the pixels inside it.

This deep learning mechanism is used to building up the entire model. Bounding boxes (bbox) is created to classify the object into different classes and then create a mask to the output objects. The multiple masking loss-function in each case is given as

$$L_{func}{=L}_{bbox}{+L}_{mask}{+L}_{class}$$ (1)

Here $L_f$ is the loss_function, $L_{bbox}$ is the regression loss of bbox, $L_{mask}$ is the prediction loss during masking, $L_{class}$ is the loss occurs during classification. These three components are composed in order to minimize the loss_function. The following Eq. (2) defines the classification loss. The loss during classification $L_{class\text{\_}loss}$ of each anchor is the loss using log function, to find the lung area which is calculated from Eq. (3)

$$L_{class}=\frac{1}{\left|N_{class\text{\_}loss}\right|}\sum _i{L}_{class\text{\_}loss}(d_i,d_i\ast )$$ (2)

$$L_{class\text{\_}loss}(d_i,d_i\ast )=-d_i\ast ,{logd}_i\ast -(1-d_i\ast )log(1-d_i\ast )$$ (3)

The following Eq. (4) explains about the bbox regression loss.

$$L_{bbox}=\frac{1}{\left|N_{bbox}\right|}\sum _i{d_i\ast L}_1(p_i,p_i\ast )$$ (4)

Here the mathematical expressions indicates that anchor index as i, detection probability that anchor the lung is denoted as $d_i$, the truth value is denoted as $d_i\ast$, if the anchor shows of the lung it produces the value as 1, if the value is 0, it does not produce any lung image. The parameterized co-ordinates of the bounding box is predicted and denoted as $p_i$. The ground truth co-ordinates of the positive anchor are represented as $p_i\ast$. Here the mini batch size is $N_{class\text{\_}loss}$ and $N_{bbox}$ denotes the number of anchor location; where $L_1$ loss is used as robust for outliers. $L_{mask}$ define the cross entropy loss for binary as shown in Eq. (5) these are the normalization terms.

$$L_{mask}=-\frac{1}{\left|m^2\right|}\sum _{1\le i,j\le m}{\left[y_{ij}log{y}_{ij}^{-k}+\left(1-y_{ij}\right)log\left(1-y_{ij}^{-k}\right)\right]}_{.}$$ (5)

Here the value of label cell is (i,j) where the size of region n $\ast$ n is given by $y_{ij}$. kth class of that particular cell is predicted using the value of $y_{ij}^{-k}$. During dataset training, the minimized loss function value nearly 0 is observed, which clearly indicates that this model performs well with no over-fitting problems.

3.2. Covid-19 severity detection

3.2.1. Few shot learning

A Few short learning is a simple method which classifies new data with very few training samples with supervised information. It mainly focuses on pre-training and fine turning the images. Initially few Chest CT images were taken to train the pretrained model which gain experiences from others that is why it is characterized as a Meta learning problem. The base classifier is mainly required to classify the images in the training stage. ResNet-50 used as base classifier to classify the images accurately, fully connected layer is replaced by 1 $\ast$ 1 convolutional layers to achieve the final classification effectively as shown in Fig. 4. It uses N-number of way K- number of shot method for classification to discriminate between N-Classes with K-Samples.

Fig. 4.

Workflow of Few shot learning with ResNet-50 base classifier.

The Support-Set samples consists of ${\mathrm{N}}_{\mathrm{l}}$-number of labels, where each and every label contains K-labeled images, a Query-Set contains query images names as Q. The average loss calculation for each task is calculated. During the classification task Q among N-classes are given and ${\mathrm{N}}_{\mathrm{l}}$$\ast$ K images in the Support-Sets. It helps to fine tune f on Support-set quickly.

The fine-tuned f in the query set is send to cosine similarity where the loss results is calculated from the classification error throughout the function and update that as ‘$\theta$’. Finally softmax function is employed throughout the training. The cosine similarities are multiplied by learnable scalar in the training to get the prediction of probability prediction. To classify the severity of lung disorder with few shot learning task using cosine similarity of base classifier ResNet-50 with the image from query set ‘f’ achieves better accuracy.

3.2.2. Base classifier

The base classifier used in this model for predicting the severity is ResNet-50, which is a pre-trained architecture (Aswathy et al., 2021). ResNet-50 is a CNN network with 50 layers deep. In the few-shot learning technique, a cosine nearest method is used to train the classifier while taking in account for loss with respect to the base classes. An encoder “f” is acquired by removing the fully connected layer from the pre-trained model and adding 1 $\ast$ 1 convolution layers on the top of it. Encoder “f” is in charge of mapping each input with respect to its embedding, which aids in providing training for the classifiers corresponding to all base classes in addition to conventional cross-entropy loss.

$$X_d=\frac{1}{\left|P_d\right|}\sum _{a\in P_d}f\left(a\right)$$ (6)

Let us take a look at the supported pair for task “P”, a few-shot. The corresponding few-shot samples over the class are taken as “P${}_{\mathrm{d}}$” and are represented as “d” for each class that is present across the dataset. The above equation describes how the base classifier calculates the average embedding, or class centroid, ‘X${}_{\mathrm{d}}$’:

$$p\left(B=d|a\right)=\frac{\exp \left(〈f\left(a\right),X_d〉\right)}{\sum _{d^{\prime }}\exp \left(〈f\left(a\right),X_{d^{\prime }}〉\right)}$$ (7)

In a few-shot task, the query “a” is used to provide a sample. As shown in Eq. (7), it aids in determining the likelihood of samples belonging to the class “d” with respect to cosine similarity and centroid. The value $〈f\left(a\right),X_d〉$ in the Chest CT scan image denotes the cosine similarity of two vector values. According to Eq. (8) cosine similarity is obtained as

$$〈f\left(a\right),X_d〉=\frac{f\left(a\right).X_d}{\Vert f\left(a\right)\Vert \ast \Vert X_d\Vert }$$ (8)

When determining logits, it may be helpful to normalize the value beforehand using the Softmax function throughout training because the confidence interval for cosine similarity is [1, 1]. The learnable scalar is multiplied by the cosine similarity and scalar during training to obtain the probability prediction.

$$p\left(B=d|a\right)=\frac{\exp \left(\theta .〈f\left(a\right),X_d〉\right)}{\sum _{t^{\prime }}\exp \left(\theta .〈f\left(a\right),X_{d^{\prime }}〉\right)}$$ (9)

A few-shot task classifies a CT image in the query set using cosine similarity as a proximity metric after evaluating the average features for samples of each classes using the support set “p” as input (9). For classifying the severity level effectively, the convergence pre-trained base classifier is further optimized.

4. Experimentation and result discussion

4.1. Dataset

In this study two distinct datasets were analyzed. The first dataset is derived from publicly available datasets which included in this research comprised of original chest CT-scan images of 377 persons. The data-set contains 15 589 and 48 260 CTScan images which belongs to 282 non covid person and 95 covid-19 persons. The first dataset for Covid-19 detection source is available in the repository https://github.com/mr7495/COVID-CTset from Rahimzadeh et al. (2021). These images are 16 bit gray-scale DICOM format with a pixel resolution of 512 $\ast$ 512. From this dataset, pre-classification process occurs into covid $+$ve and covid $-$ve.

Then the second data-sets used in this research work which is created by fusing multiple images from different source and created a new datasets, where the total dataset consisting of 12 455 images. Totally four classes is being created in this dataset which includes sepsis image for critical stage, ARDS for severe stage, Covid pneumonia for moderate stage and common bronchitis for mild stage. Each class consists of different number of images. For instance, class covid pneumonia comprises maximum of 7685 images and class sepsis comprises of only seventy two images.

Then the datasets must have to re-classify the covid-19 images into these four classes. Initially, pre-process the images and make them to same size, then it is been fused to perform different classes classification. The images contain all the four classes which is shown below in Fig. 5. The dataset’s properties are listed in Table 3, Table 4 accordingly.

Fig. 5.

Chest CT images of Sepsis, ARDS, Covid Bronchitis and Covid Pneumonia.

Table 3.

Properties of Dataset-1.

Properties of dataset	Respective values
Image size	512 $\times$ 512 pixel
Nature of image	Gray scale images
Entire classes	2
Covid $+$ve	95
Covid $-$ve	282

Table 4.

Properties of Dataset-2.

Properties of dataset	Respective values
Image size	512 $\times$ 512 pixel
Nature of image	Gray scale images
Entire classes	4
Critical (Sepsis)	72
Severe (ARDS)	856
Moderate (Covid Pneumonia)	7685
Mild (Covid Bronchitis)	3839

4.2. Evaluation metrics

Classification models have a discrete output. Evaluation metrics is needed to compare the different classes (discrete) in some form. Classification metrics can evaluate the model’s performance, each metrics evaluate in different way.

$$Accuracy=\frac{Truepositive+TrueNegative}{Truepositive+TrueNegative+FalsePositive+FalseNegative}$$ (i)

Accuracy perhaps the simplest and most important metric to implement, it is expounded as the number of true prediction divided by total number of prediction (i).

$$Sensitivity=\frac{TruePositive}{TruePositive+FalseNegative}$$ (ii)

Sensitivity metrics defined as the total proportion of observed true positive that were predicted to be a positives (ii). It helps to make better decisions with various outcomes. In simple term, the true positive divided by true positive and false negative.

$$Precision=\frac{TruePositive}{TruePositive+FalsePositive}$$ (iii)

Precision is the classification metric which evaluates the number of true positive predictions occurs (iii). It is expounded as the no. of true $+$ve divid by number of true $+$ve and false $+$ve.

$$Specificity=\frac{TrueNegative}{TrueNegative+FalsePositive}$$ (iv)

Specificity metrics defined as the total proportion of observed true positive that were predicted to be a negatives (iv). In simple term, the true negative divided by true negative and false positive.

$$MatthewsCorrelationCoefficient$$

$$=\frac{TP\times TN+FP\times FN}{\sqrt{}(TP+FP)(TP+FN)(TN+FP)(TN+FN)}$$ (v)

Matthews’s correlation coefficient (MCC) is a single value classification metric helps to summarize the confusion matrix (v). It works well in binary classification with unbalanced datasets. Here TP denotes true positive, FP denotes false positive, TN denotes true negative and FN denotes false negative.

$$F1Score=2\times \frac{Precision\times Recall}{Precision+Recall}$$ (vi)

F1 score is a Harmonic mean of precision and recall. It is a more desirable performance measure for classification than accuracy (vi). When data is unbalanced, such as when the number of samples belonging to one class far outnumbers those observed in the other class.

$$mAP=\frac{1}{N}\sum _{i=1}^N{AP}_i$$ (vii)

Mean Average Precision (mAP) is a standard metric by which the accuracy of object detection models can be judged. Recall values between 0 and 1 are used to determine the mean AP value (vii). The mAP takes into account both false positives (FP) and false negatives (FN), incorporating the trade-off between precision and recall.

$${(G-mean)}_{binary}=\sqrt{(}specificity\times recall)$$ (viii)

The G-mean is the square root of the product of the sensitivities of the different classes. This metric’s goal is to achieve the highest possible accuracy for each class while maintaining parity among them. G-mean is calculated by taking the square root of the product of the specificity and sensitivity for a binary classification (viii).

$$(G-mean)=\sqrt{TPR}\times 1-FPR$$ (ix)

It is the square root of the product of the sensitivities across classes for a multi-class situation, where TPR denotes true positive rate and FPR denotes false positive rate. Eq. (ix) is an un-biased metric for imbalanced datasets.

$$AUC=\frac{1}{2}\left(\frac{TP}{TP+FN}\right)\left(\frac{TN}{TN+FP}\right)$$ (x)

The Area under Curve (AUC) is calculated with the help of confusion matrix in Eq. (x) as discussed by Idrees et al. (2017). The AUC is calculated by summing up the area under the ROC curve (a concept from integral calculus) in a two-dimensional space extending from (0, 0) to (1, 1).

4.3. Implementation and result discussion

4.3.1. Covid-19 detection

The experimentation on the deep learning models was performed on chest CT scan images to identify whether the patient is affected by covid-19 or not. Initially Covid-19 is detected using a pre-trained Mask R-CNN algorithm with a base classifier of ResNet-50, which were trained for 100 epochs. Various ResNet models were compared for detecting covid-19 as shown in Table 5, where ResNet-50 performs well when compared other models (Podder et al., 2021).

Table 5.

Performance of Covid-19 detection using various ResNet models.

Base classifiers	F1 Score	Specificity	Accuracy
ResNet-41	0.9454	0,8903	0.9316
ResNet-50	0.9693	0.9736	0.9698
ResNet-65	0.9574	0.9179	0.9435
ResNet-101	0.9763	0.9264	0.9523

For evaluating Mask R-CNN model with ResNet-50, 5-fold cross validation is performed. In each round, we split the dataset into 5 parts, one part is for validation, and the remaining 4 parts are merged into training subsets for model evaluation. The holdout method is repeated 5 times. Performance is evaluated for each round, the average of all the metrics are computed together with their standard deviation as shown in Table 6 based on binary classification model. Binary classification (Covid-19 and not-Covid-19) task is computed and the performance of each fold is calculated. Fig. 6 shows the results of the confusion matrix for all the 5 folds.

Table 6.

Performance of Covid-19 detection using Mask R-CNN with ResNet-50.

Measures	Fold-1	Fold-2	Fold-3	Fold-4	Fold-5	Average/Mean ($\mu$)	Standard deviation ($\sigma$)
Accuracy	0.9775	0.9850	0.9930	0.9885	0.9860	0.9860	0.0050
Sensitivity	0.9870	0.9780	1.0000	0.9840	0.9820	0.9862	0.0074
Precision	0.9686	0.9919	0.9862	0.9929	0.9899	0.9859	0.0089
Specificity	0.9680	0.9920	0.9861	0.9930	0.9900	0.9858	0.0092
F1 Score	0.9777	0.9849	0.9930	0.9884	0.9859	0.9859	0.0049
False positive rate	0.0320	0.0080	0.0139	0.0070	0.0100	0.0141	0.0092
False negative rate	0.0130	0.0220	0.0000	0.0160	0.0180	0.0138	0.0074
False discovery rate	0.0314	0.0081	0.0138	0.0071	0.0101	0.0141	0.0089
Matthews correlation coefficient	0.9552	0.9701	0.9861	0.9770	0.9720	0.9728	0.0100

Fig. 6.

Comparison of Covid-19 and Not-Covid-19 confusion matrices (a) Confusion matrix of Fold-1 (b) Confusion matrix of Fold-2 (c) Confusion matrix of Fold-3 (d) Confusion matrix of Fold-4 (e) Confusion matrix of Fold-5.

The performance of the proposed work for detecting covid-19 or not-covid-19 is evaluated, metrics of the entire 5-fold is evaluated and their average is illustrated in Table 6. It achieves the performance of 98.60% for Accuracy, 98.58% of Specificity, 98.59% of Precision, 98.62% of Sensitivity, 98.59% of F1-Score, 0.0141 of False Positive Rate, 0.0138 of False Negative Rate, 0.0141 of False Discovery Rate. Metrics such as accuracy and F1 score derived from confusion matrix or error matrix are widely used in binary classification. On unbalanced datasets, these techniques can exhibit overoptimistic inflated results. So, we calculate Matthews Correlation Coefficient which is a single value classification metric helps to summarize the confusion matrix. It reveals when the classifier fails to correctly label negative-class samples. The Matthews Correlation Coefficient value is calculated using Eq. (vii) obtain the value as 0.9728. The geometric mean (G-Mean) is a single score that takes into account both sensitivity and specificity for binary classification as given in the Eq. (viii). The average value of sensitivity and specificity is 0.9862 and 0.9859. By taking squared root of both values, we achieve a G-Mean value as 0.9769 with an optimal threshold value of 0.0131.

The Quality of a classifier’s results can be measured using AUC Receiver Operating Characteristic (ROC) curve. AUC ROC curve is an efficient method to summarize the overall accuracy. The greater the area the better is the classifier. The Y-axis of ROC curve represents the true positive rate whereas X-axis represents the false positive rate. Because of this, the “ideal” point is located in the upper left corner of the plot, where the false positive rate is zero and the actual positive rate is one. We calculate the AUC values from the error matrix for all the folds and get an average value of 0.9721 for covid-19 detection. Fig. 7 shows the analysis of a classifier performance using ROC curve for binary classification. Here the binary classification occurs between covid-19 and not covid-19.

Fig. 7.

AUC-ROC curve for Covid-19 detection.

In object detection task, Intersection over Union (IOU) is focused mainly for evaluating the degree of overlap between the predicted bounding box coordinates to the ground truth box. If the IOU value is higher, the actual ground frame ${\beta }_{gf}$ i.e. ground truth of the target and predicted frame ${\beta }_{pf}$ coordinates resembles close to each other. From these two frames, we calculate the IOU using the formula $IOU=\frac{area({\beta }_{gf}\cap {\beta }_{pf})}{area({\beta }_{gf}\cup {\beta }_{pf})}$, It helps to check whether the detection frame meets the required condition. IOU ranges between 0 and 1, it is calculated with thresholding i.e. we need a threshold ($\alpha )$ to decide whether the detection is correct or not. To determine the average precision (AP), we use the weighted mean of the precisions at each threshold, where the weight represents the progress in recall from the previous threshold By using the basic definition in Eq. (vii) mAP is the average of AP in each classes. Adjusting different confidence threshold, we predict different values with different accuracies. Instead of changing the threshold value, we take a global threshold mAP@0.5.

By using the threshold of ($\alpha =0.5$), the Mask R-CNN model with base classifier of ResNet-50 achieves the value of 0.9152. It achieves greater accuracy when compared to other state of art methods as shown in Table 5.

The output of covid-19 detection using Mask R-CNN is shown in Fig. 8. Initially it creates a bounding box to an image, and then creates a mask image to classify whether the patient is affected by covid $+$ve or not covid. Fig. 9.A, Fig. 9.B shows the performance of the classification task of covid-19 detection which has an accuracy and loss value. The testing set’s loss is diminishing over time, as can be shown. Though there is a difference in training and test accuracy between epochs 20 and 40, it improves after 50.

Fig. 8.

Covid-19 detection-CT image of patients and its classification using Mask R-CNN.

Fig. 9.A.

Covid-19 detection accuracy.

Fig. 9.B.

Covid-19 detection loss.

The mean Average Precision is the standard metric for object detection model; it is calculated and compared with various states of art object detection models shown in Table 7. The mAP is calculated with the global threshold of ($\alpha =0.5$). The proposed work achieves mAP of 0.9152 which is 0.1550, 0.4479, 0.2922, and 0.0281 greater than EfficientNet (Shamila Ebenezer et al., 2022), Faster R-CNN, YOLO v5 (Qu et al., 2021) and RYOLO v4tiny (Kumar, 2022) respectively.

Table 7.

Performance of various object detection models for covid-19 detection.

Detection models	mAP@0.5	F1-Score (%)	Accuracy (%)
EfficientNet	0.7602	89.94	90.15
EfficientNet with CLAHE	–	93.02	94.56
Faster RCNN	0.4673	92.56	95.60
YOLO v5	0.6237	93.01	95.37
RYOLO v4tiny	0.8871	93.42	94.74
MaskR-CNN	0.9152	98.59	98.60

When compare to various state of arts models, the proposed Mask R-CNN with a base classifier of ResNet-50 gives better accuracy and good overall results for detecting covid-19.

4.3.2. Covid-19 severity detection

Various lung disorders may arise for the corona virus affected person. Few-short learning algorithm is implemented to find its severity with a base classifier of ResNet-50. Initially, pre-train the model and fine-tune with an evolutionary algorithm is used here to get better accuracy. For Meta-learning methods, sample episodes with 4 classes from the target dataset. For each class, k instances taken as support set sample and k-1 instances as the query set for k-shot task. In training, 12 000 images were trained for 1-shot and 8000 episodes for 5 shot task on the base datasets. In search of samples 20 episodes of images from the validation dataset. Repeated k-Fold cross validation method is performed here to minimize errors and estimate the mean model performance. Typical performance criteria in image classification investigation are employed in this study to examine the classification reliability and accuracy. It is also used to evaluate models’ performance by looking at the AUC of the receiver operation characteristic curve (AUC of ROC).

For deeper network, more number of layers required to be fine-tuned for few shot learning. Fine tuning the network with 100 iterations for support-set and evaluates the network on the query-set. In the evaluation, the fine tune layer searched configuration on the support-set and evaluate on the query-set images from novel dataset. The feature extractor is trained for 100 epochs with the batch sized of 32 on the novel dataset is maintained throughout the process. For Algorithm 1, set the maximum iteration as 40, population size as 40 and other samples like fine tuning vector random sampling, crossover images as 100.

ResNet-50 is used as ground classifier, which become trained on ImageNet database. For training, Adam optimizer is used to reduce the loss function with the learning rate of 1e⁻⁴, a momentum of 0.9, and decay factors of 0.1. In fine tuning, Adam optimizer with 0.001 learning rate is fixed throughout the classification. Also, every training batch includes a small quantity of few shot tasks for the purpose of calculating the average loss. The cosine scaling parameter is initially set to a value of 10. It is crucial to evaluate performance of classification in image classification interrogations in order acquire scientific evidence for the study’s findings. Otherwise, the classification study might be abandoned in the classroom, leaving it inadequate. It is typical practice in few-shot learning to employ the episode training strategy.

A five-way, 25-shot approach with 25 training images for each class is used to complete the classification-2 task. The training set is divided into four categories, from which a random sample of twenty images from each class was drawn. The standard geometric transformations like random flip, horizontal or vertical crop, rotation or translation of images occurs during data augmentation

The result obtained for severity of covid positive patients is detected as shown in Table 8 which evaluates the accuracy and F-1 score of various pre-trained models. The accuracy obtained by VGG-19 is $\sim$93.54%, AlexNet is $\sim$93.70%, VGG-16 is 94.64%, ResNet-101 is 95.35%, DenseNet-201 is 96.34%, ResNet-50 is 94.65% and Few short Learning + ResNet-50 is 95.47%. When compare to various existing techniques, the proposed a few short Learning with the base classifier of ResNet-50 gives better accuracy and goosrad overall results for detecting the severity covid-19 positive patients.

Table 8.

Various pre-trained network used for severity detection.

Pre-trained network used for severity detection	Accuracy	F1 Score
VGG-19	93.54%	89.23%
AlexNet	93.70%	91.45%
VGG-16	94.64%	93.94%
ResNet-101	95.35%	93.86%
DenseNet-201	96.34%	95.84%
ResNet-50	94.65%	94.01%
Few short Learning + ResNet-50	95.47%	95.65%

Various pretrained model for detection severity from the covid-19 positive patients is represented in the above chart in Fig. 10. It compares the accuracy of various pre-trained model with the proposed model which helps to analyze the results clearly

Fig. 10.

Various pre-trained network used for severity detection.

From the output of covid-19 patient, it further classifies the severity using few shot learning with a base classifier of ResNet-50. Train the model by fusion multi-model datasets which includes the CT images of sepsis, ARDS, Covid Pneumonia, Covid Bronchitis as shown in Fig. 11. The proposed algorithm prognosis the severity based on four classes, Critical class leads to Sepsis, Severe class leads to ARDS, Moderate class leads to Covid pneumonia and Mild class leads to covid Bronchitis. It classifies the severity of the patient from the positive sample and the performances obtained are shown Table 9, which helps to provide better treatment.

Fig. 11.

Four classes of Covid-19 severity images.

Table 9.

Performance evaluation for various severity detection classes.

Severity detection classes	Accuracy (%)	Specificity (%)	Sensitivity (%)	Precision (%)	F1 Score (%)	AUC
Critical (Sepsis)	98.80$\pm$0.9	99.33$\pm$0.5	97.67$\pm$1.2	97.86$\pm$1.1	97.43$\pm$1.4	0.0966$\pm$0.0472
Severe (ARDS)	92.33$\pm$ 2.1	98.75$\pm$1.2	92.84$\pm$4.3	98.34$\pm$0.8	93.53$\pm$2.9	0.0914$\pm$0.0294
Moderate (Covid Pneumonia)	94.57$\pm$3.2	97.30$\pm$2.1	94.97$\pm$3.1	97.33$\pm$1.3	95.38$\pm$2.1	0.0943$\pm$0.0388
Mild (Covid Bronchitis)	96.18$\pm$2.5	98.48$\pm$1.0	97.33$\pm$1.4	96.28$\pm$2.6	96.27$\pm$2.4	0.0934$\pm$0.0175
Average value of severity detection	95.47$\pm$2.3	98.49$\pm$0.6	95.87$\pm$1.7	97.47$\pm$0.6	95.65 $\pm$ 2.2	0.0932$\pm$0.0323

The classification task of covid-19 severity detection has been performed, where the accuracy and loss value is calculated in both training and testing phase. The testing set’s loss is diminishing over time, as can be shown in Fig. 12.a, Fig. 12.b. Though there is a difference in training and test accuracy between epochs 20 and 40, it improves after 70.

Fig. 12.a.

Covid-19 severity detection accuracy.

Fig. 12.b.

Covid-19 severity detection loss.

With the help of fusing these multi-model datasets, it is possible to predict the severity level of the covid-19 positive patients accurately. The proposed model achieves an accuracy of 98.8% for critical (sepsis), 92.3% for severe, 94.5% for moderate and 96.1% for mild. It achieves the sensitivity of 97.6% for critical, 92.8% for severe, 94.9% for moderate (covid pneumonia) and 97.3% for mild. The precision obtained here is 97.4% for critical, 93.5% for severe, 95.3% for moderate and 96.21% for mild (covid bronchitis). The specificity of this model is 99.3% for critical, 98.7% for severe (Acute Respiratory distress syndrome), 94.9% for moderate and 97.3% for mild. From the above Table, the critical stage of covid positive patients can be able to analyze accurately.

The AUC value is calculated for each class for covid-19 severity detection. From the value of True-Positive-Rate and False-Positive-Rate, ROC curve is drawn for classes 0–3 i.e. 4 classes (Critical, severe, moderate, mild) as shown in Fig. 13. The mean ($\mu$) value of severity classes are 95.47 whereas the standard deviation ($\sigma$) value is 2.3. This equation is used to calculate Standard Error of Mean (SEM) is 0.0243 at 95% confidence level. Based on the SEM, the margin of errors (or confidence intervals) of different confidence levels is calculated for all the metrics.

Fig. 13.

AUC-ROC curve for Covid-19 severity detection.

Depending on the field of study, a confidence level of 95% (or statistical significance of 5%) is typically used for data representation. It shows that the margin of error depends on its confidence level (CL), when the CL value increases, error rate will get decrease as shown in Table 10. The unbiased metric for imbalanced data is Geometric mean. The G-Mean value for multiclass can be calculated from the value of TPR and FPR for all the subsets using the formula of $\sqrt{TPR}\times 1-FPR$. G-Mean is calculated for all the four subsets, the subset-1 (Critical) achieves a G-Mean of 0.9416 and error rate of 0.0432. The subset-2 (severe) achieves a G-Mean of 0.9352 and error rate of 0.0231. The subset-3 (moderate) and subset-4 (mild) achieves a G-Mean of 0.9367, 0.9475 and error rate of 0.0185, 0.0226 respectively. Table 11 shows the average G-Mean and Error rate value for Covid-19 severity classification model i.e. 0.9402 and0.0243 respectively.

Table 10.

Margin of errors calculation.

Accuracy		Specificity		Sensitivity
Confidence level	Margin of error	Confidence level	Margin of Error	Confidence level	Margin of Error
68.3%, ${\sigma }_{\overline{x}}$	95.42$\pm$1.185 ($\pm$1.24%)	68.3%, ${\sigma }_{\overline{x}}$	98.42$\pm$0.29 ($\pm$0.30%)	68.3%, ${\sigma }_{\overline{x}}$	95.82$\pm$0.795 ($\pm$0.83%)
90%, 1.645${\sigma }_{\overline{x}}$	95.42$\pm$1.949 ($\pm$2.04%)	90%, 1.645${\sigma }_{\overline{x}}$	98.42$\pm$0.478 ($\pm$0.49%)	90%, 1.645${\sigma }_{\overline{x}}$	95.82$\pm$1.307 ($\pm$1.36%)
95%, 1.960${\sigma }_{\overline{x}}$	95.42$\pm$2.432 ($\pm$2.43%)	95%, 1.960${\sigma }_{\overline{x}}$	98.42$\pm$0.569 ($\pm$0.58%)	95%, 1.960${\sigma }_{\overline{x}}$	95.82$\pm$1.558 ($\pm$1.63%)
99%, 2.576${\sigma }_{\overline{x}}$	95.25$\pm$3.053 ($\pm$3.20%)	99%, 2.576${\sigma }_{\overline{x}}$	98.42$\pm$0.748 ($\pm$0.76%)	99%, 2.576${\sigma }_{\overline{x}}$	95.82$\pm$2.047 ($\pm$2.14%)
99.9%, 3.291${\sigma }_{\overline{x}}$	95.42$\pm$3.9 ($\pm$4.09%)	99.9%, 3.291${\sigma }_{\overline{x}}$	98.42$\pm$0.956 ($\pm$0.97%)	99.9%, 3.291${\sigma }_{\overline{x}}$	95.82$\pm$2.615 ($\pm$2.73%)
99.99%, 3.891${\sigma }_{\overline{x}}$	95.42$\pm$4.611 ($\pm$4.83%)	99.99%, 3.891${\sigma }_{\overline{x}}$	98.42$\pm$1.13 ($\pm$1.15%)	99.99%, 3.891${\sigma }_{\overline{x}}$	95.82$\pm$3.092 ($\pm$3.23%)
F1-Score		Precision		AUC
Confidence level	Margin of Error	Confidence level	Margin of Error	Confidence level	Margin of Error
68.3%, ${\sigma }_{\overline{x}}$	97.4$\pm$0.311 ($\pm$0.32%)	68.3%, ${\sigma }_{\overline{x}}$	95.6$\pm$0.712 ($\pm$0.74%)	68.3%, ${\sigma }_{\overline{x}}$	0.0935$\pm$0.00901 ($\pm$0.96%)
90%, 1.645${\sigma }_{\overline{x}}$	97.4$\pm$0.512 ($\pm$0.53%)	90%, 1.645${\sigma }_{\overline{x}}$	95.6$\pm$1.17 ($\pm$1.22%)	90%, 1.645${\sigma }_{\overline{x}}$	0.0935$\pm$0.00148 ($\pm$1.59%)
95%, 1.960${\sigma }_{\overline{x}}$	97.4$\pm$0.61 ($\pm$0.63%)	95%, 1.960${\sigma }_{\overline{x}}$	95.6$\pm$1.395 ($\pm$1.46%)	95%, 1.960${\sigma }_{\overline{x}}$	0.0935$\pm$0.00177 ($\pm$1.89%)
99%, 2.576${\sigma }_{\overline{x}}$	97.4$\pm$0.801 ($\pm$0.82%)	99%, 2.576${\sigma }_{\overline{x}}$	95.6$\pm$1.833 ($\pm$1.92%)	99%, 2.576${\sigma }_{\overline{x}}$	0.0935$\pm$0.00232 ($\pm$2.48%)
99.9%, 3.291${\sigma }_{\overline{x}}$	97.4$\pm$1.024 ($\pm$1.05%)	99.9%, 3.291${\sigma }_{\overline{x}}$	95.6$\pm$2.342 ($\pm$2.45%)	99.9%, 3.291${\sigma }_{\overline{x}}$	0.0935$\pm$0.00297 ($\pm$3.17%)
99.99%, 3.891${\sigma }_{\overline{x}}$	97.4$\pm$1.211 ($\pm$1.24%)	99.99%, 3.891${\sigma }_{\overline{x}}$	95.6$\pm$2.768 ($\pm$2.90%)	99.99%, 3.891${\sigma }_{\overline{x}}$	0.0935$\pm$0.00351 ($\pm$3.75%)

Table 11.

G-Mean and Error rate for severity classification model.

Class subsets	G-Mean value	Error rate
Critical	0.9416	0.0332
Severe	0.9352	0.0231
Moderate	0.9367	0.0185
Mild	0.9475	0.0226
Average	0.9402	0.0243

The Fig. 14- shows the performance of classification model for severity detection using Few-shot learning method with a base classifier of ResNet-50. The data labels for critical and mild are mentioned in the above figure. This model can perform both detection and access the severity of the patient condition with a single architecture and achieves excellent accuracy.

Fig. 14.

Performance graph for severity detection.

5. Conclusion

The world is still struggling with different mutants of Corona viruses. The most difficult challenge for doctors is still accurate diagnosis and prognosis of COPD and lung disorders caused by the Corona virus. Patients with deteriorating breathing or those considered at risk for illness progression must undergo chest imaging for diagnosis. As a solution to the problem of classifying coronavirus and its severity, Mask R-CNN is used to predict covid $+$ve and $-$ve from the chest computerized tomographic image, and severity is detected with a few-shot learning approach. Using a combination of pre-trained base classifiers and meta-learning, a framework is proposed. It is far simpler and more convenient than other methods for classifying the severity of positive patients. From the experimentation outcome, the proposed model achieves the G-Mean and MCC for binary classification as 97.69% and 97.28%, respectively. For multiclass detection, it achieves a G-Mean of 94.02% with an error rate of 2.43%. When building this model, the cosine similarity distance is used to estimate how far apart the trained meta-learning model and the query images are from each other. It enables healthcare professionals to accurately detect severity in order to recommend appropriate treatments. The main challenge in this model is imbalanced classification. In the future, the severity of COVID can be detected with a greater number of infections (class labels) like pneumothorax, asthma, etc. by overcoming the problem of imbalanced classification.

CRediT authorship contribution statement

Suganya D.: Conceptualization, Methodology, Writing – original draft. Kalpana R.: Visualization, Investigation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.