COVID-19 detection from lung CT-Scans using a fuzzy integral-based CNN ensemble

Abstract

The COVID-19 pandemic has collapsed the public healthcare systems, along with severely damaging the economy of the world. The SARS-CoV-2 virus also known as the coronavirus, led to community spread, causing the death of more than a million people worldwide. The primary reason for the uncontrolled spread of the virus is the lack of provision for population-wise screening. The apparatus for RT-PCR based COVID-19 detection is scarce and the testing process takes 6–9 h. The test is also not satisfactorily sensitive (71% sensitive only). Hence, Computer-Aided Detection techniques based on deep learning methods can be used in such a scenario using other modalities like chest CT-scan images for more accurate and sensitive screening. In this paper, we propose a method that uses a Sugeno fuzzy integral ensemble of four pre-trained deep learning models, namely, VGG-11, GoogLeNet, SqueezeNet v1.1 and Wide ResNet-50-2, for classification of chest CT-scan images into COVID and Non-COVID categories. The proposed framework has been tested on a publicly available dataset for evaluation and it achieves 98.93% accuracy and 98.93% sensitivity on the same. The model outperforms state-of-the-art methods on the same dataset and proves to be a reliable COVID-19 detector. The relevant source codes for the proposed approach can be found at: https://github.com/Rohit-Kundu/Fuzzy-Integral-Covid-Detection.

1. Introduction

COVID-19 is an extremely contagious disease that led to a community spread in the world. It was declared to be a public health emergency by the World Health Organization (WHO) in January 2020 and recognized as a pandemic in March 2020. Currently, more than 50 million people worldwide have been affected by the coronavirus, out of which it accounts for more than 1.25 million unfortunate deaths (2.5% mortality rate). However, many positive cases go undetected due to the lack of provision for conducting population-wise testing. The current statistics of the COVID-19 pandemic in the world are shown in Fig. 1 . All the data for the graphs have been obtained from a publicly available source by Ref. [47].

Fig. 1.

Graph showing the growth of COVID-19 disease in the world (till mid-September 2021) [47]: (a) Total cases and deaths, and (b) Daily new cases and deaths.

The available standard tests for COVID-19 detection is the Real-Time Polymerase Chain Reaction (RT-PCR) and the rapid antigen test. However, there are some drawbacks of the RT-PCR test [59]. First, it takes 6–9 h for the testing process. Secondly, there is the issue of unavailability of sufficient apparatus for population-wise testing, and thirdly, the RT-PCR test is not wholly reliable, being only 71% sensitive [67]. Due to these reasons, this virus has spread to an uncontrollable extent. On the other hand, the rapid antigen test [34] uses the human blood sample to detect the presence of IgG and IgM antibodies for COVID-19 detection and can produce the result within 15 min. However, the human body takes several days to produce these antibodies after the virus has entered the host, making the antibody test unreliable to detect COVID-19 in its early stages. This can lead to the spread of the virus even before the disease is diagnosed. Even though several vaccines have been developed by experts across the world, it will take a lot of time to vaccinate each individual on the planet, especially with the spread of the new variants of the COVID-19 virus that are emerging due to gene mutation in the virus. Therefore, there is a need to develop Computer-Aided Detection tools for faster and less laborious screening procedures to help control the pandemic. A large part of this research focuses on using deep learning techniques [9,42]. Deep learning is a machine learning technique that uses artificial neural networks for classifying structured or unstructured data through a complex decision-making process. As compared to the RT-PCR test, chest CT-scans are more sensitive [1,14], and more widely available. Although X-Ray images are being used for automated screening, CT-scan images give a more detailed view of the lungs and therefore, are more reliable for this study [39]. The most distinguishing feature of COVID-19 infected chest CT scans is the ”ground-glass opacities” (GGO) that appear inside the lungs [11,33]. They emerge when the alveoli (tiny air sacs) get filled with fluid and appear as a white frosted-glass like appearance in the chest CT-scan [43]. Fig. 2 shows the chest CT-image of a COVID-19 infected patient and a COVID-negative patient. The GGO in the lungs of the infected patient are marked in the image.

Fig. 2.

Sample chest CT-scan images of: (a) COVID-19 positive patient, and (b) COVID-19 negative patient. The images have been taken from the publicly available SARS-COV-2 dataset [54] used in this study.

Deep learning has been proven successful in biomedical supervised learning applications like [2,15]. In the current image classification task, transfer learning has been employed which refers to the method of re-using a deep learning model used for a specific task, on a separate but related task. It is used when the current problem lacks sufficient data for training the model from scratch. Here, the parameters from the previous task are loaded, and the data from the current task are used for fine-tuning the model.

The present work uses the Ensemble Learning technique for classifier fusion. In doing so, four pre-trained models, namely, VGG-11 by Simonyan et al. [53], GoogLeNet by Szegedy et al. [57], SqueezeNet v1.1 by Iandola et al. [24] and Wide ResNet-50-2 by Zagoruyko et al. [69] have been used, and Sugeno Fuzzy Integral has been used for ensembling the four classifiers and generating the final prediction scores for the images. Ensemble Learning enhances the performance of the constituent models by accounting for increased diversity in predictions. Although simple fusion schemes, such as averaging probability scores, majority voting, etc., have been used in literature, such techniques cannot account for the complex co-occurrences within the data to be classified. They do not account for the different weights of classifiers based on the decision scores obtained at testing time. Therefore, there is a need to generalize the simple fusion schemes, such that the weightage of one classifier may be conditioned upon the weightage of the others, to promote the adaptive importance of different classifiers for every sample image.

In the present work, the Sugeno fuzzy integral has been used for ensembling the aforementioned classifiers to address the shortcomings of using simple fusion techniques. Fuzzy integrals [61] are effective aggregators, that use the degree of uncertainty in the decision scores as additional information for the fusion of classifiers. It can be viewed as a generalisation of aggregation operators on a set of confidence scores that use some weightage given to each information source, called fuzzy measures. The Sugeno fuzzy integral takes into consideration the confidence of predictions by the base learners to assign adaptive weights to them for each input to the model, unlike the simple fusion schemes found in the literature that use fixed pre-determined weights. This concept and its application in the present study are discussed in detail in Section 3.5.

The rest of the paper has been organized into sections as follows: Section 2 provides a brief literature survey of existing COVID-19 detection models; Section 3 explores in detail, the methodology proposed in the current study; Section 4 describes the results obtained on implementation of the proposed model on a publicly available dataset and finally, Section 5 concludes the findings from this paper and outlines some possible extension of this work.

2. Literature survey

Since the outbreak of the COVID-19 pandemic, a diverse range of classification models have been proposed for the automated detection of COVID-19 patients [28]. A large number of these methods use chest radiology images (X-Rays) like [5,63]. Mahmud et al. [38] developed the CovXNet model for differentiating COVID-19 cases from pneumonia using chest X-ray images, wherein they achieved 90.2% accuracy on the multi-class classification problem. Ozturk et al. [42] developed the DarkNet model for the binary and multi-class classification of COVID-19 and pneumonia using chest X-Ray images. They based their model on the You Only Look Once (YOLO) architecture imbibing some modifications and achieved an accuracy of 98.08% for the binary classification problem (COVID-19 vs. No Findings) and 87.02% for the multi-class problem (COVID-19 vs. Pneumonia vs. No Findings).

However, chest CT-scan images are more sensitive as they give a clearer view of the lungs, therefore enabling a more reliable diagnosis. Shi et al. [51] segmented the lungs from the CT-scan image and used machine learning techniques for location-specific feature extraction. The extracted features were fed to their proposed ”infection Size Aware Random Forest (iSAR)” classifier for the final classification. Incorporating 5-fold cross-validation on their dataset, they obtained 87.9% accuracy, 90.7% sensitivity and 83.3% specificity. Machine learning requires the extraction of handcrafted features and often results in redundant features being extracted. Deep learning, on the other hand, learns the important features on its own and provides an end-to-end classification system, and therefore has been used for developing several COVID-19 detection models like [32,50]. Gozes et al. [16] used a fusion of two deep learning models for the classification of COVID-infected patients. One model uses 2D slices of CT scans and the other uses volumetric CT-scan images. Xu et al. [66] proposed a location-attention classification model that uses features extracted by the ResNet-18 pre-trained model from the lung CT images for classification. Burdick et al. [10] conducted a multi-centre clinical trial to predict invasive mechanical ventilation of COVID-19 infected patients within 24 h of the initial encounter with the virus. However, acquiring data from clinical trials requires authorization from several bodies and is not publicly accessible.

Hu et al. [22] developed a weakly supervised COVID classification model that first performs lung segmentation and then uses a deep CNN model that extracts features at 3 different steps for multi-scale learning. Ni et al. [41] proposed a deep learning algorithm for lesion detection, segmentation and segmentation to find pneumonia lesions in COVID-19 patients.

An elaborate study on the efficacy of different transfer learning models (popular and new models) for COVID-19 detection using chest CT images have been conducted by Ardakani et al. [6]. They found ResNet-101 to be the most accurate classifier, obtaining an AUC of 0.994 on a dataset of 1082 CT images. Amyar et al. [3] developed an encoder-decoder network for the segmentation and classification of chest CT images for COVID-19 detection. For this, they use an encoder coupled with two decoders and a multi-layer perceptron for the segmentation, reconstruction and classification tasks. They achieved an average AUC of 97% on a dataset of 1349 patients’ lung CT scans.

Carvalho et al. [12] developed a framework for COVID-19 classification using CT images wherein they extracted deep features from a CNN model and then used a Genetic Algorithm for optimal feature subset selection and classification. Vinod et al. [62] proposed a ”Deep Covix-Net” model for detecting COVID-19 from both X-ray and CT images. They tested their model in both binary-class and multi-class settings achieving accuracies of 97% and 96.8% respectively.

Most of the existing methods use a single classification model. The ensemble learning paradigm is very seldom explored for addressing the COVID-19 detection problem. Thus in this paper, we develop a deep ensemble model for the classification of chest CT images for COVID-19 detection. However, instead of using popular ensemble approaches prevalent in literature, like probability averaging, or weighted probability averaging, we use a fuzzy integral-based ensemble. This leverages adaptive priority to the classifiers on the fly instead of the classical ensemble methods that set the priority given to each classifier beforehand.

2.1. Motivation and contributions

The COVID-19 pandemic has caused the front-line workers to work tirelessly for attending to COVID-infected patients, along with patients homing other diseases, putting their lives at risk. Also, there is no sign of the pandemic stopping, and since population-wise screening is impossible with the current level of RT-PCR test kits available. So, the need for the development of an automated screening model motivated us to conduct this study.

In this paper, we propose a method to improve the efficiency of the Convolutional Neural Network (CNN) models by using a fuzzy integral based ensemble. Four distinct CNN architectures are used to ensure that there are multiple sources of information, manifested by the individual characteristics of the different features. The overall workflow of the proposed model is shown in Fig. 3 . The complementary nature of the data captured by the different CNNs is verified by comparing the statistical divergences from the decision scores of each CNN model. The CNN classifiers are fused by applying the Sugeno Fuzzy integral using which has two advantages, first, it works as a generalization of classical empirical schemes, and second, conditioning the weights of each model based on the probability scores of the individual classifiers capture the complementary information more robustly.

Fig. 3.

Overall workflow of the proposed approach.

The main contributions of this paper are as follows:

• 1.Chest CT-scan images are used to develop the proposed model in this study which is both widely available and can be performed faster compared to the tedious RT-PCR testing process.
• 2.To address the challenge of low availability of publicly available chest CT data of COVID-19, transfer learning models have been used in the first phase of the framework. The complementary natures of the features vectors obtained from the CNNs are verified using the Kullback-Leibler Divergence and Jensen-Shannon Divergence.
• 3.For using the features of the various models, an ensemble technique has been used based on a fuzzy measure: the Sugeno Integral. Fuzzy integrals are a powerful fusion technique for using the characteristics of all the input information sources.
• 4.The proposed method outperforms the existing state-of-the-art models achieving very high accuracy (98.93%) and sensitivity (98.93%) compared to the 71% sensitivity of the RT-PCR test.

3. Proposed method

The proposed framework uses four pre-trained deep learning models, which are trained on the ImageNet dataset [13]. The ImageNet dataset is a huge data source consisting of 14 million images belonging to 1000 classes. Training the models on such a large variety of images set the model parameters (weights and biases of the nodes of each layer) optimally, and only minor fine-tuning is generally required for using the trained models on a different classification task. The deep transfer learning models used in this study are:

• 1.VGG-11 by Simonyan et al. [53].
• 2.GoogLeNet by Szegedy et al. [57].
• 3.SqueezeNet v1.1 by Iandola et al. [24].
• 4.Wide ResNet-50-2 by Zagoruyko et al. [69]

VGG-11 is a very deep network whereas GoogLeNet has inception modules that contain parallel convolutions and hence, both of these networks have a high number of parameters. On the other hand, both SqueezeNet v1.1 and Wide ResNet-50-2 models boast about their computational efficiency due to a far lesser number of parameters for the number of layers they possess. The VGG-11 model can extract very deep features due to the use of linearly progressive convolution layers with small filter sizes. The GoogLeNet model can extract a diverse set of features through its inception modules. The SqueezeNet v1.1 model uses the squeeze and expand blocks to extract deep features while also controlling the number of parameters. Whereas, the WideResNet-50-2 model uses its residual connections to control the feature flow and eliminate the vanishing gradient problem, while concatenating the shallow features with the deep ones for efficient feature extraction. Thus, these four pre-trained models have been chosen for the current study to fuse the diverse properties of each model while also controlling the computational complexity. The total number of parameters in each network are shown in Table 1 .

Table 1.

Layer and Parameter details of the pre-trained models used in the current study.

CNN	Layers	Kernels	Parameters
VGG-11	11	(3x3)	30.15 M
GoogLeNet	22	(1x1), (3x3), (7x7)	11.98 M
SqueezeNet v1.1	14	(1x1), (3x3)	0.72 M
Wide ResNet-50-2	50	(3x3)	66.8 M

These pre-trained models are used for the binary classification task at hand and the prediction probability scores for the images are stored. These probability prediction scores are then used for computing the Sugeno Fuzzy Integral to calculate the final predictions on the test set and then the predictions are used to compute the evaluation metrics. The transfer learning models and the fuzzy integral ensemble method adopted in this study are described in the following subsections.

3.1. VGG-11

The VGG models developed by the Visual Geometry Group [53], are some of the deepest CNNs in literature. The introduction of several weight layers (up to 16–19 layers) was made possible by decreasing the size of the convolution filters to 3x3 kernels. The VGG group stresses the fact that the depth of a CNN model is important for visual representations and beneficial for accurately applying to several genres of classification tasks in Computer Vision. The VGG-11 CNN architecture has been shown in Fig. 4 .

Fig. 4.

Architecture of the VGG-11 CNN model used in the present work.

3.2. GoogLeNet

The GoogLeNet CNN model by Szegedy et al. [57], the name of which is inspired from LeNet-5 [31] is 22 layers deep network that consists of ”inception modules” instead of uniformly increasing networks. The inception block inspired by Lin et al. [35], allows a large number of units to be accommodated at each stage due to the parallel convolution and pooling, without facing an uncontrolled computational complexity because of the increased number of parameters. For controlling the computational complexity, the architecture uses dimensionality reduction in the inception blocks as shown in Fig. 5 (b), rather than the naive inception block (Fig. 5(a)). The GoogLeNet model proves that an optimal sparse architecture made from the available dense building blocks, i.e., the introduction of inception blocks, improves the performance of neural networks for Computer Vision tasks. The architecture of the GoogLeNet CNN model has been shown in Fig. 6 .

Fig. 5.

Block diagram illustrating the inception modules in the GoogLeNet architecture.

Fig. 6.

Architecture of the GoogLeNet model (Inception Block is as in Fig. 5(b)).

3.3. SqueezeNet v1.1

The SqueezeNet [24] is a CNN model having 50 times lesser parameters than AlexNet [26] model but achieving similar accuracies on the ImageNet data classification task. However, to employ model compression techniques [18,19] allow the SqueezeNet model to be stored under 0.5 MB space which is 510 times smaller than the space occupied by the AlexNet model.

The salient features of the SqueezeNet architecture are as follows:

• 1.Majority of 3x3 filters have been replaced with 1x1 filters, which have 9 times fewer number parameters.
• 2.To maintain a small total number of parameters, the number of input channels has been decreased to only 3x3 filters using ”squeeze layers”.
• 3.The down-sampling of the images has been performed late in the architectures for obtaining larger activation maps from the convolution layers.
• 4.A ”fire module” has been introduced, which is comprised of a ”squeeze” convolution layer that feeds into an ”expand” layer that is a mix of 1x1 and 3x3 convolutions. The fire module is shown in Fig. 7 . The squeeze layer in the fire module helps to limit the number of input channels. The following hyperparameters describe a fire module: s ₁ denotes the number of 1x1 filters in the squeeze layer; e ₁ denotes the number of 1x1 filters in the expand layer, and e ₃ denotes the number of 3x3 filters in the expand layer.

Fig. 7.

Illustration of the Fire Module: a salient feature of the SqueezeNet architecture.

The architecture of SqueezeNet v1.1 is shown in Fig. 8 .

Fig. 8.

Architecture of the SqueezeNet v1.1 CNN model.

3.4. Wide ResNet-50-2

Wide ResNets [69] are a genre of CNNs honing residual blocks in their architecture. They have lower depths and greater width than the traditional ResNets [20] achieving state-of-the-art results but consuming much lesser training time. They have proven that the 50 layers deep Wide ResNet (Wide ResNet-50-2) outperformed the 152 layers deep ResNet (ResNet-152) on the ImageNet [13] data classification task. Similar to the previous networks, the final classification layer of the Wide ResNet-50-2 architecture was changed for the binary classification task in the present study. The various residual blocks used in the Wide ResNet architecture has been shown in Fig. 9 , and the complete architecture has been shown in Fig. 10 .

Fig. 9.

Residual Blocks present in Wide ResNet.

Fig. 10.

Architecture of wide ResNet.

3.5. Ensemble: Sugeno Fuzzy Integral

Ensemble learning is a strategy for fusing the salient properties of two or more base learners. Such a framework performs more robustly than its constituent models because ensembling reduces the variance in the prediction errors. However, most of the traditional ensemble frameworks [29,30] tend to assign a pre-defined classifier weight to compute the ensemble. In this study, we use a fuzzy integral-based approach, wherein instead of assigning a fixed weight to the classifiers, we leverage adaptive importance on the fly while testing. Such a method takes into account the confidence in the predictions of the base learners for every sample to assign importance to the classifier, and thus performs more robustly than the traditional approaches in the literature.

Fuzzy integrals have proven successful in several pattern recognition problems like [36,65]. They are powerful and flexible functions for aggregating information based on a fuzzy measure. This fuzzy measure represents the relevance or importance of the constituent information sources when the aggregation is computed.

A fuzzy measure [55] is the set function f that satisfies the following properties:

• 1.f(φ) = 0, f(X) = 1
• 2.A, B ∈ β and A ⊆ B, implies f(A) ≤ f(B)
• 3.If C _n ∈ β, C ₁ ⊆ C ₂ ⊆ C ₃ ⊆…⊆ C _n, then lim_n→∞ f(C _n) = f(lim_n→∞ C _n)

where β is a Borel field [37] of an arbitrary set X.

The Sugeno fuzzy-λ measures was introduced by Tahani et al. [60]. Suppose there is a set of scores S = {s ₁, s ₂, s ₃, …, s _n} where N is the number of information sources (N = 4 in our case), and e ∈ S. The Sugeno-λ measure is the function f _λ: 2^S → [0, 1] such that it satisfies the following conditions:

• 1.f_λ(S) = 1
• 2.if e _i ∩ e _j = φ, then ∃ a λ > − 1, such that, Equation (1) holds true.

$$f_{\lambda }(e_{\mathrm{i}}\cup e_{\mathrm{j}})=f_{\lambda }(e_{\mathrm{i}})+f_{\lambda }(e_{\mathrm{j}})+\lambda .f_{\lambda }(e_{\mathrm{i}})(e_{\mathrm{j}})$$ (1)

Therefore, λ is the real root of the equation as given in Equation (2).

$$\lambda +1=\prod _{n=1}^N\left(\lambda .f(e_{\mathrm{n}})+1\right)$$ (2)

The Sugeno Integral [56] is a fuzzy measure integral [46] that can be defined as follows: Suppose (A, μ) is a measurable (Borel) space, and f: X → [0, 1] is a μ-measurable function, then the Sugeno Integral of the measurable function f with respect to a fuzzy measure Ω is given as in Equation (3).

$$\int f(x)d\mathrm{\Omega }=\underset{1\le i\le n}{\max }(\min (f(x_{\mathrm{i}}),\mathrm{\Omega }(A_{\mathrm{i}})))$$ (3)

where Ω(A _i) = Ω({x _i, x _i+1, x _i+2, …, x _n}) and.

{f(x ₁), f(x ₂), f(x ₁), …, f(x _n)} are the ranges defined as f(x ₁) ≤ f(x ₂) ≤ f(x ₃) ≤ … ≤ f(x _n) The algorithm for computing the Sugeno Integral is shown in Algorithm 1.

Algorithm 1

Pseudo code for Sugeno Integral Ensemble

4. Results and discussion

This section describes the dataset used in the current study, and the results obtained by implementing the proposed approach on the dataset. Comparisons to existing standard models in literature have also been drawn to validate the efficacy of the framework.

4.1. Dataset description

The dataset used in the present work is proposed by Soares et al. [54] which they generously made publicly available on Kaggle.2 It consists of a total of 2481 chest CT-scan images divided into two categories: COVID and Non-COVID, distributed unevenly among the two classes. The distribution of images in the training and test set from each class are shown in Table 2 . The images in the dataset have been classified by Soares et al. [54] according to the results from the RT-PCR tests conducted on the patients. Patients with confirmed COVID or Non-COVID cases are only present in the dataset and thus, there is little or no risk of unnecessary bias through the non-representative selection of control patients.

Table 2.

Class-wise distribution of images in the train and test set of the SARS-COV-2 dataset [54].

Class	Category	Total	Train set	Test set
1	COVID	1252	876	376
2	Non-COVID	1229	860	369

4.2. Evaluation criteria

The current image classification problem is a binary classification task and the metrics that have been used for evaluating the performance of the proposed approach on this task are accuracy, specificity, precision, recall or sensitivity and f1-score. The definitions for ”True Positive (TP)”, ”False Positive (FP)”, ”True Negative (TN)” and ”False Negative (FN)” are required to understand and formulate these parameters.

The two classes in the binary classification task are the COVID-19 positive class and the COVID negative class. If the lung CT-image is of a COVID-19 positive patient and the model classifies it to be COVID positive class (i.e., correct classification), then the sample image is said to be True Positive (TP). Now, if the lung CT image is of a COVID positive patient, but the model classifies it to be COVID negative, the sample is called False Positive (FP). Similarly, a COVID negative lung CT image when classified correctly is called True Negative (FN). And, a COVID negative CT-scan image misclassified as COVID positive is called False Negative (FN). The evaluation metrics can now be formulated as Equations (4), (5), (6), (7), (8).

$$Accuracy=\frac{TP+TN}{TP+FP+TN+FN}$$ (4)

$$Specificity=\frac{TN}{TN+FP}$$ (5)

$$Precision=\frac{TP}{TP+FP}$$ (6)

$$RecallorSensitivity=\frac{TP}{TP+FN}$$ (7)

$$F1-Score=\frac{2\times Precision\times Recall}{Precision+Recall}$$ (8)

In the COVID-19 classification task, the number of FP cases is more dangerous, because COVID-19 is an extremely contagious disease, and a COVID-19 positive patient whose diagnosis is determined as COVID negative will infect other people as a super-spreader. However, the FP case is not that big a problem, since a COVID negative patient diagnosed as a positive needs to just maintain precautions under prescribed guidelines since it is a disease with a very low mortality rate. The contagiousness of a virus is inversely related to its deadliness.

4.3. Implementation

The deep transfer learning models used in the current study have been trained for 100 epochs on the COVID-19 binary classification dataset of lung CT-scan images. The hyperparameters used for the model training are shown in Table 3 which have been set experimentally. The loss curves obtained on training by the four models are shown in Fig. 11 .

Table 3.

Hyperparameters set used for model training in the proposed framework.

Hyperparameter	Value
Optimizer	Stochastic Gradient Descent
Loss function	Cross-entropy
Batch Size	16
Initial Learning Rate	0.0001
Momentum	0.99
Period of Learning Rate Decay	10 epochs
Number of Epochs	100

Fig. 11.

Loss curves obtained by the pre-trained models on the SARS-COV-2 dataset [54]: (a)VGG-11 (b) GoogLeNet (c)SqueezeNet1.1 and (d) Wide ResNet-50-2, on the Kaggle COVID-19 dataset.

The probability distribution from the four classifiers has been fused using the Sugeno Fuzzy Integral as described in Section 3.5. The confusion matrix of the final predictions on the test set is shown in Fig. 12 .

Fig. 12.

Confusion matrix obtained by the proposed framework on the test set of the SARS-COV-2 dataset [54] used in this study.

The class-wise results obtained by the proposed framework on the test set of the SARS-COV-2 dataset are shown in Table 4 .

Table 4.

Class-wise results obtained by the proposed method on the test set of the SARS-COV-2 dataset [54].

Class	Accuracy(%)	Specificity(%)	Precision(%)	Sensitivity(%)	F1-Score(%)
COVID	98.94	98.92	98.94	98.94	98.94
Non-COVID	98.92	98.94	98.92	98.92	98.92
Aggregate	98.93	98.93	98.93	98.93	98.93

The most commonly used tests for COVID-19 detection are the RT-PCR test and the rapid antigen-antibody test. The RT-PCR test takes a long time to produce results and is not sufficiently sensitive. The rapid antigen-antibody test relies on the detection of formed antibodies in the human body and produces results in 15 min. However, the human body requires several days to form antibodies, and thus in the initial stage, an infected patient may go undetected for days. The proposed automated framework gives a high classification accuracy and sensitivity by using chest CT-scan images while also being a much faster method. The proposed method can be used as a plug-and-play model, where new test images can be passed through the model to get the predictions by the ensemble mechanism. Thus, the incorporation of the proposed method in the field under consideration is simplistic and COVID-19 cases can be screened more efficiently.

4.4. Verification of complementary features

For establishing the complementary nature (dissimilarity) in the probability vectors generated by the four pre-trained CNN models, two statistical divergence methods are incorporated: the Kullback-Leibler Divergence (KLD) [27] and the Jensen-Shannon Divergence (JSD) [40].

KLD, is closely related to relative entropy, is a measure of dissimilarity between two probability distributions, and is an non-symmetric measure. Suppose we have a probability space X. For every discrete random variable x in X, suppose there are two discrete probability distributions p(x) and q(x) on this same space obtained from two classifiers, i.e., p(x) + q(x) = 1 and p(x), q(x) > 0. Then the KLD (discrete form) from q(x) to p(x) is given as in Equation (9).

$$D_{\mathrm{K}\mathrm{L}}\left(p(x)\Vert q(x)\right)=\sum _{x\in X}p(x)ln\left(\frac{p(x)}{q(x)}\right)$$ (9)

Since $D_{\mathrm{K}\mathrm{L}}\left(p(x)\Vert q(x)\right)\ne D_{\mathrm{K}\mathrm{L}}\left(q(x)\Vert p(x)\right)$, the symmetrical statistical divergence known as JSD has been proposed, which is derived from the KLD, and is a smoothed version of it. For probability distributions P = p(x) and Q = q(x) on the probability space X, suppose we have another probability distribution $M=\frac{1}{2}(P+Q)$, then the discrete form of JSD is given as in Equation (10).

$$D_{\mathrm{J}\mathrm{S}}(P\Vert Q)=\frac{1}{2}\left[D_{\mathrm{K}\mathrm{L}}(P\Vert M)+D_{\mathrm{K}\mathrm{L}}(Q\Vert M)\right]$$ (10)

The KLD and JSD values between every pair of classifiers from the four pre-trained CNN models are shown in Table 5 .

Table 5.

KL and JS divergences among the four pre-trained CNN classifiers on the SARS-COV-2 dataset [54].

Distribution P	Distribution Q	DKL(P‖Q)	DJS(P‖Q)
VGG-11	GoogLeNet	0.108	0.134
GoogLeNet	VGG-11	0.456
VGG-11	SqueezeNet v1.1	0.651	0.105
SqueezeNet v1.1	VGG-11	0.164
VGG-11	Wide ResNet-50-2	0.314	0.104
Wide ResNet-50-2	VGG-11	0.480
GoogLeNet	SqueezeNet v1.1	0.386	0.131
SqueezeNet v1.1	GoogLeNet	0.105
GoogLeNet	Wide ResNet-50-2	0.226	0.128
Wide ResNet-50-2	GoogLeNet	0.194
SqueezeNet v1.1	Wide ResNet-50-2	0.297	0.106
Wide ResNet-50-2	SqueezeNet v1.1	0.326

4.5. Comparison with existing methods

The proposed approach is compared with some standard deep transfer learning models, including the ones used for the ensemble in the present study, and the results thus obtained are tabulated in Table 6 . DenseNet161 performs closest to the proposed ensemble model, obtaining an accuracy of 96.91%, but still, the performance is 2.2% lower than the proposed method.

Table 6.

Comparison of the proposed framework with some standard pretrained models on the SARS-COV-2 dataset [54].

Model	Accuracy
DenseNet161 [23]	96.91%
Wide ResNet-50-2 [69]	96.78%
VGG-11 [53]	96.38%
SqueezeNet v1.1 [24]	96.24%
GoogLeNet [57]	96.11%
Inception v3 [58]	92.15%
Proposed Method	98.93%

The results obtained by using different ensemble techniques on the SARS-COV-2 dataset [54] are shown in Table 7 . The fuzzy measures used for the Sugeno Integral has been found through extensive experimentation to be {0.35, 0.35, 0.02, 0.28} for {VGG-11, GoogLeNet, SqueezeNet v1.1, Wide ResNet-50-2} giving the best performance on several runs. The weights assigned for computing the weighted average are {0.27, 0.32, 0.23, 0.18}. Other than the weighted average approach, other ensembles that have been used for comparison are the element-wise Multiplication Rule, Maximum Probability Rule, Majority Voting Ensemble, and Average Probability Rule.

Table 7.

Comparison of Fuzzy Integral fusion of classifiers with other popular ensemble techniques on the SARS-COV-2 dataset [54].

Ensemble technique	Accuracy
Multiplication Rule	95.82%
Maximum	96.78%
Majority Voting	97.65%
Average	97.83%
Weighted Average	98.12%
Proposed Method	98.93%

Among the explored ensemble techniques, the weighted average approach gives the result closest to the fuzzy integral ensemble, where the weights have also been assigned experimentally. This weighted average ensemble is a static computation process since there is no scope for dynamically refactoring the weights at the prediction time. Fuzzy integral fusion, on the other hand, can use the confidence scores of each classifier and tailors the weights on the fly, which is a different set for each sample data point. This enables further refinement of predictions even after setting the fuzzy measures.

Existing approaches for the chest CT classification for COVID-19 detection on the dataset used in this study mostly use a single CNN model for classification. Halder et al. [17] and Jaiswal et al. [25] used transfer learning with DenseNet201, for classification. Panwar et al. [44] used transfer learning with VGG-19 along with adding 5 more fully connected layers before the final classification layer. In their experiments on chest X-Ray images, they employed a Grad-CAM based colour visualization approach for interpreting the X-ray scans for further diagnosis. Angelov et al. [4] used the GoogLeNet architecture (non-pretrained model) for extracting deep features, which were used by an MLP classifier for the final classification on the chest CT-images dataset and Sen et al. [49] used a bi-stage deep feature selection approach for the COVID-19 detection problem. The results obtained by these methods on the SARS-COV-2 dataset [54], used in the current study, have been compared with the model proposed by us in Table 8 . The performance of the proposed model is seen to be appreciably better than the existing methods in the literature. As compared to the single CNN model-based approaches by Jaiswal et al. [25], Panwar et al. [44] and Angelov et al. [4], the proposed approach performs significantly better justifying the use of an ensemble framework.

Table 8.

Comparison of the proposed framework with existing methods in the literature on the SARS-COV-2 dataset [54].

Method	Accuracy(%)	Specificity(%)	Precision(%)	Sensitivity(%)	F1-Score(%)
Yazdani et al. [68]	–	96.20	–	85.00	90.00
Silva et al. [52]	87.60	–	–	–	86.19
Angelov et al. [4]	88.60	–	89.70	88.60	89.15
Wang et al. [64]	90.83	–	95.75	85.89	90.87
Panwar et al. [44]	94.04	95.86	95.00	94.00	94.50
Sen et al. [49]	95.32	–	95.30	95.30	95.30
Jaiswal et al. [25]	96.25	96.21	96.29	96.29	96.29
Halder et al. [17]	97.00	95.00	95.00	98.00	97.00
Horry et al. [21]	97.40	–	99.10	95.50	97.30
Kundu et al. [30]	97.81	–	97.77	97.81	97.77
Pathak et al. [45]	98.37	–	98.74	98.87	98.14
Biswas et al. [8]	98.79	–	98.79	98.79	98.79
Banerjee et al. [7]	98.85	–	–	–	–
Proposed Method	98.93	98.93	98.93	98.93	98.93

4.6. GradCAM analysis

Gradient-weighted Class Activation Maps or GradCAM [48] provides a visual representation of the regions in an image that is focused on by the CNN model. The features from the convolution layers of the trained CNN model are used to produce a gradient-based weighted activation map which is superimposed on the input image to highlight the regions that are considered discerning by the CNN model.

Fig. 13 shows the GradCAM activations by the four different models used to form the ensemble on some sample images taken from the SARS-COV-2 dataset [54]. As seen from the figure, different models focus on different parts of the same chest CT image. For example, Fig. 13(a) shows a CT image of a COVID-19 infected patient. Fig. 13(b), (c), (d) and (e) respectively show the activations produced by VGG-11, GoogLeNet, SqueezeNet v1.1 and Wide ResNet-50-2. VGG-11 focuses on the lower part of the right lung (left side of the image) and the middle part of the left lung. GoogLeNet focuses on the middle to lower part of the left lung. SqueezeNet v1.1 focuses their attention on the GGO appearing on the upper and lower regions of the right lung and middle of the left lung, and Wide ResNet-50-2 focuses on the entirety of the right lung (primarily on the lower side) where the GGO is more concentrated.

Fig. 13.

Some sample CT-scan images (taken from the SARS-COV-2 dataset [54]) along with their GradCAM activations by the four models used for forming the ensemble in this study.

The different models focusing on the different regions of the chest CT images justify that complementary information is being extracted, making the ensemble successful. From the examples shown, it is clear that all the models focus on different regions inside the lungs of the patients. Since GGO starts appearing in the lungs of COVID-19 infected patients from the onset of the disease, the models can classify the images correctly most of the time by looking for the white opacities in the lungs. Thus, the proposed framework is fit for detecting the early stages of COVID-19, unlike the rapid antigen test which is only effective after several days of the infection. This makes the proposed framework a good COVID-19 predictor for use in the practical field.

4.7. Error analysis

The proposed framework works reliably well for even heavily noisy images examples of which are shown in Fig. 14 (a) and (f). Both the images in the figure have been taken in bad imaging conditions, the images being hazy and having improper contrast. In the CT-scan image shown in Fig. 14(f), both the lungs of the patient are not prominent, but even so, the proposed approach could classify the images correctly. For further justification, the GradCAM activations by the four base learners have been shown along with the images. In both cases, the models were able to focus on the correct regions inside the lungs to compute the predictions. And since different regions were focused on the different models, the ensemble was able to incorporate the complementary information provided by each.

Fig. 14.

Correctly classified noisy samples (from the SARS-COV-2 dataset [54]) of: (a) COVID-19 infected case with its corresponding GradCAM activation by the four models shown in (b)–(e) and (f) Non-COVID case with its corresponding GradCAM activation by the four models shown in (g)–(j).

However, Fig. 15 (a) and (f) shows two misclassified samples from each category- COVID and Non-COVID. Fig. 15(a) shows the lung CT-scan of a COVID-19 positive patient but has been classified as a Non-COVID case by the proposed model. The image when examined can be seen that does not prominently have GGO in either lung and possibly is a very mildly affected COVID case. The image has most likely been taken in a very early stage, where the COVID-19 virus has not even started affecting the lungs, leading to a wrong prediction. The GradCAM activations in Fig. 15(b)–(e) shows that the base learners were not able to localize on the correct regions of the lungs to extract discerning information. Fig. 15(f) shows the lung CT-scan image of a Non-COVID category patient which has been misclassified to be a COVID case by the proposed model. The bottom portion of the right (image perspective) lung has an erroneous white region, which might have been considered to be the GGO by the model (which is seen in the GradCAM activation maps in Fig. 15(g)–(j)), resulting in a wrong prediction.

Fig. 15.

Misclassified samples (from the SARS-COV-2 dataset [54]) of: (a) COVID-19 infected case with its corresponding GradCAM activation by the four models shown in (b)–(e) and (f) Non-COVID case with its corresponding GradCAM activation by the four models shown in (g)–(j).

Even so, only a few images have been misclassified by the proposed model, and the false positive rate is also low, which is important for a COVID classifier. Although the model fails to accurately classify all the samples provided, it still performs reliably well and is very sensitive to changes in chest CT scans.

5. Conclusion and future work

COVID-19 is an extremely contagious disease whose alarming spread on the global scale has crippled the world health care systems. It has set back the global economy to a near stand-still as the countries of the world impose lockdown throughout the country to try and halt the spread of the virus. As the human toll increases, the need for a conclusive solution becomes direr. The primary reason for the rapid spread of the virus is the lack of equipment for population-wide screening of infected patients. Coupled with this, the gold-standard RT-PCR test is not all-reliable either, being only 71% sensitive, and takes 6–9 h for a single test as well. This demands a fully automated system that can detect COVID-19 through other modalities in a fast and reliable manner.

In this paper, we have proposed a fully automated, computer-based detection method that uses lung CT-scan images for the classification of COVID-19 infected patients. Chest CT scans are more sensitive than RT-PCR tests and X-Rays, while also being more widely available than RT-PCR test kits. The proposed method uses deep learning techniques along with a fuzzy measure based integral, the Sugeno Integral for ensembling the features of four pre-trained deep transfer learning models.

The results from the proposed approach on the SARS-COV-2 dataset [54] outperforms the current state-of-the-art results on the same dataset. From the confusion matrix in Fig. 12, it is clear that the False Positive samples are very less which is important as explained in Section 4.2. Also, from Table 4, it is evident that the accuracy and sensitivity are very high (compare with the 71% sensitivity of RT-PCR). So, it can be concluded that the proposed approach is reliable and can be used for COVID-19 detection.

As discussed before, the proposed model was unable to classify a few instances of mild COVID situations. Since the COVID-19 infection only affects the inside of the lungs, and there are no pleural effusions, a patch-based approach that selects patches from the image for training, or an attention mechanism can be developed in future. Choosing the fuzzy measures is an optimization problem, but for this study, we set the measures through extensive experiments. We aim to address this problem in future as well.

Declaration of competing interest

The authors declare to have no conflicts of interest.