A framework for in-vivo human brain tumor detection using image augmentation and hybrid features

Abstract

Brain tumor is caused by the uncontrolled and accelerated multiplication of cells in the brain. If not treated early enough, it can lead to death. Despite multiple significant efforts and promising research outcomes, accurate segmentation and classification of tumors remain a challenge. The changes in tumor location, shape, and size make brain tumor identification extremely difficult. An Extreme Gradient Boosting (XGBoost) algorithm using is proposed in this work to classify four subtypes of brain tumor—normal, gliomas, meningiomas, and pituitary tumors. Because the dataset was limited in size, image augmentation using a conditional Generative Adversarial Network (cGAN) was used to expand the training data. Deep features, Two-Dimensional Fractional Fourier Transform (2D-FrFT) features, and geometric features are fused together to extract both global and local information from the Magnetic Resonance Imaging (MRI) scans. The model attained enhanced performance with a classification accuracy of 98.79% and sensitivity of 98.77% for the test images. In comparison to state-of-the-art algorithms employing the Kaggle brain tumor dataset, the suggested model showed a considerable improvement. The improved results show the prominence of feature fusion and establish XGBoost as an appropriate classifier for brain tumor detection in terms on testing accuracy, sensitivity and Area under receiver operating characteristic (AUROC) curve.

Introduction

Brain tumors develop when irregular cells grow inside the brain. Tumors are divided into two types: benign (non-cancerous) and malignant (cancerous). These categories can be further classified as primary tumors, which begin in the interior segment of the brain, and secondary tumors, that are located outside the brain. The symptoms produced by brain tumors depends upon its size and location in the brain. The symptoms and signs of these tumors are widespread. Depending on the type, size, and location of the tumor, people may suffer a variety of symptoms [1]. Most common and early symptom of brain tumor is headache. Headaches, due to brain tumors, are mainly because of the intracranial pressure exerted by the fluids on the skull surface. Other symptoms that are reported, in association with brain tumors, are vision problems, vomiting, behavioural changes, speech disorder, unconsciousness and seizures. The cause of the majority of brain tumors is currently unknown [2]. Aside from ionising radiation and vinyl chloride exposure, no known environmental factors have been linked to the development of brain tumors. The deletion and mutation of P53 tumor suppressor genes is another cause of various forms of brain cancers. Smoking and people with genetic illnesses such tuberous sclerosis, Von Hippel–Lindau syndrome, and endocrine neoplasia have been linked to an increased risk of developing brain tumors in a few studies. Although the World Health Organization (WHO) has indicated that mobile phone radiation may be carcinogenic to the brain, there is no conclusive data in this regard [3, 4].

Despite the lack of a specific symptom, the occurrence of a set of symptoms and the lack of persistent signs of other causes can be a pointer to a brain tumor. The diagnosis begins with a medical history that includes medical backgrounds and clinical manifestations. Laboratory investigations serve to eliminate other infections as the cause of the symptoms [5]. Electro-encephalography (EEG), eye examinations, and otolaryngological (or ENT) exams may be conducted at this stage. Thus, brain tumors pose a challenge for diagnosis in comparison to other tumors of the body. Due to the strict control of blood–brain barrier (BBB) membrane over brain, many tracers would be unable to reach brain tumors and thus cause complications in diagnosis through radioactive indicators. Interruption of the BBB due to larger tumors is efficiently imaged through an MRI or computed tomography (CT) scan, and hence regarded as the core diagnostic method for malignant gliomas, meningiomas, and pituitary tumors [6–8]. Medical imaging plays a dominant role in the early detection of brain tumors. Conventional methods such as biopsy, pneumoencephalography and cerebral angiography have been abandoned due to their long interpretation time, danger, cost and pain. Non-invasive methods especially MRI and CT scans, provide high resolution and fast results [9].

Primary brain tumors occur in around 250,000 people a year globally. Brain tumors come second after acute lymphoblastic leukaemia as the most common type of cancer in children above 15 years of age. The number of incidences of brain tumors has a huge difference between developed and developing countries (developing countries have lesser incidences). This is due to undiagnosed tumor-related deaths. Patients in extreme poverty and poor medical facilities do not get access to early diagnostic facilities required to diagnose a brain tumor at the initial stage [10]. Considering this scenario, automatic disease detection and classification are crucial for decreasing mortality due to brain tumors, as well as carrying out effective treatment and prognosis. In recent medical computing research, machine learning and deep learning contribute a significant number of solutions toward early disease prediction or classification through efficient image processing. These automated systems detect a tumor in the MR images without any human interference [11]. Medical image processing involves pre-processing of images such as resizing, enhancement, filtering, segmentation, feature extraction and feature selection. Post-processing includes the identification and classification of abnormal case images. These steps can be implemented by deep learning as well as the conventional machine learning approach. In the traditional machine learning approach, hand-crafted features are extracted to attain results from the test images and thus the method produce rapid results [12–16]. In deep learning, models are modified by aptly choosing the number of layers, activation and pooling. However, in several literatures, both machine learning and deep learning algorithms are used to improve classification accuracy. The proposed work in this article used hybrid feature extraction of brain tumor MRI images and a XGBoost algorithm for the classification of images into normal and tumorous.

The main contributions of the proposed study include 4 phases, shown in Fig. 1 and précised below:

1. The brain tumor image dataset has been collected, pre-processed and augmented through cGAN to increase the quantity for better training of the classification model.

2. Unlike traditional handcrafted feature extraction techniques, deep features using VGG16, ResNet50 and MobileNet, time–frequency features using 2D-FrFT and geometric features using Eccentricity and Curvature descriptor have been implemented for obtaining a broader spectrum of features, shown in Table 1.

3. A highly efficient gradient boosting framework based on ensemble learning, XGBoost, has been used for the classification of four classes of brain tumors—normal, gliomas, meningiomas, and pituitary tumors.

4. The results have been compared with other existing models available for brain tumor detection or classification. Another comparative analysis has been made with a few machine learning classifier to analyse the performance of XGBoost.

Fig. 1.

Flowchart of the proposed system

Table 1.

Detail of features extracted from brain tumor MRI images

Deep features	2D-FrFT features	Geometric features
VGG16 ResNet50 MobileNet	–	Eccentricity Curvature descriptor

Related work

The introduction of deep learning technology has led to a significant improvement in a broad range of medical diagnostic problems. Deep learning models hierarchically learn features from the input data by the convolutional kernel as the network goes deeper. Kavitha et al. [17] employed a genetic algorithm for the purpose of segmentation and a Support Vector Machine (SVM) classifier to classify the tumors in the brain. Ghotekar and Mahajan [18] used a Gray-level covariance matrix to extract the features and SVM to classify the brain tumor MRI images. Tamijeselvy et al. [19] used a probabilistic neural network in combination with various pre-processing steps such as RGB (Red, Green and Blue) gray conversion, gaussian noise removal, and Blob detection and erosion-dilation for noise removal. The model achieved an accuracy of 90.0%. Sajjad et al. [20] presented a Deep Neural Network (DNN) for the detection of multigrade brain tumors. The model involved data augmentation and a pretrained Convolutional Neural Network (CNN) for training. Sajid et al. [21] described a deep learning model for dissimilar brain tumor segmentation. A hybrid CNN model was trained for the extraction of contextual information of tumors and passed to feed-forward NN for training. Saba et al. [22] used a GrabCut strategy to extract the textural features from images for brain tumor classification. It intricated VGG-19 as the classifier and gained 99.1% accuracy for 2-class prediction.

Ejaz et al. [23] proposed hybrid pixel labelling with deterministic feature clustering for brain tumor identification using BraTS dataset. Kumar et al. [24] presented an automated segmentation and classification based on ANN using 200 MRI scans. Hao et al. [25] proposed a method that includes spectral phasor analysis, 1D-CNN, 2D-CNN, edge-preserving filtering and a fully convolutional network for background segmentation. This model gained a four-class classification accuracy of 96.34%. Chanu and Thongum [26] presented a computer-aided solution using a 2D convolutional neural network for classifying the brain MRI images into normal and tumor classes with 97% accuracy. These research studies focus on spatial features, gaussian noise removal and traditional machine learning algorithms such as KMeans, SVM and Random Forest. The major limitations of these existing models include complexity and lower performance with a larger brain MRI database. Also, all these methods are based on two-class predictions—the presence and absence of tumors. The accuracy and sensitivity reduce drastically while classifying multiple classes or subtypes of brain tumors. Therefore, it is vital to implement a consistent and precise method for tumor detection with more than two classes using deep learning features.

Materials and methods

This section presents details about the dataset collection, pre-processing, feature extraction and classification of brain tumors into four classes. The performance of the proposed method has been measured using evaluation metrics: accuracy, precision, recall, F1-score and AUROC.

Dataset collection and description

The work uses Kaggle Brain Tumor database for training and testing the hybrid model. This dataset of contrast-enhanced T1-weighted MRI images of Brain consists of 3458 slices, including 394 normal, 708 meningiomas, 1426 gliomas, and 930 pituitary tumors, which are publicly available at (https://www.kaggle.com/denizkavi1/brain-tumor?select=3) [27]. Dataset has an image size of 224 × 224. All the images are further rescaled, resized and pre-processed using gaussian filter and adaptive histograms, to make them compatible to the model architecture. Figure 2 shows the sample images.

Fig. 2.

Sample MRI images

Dataset expansion using cGAN

The cGAN, is a type of GAN that involves the conditional generation of images by a generator model. Image generation is conditional on a class label, allowing the targeted generation of synthetic images of a given category [28]. The architecture represented in Fig. 3 consists two components—Generator and Discriminator. The job of the generator is to generate new images, which are as close/similar to the dataset that is provided as input, and the discriminator classifies between the generated and real data. Generator can only impact the distribution of fake images and thus attempts to minimize the following function:

$$\frac{minmax}{\mathit{GD}}V\left(D,G\right)=E_{x\sim Pdat{a}^{\left(x\right)}}\left[log\left(D\left(x\right)\right)\right]+E_{z\sim P_z\left(Z\right)}\left[log\left(1-D\left(G\left(z\right)\right)\right)\right]$$ 1

In this function: D(x)—Discriminator’s probability estimation that real data (x) is real. $E_x$—Expected value over real data instances. G(x)—Output of Generator when noise is provided. D(G(z))—Discriminator’s probability estimation that fake data (z) is real. $E_z$—Expected value over fake data instances.

Fig. 3.

Architecture of cGAN for dataset augmentation

As given in Eq. (1), Generator cannot affect the term log (D(x)) directly, so it tries to minimize the loss (log(1 − D(G(z))), whereas the discriminator tries to maximize it. In 2014, conditional GAN has been introduced by adding label y as a parameter to the input and corresponding data points are generated [29]. It also helps the discriminator by adding labels to the input. In the architecture of cGAN, the random input noise z is combined with the label y in the joint hidden representation, and the GAN training framework allows flexibility in how it receives the input. The data points X and Y are passed into the input of the discriminator along with generative output G(z), similar to vanilla GAN architecture. The loss function of cGAN is shown below:

$$\frac{minmax}{\mathit{GD}}V\left(D,G\right)=E_{x\sim Pdat{a}^{\left(x\right)}}\left[log\left(D\left(x|y\right)\right)\right]+E_{z\sim P_z\left(Z\right)}\left[log\left(1-D\left(G\left(z|y\right)\right)\right)\right]$$ 2

The implementation of cGAN on Brain Tumor dataset includes a total of 3458 images of size 224*224 and pixel values between 0 to 255. The dataset is divided into four labels: 0 for Normal, 1 for meningiomas, 2 for gliomas and 4 for pituitary tumors. The generator takes up the brain tumor images with corresponding labels along with the latent noise vector as the input for the augmentation process. The discriminator takes an image and label as the input and provide the probabilities of it being real or fake. As the training starts, the generator gets updated through discriminator. Thus, it is stacked above the discriminator model. At the initial stage, the training of discriminator is set to False. cGAN uses all normal, meningiomas, gliomas and pituitary tumor data to produce the synthetic images. Adam optimizer is used and hyperparameters are set to: batch size = 16, learning rate = 0.0001, beta = 0.9, number of epochs = 100. It takes approximately 4 h to train 20 million parameters of cGAN architecture. The architecture generated 1000 synthetic images of each class creating 1394 Normal, 1708 meningiomas, 2426 gliomas and 1930 pituitary tumors.

Feature extraction

Deep features

In recent literature of automated diagnosis, CNNs are considered to be highly effective and comprehensible [30]. As an end-to-end model, it has shown high performance in extracting features from the available data using deep convolution layers. Various hybrid approaches that employ deep features with traditional machine learning classifiers provided magnificent results in edge detection, image segmentation and image classification. The three primary layers of CNN networks are convolution layer, pooling layer, and fully linked layer. The convolution layer collects patterns, which are then passed through filters to map features. With a higher number of convolution layers, deeper features could be obtained. The pooling layer determines the size of feature maps and minimises the number of parameters, whilst the fully connected layer converts the maps into one-dimensional vectors [31]. In literature, many algorithms comprising a different set of convolution, pooling and fully connected layers are present to perform specific tasks.

In this proposed work, VGG16, ResNet50 and MobileNet architecture have been implemented to extract the deep features from Brain MRI images. Unlike traditional CNN architecture, VGG-16 is a deep network with 16 layers. A pretrained version of this network was found to be the best performing model when trained on ImageNet database comprising more than a million images [32]. VGG16 also provides high prediction accuracies for object localization in multi-class datasets. The limitation of this model includes exploding gradient problem due to its 138 million parameters. Due to this, the second feature extractor model, deep residual network (ResNet) has been utilized. ResNet50 is a residual block model with 50 layers and convolution stages of 1 × 1, 3 × 3, and 1 × 1. There are 25.6 million parameters in the ResNet50 architecture [33]. This deep model prevents the vanishing gradient problem to occur. Since the skip connections serve as gradient highways, a smooth flow of gradient is achieved. The network also has a capability to learn rich features for an extensive range of images. Third model, MobileNet, contains a total of 28 deep layers and only 4.2 million parameters. The idea behind implementing MobileNet is to carryout depth wise separable convolutions, which is required for obtaining channel-wise attributes of any image. MobileNet apply standard convolution in the first layer, that is followed by a combination of depth wise and point wise convolution. While depth wise convolution filters the input channels, pointwise convolution merges the output channels of the depth wise convolution. By doing so, the computational work is significantly reduced and a new set of features have been extracted efficiently.

The input image size for all three architectures is 224 × 224 pixels. The completely connected layers in each model are used to obtain the final features. A total of 3000 characteristics have been obtained [34].

2D-FrFT features

To obtain the time–frequency features, 2D-FrFT has been applied to the images. The 2D-FrFT transform is a set of complex exponentials with varying magnitudes, frequencies, and phases in a fractional domain that represents an image [35]. Enhancement, analysis, restoration, and compression are a few crucial image processing applications where this transformation becomes accessible. For one-dimensional FrFT (1D − FrFT), the mathematical representation of the FrFT of signal f(t) is given by:

$$T_F^{\phi }\left[f\left(t\right)\right]=F^{\phi }\left(u_{\phi }\right)=\underset{-\infty }{\overset{\infty }{\int }}f\left(t\right)K_{\phi }\left(t,u_{\phi }\right)dt,$$ 3

where,

$$K_{\phi }\left(t,u_{\phi }\right)=\frac{1}{2\pi }\sqrt{1-jcot\phi }exp\left[\frac{j}{2}\left(t^2+u_{\phi }^2\right)cot\phi -jt{u}_{\phi }csc\phi \right]$$ 4

$K_{\phi }$ is the kernel of FrFT. $T_F^{\phi }$ and $\phi$ represents the operator and rotational angle, respectively.

The inverse of signal f(t) can be obtained by taking backward rotation angle ‘− $\phi$’, as shown below:

$$f\left(t\right)=\underset{-\infty }{\overset{\infty }{\int }}{K}_{-\phi }\left(t,u_{\phi }\right)F^{\phi }\left(u_{\phi }\right)d{u}_{\phi }$$ 5

The FrFT expression can be easily modified to two dimensions by repeating the transform in x and y directions independently and considering a separate kernel. The separable 2D-FrFT of orders φ(x-axis) and γ(y-axis) is shown in Eq. (6).

$$T_F^{\phi ,\gamma }\left[f\left(x,y\right)\right]=F^{\phi ,\gamma }\left(u_{\phi },v_{\gamma }\right)=\int \underset{-\infty }{\overset{\infty }{\int }}f\left(x,y\right)K_{\phi ,\gamma }\left(x,y,u_{\phi },v_{\gamma }\right)dxdy$$ 6

where,

$$K_{\phi ,\gamma }\left(x,y,u_{\phi },v_{\gamma }\right)=K_{\phi }\left(x,u_{\phi })v_{\gamma }\right)K_{\phi }\left(y,v_{\gamma }\right)$$ 7

$$\begin{array}{c}=\frac{1}{2\pi }\sqrt{1-jcot\phi }\sqrt{1-jcot\gamma }exp\left[\frac{j}{2}\left(x^2+u_{\phi }^2\right)cot\phi -jx{u}_{\phi }csc\phi \right]\\ exp\left[\frac{j}{2}\left(y^2+v_{\gamma }^2\right)cot\gamma -jy{v}_{\gamma }csc\gamma \right]\\ \end{array}$$ 8

Now, for a M*N matrix, the 2D discrete FrFT can be computed through Eq. (9):

$$\begin{array}{cc}\hfill F^{\phi ,\gamma }\left(u_{\phi },v_{\gamma }\right)& =\sum _{x=0}^{M-1}\sum _{y=0}^{N-1}f\left(x,y\right)exp\left[\frac{j}{2}\left(x^2+u_{\phi }^2\right)cot\phi -jx{u}_{\phi }csc\phi \right]\hfill \\ \hfill & exp\left[\frac{j}{2}\left(y^2+v_{\gamma }^2\right)cot\gamma -jy{v}_{\gamma }csc\gamma \right]\hfill \\ \hfill \end{array}$$ 9

The generalization of discrete FrFT is considered by taking the transform of rows and columns individually. 2D-FrFT is a detailed transformation of classical fourier transform with a fractional parameters φ, γ and operator a, related as φ = γ = a$\frac{\pi }{2}.$ The FrFT operator ‘a’ takes value between 0 to 1. This value depends on the presence of noise in the signal or image to be processed through 2D-FrFT. A denoising analysis with random 200 brain tumor MRI images has been done to find the optimum value of ‘a’ in the proposed work. The results yield that at a = 0.7, minimum mean square error (MSE) and maximum peak-to-noise-ratio (PSNR) have been obtained for maximum images. Therefore, for brain tumor MRI scans, the features have been extracted at a = 0.7. In the proposed feature extraction method, images are initially transformed by encompassing convolution function to obtain the magnitude spectrum. The generated magnitudes—mean (m) and standard deviation (sd) are then calculated and stored in a matrix format with the same number of rows and columns as the input image of size 224*224. The magnitude of the 2D-FrFT is now divided into a vast number of 16 × 16 non-overlapping blocks. Each block’s mean and standard deviation are computed and saved as a feature vector, yielding a total of 224 features.

Geometric features

Two geometric features—eccentricity and curvature descriptor [36] have been extracted from the images. Eccentricity is defined as the shortest path length in a connected network from one vertex x to any other vertex y. When computed for each vertex x, it translates the graph's connection structure into a set of values. A linked region of a digital image is defined using the neighbourhood graph and the provided metric. Eccentricity's mathematical formula is given below:

$$ECC=\sqrt{1-\frac{x^2}{y^2}}$$ 10

where x and y are the lengths of the nodule region of interest's semi-major and semi-minor axes, respectively.

Curvature descriptor, on the other hand, is calculated in relation to intensity inside the nodule region of interest, which is dependent on intensity change. It can be defined as follows:

$$Cdd={tan}^{-1}\left(\frac{\sqrt{{\alpha }_1^2}+\sqrt{{\alpha }_2^2}}{1+f\left(x,y\right)}\right)$$ 11

Concatenation of both deep, frequency and geometric features, as shown in Fig. 4, has been done to obtain a total of 3226 features from 7458 images. This feature vector has been fed on to the classifier model.

Fig. 4.

Concatenation of hybrid features to obtain a feature pool

Classification of images in to four classes

In this proposed work, XGBoost has been implemented to classify the four classes of brain tumors. XGBoost is an optimized gradient boosting tool that is designed and distributed to be highly flexible, portable and efficient [37]. The salient features of XGBoost that differentiate it from other gradient boosting algorithms include automatic feature selection, Extra randomization parameter, Newton Boosting, proportional shrinking of leaf nodes and penalization of leaf nodes. The minimum information needed for classification are input features, target variables, objective function and number of iterations. The training objective function includes the model as ${\sum }_{k=1}^K{f}_k$ where each $f_k$ is the prediction from a decision tree. The predictions of all decision trees and prediction at the tth step are $\widehat{y_i}={\sum }_{k=1}^K{f}_k\left(x_i\right)$ and $\widehat{y_i^t}={\sum }_{k=1}^K{f}_k\left(x_i\right)$, where $x_i$ is the feature vector for the i-th data point. To train the model, LogLoss function is considered for binary classification. The loss function is defined as:

$$L=-\frac{1}{N}\sum _{i=1}^N\left(y_i\text{log}\left(p_i\right)+\left(1-y_i\right)\text{log}\left(1-p_i\right)\right)$$ 12

To avoid overfitting while training, XGBoost has an efficient regularization function, defined as:

$$\mathrm{\Omega }=\gamma T+\frac{1}{2}\lambda \sum _{j=1}^T{w}_j^2$$ 13

where T is the number of leaves present in the decision tree and $w_j^2$ is the score on the j-th leaf.

The objective function of the model becomes $Obj=L+\mathrm{\Omega }$, when both the loss function and the regularisation term are combined. Here, loss function controls the prediction and regularization simplifies the computation. After defining the objective function, gradient descent is computed in the case of XGBoost to improve the direction of $\widehat{y}$ to minimize the loss. As XGBoost works as Newton Raphson in function space, a second order Taylor approximation is used for the derivation [38]. The improved objective function is represented below:

$$Ob{j}^t\cong \sum _{i=1}^N\left[L\left(y_i,{\widehat{y}}^{\left(t-1\right)}\right)+g_i{f}_t\left(x_t\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)\right]+\sum _{i=1}^t\mathrm{\Omega }\left(f_i\right)$$ 14

where, $g_i={\partial }_{{\widehat{y}}^{t-1}}l\left(y_i,{\widehat{y}}^{\left(t-1\right)}\right),h_i={\partial }_{{\widehat{y}}^{t-1}}^{2}l\left(y_i,{\widehat{y}}^{\left(t-1\right)}\right)$.

After eliminating the constant terms, Eq. (15) shows the final objective function that must be optimised during training:

$$Ob{j}^t\cong \left[g_i{f}_t\left(x_t\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)\right]+\mathrm{\Omega }\left(f_i\right)$$ 15

The optimal splitting point must be found iteratively until the maximum depth is reached in order to build the tree. The pruning of nodes is then performed by providing a negative gain in bottom-up order. Many parameters are included in the XGBoost function, which are divided into three categories: general parameters, booster parameters, and task parameters. Table 2 shows a description of these parameters together with their default settings. In Jupyter Notebook, the XGBoost algorithm was implemented using the Sklearn module.

Table 2.

Optimal values for critical hyperparameters

General parameters	Booster parameters	Task parameters
booster (gbtree)	learning_rate (0.3)	objective (reg:squarederror)
verbosity (1)	gamma (0)	binary:logistic
validate_parameters (false)	max_depth (6)	multi:softmax
nthread (max)	min_child_weight (1)	rank:pairwise
disable_default_eval_metric	max_delta_step (0)	base_score (0.5)
num_feature (autoset)	lambda (1)	eval_metric
	alpha (0)	seed (0)
	scale_pos_weight (1)

Evaluation metrices

The metrices, defined in Eqs. (16)–(21), are calculated to evaluate the performance of proposed model for classification are as follows:

$$\text{Accuracy}=\left(\text{TP}+\text{TN}\right)/\left(\text{TS}\right)$$ 16

$$\text{Precision}=\text{TP}/\left(\text{TP}+\text{FP}\right)$$ 17

$$\text{Recall}/\text{sensitivity}\left(\text{R}\right)=\text{TP}/\left(\text{TP}+\text{FN}\right)$$ 18

$$\text{F1 - score}=2\ast \left(\text{R}\ast \text{P}\right)/\left(\text{R}+\text{P}\right)$$ 19

$$\text{Specificity}=\text{TN}/\left(\text{TN}+\text{FP}\right)$$ 20

$$\text{Mathews}\text{correlation}\text{coefficient}\left(\text{MCC}\right)=\frac{\text{TP*TN}-\text{FP*FN}}{\sqrt{\left(\text{TP}+\text{FP}\right)\left(\text{TP}+\text{FN}\right)\left(\text{TN}+\text{FP}\right)\left(\text{TN}+\text{FN}\right)}}$$ 21

where, TP = True Positives, TN = True Negatives, FP = False Positives, FN = False Negatives, TS = Total Samples.

Experimental results

In this study, the pre-processing, image augmentation, feature extraction and classification tasks are implemented using Python’s Jupyter Notebook on a system with configuration Intel(R) Xeon (R) CPU @ 3.30 GHz 8 GB RAM x-64 processor. The pre-processing of Kaggle dataset of four types of brain tumors includes resizing, rescaling and denoising. The original and pre-processed images are shown in Fig. 5.

Fig. 5.

Qualitative effect of denoising the dataset

After pre-processing, cGAN is applied to augment the brain tumor images to increase the size of dataset. The details of original and augmented images and their training has been shown in Table 3. Architectures of discriminator and generator models have been presented in Fig. 6. The updated dataset was then used to extract the features through 3 DNN models, 2D-FrFT, eccentricity and curvature descriptor.

Table 3.

Details of original and augmented images in both original and augmented dataset

	Normal	Pituitary tumors	Meningiomas	Gliomas	Total trainable images
Original dataset	394	930	708	1426	3458
Augmented	1394	1930	1708	2426	7458

Fig. 6.

Individual architecture of discriminator and generator models of cGAN

For each DNN model, 2D-FrFT, and geometric descriptors, Table 4 displays the total number of features obtained and the execution duration in seconds. In terms of complexity, the ResNet50 model takes the longest to extract features compared to the other models. MobileNet is the fastest model when it comes to deep feature extraction. 2D-FrFT and geometric features are extracted in relatively much reduced amount of time than the deep features.

Table 4.

Features obtained in the study and their execution times

Features	Total number of features	Execution time for feature extraction (s)
VGG16	1000	522
ResNet50	1000	2563
MobileNet	1000	257
2D-FrFT	224	85
Eccentricity	1	60
Curvature descriptor	1	75
Combined	3326	3562

Feature extraction is followed by the classification of brain tumors into four classes—normal, gliomas, meningiomas, and pituitary tumors. XGBoost model is trained with 7458 images of size 224*224. The dataset is split into 80:20 ratio for training and testing. Figure 7 illustrates the confusion matrix with true and predicted labels. A total of 1492 testing images, features, tuned hyperparameters and model specifications are fed into the model to get the desired predicted output of brain tumor class.

Fig. 7.

Confusion matrix for predicted and true class labels of augmented dataset at 80:20 split

As reflected in the Table 5, the fusion of deep features (3000 × 1) with 2D-FrFT coefficients (224 × 1) and geometric features (2 × 1) gives the best efficiency. Thus, the final experimentation for the detection of brain tumor type is done by using all the concatenated features of vector size of 3226 × 1. Lower accuracy in the case of individual features such as, only deep features or only 2D-FrFT coefficients, is evident due to the lack of features and wrongly predicted class of tumor due to its size and shape.

Table 5.

Performance results of XGBoost classifier using different combinations of extracted features

Features	Accuracy	Precision	Sensitivity	Specificity	F1-Score	MCC	AUROC
Only deep features	91.67	91.64	91.62	92.70	91.63	0.9201	0.90
Only 2D-FrFT features	83.60	82.79	82.79	83.21	82.68	0.8256	0.82
Deep Features + Geometric Features	92.08	92.5	92.73	92.70	92.57	0.9305	0.90
Deep Features + 2D FrFT	94.97	94.95	94.29	95.01	94.61	0.9561	0.95
Deep features + 2D-FrFT + Geometric features	98.79	98.77	98.77	99.85	98.72	0.9846	0.99

The spatial information of images is captured by the convolutional layers in deep learning models, whereas the discriminative fractional time–frequency features are captured by the 2D-FrFT. Geometric features deliver primitive features associated with the edge, shape, curve, area or blob of the region of interest in any image. For tumor detection, eccentricity and curve descriptors provide essential information about the shape and size of a specific class, thus, beneficial in categorizing the dataset into four tumor classes with high accuracy. When the information from all domains is combined, it creates a more thorough representation of the input data, which improves the research outcomes. It also demonstrates the utility of fractional frequency domain characteristics in deep structures. The proposed model incorporates global characteristics that explain the overall structure of all tumors and spots. As XGBoost provides efficient results, a comparison with other machine learning algorithms such as Support Vector Machine, Logistic Regression, Random Forest, Naïve Bayes and Decision Tree was done to project the increased performance. Table 6 also provides a comparison of the proposed hybrid model with state-of-the-art methodologies found in the literature. All these models were implemented in a similar simulation environment with same dataset.

Table 6.

Comparison of the proposed hybrid model with traditional machine learning algorithms and state-of-the-art methodologies found in the literature

Model	Accuracy (%)	Precision (%)	Sensitivity (%)	Specificity (%)	F1-Score (%)	MCC	AUROC
Support Vector Machine [11]	65.04	65.01	65.01	66.15	65.01	0.69	0.634
Logistic Regression [12]	69.99	68.97	68.84	70.02	72.04	0.70	0.7
Random Forest [13]	71.26	72.50	71.60	73.58	72.04	0.73	0.7
Naïve Bayes [14]	76.29	74.09	74.09	75.28	74.09	0.756	0.752
Decision Tree [15]	81.39	81.35	81.30	82	81.32	0.8	0.85
Neural Network [16]	85.09	85.15	85.15	87.42	85.15	0.85	0.85
Hao et al. [25]	86.34	86.09	87.02	87.78	87.30	0.878	0.9
Chanu and Thongum [26]	87.00	86.05	86.51	87.20	86.27	0.88	0.928
XGBoost (without feature extraction) [38, 39]	90.49	92.95	92.67	93.68	92.80	0.92	0.95
XGBoost + Hybrid features (original dataset)	95.07	97.09	97.07	96.56	97.07	0.967	0.95
XGBoost + Hybrid features (augmented dataset) (Proposed)	98.79	98.77	98.77	99.85	98.72	0.98	0.99

Discussion

The performance of machine learning algorithms is dependent on the availability of larger datasets for training. To overcome this issue, data augmentation has become a common approach for expanding the size of a training dataset, particularly in areas where huge datasets are often not available, such as medical imaging. Data augmentation aims to generate more data that is used to train the model, and it has been shown to enhance performance when tested on a separate dataset. Conditional GAN has been used to create synthetic brain tumor images for all four classes in this study. A framework for brain tumor detection using feature fusion is then proposed in the study. All the experiments were carried out by using the Kaggle brain tumor dataset, comprising four classes—gliomas, meningiomas, pituitary tumors and normal brain MRI scans. The study presented an efficient utilization of deep learning potential with time–frequency domain fractional fourier transform. The hybrid features are extracted from these images to check the efficacy of the model in handling the global as well as local structure. In the first part of the experimentation, the individual classification performance of different feature sets is obtained by implementing XGBoost classifier. Results indicate that the fusion of all features gave the best results compared to individual feature sets. In the next phase of the experimentation, XGBoost classifier is used and its performance is checked with other models suggested in literature. The model outperforms the traditional machine learning algorithms and suggested models in literature. A separate analysis with only two classes—Glioma and Normal, has been conducted to establish a comprehensive validation of results. With two classes, the proposed model achieved an accuracy of 99.60% and thus superior to the available state-of-the-art methods related to binary classification.

The findings reveal that by utilising suggested XGBoost classification system, clinicians can accurately distinguish between brain tumor patients and healthy patients. The XGBoost framework achieves a maximum classification accuracy of 98.79%, highlighting the potential clinical application of MRI data to classify brain tumor patients. The comparison between machine learning models shows a considerable improvement that XGBoost has produced over the other methods analysed. When compared to the other systems, SVM and LR approaches performed the worst. The system that most closely approximates the recall accuracy values of the proposed method is XGBoost + Hybrid features on original dataset. The model developed by Hao et al. achieved 86.34% accuracy, 86.09% precision and 87.02% sensitivity through a novel transfer learning–based active learning framework. Chanu and Thongum describe a computer-assisted method for identifying brain MRI images using a 2D convolutional neural network. This model has an accuracy of 86.34%, precision of 86.05%, and sensitivity of 86.61%. The proposed XGBoost system is capable of handling large data dimensions and avoids overtraining. As a result, the proposed method provides a credible tool for automatic analysis to aid in the diagnosis of brain tumors. Future work will be focused on testing real-time datasets of X-ray, PET scans and MRIs in order to ensure that the model is robust across all image modalities.

Conclusion

The study presented an efficient utilization of deep learning potential with traditional machine learning methods by combining deep features with 2D-FrFT and geometric descriptors. A framework for automatic brain tumor detection using feature fusion is proposed in the study. All the experiments were carried out by using the Kaggle Brain tumor dataset, comprising four classes of brain tumors—normal, gliomas, meningiomas, and pituitary tumors. At first, the dataset is pre-processed and augmented through cGAN in order to increase the training efficiency. In the next phase, deep, frequency and geometric features have been extracted and concatenated to feed them into the training model. XGBoost model is designed with tuned parameters to classify the tumors into four classes. The current results out-performed the available state-of-the-art models and overcomes the limitation of using spatial or frequency features individually. In future, it is intended to use feature selection methods such as Principle Component Analysis (PCA) and Recursive Feature Elimination (RFE) to reduce the redundancy of few features.

Funding

No funds, grants, or other support was received.

Data availability

The datasets generated during and/or analysed during the current study are available in the Kaggle Brain Tumor repository at https://www.kaggle.com/denizkavi1/brain-tumor?select=3

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.