Enhancing early detection of Alzheimer’s disease through hybrid models based on feature fusion of multi-CNN and handcrafted features

Abstract

Alzheimer’s disease (AD) is a brain disorder that causes memory loss and behavioral and thinking problems. The symptoms of Alzheimer’s are similar throughout its development stages, which makes it difficult to diagnose manually. Therefore, artificial intelligence (AI) techniques address the limitations of manual diagnosis. In this study, the images were enhanced and the active contour algorithm (ACA) was used to extract regions of interest (ROI) such as soft tissue and white matter. Strategies have been developed to diagnose AD and differentiate its stages. The first strategy is using XGBoost and ANN networks with the features of MobileNet, DenseNet, and GoogLeNet models. The second strategy is by XGBoost and ANN networks with combined features of MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet. The third strategy combines XGBoost and ANN networks with combined features of MobileNet-DenseNet121-Handcrafted, DenseNet121-GoogLeNet-Handcrafted, and MobileNet-GoogLeNet-Handcrafted leading to improved accuracy of the strategies and improved efficiency. XGBoost with hybrid features of DenseNet-GoogLeNet-Handcrafted achieved an AUC of 98.82%, accuracy of 98.8%, sensitivity of 98.9%, accuracy of 97.08%, and specificity of 99.5%.

Introduction

Alzheimer’s disease is a neurodegenerative disorder with severe behavioral and cognitive consequences¹. AD presents a series of complex clinical signs, extending from forgetfulness to advanced stages characterized by memory loss, cognitive decline, behavioral changes and eventually dementia². Some factors cause AD, such as genetics, lifestyle variables, and age-related cognitive changes³. AD begins with vague clinical symptoms and progresses slowly, making diagnosis in the moderate to severe stages⁴. AD represents 60–80% of dementia cases. Dementia cases increase annually by 10 million new cases⁵. The economic repercussions of AD are staggering, with dementia-related costs reaching 1.6 trillion US dollars in 2019, underscoring the imperative need for early detection to mitigate healthcare expenses⁶. Nevertheless, the heterogeneous nature of AD, manifesting in varied etiologies, clinical presentations, and progression rates, poses a formidable barrier to effective therapeutic intervention, with no definitive cure currently available⁷. The heterogeneous nature causes an obstacle to the development of Alzheimer’s treatments. There is no effective treatment for Alzheimer’s. Still, some medications slow down the symptoms of the advanced stages of Alzheimer’s and slow down the progression to dementia⁸. It is indicated that there are genetic and environmental factors for neurodegenerative conditions. According to the most recent diagnoses, Alzheimer’s appears in some people with pathological features that include vascular disease, which is beta-amyloid(Aβ) buildup in brain cells, plaques, which are Aβ deposition in the brain parenchyma, and neurofibrillary tangles, which are deposits inside neurons from the phosphorylation tau protein⁹. Approximately 20% of elderly people with Alzheimer’s show symptoms of Aβ and tau disease because of the biomarkers characteristics of incurable and irreversible AD. Therefore, it is essential to identify early AD and take the necessary measures to delay the development of the disease and reach the stage of dementia. Early Alzheimer’s, or Mild Cognitive Impairment (MCI), is characterized by small changes in cognitive abilities¹⁰ and falls between health and injury¹¹. About 80% of people with MCI progress to AD within six years¹². This research aligns with the overarching goal of timely AD identification and staging, as early intervention can potentially mitigate disease progression. Traditional techniques for diagnosing AD include neuropsychological assessments, genetic testing, and biomarker analysis from cerebrospinal fluid or blood samples. Structural imaging methods, such as CT and MRI, have also been extensively used to observe brain atrophy linked to AD. Although these approaches provide insights into disease progression, they are often limited by the need for advanced expertise, time constraints, and the risk of subjective variability. Functional magnetic resonance imaging (fMRI) has emerged as a valuable diagnostic method, since it provides an accurate description of the functional and structural brain changes of AD¹³. AD involves disturbances in brain networks. So, fMRI is well suited for network analysis, allowing identification of connectivity patterns and abnormalities in functional brain networks associated with AD. The use of artificial intelligence techniques for analyzing fMRI images holds significant promise in advancing the early diagnosis of AD development stages¹⁴. Early detection of AD enables healthcare providers to begin intervention and treatment strategies at the earliest stages, potentially slowing down the progression of the disease¹⁵. While there is no cure for Alzheimer’s, some treatments help manage symptoms and improve the patient’s quality of life. AD is emotionally challenging for both patients and their families. An early diagnosis allows individuals and families to seek support, understand what to expect, and plan for the future, including making legal and financial arrangements. Some of the symptoms of AD overlap with other conditions, such as depression or certain vitamin deficiencies¹⁶. Artificial intelligence helps in more accurate differentiation and reduces the risk of misdiagnoses. Here, artificial intelligence techniques play a vital role in achieving early diagnosis: fMRI images are complex, and detecting subtle changes is challenging for human radiologists. AI models automatically process these images, detecting patterns and anomalies that might indicate AD. AI techniques process large datasets of fMRI images, making it possible to identify patterns and trends across a broad spectrum of patients¹⁷. This reveals early markers of AD that might not be apparent in individual cases¹⁸. The main objective of this study is to extract fMRI biomarkers from the ADNI dataset. The study develops hybrid techniques that combine features from several CNNs to enhance the accuracy of early detection of AD and distinguish its stages. There is a challenge due to the similarity of age-related degenerative changes and changes observed on brain imaging. This study addressed this challenge by extracting hand-crafted features and combining them with CNN features. The high-level features are passed to the RFE algorithm to select the features according to their correlation with the target features.

The primary contributions of this research encompass several critical facets:

• Enhancement of fMRI images of ADNI dataset through average filters and the contrast-limited adaptive histogram equalization (CLAHE) method.

• Segmentation and isolation of ROI, a pivotal step accomplished through the ACA algorithm.

• Dimensionality reduction within Convolutional Neural Network (CNN) models, selecting the most pertinent features in order of priority through the RFE technique.

• AD was diagnosed using fMRI images using XGBoost and ANN based on combining the features extracted from CNN.

• Diagnosis of fMRI images of AD by XGBoost and ANN based on CNN models’ fused features and handcrafted features.

The paper is structured in the following manner:

The second section discusses a collection of prior studies and presents their techniques and results. The third section elucidates the tools and techniques utilized for analyzing fMRI images in the diagnosis of AD. The fourth section provides a summary of the results obtained from the diagnostic techniques and distinguishes between the different stages of AD. The fifth section compares the performance of the proposed systems with that of previous studies. Finally, the sixth section concludes the study.

Related work

This section discusses the techniques and literature for analyzing brain tissue patterns for early detection of AD stages to receive appropriate treatment and health care before it develops into dementia.

Tao-Ran et al.¹⁹ used Voxel-based Computed Tomography Analysis to detect AD based on PCA/ subprofile model. Receptivity curves and pattern scores were plotted, and cross-checked competency and correlation analyses were performed to find the relationship between traits associated with Alzheimer’s developmental stages. The method reached an AUC of 81.5%. Janani et al.²⁰ a CNN model for analyzing MRI images and clinical data for ADNI dataset classification. Automatic encoders and feature extraction of clinical data achieved noise reduction. The extracted high-performance features were determined by clustering analysis and CNN models. The CNN model achieved an accuracy of 88%, recall of 89%, and precision of 92%. Ahila et al.²¹ used an improved system based on the CNN model to distinguish between Alzheimer’s and normal control ADNI dataset. Improved the quality of the images through enhancement techniques and subsequently extracted pertinent features from the ADNI dataset. The model achieved an accuracy of 96%, a sensitivity of 96%, and a specificity of 94%. Louise et al.²² used three machine learning algorithms to train an ADNI dataset. Features from 34 cortical regions and 34 subcortical regions were extracted by pipeline. The selected feature set was utilized as input for training and evaluating RF and XGBoost classifiers. RF and XGBoost achieved an accuracy of 62.5% and 68.06%. Bin et al.²³ a pre-trained deep learning model for classifying MRI images of an ADNI dataset and then building a gender classifier based on Inception-ResNet-V2, a basic learning transfer model, reached an accuracy 94.9%. After transfer learning, the model achieved an accuracy of 90.9%. Minseok et al.²⁴ used CNN and machine learning algorithms to classify the ADNI dataset. FreeSurfer extracted 63 and 22 features and fed them into RF, MLP, SVM, and CNN classifiers. When feeding the networks with 63 features, RF, CNN, and MLP achieved an accuracy of 90.2%, 90.5%, and 89.6%. Eric et al.²⁵ reduced the features and biomarkers from MRI images to analyze the effect of redundant features on classification systems. By α-corrected Bonferroni identified biomarkers 93.45%, and genetic 92.54% closely correlated. Thus, the features, genetics and biomarkers were filtered, and only one of the highly correlated features was selected. Run-Hsin et al.²⁶ developed a method for selecting features and identifying biomarkers from images of the ADNI data set to predict MCI disease by machine learning methods. With 29 gene biomarkers, the RF classifier achieved an AUC of 84.1%. Janghel et al.²⁷ used an approach based on VGG-16 for feature extraction and classification by SVM. The features of the fMRI images of the ADNI dataset were extracted, and the proposed method achieved an accuracy of 73.46%. Afnan et al.²⁸ a method for OTSU segmentation with fuzzy elephant herding optimization (FEHO). FEHO identifies the edges of affected areas and feeds them into a dual attention multi-instance CNN network for Alzheimer’s diagnosis. The method reached a sensitivity of 92.3%. Saeda et al.²⁹ transformational networks based on freezing features for the classification of AD for the ADNI data set. VGG showed better results than the other models in classifying each class against the other. Where VGG reached an accuracy of 97.06% for classifying MCI vs. CN class and an accuracy of 98.89% for classifying CN vs. AD class. Nitsa et al.³⁰ a pipeline for data processing, then analyze changes in brain structures to extract asymmetry features from the MRI of the ADNI data set. CNN and the machine learning algorithm reached an accuracy of 75% and 92.5% for EMCI classification versus CN and 90.5% and 93% for AD class classification versus NC, respectively. Haijing et al.³¹ extraction of correlation in feature space by ResNet50 based on Attention Mechanism and Spatial Switched Networks (STN). Relu function is replaced by the ResNet50 model by STN. An attention mechanism is built into the backbone of ResNet50. ResNet50 extracts more spatial information from the fMRI images of the ADNI dataset. STN converts spatial information into another space and finds the relationship between infected and normal areas. The methodology achieved an accuracy of 95.3%. Badiea et al.³² employed a pair of CNN models to extract discriminative features from MRI scans related to AD. These extracted features were subjected to classification using SVM. The images have been optimized, and data augmentation has been applied to balance the data set. AlexNet with SVM achieved an accuracy of 94.8%, a sensitivity of 93%, and a specificity of 97.75%. Saman et al.³³ introduced two DL architectures, modified CNN and Conv-AE, for classifying subjects into AD based on scalp EEG recordings. To extract relevant features from EEG signals, time-frequency representation (TFR) is employed. The average accuracy of 92% for the modified CNN and 89% for the convolutional autoencoder network. Lisa et al.³⁴ a 3D-CNN was developed for the multiclass classification of 18 F-FDG PET images in AD diagnosis. Two post hoc explanation techniques, Saliency Map (SM) and Layerwise Relevance Propagation (LRP), were employed to interpret CNN’s decisions. The CNN achieved competitive performance on the test set with an average AUC of 0.81 for CN, 0.63 for MCI, and 0.77 for AD prediction. Pakize et al.³⁵ a CNN for cost-effective and speedy AD detection. The model was trained and tested on the DARWIN dataset, which originally contained 1D features extracted from handwriting data. These 1D features were transformed into 2D features to be compatible with the proposed model. The experimental results showed that the novel model achieved an impressive accuracy of 90.4%.

Illakiya et al.³⁶ presented an Adaptive Hybrid Attention Network (AHANet) integrates Enhanced Non-Local Attention (ENLA) and Coordinate Attention modules to extract global and local features separately from brain MRI, enhancing feature extraction. Illakiya et al.³⁷ presented CNNs, RNNs, and TL for accurate AD detection, segmentation, and severity grading through an in-depth examination of radiological features. The analysis focuses on diverse deep learning methods applied to AD detection using neuroimaging modalities like PET, MRI, etc., specifically in the context of radiological imaging data. Amir et al.³⁸ presented deep sequence-based networks to model the sequence of MRI features generated by a CNN for AD detection. Prasath et al.³⁹ presented various MRI imaging techniques are employed to detect protein accumulation in the nervous system, indicative of AD. Utilizing imaging and data processing, system aim to identify biomarkers and gain insights into the disease’s etiology. Senthilkumar et al.⁴⁰ presented approach to detect AD using Neuroimaging techniques. Various preprocessing methods ensure dataset readiness for subsequent feature extraction and categorization. Noise reduction involves converting and averaging real scan frames to DCT space. InceptionV3 and DenseNet201 pre-trained models are employed. Lanjewaret al.⁴¹presented an efficient framework combining CNN and KNN to detect AD from MRI images. The CNN extracts feature, then trains the KNN model. Evaluated with ROC, stratified K-fold, MCC, and CKC, the CNN-KNN approach achieved metrics: 99.58% accuracy, 99.63% precision, and a CKC of 0.9931.

The recent literature reflects several advanced approaches to AD diagnosis and highlights critical gaps that this study aims to address. For example, Tao Ran et al. used voxel-based CT analysis combined with principal component analysis models, achieving an AUC of 81.5%, emphasizing the importance of feature patterns but lacking integration with high-dimensional neural network models to improve spatial feature representation. Other approaches, such as those by Janani et al. and Ahila et al., used CNN models and clinical data on the ADNI dataset, achieving 88–96% accuracies by applying feature pooling and noise reduction. Despite their success, they were limited in feature diversity, which may limit generalizability across data sources. Additionally, studies such as those by Louis et al. and Ben et al. used machine learning algorithms such as RF and XGBoost, achieving 62.5–94.9% accuracies, and demonstrated that additional data augmentation and fine-tuning of the model can improve performance. This supports the potential of our technique, which integrates multiple CNN models, combines their feature extraction processes with handcrafted features, and applies the RFE algorithm to prioritize relevant features, thus aiming to mitigate redundancy and improve diagnostic accuracy.

The proposed method addresses these gaps by using advanced CNN architectures to extract features from fMRI images, integrating features from diverse models to enhance accuracy and robustness, and applying redundant feature removal to select features to improve the efficiency of the classification process. This integrated approach enhances the interpretation of spatial features and is in line with recent advances while introducing a new set of methodologies to push performance metrics even further.

Materials and methods

This flow diagram is Fig. 1. Hybrid Approach Flow Diagram for AD Detection Using XGBoost and ANN with Fusion Features. The fMRI Images are from the ADNI Dataset. Average Filters and CLAHE were used for better visualization. ACA was applied to isolate ROIs (critical regions in fMRI images). Applied CNN Models (MobileNet, DenseNet121, GoogLeNet) on ROI images to extract deep feature maps: MobileNet, DenseNet121and GoogLeNet. Fused Features from each model pair in sequence for comprehensive representation: MobileNet-DenseNet121, DenseNet121-GoogLeNet, and MobileNet-GoogLeNet. Applied RFE to select the most essential features. Extract features using traditional techniques like FCH, DWT, GLCM, and LBP. Combined CNN Features and Handcrafted Features to create a complete feature set. Applied XGBoost and ANN to classify the fused features and determine AD stages. Output: AD Diagnosis and Staging.

Fig. 1.

Flowchart of the proposed systems for classifying Alzheimer’s disease.

Description of ADNI dataset

The AD Neuroimaging Initiative (ADNI) is a large-scale, longitudinal study that has collected a wealth of data on the brain changes associated with AD. This data includes fMRI images. ADNI is a multi-site study funded by the National Institute on Aging (NIA) and the National Institutes of Health (NIH). The ADNI dataset aims to design systems for early diagnosis of AD. The ADNI dataset is public and available to researchers to develop their systems⁴². The ADNI consists of 1296 fMRI images divided into five categories as follows: 171 images from the AD category, 580 images from the normal cognitive impairment (CN) category, and 233 images from the mild cognitive impairment (MCI) category, 240 images of early MCI category (EMCI category), and 72 images of late MCI category (LMCI).

Enhancement fMRI images of the ADNI dataset

The fMRI images are used to measure brain activity by detecting changes in blood flow. These changes are used to identify brain regions active during different cognitive tasks. However, fMRI images are noisy and difficult to interpret. This is because they are affected by several factors, such as head motion, scanner noise, and physiological noise. Several methods are used to enhance fMRI images. These methods reduce noise, improve image contrast, and correct artifacts. In this study, an average filter was used to remove unwanted artifacts. The CLAHE method was also used to show non-contrast edges within brain structures.

The average filter is a filter used to enhance fMRI images. The filter helps remove noise by replacing each value with the average of neighboring values.

The filter operator was set to a size of 5 × 5 and the target pixel was selected and replaced with the values ​​of 25 neighboring pixels⁴³. The filter continues until each pixel in the image is replaced by the average of its neighbors as in Eq. 1. For an input image represented as h(x, y) and a filter size of N = i × j = 5 × 5, the resultant output of the averaging filter, represented as f(x, y), is denoted as follows:

1

The variables x and y are adjusted to ensure that the filter covers every pixel in the output.

CLAHE is a technique for contrast enhancement of fMRI images for AD. It is an effective way to increase the contrast of low-contrast areas of the brain. CLAHE works by dividing an image into small areas called tiles and then performing processing on each tile. This leads to increased contrast within each tile⁴⁴. CLAHE improves the contrast of fMRI images of AD by increasing the visibility of areas of the brain affected by the disease. For example, CLAHE improves the visibility of amyloid plaques and protein deposits associated with AD. Thus, the CLAHE method helps improve the accuracy of AD diagnosis by increasing the visibility of brain areas affected by the disease.

Finally, improved fMRI images have been effectively obtained to be submitted to the next stages of medical image processing. Figure 2 illustrates random samples of images from the ADNI dataset, representing various stages of AD progression, both before and after undergoing image enhancement techniques.

Fig. 2.

presents a selection of images extracted from the ADNI dataset pertaining to AD. Subfigure (a) displays the original images (b) showcases the same images following an enhancement process.

Active contour algorithm

The ACA is a popular edge-detection method used to segment interest regions in fMRI images of AD. The algorithm works by iteratively moving a curve towards regions of high image intensity. The curve is initialized at a user-specified location and then moved in a direction that minimizes the system’s energy⁴⁵. The system’s energy is a measure of the fit of the curve to the image data. The ACA segments various interest regions in fMRI images, including amyloid plaques, tau tangles, and white matter hyperintensities. The algorithm is relatively simple and used to segment images quickly and efficiently. The energy function used in the Active Contour Model consists of two main terms: internal and external energy. The internal energy encourages smoothness and regularity in the contour, while the external energy attracts the contour towards image features that correspond to the ROI. Let’s denote the contour as C(s) = (x(s), y(s)), where s represents the curve parameterization⁴⁶. The contour evolves, and the minimization of the energy function governs its evolution. The energy functional is defined as Eq. 2:

2

Internal Energy:

Internal Energy: The internal energy component , serves to maintain the smoothness of the contour by penalizing excessive curvature, thereby ensuring regularity. The fitness function as Eq. 3:

3

where α is a parameter that controls the influence of the internal energy, κ(s) is the curvature of the contour at point s, and is the desired curvature.

External Energy:

The external energy term attracts the contour towards image features corresponding to the interest regions. This term is usually derived from the image gradient or other image features. This is computed based on the gradient of the image, using a fitness function defined Eq. 4:

4

where β is a parameter that controls the influence of the external energy, ∇I (x, y) represents the image gradient at point (x, y), and G (|∇I (x, y)|) is a function that assigns higher values to regions with stronger gradients, indicating ROI.

Value update mechanism: ACA updates the contour iteratively through stepwise descent, allowing the contour to evolve in a way that minimizes the total energy. The evolution is controlled by Eq. 5:

5

where are the external and internal forces, respectively. These forces iscomputed from the energy functional derivatives with respect to the contour coordinates as Eqs. 6 and 7:

6

7

where and are the gradients of the external and internal energy functionals, respectively.

ACA combines ROI contours in fMRI by iteratively updating the contour. Figure 3 shows sample images of the ADNI dataset after applying the ACA method. The ROI is then fed to CNN models to extract features.

Fig. 3.

Samples from all classes of the ADNI dataset for AD after segmentation by the ACA method (a) Original images (b) Segmentation (c, d) Region of interest.

Training of hybrid systems with fused CNN features

Deep feature extraction

CNNs learn features from images without human intervention which makes them powerful for extracting fMRI features of the ADNI dataset⁴⁷. CNN models work by convolving a series of filters on an image. Convolution is the process of convolving an image with a filter⁴⁸.

Each layer has several filters of different sizes. The filter f(t) slides over a small area of ​​the image x(t) and the filter moves to another area as in Eq. 8⁴⁹.

8

Where f (t), x(t) and y(t) refer to the filter, input and output image.

Pooling layers are CNN layers used to reduce the feature extracted by convolutional layers⁵⁰. There are two main types of pooling layers⁵¹namely max pooling and mean pooling. Maximum pooling takes the maximum value from each patch in a feature map, as in Eq. 9. Average pooling takes the average value from each patch in the feature map, as in Eq. 10⁵². Pooling layers help improve the efficiency of CNN by reducing feature maps while preserving important features⁵³.

9

10

where f denotes filter, m, n denotes the matrix location, p denotes Filter wrap, and k denotes the vectors.

Output Layer: The final fully connected layer is connected to an output layer, typically consisting of one or more neurons, depending on the desired task. In the case of AD, the output layer may represent the classification of the image into different classes (e.g., in this work, five neurons, each class in the dataset in one neuron⁵⁴.

Training and Optimization: The CNN model is trained using labeled fMRI images, with appropriate loss functions to measure the difference between predicted and actual labels⁵⁵. Optimization techniques, such as gradient descent, are employed to update the model’s weights and biases, minimizing the loss and improving the model’s performance⁵⁶.

CNNs are a promising tool for extracting characteristics from fMRI images of AD. They learn features from images in an automated method, and they are effective at identifying regions of the brain affected by the disease. This led to new and improved methods for diagnosing and treating AD.

By useful the hierarchical and local receptive field properties of CNN models, they effectively extract discriminative features from fMRI images, aiding in identifying and classifying AD.

This approach retained the essential layers—convolutional, pooling, and auxiliary layers—of each CNN model (MobileNet, DenseNet121, and GoogLeNet) for feature extraction, excluding only the classification layers. Below are the specific layer-wise feature sizes:

MobileNet: Convolutional layers: The initial layers begin with (224, 224, 3), down sampling through various stages. Key intermediate convolutional layers produce feature maps like (56, 56, 24), (28, 28, 32), (14, 14, 96), and finally, (7, 7, 1024) in the high-level layers.

DenseNet121: Dense blocks yield increasing feature map sizes. The first layers start at (224, 224, 64). The dense layers produce sizes such as (56, 56, 256), (28, 28, 512), (16, 16, 1024), and finally (16, 32, 512), representing complex hierarchical feature extraction.

GoogLeNet: The initial layers output feature maps of sizes like (224, 224, 64), followed by down-sampled intermediate sizes of (56, 56, 192). Higher-level feature maps include sizes like (28, 28, 480), (14, 14, 832), culminating at (7, 7, 512) in the final inception layers.

Importance of Detailed Layer Sizes: These layer-wise dimensions allow readers to understand the gradual spatial reduction and feature expansion across the models. This breakdown illustrates the level of detail each model captures, which is essential for understanding how distinct aspects of the fMRI data contribute to the final feature representation.

Number of Trained Parameters: MobileNet of Approximately 4.2 million parameters, DenseNet121of Approximately 7 million parameters and GoogLeNet of Approximately 6.8 million parameters as shown in Table 1. Also, Table shown hyperparameters for MobileNet, DenseNet121, and GoogLeNet models. These hyperparameters were selected through experiments to identify the optimal balance between model performance and training stability to the fMRI data.

Table 1.

Number of trained parameters and hyperparameters for MobileNet, DenseNet121, and GoogLeNet models.

Model	Number of trained parameters	Learning rate	Batch size	Optimizer	Dropout rate
MobileNet	~ 4.2 million	0.001	32	Adam	0.3
DenseNet121	~ 7 million	0.0001	16	Adam	0.4
GoogLeNet	~ 6.8 million	0.0001	32	SGD	0.2

Recursive feature elimination algorithm

RFE is a technique to select features that are strongly correlated with target features and delete the least correlated ones to improve systems performance. The RFE algorithm is fed with features from multiple CNNs and handcrafted features to select the most important ones. RFE helps identify and remove less relevant features, potentially improving the robustness of the diagnostic model. RFE contributes to better generalization performance by focusing on the most informative features. This is particularly important when dealing with diverse feature sets from different modalities or sources. RFE helps prevent overfitting by iteratively eliminating features, ensuring the model generalizes well to new, unseen data.

General overview of how RFE is applied to reduce the dimensions of features extracted by CNN models from fMRI images in AD: After training the CNN model, use it to extract features from the fMRI images in the dataset. The CNN model captures relevant patterns and discriminative features from the images and saves them in feature vectors. Apply the RFE algorithm to the extracted features⁵⁷. RFE iteratively removes features with the most minor importance or relevance to the target variable. Rank the features based on their importance scores obtained from the RFE algorithm. The RFE algorithm typically ranks features based on their relevance to the target variable. Select a desired number of top-ranked features based on the ranking obtained from the RFE algorithm. These top-ranked features are considered the most informative and relevant for the classification task⁵⁸. Therefore, the size of the ADNI dataset becomes 1296 × 520, 1296 × 430, and 1296 × 670 for MobileNet, DenseNet121, and GoogLeNet, respectively. Applying RFE to the features extracted by the CNN model effectively reduces the dimensionality of the feature set while retaining the most informative features for AD classification. This helps in improving the efficiency of the subsequent classification model and potentially enhances the interpretability of the extracted features.

Handcrafted features

This section discusses Fuzzy Color Histogram (FCH), discrete wavelet transforms (DWT), Gray Level Co-occurrence Matrix (GLCM) and Local Binary Pattern (LBP) algorithms for extracting color, shape, and texture features from fMRI images of AD.

FCH is mainly used for color-based image analysis. Therefore, it expands color histogram methods allowing more representation of color information. FCH extracts 16 color features from fMRI images, through the distribution of fuzzy color values.

The DWT method decomposes fMRI images using wavelets into different frequency components. DWT extracts 12 features from fMRI images, which represent the data for each image.

The GLCM method analyzes the spatial relationships of an image to analyze texture. GLCM extracts 24 features from fMRI images for the spatial distribution of pixel intensity.

The LBP method describes the texture to capture local patterns in the image by replacing the value of each pixel with its neighboring pixels. LBP extracts 203 features from fMRI images for tissue and local patterns of image regions.

Each algorithm contributes to the feature extraction process by extracting different features from fMRI images. FCH focuses on color information, DWT on frequency components, GLCM on spatial relationships, and LBP on local patterns. Combining these features can provide a comprehensive representation of the features in fMRI data, which is valuable for tasks such as AD detection and classification. After collecting the features of the four algorithms, a feature vector with 255 features is formed for each image, called hand-crafted features.

Classification

AD is a progressive neurodegenerative disorder that primarily affects memory, thinking, and behavior. There are different methods used to classify the stages of AD, and two popular algorithms for classification tasks are XGBoost and ANN. Brief profile of how these algorithms is used for AD stage classification:

XGBoost (extreme gradient boosting)

XGBoost is a powerful machine learning algorithm widely used for classification and regression tasks. It is an ensemble learning method that combines the predictions of multiple weak models (decision trees) to make more accurate predictions⁵⁹. Relevant features such as cognitive test scores, genetic markers, brain imaging data (e.g., MRI or PET scans), and clinical assessments are extracted from patients. These features are preprocessed and transformed appropriately for training the model. Data Splitting: The dataset has been partitioned into distinct subsets, namely the training, validation, and testing sets. The training set serves the purpose of training the XGBoost model, the validation set is employed for fine-tuning hyperparameters, and the testing set is instrumental in assessing the ultimate performance of the model⁶⁰. XGBoost is trained on the training set by iteratively building decision trees. Each subsequent tree tries to correct the mistakes made by the previous trees. The training process continues until a predefined stopping criterion is reached. XGBoost has several hyperparameters that control its behavior. Techniques like grid or random search are employed to find the optimal combination of hyperparameters that maximize the model’s performance on the validation set. The trained XGBoost model is evaluated using the testing set to assess its performance in classifying the stages of AD.

Artificial neural networks

ANNs are computational models that draw inspiration from the architecture and operation of biological neural networks. Interconnected artificial neurons are structured into input, hidden, and output layers. ANNs can learn complex patterns from data and are employed for AD stage classification as follows:⁶¹ The architecture of the ANN is designed, specifying the number of layers, the number of neurons in each layer, and the activation function to be used. The choice of architecture depends on the complexity of the problem and the available data⁶².

An ANN typically consists of three main layers: The input layer receives the features or input data. In the fMRI image of AD analysis, these features represent various extracted characteristics from the images, such as texture, intensity, or shape features. Hidden layers are the intermediary layers between the input and output layers. These layers contain neurons (nodes) that process the input data through weighted connections, applying activation functions to introduce non-linearity. The output layer produces the final results of the network. In AD diagnosis, the output layer might represent the predicted class labels or probabilities associated with different AD stages.

The ANN receives the features sent by the RFE algorithm, which contains input units the same size as the extracted features. The ANN contains 15 input layers that perform network tasks. The output layer consists of five neurons based on the number of classes in the ADNI dataset. The ANN model is trained using a labeled dataset through an iterative process called backpropagation. During training, the model adjusts the weights and biases of the neurons to minimize the difference between predicted and actual outputs. Like XGBoost, ANNs feature a range of hyperparameters that influence their performance. Approaches such as grid search or random search may be deployed to identify the most advantageous hyperparameter configuration⁶³. Next, the efficiency of the ANN is evaluated by using a test dataset to measure its effectiveness and generalization.

Strategy of implementation sequence

Hybrid approach of machine and deep learning models

The proposed approach of XGBoost networks, ANNs, and CNN models is a way to combine the strengths of classification and feature extraction networks. The hybrid approach is more efficient than the separate CNN, XGBoost, or ANN models. The CNN model extracts complex and hidden features, while XGBoost or ANN networks classify the image based on those features. Overall, the hybrid approach is powerful image classification techniques and is generalizable to a new dataset.

The procedure for analyzing fMRI images related to AD involves a series of operations, as shown in Fig. 4. Feature extraction:

Fig. 4.

Strategies of fMRI image analysis for the ADNI data set for AD by CNN-ANN and CNN-XGBoost.

MobileNet, DenseNet121, and GoogLeNet models are individually trained on the ADNI dataset to extract unique, high-dimensional features from the enhanced fMRI images.

Redundant feature removal:

The features extracted by each CNN model undergo feature selection using the RFE algorithm. This process reduces feature sets to the most important, resulting in 520 features from MobileNet, 430 features from DenseNet121, and 670 features from GoogLeNet.

Feature fusion:

The selected features from the three CNN models are sequentially combined to form a comprehensive feature set. This fusion leverages the complementary strengths of each CNN architecture, ensuring a more robust and representative feature vector for classification.

Classification using XGBoost and ANN:

The combined feature is fed into XGBoost and ANN classifiers independently. This implementation allows each classifier to independently analyze the comprehensive feature vector independently, enhancing diagnostic accuracy and overall reliability.

This sequential integration—from feature extraction, selection, and fusion to classification—ensures that our methodology effectively leverages deep learning and machine learning techniques to enhance early diagnosis and staging of Alzheimer’s disease.

Hybrid approach of XGBoost and ANN Based on fusion features of CNN

The fMRI image analysis is an essential technique used in the early diagnosis of AD. In this study, two machine learning algorithms, XGBoost and ANN, combined with fused features extracted from MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet models. A brief explanation of how these algorithms contribute to the early diagnosis of AD using fused CNN features in fMRI image analysis: Multiple CNN (MobileNet, DenseNet121 and GoogLeNet) models are trained on the ADNI dataset, resulting in each model capturing unique features. The features extracted by these MobileNet, DenseNet121 and GoogLeNet models are fused to represent the fMRI images comprehensively. This fused feature of MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet models representation combines the strengths and diverse information learned by each CNN model. After extracting the combined features of the MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet models, they are passed to the XGBoost and ANN networks. XGBoost and ANN networks accurately classify hybrid features and distinguish whether an individual has AD and determine at what stage. This methodology leverages the power of CNN and machine learning techniques to extract accurate features and make high predictions, which contributes to diagnosing the stages of AD progression.

The methodology for examining fMRI images associated with AD comprises a series of procedural steps, illustrated in Fig. 5. The proposed method uses features extracted from MobileNet, DenseNet121, and GoogLeNet models. These models were trained independently on the ADNI dataset to capture distinct feature sets for comprehensive representation. Then, the RFE algorithm prioritizes and reduces these features, resulting in 520, 430, and 670 feature subsets for each CNN model. Hybrid features are applied to XGBoost and ANN classifiers separately.

Fig. 5.

Strategies of fMRI image analysis for ADNI dataset for AD by XGBoost and ANN with integrated CNN features.

Feature Fusion and Application: Hybrid features are applied separately to XGBoost and ANN classifiers. - The features extracted from the CNN are sequentially fused and fed to XGBoost and ANN classifiers independently. This fused representation includes diverse information from each CNN, leveraging the strengths of MobileNet, DenseNet121, and GoogLeNet.

Use of XGBoost and ANN: Post-fusion, the hybrid feature sets are applied to XGBoost and ANN independently, enabling a comprehensive classification of the AD stages based on this multi-model fusion. XGBoost and ANN provide robust classification performance, with XGBoost leveraging gradient boosting to handle complex, structured data and ANN adding nonlinear classification capabilities.

Hybrid approach of XGBoost and ANN with fusion feature CNN and Handcrafted

In this study, two machine learning algorithms, XGBoost and ANN, have been utilized with fused features derived from CNN models and handcrafted features. A brief explanation of how these algorithms contribute to the early diagnosis of AD using fused features CNN and handcrafted features in fMRI image analysis: CNN models are employed to extract meaningful features from fMRI images. Multiple CNN (MobileNet, DenseNet121 and GoogLeNet) models are trained on the ADNI dataset, resulting in each model capturing unique features. The features extracted by these MobileNet, DenseNet121 and GoogLeNet models are fused to represent the fMRI images comprehensively. Handcrafted features are also extracted from the fMRI images. These features are carefully designed and engineered based on traditional feature extraction methods. They capture specific features of the fMRI data indicative of AD using FCH, DWT, GLCM and LBP methods. The fused feature representation combines the extracted fusion CNN features (MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet) with the handcrafted features.

The fusion features are fed into the XGBoost and ANN networks to classify each image and determine the stage of Alzheimer’s.

The process of analyzing fMRI images to detect AD stage using XGBoost and ANN with hybrid CNN and hand-crafted features involves a series of implementation steps, as shown in Fig. 6:

Fig. 6.

Strategies of fMRI image analysis for ADNI dataset for AD by XGBoost and ANN with integrated CNN features and handcrafted features.

Input fMRI images: Input fMRI images from the ADNI dataset, which represent different stages of Alzheimer’s disease.

Image enhancement: Improve image quality using median and CLAHE filters to improve contrast and detail.

Region of interest extraction: Apply the active contour algorithm to identify and isolate regions of interest in the images, which represent important regions for diagnosing Alzheimer’s disease.

CNN model feature extraction:

• Use three CNN models—MobileNet, DenseNet121, and GoogLeNet—to extract high-dimensional features from the identified regions of interest.

• Each CNN model captures unique features, allowing for a variety of representations from fMRI images.

Sequential feature fusion:

• Sequentially fuse the features extracted by CNN models into pairs:

• First, fuse the MobileNet-DenseNet121,

• DenseNet121-GoogLeNet,

• MobileNet-GoogLeNet features.

• This sequence-specific integration captures diverse patterns from fMRI images.

• Feature selection using RFE:

• Apply RFE to select only the most informative features, enhancing the efficiency of the classifier by focusing on those with the strongest associations with AD stages.

• Handcrafted feature extraction:

• Extract traditional (handcrafted) features from fMRI images using four methods:

• FCH to capture color-based patterns and their distribution in a fuzzy manner, DWT to analyze multiscale signals, GLCM to describe texture, and LBP (Local Binary Pattern) for texture features.

• Feature fusion:

• Combine CNN-based features and handcrafted features to form a unified and highly descriptive feature set that comprehensively represents fMRI images.

Classification:

• Input the combined feature into XGBoost and ANN classifiers independently.

• Both algorithms analyze the feature vector to determine the AD stage, improving the diagnostic accuracy by leveraging the strengths of both models.

Output: The framework generates a diagnosis for each image, indicating the stage of Alzheimer’s disease based on the hybrid analysis.

This approach combines deep learning-derived CNN features with traditional handcrafted features to effectively enhance the detection of Alzheimer’s disease stages from fMRI images, leveraging both XGBoost and ANN classifiers for robust classification.

Comparing the performance of pre-trained CNN models with the proposed systems

The performance of the pre-trained convolutional neural networks on the ADNI dataset, the results were initially rather modest. Although MobileNet had the highest AUC of 45.9%, its accuracy was low at 42.5% and its sensitivity was 35.68%. Although DenseNet121 had a slightly higher AUC of 45.92%, its accuracy was similar (42.1%) and its sensitivity was 35.98%. GoogLeNet had the highest accuracy among the previous networks (51.7%), but it had a low AUC of 45.44% and a poor sensitivity of 33.28%.

However, the situation changed when the ACA method (which identifies regions of interest related to Alzheimer’s disease) was used with these networks. Using this method, the results improved significantly. For example, MobileNet achieved an AUC of 84%, accuracy of 88.4%, and sensitivity of 81.76%, showing a significant improvement over the initial performance. DenseNet121 and GoogLeNet also showed significant performance improvements, with GoogLeNet achieving an AUC of 82.36% with accuracy of 86.9% and good sensitivity.

When we move to using classification algorithms such as XGBoost and ANN using manually extracted features such as FCH, DWT, GLCM, and LBP, we see another improvement. For example, XGBoost using these features showed an AUC of 79.42%, accuracy of 86.1%, and sensitivity of 81.1%, while ANN achieved an AUC of 77.59%, accuracy of 84.2%, and sensitivity of 79.9%. These results suggest that using manually extracted features enhances the performance of the system compared to using convolutional neural networks directly.

When using deep CNN features extracted from pre-trained neural networks such as MobileNet, DenseNet121, and GoogLeNet, the results showed significant improvement. XGBoost using MobileNet features, for example, achieved an AUC of 88.96%, accuracy of 90.7%, and sensitivity of 86.16%. While ANN using the same features achieved an AUC of 91.88%, accuracy of 91.9%, and sensitivity of 86.16%. These results confirm that combining features extracted from convolutional neural networks significantly improves performance. Finally, when combining CNN features from different networks such as MobileNet and DenseNet121, the best results were achieved. For example, XGBoost using the combined features achieved an AUC of 96.48%, accuracy of 96.1%, and sensitivity of 92.8%. While ANN using the same combined features achieved AUC of 96.18%, accuracy of 93.1% and sensitivity of 91.6%. These results show that combining features from different networks contributes significantly to improving diagnostic performance. Based on this comparison, it can be concluded that combining optimization methods such as ACA with pre-trained convolutional neural networks as well as using XGBoost and ANN with features extracted or combined from these networks leads to significant performance improvements.

Results of systems

Split of ADNI dataset

In this study, the ADNI data set for AD was used to evaluate the performance of the proposed systems. The dataset comprises 1296 images distributed across five distinct classes of AD progression stages. The ACA method was applied to extract the ROI, which was fed to the CNN models to extract the deep feature maps. The feature maps are provided as input to both the XGBoost model and the ANN. This study performed 5-fold cross-validation to ensure a comprehensive and unbiased assessment of model performance across different subsets of the dataset. As shown, the dataset was split into five folds, each serving as a validation set once, while the remaining four folds were used for training in successive iterations. To further validate the proposed models’ generalizability, we allocated 20% of the dataset exclusively for testing after completing 5-fold cross-validation, as detailed in Table 2. This separate test set was never used during the training or validation phases and served as an independent assessment of the proposed models’ applicability to a new dataset. This approach enabled us to calculate performance metrics, such as AUC, accuracy, sensitivity, precision and specificity, by averaging the results across all five iterations, reducing the potential bias from a single training-test split⁶⁴.

Table 2.

Splitting the ADNI data set for AD.

Phase	80% (80:20)		Testing 20%
Classes	Training (80%)	Validation (20%)
AD	110	27	34
CN	371	93	116
MCI	149	37	47
EMCI	154	38	48
LMCI	46	12	14

Performance metrics

The systems we developed assessed the ADNI dataset using a confusion matrix, widely regarded as the benchmark for evaluating system performance. This matrix presents a breakdown of samples classified as either true positives (TP) or true negatives (TN) for correct classifications and false negatives (FN) or false positives (FP) for incorrect classifications. We evaluated the system’s performance by employing Eq. 11 to 15, utilizing values obtained from the confusion matrix.

This study evaluated systems developed using MATLAB 2018b on CPU: Core i7 10th, GPU: GTX 8GB, and RAM:16GB.

11

12

13

14

15

Dataset balancing and overfitting processing

Artificial intelligence systems need a large data set to train them to classify a new data set. Unfortunately, many medical datasets do not have enough images to train the systems; thus, the systems face the problem of overfitting. Furthermore, an additional challenge faced by the systems is the imbalance in the number of images among the various classes within the dataset, potentially leading to accuracy biases toward the class with a larger image count. Consequently, this study simultaneously addressed both challenges by employing the data augmentation technique. First, to handle the problem of overfitting by artificially increasing each image of the data set through many operations of the data augmentation method. The processes of rotating images are applied through many angles, displacement, flipping, changing the length and height, and other operations. Secondly, when applying the operations of the data augmentation method, the increment of images in each class must be different from the other class to achieve the balance of the data set. Thus, the overfitting problem was addressed, and the data set was balanced. Table 3 displays the number of images within the ADNI dataset related to AD both before and after the implementation of data augmentation procedures. It is noted from the table that the images of each class of the ADNI data set for AD are incremented differently from the others as follows: For class AD, nine images were artificially incremented from each image; for class CN, two images were artificially incremented from each image, for classes MCI and EMCI six images were artificially incremented from each image, finally for the LMCI class twenty-one images were artificially incremented from each image.

Table 3.

Balancing the ADNI data set for AD.

Phase	Training and validation 80% dataset
Classes	AD	NC	MCI	EMCI	LMCI
Bef-augmen	137	464	186	192	58
Aft-augmen	1370	1392	1302	1344	1276

Results of pre-trained CNN models

This section provides an overview of the performance of pre-trained MobileNet, DenseNet121, and GoogLeNet models when applied to the ImageNet dataset. The knowledge acquired by these models from the ImageNet dataset was subsequently transferred to new tasks, specifically the classification of the ADNI dataset for AD. Table 4 outlines the performance metrics of these pre-trained models. MobileNet achieved an AUC of 45.9%, accuracy of 42.5%, a sensitivity of 35.68%, precision of 41.16%, and specificity of 83.36%. DenseNet121 reached an AUC of 45.92%, accuracy of 40.2%, sensitivity of 35.98%, precision of 38.58%, and specificity of 84.4%. GoogLeNet attained an AUC of 45.44%, accuracy of 51.7%, sensitivity of 33.28%, precision of 44.02%, and specificity of 84.06%.

Table 4.

Results of pre-trained MobileNet, DenseNet121 and GoogLeNet models for diagnosis ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet	AD	42.1	32.4	32.2	61.1	96.5
	CN	55.2	58.6	59.4	51.5	55.2
	EMCI	39.5	12.5	13.1	46.2	97.1
	LMCI	41.9	28.6	28.9	19	93.4
	MCI	50.8	44.7	44.8	28	74.6
	Average ratio	45.9	42.5	35.68	41.16	83.36
DenseNet121	AD	41.6	26.5	26.4	52.9	95.6
	CN	48.6	39.7	40.3	64.8	83.2
	EMCI	58.4	50	50.2	32.4	76.3
	LMCI	29.8	14.3	13.9	15.4	96.1
	MCI	51.2	48.9	49.1	27.4	70.8
	Average ratio	45.92	40.2	35.98	38.58	84.4
GoogLeNet	AD	49.1	26.5	25.8	75	98.7
	CN	60.1	84.5	83.6	51.9	36.1
	EMCI	55.2	41.7	41.7	46.5	89.4
	LMCI	10.4	0	0	0	100
	MCI	52.4	14.9	15.3	46.7	96.1
	Average ratio	45.44	51.7	33.28	44.02	84.06

Results of pre-trained CNN based on ACA method

This section summarises the performance of pre-trained MobileNet, DenseNet121, and GoogLeNet models, focusing on their utilization in conjunction with the ACA method. The images were first enhanced to increase the contrast of important areas. Secondly, the ACA method was used to extract the important areas to be analyzed later. The ACA method isolates the important regions, called ROIs, from the undesirable regions, known as healthy regions. Third, the MobileNet, DenseNet121 and GoogLeNet models receive a post-segmentation dataset called ADNI-ROI for AD and parse it through convolutional and feature reduction layers through pooling layers. The fully connected layers classify the ADNI-ROI dataset and convert the high-level features into vectors. Finally, the SoftMax function has five neurons same number of ADNI-ROI classes, which select each feature vector to a neuron. Table 5 presents the performance of the pre-trained MobileNet, DenseNet121 and GoogLeNet models. The MobileNet achieved an AUC of 84%, accuracy of 88.4%, sensitivity of 81.76%, precision of 84.22%, and specificity of 95.52%. The DenseNet121 achieved an AUC of 81.96%, accuracy of 84.6%, sensitivity of 76.82%, precision of 78.48%, and specificity of 95.86%. GoogLeNet achieved an AUC of 82.36%, accuracy of 86.9%, sensitivity of 82.06%, precision of 82.78%, and specificity of 96.74%. It is noted that the results of MobileNet, DenseNet121 and GoogLeNet models based on the ACA hash algorithm improved effectively.

Table 5.

Results of pre-trained MobileNet, DenseNet121 and GoogLeNet based on the ACA segmentation method for diagnosis ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet	AD	85.2	76.5	75.8	74.3	96.1
	CN	88.4	96.6	97.2	93.3	93.7
	EMCI	82.9	81.3	80.9	90.7	97.7
	LMCI	78.2	64.3	64.3	75	98.8
	MCI	85.3	91.5	90.6	87.8	91.3
	Average ratio	84	88.4	81.76	84.22	95.52
DenseNet121	AD	81.2	79.4	79.2	75	96.4
	CN	90.6	93.1	93.1	93.1	93.7
	EMCI	80.9	79.2	78.6	74.5	94.1
	LMCI	71.6	50	50.4	70	98.5
	MCI	85.5	83	82.8	84.8	96.6
	Average ratio	81.96	84.6	76.82	79.48	95.86
GoogLeNet	AD	76.8	73.5	74.1	67.6	95.2
	CN	86.1	84	94.3	96.5	97.1
	EMCI	80.6	75	74.9	80	96.1
	LMCI	75.4	71.4	71.2	83.3	98.6
	MCI	92.9	95.7	95.8	86.5	96.7
	Average ratio	82.36	86.9	82.06	82.78	96.74

Results of XGBoost and ANN with handcrafted feature

In this section, the performance results of XGBoost and ANN networks based on the ACA algorithm are presented. These networks were trained and their performance was evaluated using hand-crafted features from FCH, DWT, GLCM, and LBP. The XGBoost classifier reached an AUC of 79.42%, accuracy of 86.1%, sensitivity of 77.8%, precision of 80.23%, and specificity of 93.7%. In contrast, the ANN achieved an AUC of 77.59%, accuracy of 84.2%, sensitivity of 76.2%, precision of 78.36%, and specificity of 92.8%.

Results of XGBoost and ANN with CNN feature

This section discusses the performance evaluation of XGBoost and ANN models, with features of MobileNet, DenseNet121, and GoogLeNet models. The methodology goes through several steps: First, the image is enhanced to improve contrast within the ROI. Second, the ACA method is subsequently applied to identify and isolate significant (infected) regions from healthy ones for subsequent analysis. Third, the MobileNet, DenseNet121, and GoogLeNet models receive only affected regions from fMRI images of AD and analyze them through many convolutional layers to extract and reduce features through pooling layers. The RFE method then takes the features extracted from the GAP layer and identifies strongly correlated features, ultimately selecting the most informative ones. The selected features are fed into the XGBoost and ANN networks, facilitating an effective and efficient classification process.

Table 6 presents the XGBoost performance with features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method. The XGBoost with MobileNet achieved an AUC of 88.96%, accuracy of 90.7%, sensitivity of 86.16%, precision of 88.2%, and specificity of 97.34%. The XGBoost with DenseNet121 achieved an AUC of 88.2%, accuracy of 91.5%, sensitivity of 85.88%, precision of 87.2%, and specificity of 97.6%. The XGBoost with GoogLeNet achieved an AUC of 87.72%, accuracy of 90%, sensitivity of 83.18%, precision of 81.72%, and specificity of 97.48%. It should be noted that the results of the XGBoost network with the features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method improved effectively to distinguish between Alzheimer’s stages.

Table 6.

Results of the XGBoost network with CNN features for the diagnosis of the ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet - XGBoost	AD	86.2	85.3	85.2	85.3	97.7
	CN	96.7	95.7	95.4	94.9	95.6
	EMCI	88.6	87.5	88.1	84	96.2
	LMCI	80.2	71.4	71.2	83.3	98.7
	MCI	93.1	91.5	90.9	93.5	98.5
	Average ratio	88.96	90.7	86.16	88.2	97.34
DenseNet121- XGBoost	AD	82.1	79.4	78.6	87.1	98.1
	CN	97.1	95.7	96.4	94.1	95.3
	EMCI	92.4	91.7	92.5	91.7	97.8
	LMCI	70.9	64.3	64.3	69.2	98.1
	MCI	98.5	97.9	97.6	93.9	98.7
	Average ratio	88.2	91.5	85.88	87.2	97.6
GoogLeNet- XGBoost	AD	80.6	73.5	74.1	80.6	96.5
	CN	96.2	95.7	96.4	98.2	98.7
	EMCI	93.9	89.6	89.8	87.8	97.2
	LMCI	69.8	57.1	57.2	50	96.7
	MCI	98.1	97.9	98.4	92	98.3
	Average ratio	87.72	90	83.18	81.72	97.48

Table 7 presents the ANN performance with features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method. The ANN with MobileNet achieved an AUC of 91.88%, accuracy of 91.9%, sensitivity of 89.48%, precision of 89.5%, and specificity of 97.96%. The ANN with DenseNet121 achieved an AUC of 90.5%, accuracy of 93.8%, sensitivity of 87.72%, precision of 92.54%, and specificity of 98.7%. The ANN with GoogLeNet achieved an AUC of 89.86%, accuracy of 90.7%, sensitivity of 84.76%, precision of 87.38%, and specificity of 97.6%.It should be noted that the results of the ANN network with the features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method improved effectively to distinguish between Alzheimer’s stages.

Table 7.

Results of the ANN network with CNN features for the diagnosis of the ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet-ANN	AD	84.6	82.4	82.1	84.8	98.2
	CN	96.2	95.7	96.4	95.7	97.1
	EMCI	94.8	91.7	91.7	91.7	97.8
	LMCI	90.5	85.7	85.9	85.7	98.6
	MCI	93.3	91.5	91.3	89.6	98.1
	Average ratio	91.88	91.9	89.48	89.5	97.96
DenseNet121-ANN	AD	91.2	88.2	88.4	90.9	99.1
	CN	99.2	100	100	97.5	98.6
	EMCI	90.6	87.5	87.9	95.5	98.8
	LMCI	72.8	64.3	64.2	90	99.8
	MCI	98.7	97.9	98.1	88.8	97.2
	Average ratio	90.5	93.8	87.72	92.54	98.7
GoogLeNet-ANN	AD	84.1	79.4	78.8	81.8	97.2
	CN	97.7	96.6	96.5	95.7	96.8
	EMCI	92.9	89.6	90.2	89.6	98.1
	LMCI	79.8	64.3	64.4	81.8	98.6
	MCI	94.8	93.6	93.9	88	97.3
	Average ratio	89.86	90.7	84.76	87.38	97.6

When implementing systems, they produce a confusion matrix to represent their performance, as the confusion matrix is considered the essential criterion for evaluating the performance of systems. Figure 7 illustrates the confusion matrices generated by the XGBoost system when utilizing features from the MobileNet, DenseNet121, and GoogLeNet models individually. The XGBoost model, in conjunction with MobileNet features, achieved diagnostic accuracies for all classes within the ADNI dataset, with the following results: 85.3% for the AD class, 95.7% for the CN class, 87.5% for the EMCI class, 71.4% for the LMCI class, and 91.5% for the MCI class. When considering the XGBoost model with DenseNet121 features, it produced diagnostic accuracies for each ADNI dataset class as follows: 79.4% for the AD class, 95.7% for the CN class, 91.7% for the EMCI class, 64.3% for the LMCI class, and 97.9% for the MCI class. Furthermore, the XGBoost model employing GoogLeNet features resulted in diagnostic accuracies across all ADNI dataset classes: 73.5% for the AD class, 95.7% for the CN class, 89.6% for the EMCI class, 57.1% for the LMCI class, and 97.9% for the MCI class.

Fig. 7.

Production of confusion matrix from the XGBoost network with CNN features for diagnosing the ADNI data set for AD.

When implementing systems, they produce a confusion matrix to represent their performance, as the confusion matrix is considered the essential criterion for evaluating the performance of systems. Figure 8 depicts the confusion matrices generated by the ANN system when utilizing features extracted from the MobileNet, DenseNet121, and GoogLeNet models separately. The ANN model, in conjunction with MobileNet features, achieved diagnostic accuracies across all classes within the ADNI dataset, with the following results: 82.4% for the AD class, 95.7% for the CN class, 91.7% for the EMCI class, 85.7% for the LMCI class, and 91.5% for the MCI class. When considering the ANN model with DenseNet121 features, it produced diagnostic accuracies for each ADNI dataset class as follows: 88.2% for the AD class, 100% for the CN class, 87.5% for the EMCI class, 64.3% for the LMCI class, and 97.9% for the MCI class. Furthermore, the ANN model employing GoogLeNet features resulted in diagnostic accuracies across all ADNI dataset classes: 79.4% for the AD class, 96.6% for the CN class, 89.6% for the EMCI class, 64.3% for the LMCI class, and 93.6% for the MCI class.

Fig. 8.

Production of confusion matrix from the ANN network with CNN features for diagnosing the ADNI data set for AD.

Results of XGBoost and ANN with fusion CNN feature

This section summarizes the performance of XGBoost and ANNs with fused features for MobileNet, DenseNet121 and GoogLeNet models based on the ACA method. First, the images are optimized to increase the contrast of the ROI. Secondly, the ACA method was used to extract significant (infected) regions and separate them from healthy regions for analysis at later stages. Third, the MobileNet, DenseNet121, and GoogLeNet models only receive ROI from fMRI images of AD and analyze them through many convolutional layers to extract and reduce features through pooling layers. RFE receives features from the GAP layer and selects one characteristic from highly correlated and essential features. CNN features are serially integrated into vectors: MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet, and then fed to XGBoost and ANN networks for effective and efficient classification.

Table 8 presents the XGBoost performance with combined features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method. The XGBoost with features of MobileNet-DenseNet achieved an AUC of 96.48%, accuracy of 96.1%, sensitivity of 95.1%, precision of 93.18%, and specificity of 98.36%. The XGBoost with features of DenseNet-GoogLeNet achieved an AUC of 95.36%, accuracy of 93.8%, sensitivity of 89.04%, precision of 92.4%, and specificity of 98.4%. The XGBoost with features of MobileNet-GoogLeNet achieved an AUC of 95.9%, accuracy of 95%, sensitivity of 93.16%, precision of 92.18%, and specificity of 98.54%.

Table 8.

Results of the XGBoost network with combined CNN features for the diagnosis of the ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet-DenseNet121- XGBoost	AD	94.2	91.2	91.4	93.9	98.9
	CN	98.3	97.4	97.3	99.1	98.7
	EMCI	96.4	95.8	96.2	95.8	99.1
	LMCI	94.9	92.9	92.9	81.3	99.3
	MCI	98.6	97.9	97.7	95.8	95.8
	Average ratio	96.48	96.1	95.1	93.18	98.36
DenseNet-GoogLeNet- XGBoost	AD	93.3	88.2	88.2	90.9	98.6
	CN	98.9	98.3	98.1	96.6	96.6
	EMCI	95.7	93.8	93.6	90	98.2
	LMCI	92.8	71.4	70.9	90.9	99.7
	MCI	96.1	93.6	94.4	93.6	98.9
	Average ratio	95.36	93.8	89.04	92.4	98.4
MobileNet -GoogLeNet- XGBoost	AD	96.1	94.1	93.8	94.1	98.5
	CN	98.8	96.6	96.7	97.4	98.1
	EMCI	98.6	97.9	98.1	97.9	99.8
	LMCI	93.1	85.7	86.3	80	98.6
	MCI	92.9	91.5	90.9	91.5	97.7
	Average ratio	95.9	95	93.16	92.18	98.54

Table 9 presents the ANN performance with combined features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method. The ANN with features of MobileNet-DenseNet achieved an AUC of 96.18%, accuracy of 93.1%, sensitivity of 90.24%, precision of 89.58%, and specificity of 98.46%. The ANN with features of DenseNet121-GoogLeNet achieved an AUC of 96.52%, accuracy of 94.6%, sensitivity of 91.64%, precision of 94.76%, and specificity of 98.7%. The ANN with features of MobileNet-GoogLeNet achieved an AUC of 95.72%, accuracy of 94.6%, sensitivity of 91.62%, precision of 94.76%, and specificity of 98.82%. It should be noted that the XGBoost and ANN results with the combined features of the MobileNet, DenseNet121 and GoogLeNet models based on the ACA method effectively improved the previous techniques for distinguishing the stages of AD.

Table 9.

Results of the ANN network with combined CNN features for the diagnosis of the ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet-DenseNet121-ANN	AD	92.1	79.4	79.4	96.4	99.8
	CN	98.3	96.6	97.1	98.2	99.1
	EMCI	99.1	97.9	97.6	88.7	97.3
	LMCI	93.8	85.7	85.9	75	97.8
	MCI	97.6	91.8	91.2	89.6	98.3
	Average ratio	96.18	93.1	90.24	89.58	98.46
DenseNet-GoogLeNet- ANN	AD	93.2	82.4	82.3	90.3	98.5
	CN	99.3	98.3	98.1	95.8	96.8
	EMCI	98.7	95.8	95.9	93.9	98.9
	LMCI	94.6	85.7	86.2	100	100
	MCI	96.8	95.7	95.7	93.8	99.3
	Average ratio	96.52	94.6	91.64	94.76	98.7
MobileNet -GoogLeNet- ANN	AD	92.1	82.4	82.1	90.3	99.2
	CN	98.9	98.3	98.2	95.8	97.1
	EMCI	97.3	95.8	95.9	93.9	98.8
	LMCI	93.7	85.7	86.1	100	99.7
	MCI	96.6	95.7	95.8	93.8	99.3
	Average ratio	95.72	94.6	91.62	94.76	98.82

Implementing the XGBoost algorithm produces a confusion matrix to represent their performance, as the confusion matrix is considered the essential criterion for evaluating the performance of systems. Figure 9 illustrates the confusion matrix generated by the XGBoost system when employing combined features derived from the MobileNet, DenseNet121, and GoogLeNet models. The XGBoost model, useful the features from MobileNet-DenseNet, achieved diagnostic accuracies across all classes within the ADNI dataset, with the following results: 91.2% for the AD class, 97.4% for the CN class, 95.8% for the EMCI class, 92.9% for the LMCI class, and 97.9% for the MCI class. Similarly, when using features from DenseNet121-GoogLeNet, the XGBoost model yielded diagnostic accuracies for each ADNI dataset class as follows: 88.2% for the AD class, 98.3% for the CN class, 93.8% for the EMCI class, 71.4% for the LMCI class, and 93.6% for the MCI class. Moreover, when utilizing features from MobileNet-GoogLeNet, the XGBoost model achieved diagnostic accuracies across all ADNI dataset classes: 94.1% for the AD class, 96.6% for the CN class, 97.9% for the EMCI class, 85.7% for the LMCI class, and 91.5% for the MCI class.

Fig. 9.

Production of confusion matrix from the XGBoost network with combined CNN features for diagnosing the ADNI data set of AD.

Implementing the ANN algorithm produces a confusion matrix to represent their performance, as the confusion matrix is considered the essential criterion for evaluating the performance of systems.

Figure 10 presents the confusion matrix the ANN system generated when combining features derived from the MobileNet, DenseNet121, and GoogLeNet models. The ANN model, incorporating features from MobileNet-DenseNet, achieved diagnostic accuracies across all classes within the ADNI dataset, with the following results: 88.2% for the AD class, 97.4% for the CN class, 95.8% for the EMCI class, 92.9% for the LMCI class, and 95.7% for the MCI class. When using features from DenseNet121-GoogLeNet, the ANN model achieved per-class accuracy: 82.4% for AD class, 98.3% for CN class, 95.8% for EMCI class, and 85.7% for LMCI class. MCI class of 95.7%. When using features from MobileNet-GoogLeNet, the ANN model achieved accuracy: AD class 79.4%, CN class 96.6%, EMCI class 97.9%, LMCI class 85.7%, and MCI class 91.5%.

Fig. 10.

Production of confusion matrix from the ANN network with combined CNN features for diagnosing the ADNI data set of AD.

Results of XGBoost and ANN with fusion feature CNN and handcrafted

This section summarizes the performance achieved by combining XGBoost and ANNs with the fused features of the MobileNet, DenseNet121, and GoogLeNet models and hand-crafted features. To begin with, the images are optimized to enhance the contrast of the region of interest (ROI). Subsequently, the ACA method is employed to extract significant (infected) regions and separate them from healthy regions for later analysis. Next, the ROI is inputted into the MobileNet, DenseNet121, and GoogLeNet models, which employ multiple convolutional layers to extract and reduce features using pooling layers.

The CNN features are combined sequentially: MobileNet-DenseNet121-Handcrafted, DenseNet121-GoogLeNet-Handcrafted, and MobileNet-GoogLeNet-Handcrafted. The RFE algorithm is used to select the most important features according to their correlation with the objective feature. These refined features are introduced into XGBoost and ANN networks for effective classification. Table 10 presents the performance of the XGBoost algorithm with features of MobileNet, DenseNet121, and GoogLeNet models along with hand-crafted features. Performance comparison of the XGBoost algorithm with different features: XGBoost with MobileNet-DenseNet-Handcrafted features reached an AUC of 96.1%, accuracy of 96.9%, sensitivity of 94.88%, precision of 95.7%, and specificity of 99.14%. XGBoost with DenseNet-GoogLeNet-Handcrafted features achieved an AUC of 98.82%, accuracy of 98.8%, sensitivity of 98.9%, precision of 97.08%, and specificity of 99.5%. XGBoost with MobileNet-GoogLeNet-Handcrafted features reached an AUC of 97.5%, accuracy of 97.7%, sensitivity of 95.98%, precision of 97.36%, and specificity of 99.38%.

Table 10.

Results of the XGBoost network with combined CNN features and handcrafted for the diagnosis of the ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet-DenseNet-Handcrafted	AD	96.5	97.1	97.2	94.3	98.5
	CN	99.2	99.1	99.3	98.3	98.7
	EMCI	97.6	97.9	97.7	97.9	99.6
	LMCI	91.8	85.7	86.4	92.3	99.8
	MCI	95.4	93.6	93.8	95.7	99.1
	Average ratio	96.1	96.9	94.88	95.7	99.14
DenseNet-GoogLeNet-Handcrafted	AD	99.2	100	99.6	100	99.5
	CN	99.3	99.1	99.1	100	99.8
	EMCI	97.8	95.8	96.4	100	99.7
	LMCI	98.7	100	99.7	87.5	98.8
	MCI	99.1	100	99.7	97.9	99.7
	Average ratio	98.82	98.8	98.9	97.08	99.5
MobileNet-GoogLeNet-Handcrafted	AD	99.5	100	99.5	91.9	98.8
	CN	98.5	98.3	98.2	99.1	99.1
	EMCI	97.8	97.9	98.3	97.9	99.8
	LMCI	93.8	85.7	86.1	100	99.5
	MCI	97.9	97.9	97.8	97.9	99.7
	Average ratio	97.5	97.7	95.98	97.36	99.38

Table 11 provides a summary of the performance of the ANN when using combined features from the MobileNet, DenseNet121, and GoogLeNet models with hand-crafted features. MobileNet-DenseNet-Handcrafted with ANN reached an AUC of 96.8%, accuracy of 97.3%, sensitivity of 95.78%, precision of 95%, specificity of 99.3%. DenseNet-GoogLeNet-Handcrafted with ANN reached an AUC of 98.7%, accuracy of 98.5%, sensitivity of 97.2%, precision of 98.6%, specificity of 99.4%. MobileNet-GoogLeNet-Handcrafted with ANN reached an AUC of 96.54%, accuracy of 96.5%, sensitivity of 97.68%, precision of 93.46%, specificity of 99.34%.

Table 11.

Results of the ANN network with combined CNN features and handcrafted for the diagnosis of the ADNI data set for AD.

Systems	Classes of ADNI data set	AUC %	Accuracy %	Sensitivity %	Precision %	Specificity %
MobileNet-DenseNet-Handcrafted	AD	98.7	100	99.8	94.4	99.1
	CN	97.8	98.3	98.3	99.1	98.9
	EMCI	96.5	95.8	96.4	97.9	99.7
	LMCI	92.8	85.7	86.2	85.7	99.2
	MCI	98.2	97.9	98.2	97.9	99.6
	Average ratio	96.8	97.3	95.78	95	99.3
DenseNet-GoogLeNet-Handcrafted	AD	97.9	94.1	94.2	100	99.8
	CN	99.8	99.1	99.3	99.1	99.2
	EMCI	99.6	100	99.8	98	99.6
	LMCI	97.8	92.9	92.9	100	99.7
	MCI	98.4	100	99.8	95.9	98.7
	Average ratio	98.7	98.5	97.2	98.6	99.4
MobileNet-GoogLeNet-Handcrafted	AD	94.2	91.2	91.2	91.2	99.2
	CN	97.8	98.3	98.8	99.1	98.8
	EMCI	98.3	97.9	99.8	97.9	99.7
	LMCI	95.2	92.9	99.1	81.3	99.1
	MCI	97.2	95.7	99.5	97.8	99.9
	Average ratio	96.54	96.5	97.68	93.46	99.34

The XGBoost algorithm generated a confusion matrix is ​​generated by XGBoost with fused features to evaluate its performance. Figure 11 presents the confusion matrix resulting from the XGBoost algorithm with features of MobileNet, DenseNet121, and GoogLeNet models, along with handcrafted features. The XGBoost algorithm, when fusing MobileNet-DenseNet-Handcrafted features achieved accuracy: AD class of 97.1%, CN class of 99.1%, EMCI class of 97.9%, LMCI class of 85.7%, and MCI class of 93.8%. With DenseNet121-GoogLeNet-Handcrafted features, the XGBoost algorithm reached accuracies: AD class of 100%, CN class of 99.1%, EMCI class of 95.8%, LMCI class of 100%, and MCI class of 100%. With MobileNet-GoogLeNet-Handcrafted features, the XGBoost algorithm reached accuracies: AD class of 100%, CN class of 98.3%, EMCI class of 97.9%, LMCI class of 85.7%, and MCI class of 97.9%.

Fig. 11.

Production of confusion matrix from the XGBoost network with combined CNN features along with handcrafted features for diagnosing the ADNI data set of AD.

The ANN algorithm generated a confusion matrix is ​​generated by ANN with fused features to evaluate its performance. Figure 12 presents the confusion matrix resulting from the ANN algorithm with features of MobileNet, DenseNet121, and GoogLeNet models, along with handcrafted features. The ANN algorithm, when fusing MobileNet-DenseNet-Handcrafted features achieved accuracy: AD class of 100%, CN class of 98.3%, EMCI class of 95.8%, LMCI class of 85.7%, and MCI class of 97.9%. With DenseNet121-GoogLeNet-Handcrafted features, the ANN algorithm reached accuracies: AD class of 94.1%, CN class of 99.1%, EMCI class of 100%, LMCI class of 92.9%, and MCI class of 100%. With MobileNet-GoogLeNet-Handcrafted features, the ANN algorithm reached accuracies: AD class of 91.2%, CN class of 98.3%, EMCI class of 97.9%, LMCI class of 92.9%, and MCI class of 95.7%.

Fig. 12.

Production of confusion matrix from the ANN network with combined CNN features along with handcrafted features for diagnosing the ADNI data set of AD.

Discussion the implementation of the systems

AD is a progressive neurodegenerative disorder that affects the brain, causing a decline in cognitive function and memory loss. Early diagnosis and staging of AD are crucial for effective treatment and intervention strategies. The primary objective of this study is to develop multiple hybrid systems that integrate features derived from CNNs with handcrafted features. The optimization of fMRI images was conducted specifically for the ADNI data set, utilizing the ACA method to segment ROI and isolate them from the image.

The first methodology to distinguish early stages of AD by a hybrid technique based on the two algorithms ACA and REF between MobileNet-XGBoost, DenseNet121-XGBoost and GoogleNet-XGBoost, which reached an accuracy of 90.7%, 91.5% and 90%, respectively. While the hybrid technology MobileNet-ANN, DenseNet121-ANN and GoogleNet-ANN reached an accuracy of 91.9%%, 93.8% and 90.7%, respectively. The time taken to execute the models MobileNet-XGBoost, DenseNet121-XGBoost, and GoogleNet-XGBoost was 5 min: 20 s, 7 min: 40 s, and 12 min: 10 s respectively.

The second methodology for diagnosing AD and distinguishing between its stages of development using XGBoost and ANN networks based on the integration of features of CNN models. The XGBoost network based on the hybrid CNN features of MobileNet-DenseNet121, DenseNet121-GoogleNet and MobileNet-GoogleNet achieved an accuracy of 96.9%, 93.8% and 95%, respectively. The time taken to execute the models MobileNet-DenseNet121-XGBoost, DenseNet121-GoogleNet-XGBoost, and MobileNet-GoogleNet-XGBoost were 7 min: 25 s, 8 min: 32 s, and 10 min: 25 s respectively.

While the ANN based on the hybrid CNN features of MobileNet-DenseNet121, DenseNet121-GoogleNet and MobileNet-GoogleNet achieved an accuracy of 95.4%, 94.6% and 93.1%, respectively. The time taken to execute the models MobileNet-DenseNet121-ANN, DenseNet121-GoogleNet-ANN, and MobileNet-GoogleNe-ANN were 5 min: 10 s, 6 min: 38 s, and 7 min: 42 s respectively.

The third methodology to analyze fMRI images with high accuracy for early diagnosis and distinguishing Alzheimer’s stages using XGBoost and ANN networks based on integrating features of fused CNN models with handcrafted features. XGBoost based on the hybrid CNN features of MobileNet-DenseNet121-Handcrafted, DenseNet121-GoogleNet-Handcrafted and MobileNet-GoogleNet-Handcrafted achieved an accuracy of 96.9%, 98.8% and 97.7%, respectively. The time taken to execute the models MobileNet-DenseNet121-Handcrafted-XGBoost, DenseNet121-GoogleNet-Handcrafted-XGBoost, and MobileNet-GoogleNe-Handcrafted-XGBoost were 8 min: 12 s, 9 min: 25 s, and 11 min: 40 s respectively.

While the ANN based on the hybrid CNN features of MobileNet-DenseNet121-Handcrafted, DenseNet121-GoogleNet-Handcrafted and MobileNet-GoogleNet-Handcrafted achieved an accuracy of 97.3%, 98.5% and 96.5%, respectively. The time taken to execute the models MobileNet-DenseNet121-Handcrafted-ANN, DenseNet121-GoogleNet-Handcrafted-ANN, and MobileNet-GoogleNe-Handcrafted-ANN were 7 min: 55 s, 8 min: 45 s, and 10 min: 24 s respectively.

Table 12 shows the methods and results of previous studies relevant to the diagnosis of AD achieved by the systems.

Table 12.

Comparison of the performance of the proposed systems with previous relevant studies.

Study	Method	Results
Tao-Ran et al.	Voxel-based computed tomography analysis	AUC of 81.5%
Janani et al.	Deep learning model	Accuracy of 88%, recall of 89%, and precision of 92%
Ahila et al.	Improved system based on the CNN model	Accuracy of 96%, sensitivity of 96%, and specificity of 94%
Louise et al.	Three machine learning algorithms	Accuracy of 62.5% and 68.06%
Bin et al.	Pre-trained deep learning model	Accuracy of 94.9% after transfer learning
Minseok et al.	CNN and machine learning algorithms	Accuracy of 90.2%, 90.5%, and 89.6%
Eric et al.	Reduced features and biomarkers from MRI images	Accuracy of 93.45%, 100%, and 92.54%
Run-Hsin et al.	Method for selecting features and identifying biomarkers	AUC of 84.1%
Janghel et al.	Approach based on VGG-16 for feature extraction and classification by SVM	Accuracy of 73.46%
Afnan et al.	Method for OTSU segmentation with enhanced fuzzy elephant herding optimization (EFEHO)	Sensitivity of 92.3%
Saeda et al.	Transformational networks based on freezing features	Accuracy of 97.06% for classifying MCI vs. CN class and 98.89% for classifying CN vs. AD class
Nitsa et al.	Pipeline for data processing and analysis of changes in brain structures to extract asymmetry features	Accuracy of 75% and 92.5% for EMCI classification versus CN and 90.5% and 93% for AD class classification versus NC, respectively
Haijing et al.	Extraction of correlation in feature space by ResNet50 based on Attention Mechanism and Spatial Switched Networks (STN)	Accuracy of 95.3%
Badiea et al.	Two CNN models to extract features of MRI images of AD and their classification by SVM	Accuracy of 94.8%, sensitivity of 93%, and specificity of 97.75%
Proposed system	Hybrid Approach of XGBoost Based on Fusion Features of CNN with Handcrafted	an AUC of 98.82%, accuracy of 98.8%, sensitivity of 98.9%, precision of 97.08%, and specificity of 99.5%

In comparing the study’s method and results with prior research, it’s evident that the proposed hybrid approach, combining XGBoost with fusion features from CNN and handcrafted features, outperforms many existing methods regarding diagnostic accuracy. It showcases superior performance with an AUC of 98.82% and impressive values for accuracy, sensitivity, precision, and specificity. Several previous studies employed deep learning models, CNNs, or machine learning algorithms, achieving 62.5–96.0% accuracy. While these methods demonstrate competence, they generally fall short of the comprehensive performance achieved by the proposed hybrid system. Notably, some studies focused on feature extraction from MRI images, achieving accuracies in the 73.46–97.06% range. While these approaches are valuable, the proposed hybrid method provides more robust results across multiple performance metrics. In summary, the hybrid approach presented in this study stands out as a promising and highly effective methodology for AD diagnosis, boasting exceptional accuracy and diagnostic capabilities compared to existing field techniques.

The hybrid approach of XGBoost based on fusion features of CNN with handcrafted features has outperformed previous systems for analyzing fMRI images for AD. The hybrid approach works by extracting features from fMRI images using CNN and fusing them.

Multiple CNN features are combined with handcrafted features and dimensionality reduction by RFE method. The revised features are used to train and evaluate the XGBoost classifier. On the other hand, deep learning methods automatically learn and extract features directly from images and combine them into vector features. The hybrid approach has advantages over previous methodologies, which is combining the features of multiple CNN models with hand-crafted features. The hybrid approach is a new, highly efficient and promising method for analyzing fMRI images of Alzheimer’s disease. The strengths of this study lie in the development of validated methodologies for the diagnosis of Alzheimer’s disease. The study uses three distinct methodologies, each with hybrid techniques such as XGBoost and ANN, and the MobileNet, DenseNet and GoogLeNet models. Feature Fusion: Combine CNN features with hand-crafted features. Exceptional performance: The hybrid model, which combines DenseNet-GoogLeNet-Handcrafted features, achieves superior performance (AUC of 98.82%), accompanied by an accuracy rate of 98.8%, a sensitivity of 98.9%, a precision of 97.08%, and a remarkable specificity of 99.5%.

Conclusions

Artificial intelligence techniques appear as a promising, efficient and effective tool for diagnosing the developmental stages of AD. In this study, several hybrid methodologies were developed to analyze fMRI data in diagnosing the developmental stages of AD. For all methodologies, images were enhanced and important brain regions were isolated from the rest of the brain structure by the ACA algorithm. The first methodology is based on XGBoost and ANN networks, with features of MobileNet, DenseNet, and GoogLeNet models. The second methodology is based on XGBoost and ANN with features from several models MobileNet-DenseNet121, DenseNet121-GoogLeNet and MobileNet-GoogLeNet in the XGBoost and ANN frameworks. The third methodology is based on XGBoost and ANN networks with features from several CNN models and handcrafted features as follows: MobileNet-DenseNet121-handcrafted, DenseNet121-GoogLeNet-handcrafted and MobileNet-GoogLeNet-handcrafted. The XGBoost and ANN networks with multiple handcrafted CNN features achieved better results than other methodologies. The MobileNet-GoogLeNet-handcrafted-XGBoost methodology reached an AUC of 98.82%, an accuracy of 98.8%, a sensitivity of 98.9%, a precision of 97.08%, and a specificity of 99.5%.

There are also limitations to consider: one of the main challenges faced by this study is the size of the data set and the imbalance of the data set. These limitations have been addressed by the use of data augmentation techniques.

Future works of this study include the integration of Internet of Things (IoT) technologies to enable real-time data collection and monitoring, as well as evaluating the proposed models on clinical images to further validate their effectiveness in real-world scenarios. Additionally, we plan to perform statistical analyses model performance across varied datasets.

Author contributions

Conceptualization, A.M.A and E.M.S; methodology, A.M.A and E.M.S; software, E.M.S; validation, J.S.A, and E.M.S; formal analysis, A.M.A, J.S.A and E.M.S; investigation, A.M.A; resources, E.M.S; data curation, J.S.A and E.M.S; writing—original draft preparation E.M.S; writing—review and editing, A.M.A and J.S.A; visualization, E.M.S and A.M.A; supervision, A.M.A and E.M.S; project administration, A.M.A, E.M.S, and J.S.A; funding acquisition, A.M.A and J.S.A; All authors have read and agreed to the published version of the manuscript.

Data availability

The fMRI images used as data for this study were collected from the ADNI AD Data Set, accessible through the following link: https://www.kaggle.com/datasets/abdulelahkhalaf/adni-5-classes.

Declarations

Competing interests

The authors declare no competing interests.