An EEG-fNIRS Hybridization Technique in the Four-Class Classification of Alzheimer’s Disease

Abstract

Background:

Alzheimer’s disease (AD) is projected to become one of the most expensive diseases in modern history, and yet diagnostic uncertainties exist that can only be confirmed by postmortem brain examination. Machine Learning (ML) algorithms have been proposed as a feasible alternative to the diagnosis of several neurological diseases and disorders, such as AD. An ideal ML-derived diagnosis should be inexpensive and noninvasive while retaining the accuracy and versatility that make ML techniques desirable for medical applications.

New Methods:

Two portable modalities, Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) have been widely employed in constructing hybrid classification models to compensate for each other’s weaknesses. In this study, we present a hybrid EEG-fNIRS model for classifying four classes of subjects including one healthy control (HC) group, one mild cognitive impairment (MCI) group, and, two AD patient groups. A concurrent EEG-fNIRS setup was used to record data from 29 subjects during a random digit encoding-retrieval task. EEG-derived and fNIRS-derived features were sorted using a Pearson correlation coefficient-based feature selection (PCCFS) strategy and then fed into a linear discriminant analysis (LDA) classifier to evaluate their performance.

Results:

The hybrid EEG-fNIRS feature set was able to achieve a higher accuracy (79.31%) by integrating their complementary properties, compared to using EEG (65.52%) or fNIRS alone (58.62%). Moreover, our results indicate that the right prefrontal and left parietal regions are associated with the progression of AD.

Comparison with existing methods:

Our hybrid and portable system provided enhanced classification performance in multi-class classification of AD population.

Conclusions:

These findings suggest that hybrid EEG-fNIRS systems are a promising tool that may enhance the AD diagnosis and assessment process.

1. Introduction

Alzheimer’s disease (AD), the most common and severe age-associated neurodegenerative disease, currently affects 5.8 million Americans and is expected to affect 13.8 million individuals in the US alone by 2050, with individual end-of-life costs exceeding those of heart disease and cancer (Assoc, 2018; Kelley et al., 2015). The only medically confirmed diagnosis for AD has been through autopsy, highlighting the urgent need to develop innovative and effective diagnostic tools (Assoc, 2018). While deposits of aggregated amyloid beta (Aβ) and neurofibrillary tangles of hyper-phosphorylated tau protein are the two hallmarks of AD, disruption of synaptic spines due to the detrimental binding of small Aβ oligomers is key in triggering cognitive decline (Serrano-Pozo et al., 2011). The synaptic failure induced by the binding of Aβ oligomers is widely considered to be one of the first dysfunctional events driving symptomatic AD, suggesting that cortical imaging techniques could capture information that is unique to AD patients (Selkoe, 2002; Teipel et al., 2015).

One of the main interests in machine learning (ML) research is the rapid and accurate diagnosis and assessment of neurodegenerative diseases, such as AD, through relatively simple experimental setups (Ahmadian et al., 2018; An et al., 2018). Imaging modalities that can collect the data needed to train the classifiers are preferred to be non-invasive, versatile, and inexpensive; qualities that would make them realistically applicable in the clinical setting. Several groups have shown that the implementation of an ML model in AD classification studies could achieve clinically acceptable accuracies (Katako et al., 2018; Klöppel et al., 2008; Wang et al., 2015). For instance, Klöppel et al. were able to achieve 89% accuracy when classifying patients as AD positive or healthy controls by using magnetic resonance imaging (MRI) data (Klöppel et al., 2008). Although they achieved a clinically compatible result with their classification accuracy, the high cost and environmental sensitivity associated with MR imaging can potentially compromise the clinical applicability of their method (Biasutti et al., 2012; Turner, 2016). Positron emission tomography (PET) has also been investigated as a potential source of data with Katako et al. achieving an accuracy of 84% through the support vector machine (SVM) (Katako et al., 2018). While PET imaging provided highly sensitive and accurate data, it is difficult to effectively combine the technique with other imaging modalities such as computed tomography (CT) and MRI (Vaquero and Kinahan, 2015). This therefore limits the applicability of PET in hybrid models that combine data from various imaging modalities to enhance overall classification performance.

Advanced neuroimaging techniques, including Electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS) and functional magnetic resonance imaging (fMRI), have been extensively utilized by the research community to develop high-performance classification models (Houmani et al., 2018; Karamzadeh et al., 2016; Li et al., 2017; Sato et al., 2015; Wang et al., 2015). All these imaging modalities present specific challenges which can often compromise their applicability in a clinical setting. Electroencephalography (EEG) utilizes scalp electrodes to measure fluctuations in voltage caused by the electrical activity of the neurons present on the cortical surface of the brain (Binnie and Prior, 1994). EEG recordings offer a high temporal resolution while suffering from a relatively low spatial resolution that is primarily caused by the conductivity distribution of the human head (Lachaux et al., 1999; Schoffelen and Gross, 2009). EEG is also noted for being sensitive to environmental noise and easily corruptible by motion artifacts (Reis et al., 2014), making the technique difficult to independently implement in the field of neurodegenerative disease research.

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique that measures hemodynamic responses on the cortical surface associated with neuronal behavior. It accomplishes this by dynamically measuring the changes in both oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) concentrations with optical signals of distinct wavelengths (Scholkmann et al., 2014). The portability and low cost of fNIRS systems have accentuated the potential clinical value of this imaging modality in the research community (Arenth et al., 2007; Mihara et al., 2010; Obrig, 2014). In contrast to EEG, fNIRS is noted for having a higher spatial resolution but a lower temporal resolution (Quaresima et al., 2012). A significant advantage that fNIRS holds over EEG is the negligible cross-talk effect when the activation region distance is greater than one centimeter (Strangman et al., 2003). The imaging modality is made resilient to motion artifacts through signal processing, making it an attractive tool to collect clinical data from patients with neurological illnesses (Balardin et al., 2017). However, a critical shortcoming of fNIRS with respect to EEG is the delay in the captured hemodynamic signal; for instance, it usually takes four to six seconds after stimulus onset to reach its peak response and another six to ten seconds to reach its negative post-stimulus undershoot (PSU) peak (Buxton et al., 2004; Cohen, 1997; Mayer et al., 2014). The low penetration of infrared light further limits the detection of the hemodynamic response to the superficial volume of the cortical mass (Quaresima et al., 2012).

Despite the limitations of these two imaging modalities, EEG coupled with fNIRS yields an augmented temporal and spatial resolution, thus allowing researchers to develop novel ML algorithms that utilize the complementary information that is collected (Li et al., 2017). In addition, both the EEG and fNIRS systems are highly portable and compatible with each other, which greatly reduce the measurement constraints to the patients and significantly expand the application scenarios of the multi-modal system (Li et al., 2019). Therefore, integrated EEG and fNIRS data has been applied in a number of classification studies, including workload assessment, motor function and auditory function (Hong and Santosa, 2016; Li et al., 2017; Omurtag et al., 2017).

Several groups using EEG have been able to accurately distinguish between healthy controls and mild cognitive impairment (MCI) subjects, or AD patients (Houmani et al., 2018; Wang et al., 2015). Other groups have shown that fNIRS could be a valuable source of information for AD studies (Li et al., 2018a; Li et al., 2018b). Houmani et al. were able to achieve a binary (AD and Subjective Cognitive Impairment (SCI)) classification accuracy of 91.6% using EEG data while achieving a three class (AD, SCI, and other patients) classification accuracy of 81.8% to 88.8% (Houmani et al., 2018). Li et al. were able to show that fNIRS data could be utilized to distinguish between various stages of AD when comparing the hemodynamic response of mild and severe cases (Li et al., 2018b). These works highlight the importance of the AD-specific information that can be captured using the EEG and fNIRS imaging modalities. Utilizing fNIRS and EEG to train multiclass ML models should, therefore, be investigated further to expand upon previous findings in the field.

The primary objective of this study is to evaluate the feasibility of utilizing hybrid EEG-fNIRS data to classify subjects at different stages of AD. Specifically, subjects from four groups were recruited, including healthy controls (HC), patients with mild cognitive impairment (MCI), mild AD (MAD), and moderate/severe AD (MSAD). Concurrent EEG and fNIRS measurement were employed to collect data from subjects during a random digit memorization task. We utilized a computationally inexpensive Pearson correlation coefficient-based feature selection (PCCFS) algorithm to optimize feature selection and achieve higher classification accuracy using hybrid EEG-fNIRS features. To our knowledge, this is the first hybrid EEG-fNIRS-based study to perform classification among healthy controls and patients at different stages of AD.

2. Materials and method

2.1. Participants

Twenty-nine subjects were recruited and participated in the study, including eight healthy controls, eight MCI patients, six mild AD patients, and seven moderate/severe AD patients. All subjects were right-handed and above 50 years of age. The experiment was approved by the local ethics committee (Guangdong Provincial Work Injury Rehabilitation Center, China), and was performed in accordance with the Declaration of Helsinki. Each subject (or caregivers in severe cases) was fully informed about the purpose of the research and provided written, informed consent prior to the beginning of the experiment. All patients recruited in this study were able to follow the study instructions independently. All participants were naïve to the experimental task and to the recording systems. The mental state of each subject was examined using the Mini-Mental State Examination (MMSE), and all rating scores were recorded. The demographic information and clinical rating scores of all subjects are summarized in Table1.

TABLE I.

DEMOGRAPHIC INFORMATION OF ALL SUBJECTS.

Characteristic	HC (n = 8)	MCI (n = 8)	MAD (n = 6)	MSAD (n = 7)
Ages (years)	63.6 ± 6.5	70.3 ± 5.4	72.5 ± 7.3	76.0 ± 4.8
Gender (M/F)	6/2	6/2	2/4	3/4
MMSE	28.2 ± 2.2	26.0 ± 2.2	19.7 ± 3.0	9.4 ± 1.7
Education (years)	11.0± 2.5	10.0 ± 3.1	11.2 ± 2.8	10.3 ± 2.9
Performance	30.0 ± 0.0	27.6 ± 1.8	24.0 ± 6.0	19.0 ± 2.8

2.2. Experimental Paradigm

All experiments were conducted in a shielded room meant to minimize environmental noise. The subjects were instructed to sit in a comfortable chair and were told to remain still and relaxed. The subjects were given visual instruction from a screen placed one meter in front of their eyes. The paradigm used in this study consisted of 30 random trials. Each trial began with a ten-second encoding task, where the subject was asked to memorize a number sequence (four, five, and six digits, ten trials for each digit length) presented on the screen followed by a ten second resting period. The subject would then be instructed to retrieve the number sequence from the previous encoding task and verbally repeat the sequence within ten seconds, followed by an additional ten seconds of rest (Figure 1(A)). We define patient performance as being the number of correct responses during the retrieval task. We defined a block of data as the EEG and fNIRS signals collected for each of the memorization digit lengths (for a total of 3 blocks × 10 trials).

Fig. 1.

Experimental Paradigm and setup. (A) The paradigm used for this experiment. (B) The locations of the EEG electrodes (grey), fNIRS sources (red) and fNIRS detectors (green).

2.3. Experimental Setup

EEG and fNIRS data were simultaneously recorded during the entire experiment. EEG data were recorded using a BrainAmp DC EEG recording system (Brain Products GmbH, Germany) with a sampling rate of 500 Hz. As shown in Figure 1(B), we selected channel FCz as the reference and CPz as the ground channel. Thirty-two additional EEG electrodes were placed on the scalp over the left and right hemispheres. A NIRScout system (NIRx Medizintechnik GmbH, Germany) with 16 sources and 15 detectors was used to collect fNIRS data. The inter-optode distance was 3 cm, and a total of 46 fNIRS channels were symmetrically distributed over the left and right hemispheres, encompassing the frontal and parietal regions of the cortical surface (Hong et al., 2018). Lights with wavelengths of 760 and 850 nm were used to detect the change in oxygenated and deoxygenated hemoglobin concentrations. fNIRS data were collected at a sampling rate of 3.91 Hz.

2.4. Data Preprocessing

Raw EEG signals were preprocessed using BrainVision Analyzer software (Brain Products GmbH, Germany). Data was first filtered from 0.5 to 40 Hz using a third order Butterworth band-pass filter. Powerline noise (50 Hz) was removed using a third order notch filter. Ocular artifact correction was accomplished using independent component analysis (ICA) followed by the manual removal of the artifact signals. Re-referencing was conducted using the common average reference method and data from the FCz channel was retrieved for further analysis. Single-trial EEG data for each encoding task was segmented from two seconds before task onset to ten seconds after the task onset. All segmented trails were manually inspected; trials with large spikes were considered “noisy” and were excluded from further analysis. On average, fewer than 10% of the total trials (4.14% ± 5.09%) were rejected for each subject. Baseline correction was performed by subtracting the mean value of baseline interval (−2 to 0 seconds) from each EEG channel’s corresponding segmented trial. Block averaging was done for each EEG channel with respect to different digit lengths, including 4-digit, 5-digit, 6-digit and all number sequences, yielding 33 × 4 (channels × blocks) averaged EEG signals for each subject.

Raw fNIRS signals were preprocessed using the nirsLAB software (NIRx Medizintechnik GmbH, Germany). We applied a fourth order Butterworth band-pass filter from 0.01 to 0.2 Hz to remove artifacts such as EKG (~0.8 Hz) and respiration (0.2–0.3 Hz). The changes in HbO and HbR concentrations were calculated according to the Modified Beer-Lambert Law with a differential path length factor of 7.25 and 6.38 for the 760 nm and 850 nm wavelengths, respectively (Essenpreis et al., 1993; Scholkmann et al., 2014). We then segmented the single trial fNIRS data from the onset of the encoding task to 20 seconds after the onset (0 to 20 seconds). All segmented trails were manually inspected, and trials with apparent motion artifacts were excluded. On average, fewer than 10% of the total trials (1.95% ± 6.93%) were rejected for each subject. Baseline correction was performed by subtracting the data point before the onset from each fNIRS channel’s corresponding segmented trial. Block average was then performed for each fNIRS channel with respect to different digit lengths, including 4-digit, 5-digit, 6-digit, and all number sequences, resulting in 46 × 4 × 2 (channels × blocks × Hb) averaged fNIRS signals for each subject.

2.5. Feature Extraction and Selection

Block averaged EEG data (10 seconds) was used to compute the relative band power for each channel across six bands of interest (Delta: 0.5 to 4 Hz, Theta: 4 to 7 Hz, low Alpha: 8 to 10 Hz, high Alpha: 10 to 13 Hz, Beta: 14 to 25 Hz, Gamma: 26 to 40 Hz). The power spectrum density (PSD) of the signal was calculated based on a 2-second window (with no overlapping) for four different block averaged signals. PSD was calculated using the Fast Fourier Transform (FFT). The relative band power of a specific frequency band, defined as the percentage of that frequency’s band power with respect to the total power (power from 0.5 Hz to 40 Hz), was extracted as features for classification. In summary, 3,960 EEG features (33 channels × 6 bands × 5-time windows × 4 blocks) were extracted for each of the subjects.

For the fNIRS signals, we focused on the main time frame from 3 to 12 seconds after the onset of the encoding task as suggested by the literature (Buxton et al., 2004; Cohen, 1997; Mayer et al., 2014). Average changes of HbO and HbR concentrations were then calculated as features by taking the average of the data every 3 seconds (with no overlapping) for four different block averaged signals. This yielded a total of 1,104 fNIRS features (46 channels × 2 Hb × 3 time windows × 4 blocks) per subject.

2.6. Feature Optimization and Classification

After the feature extraction, we ended up with two different feature sets to evaluate the classifier performance: EEG band power (29 subjects × 3,960 features) and fNIRS hemoglobin concentrations (29 subjects × 1,104 features). To select the optimal feature set for classification and to enhance the computational efficiency, we implemented a Pearson correlation coefficient-based feature selection (PCCFS) strategy with the EEG and fNIRS feature sets prior to the classification. The principle of PCCFS is demonstrated in Figure 2 (Aghajani et al., 2017; Guyon and Elisseeff, 2003). The Pearson correlation coefficient between each feature and all class labels (Healthy = 0, MCI = 1, MAD = 2, MSAD = 3) were individually calculated. All features were then sorted based on their absolute Pearson correlation coefficients in a descending manner, wherein the features that yielded a higher coefficient were given higher priority in the classification of the four groups. With the sorted feature set, we then iteratively (in a forward manner) added the features and evaluated the classifier performance of each iteration. The optimal feature set was defined as the subset that yielded the highest accuracy. The optimal EEG feature set and fNIRS feature set were then combined and re-sorted using PCCFS to form the EEG-fNIRS sorted hybrid feature set. The optimal hybrid feature set was defined in the same way as the optimal unimodal feature sets. In this study, considering the large number of features, the classification accuracy was assessed using the Leave One Out Cross Validation (LOOCV) with a Linear Discriminant Analysis (LDA) classifier due to its simplicity and low computational requirements as well as high popularity in EEG/fNIRS classification studies (Hong et al., 2018; Lahmiri et al., 2018). A flowchart is shown in Figure 3 to summarize the overall design of the study.

Fig. 2.

The flowchart of feature optimization using the Pearson correlation coefficient-based feature selection (PCCFS) method.

Fig. 3.

The overall design of the study. The subject presented in the figure provided verbal and written informed consent forthe publication of this figure.

3. Results

The goal of this study was to effectively classify subjects based on the degree of their dementia and to evaluate the performance of the hybrid EEG-fNIRS model relative to the EEG and fNIRS standalone model. Figure 4 demonstrates the changes in classification performance during the PCCFS processing for the EEG feature set, the fNIRS feature set and the hybrid feature set, respectively. After PCCFS optimization, we were able to reduce the size of each respective feature set according to the optimal classification performance. Overall, the hybrid EEG-fNIRS feature set provided the best performance (Mean: 79.31%, Standard Error: 7.66%), outperforming the EEG unimodal feature set (Mean: 65.52%, Standard Error: 8.98%) and fNIRS unimodal feature set (Mean: 58.62%, Standard Error: 8.98%). Specifically, the PCCFS-sorted EEG feature set (Figure 4(a)) achieved its highest accuracy with the top 21 features, while the PCCFS-sorted fNIRS feature set (Figure 4(b)) reached its highest accuracy with the top 48 features. Finally, the PCCFS-sorted hybrid feature space (Figure 4(c)) achieved its highest accuracy with the top 31 features, including all 21 optimal EEG features and the top 10 optimal fNIRS features. The confusion matrix and several commonly used metrics (precision, sensitivity, specificity and F-score) were calculated manually to evaluate the performance of each model further, as displayed in Tables 2, 3 and 4. Precision is defined as the ratio of true positives over true positives and false positives, sensitivity is defined as the ratio of true positives over true positives and false negatives, specificity is defined as the ratio of true negatives over false positives and true negatives, and F-score is defined as the harmonic mean of precision and recall.

Fig. 4.

Performance evaluation of EEG features (A), fNIRS features (B) and hybrid features (C) using PCCFS. In each sub-figure, the black solid line denotes average accuracy while shaded area denotes the standard error.

TABLE II.

THE CONFUSION MATRIX AND CLASSIFICATION PERFORMANCE OBTAINED BY THE OPTIMAL EEG FEATURES.

True Class	Predicted Class				Model Performance
	HC	MCI	MAD	MSAD	Precision	Sensitivity	Specificity	F-Score	Accuracy
HC	100%	0%	0%	0%	73%	100%	50%	84%	65.52%
MCI	25%	37.5%	25%	12.5%	50%	38%	73%	43%
MAD	16.7%	50%	33.3%	0%	40%	33%	77%	36%
MSAD	0%	0%	14.3%	85.7%	86%	86%	65%	86%

TABLE III.

THE CONFUSION MATRIX AND CLASSIFICATION PERFORMANCE OBTAINED BY THE OPTIMAL FNIRS FEATURES.

True Class	Predicted Class				Model Performance
	HC	MCI	MAD	MSAD	Precision	Sensitivity	Specificity	F-Score	Accuracy
HC	87.5%	12.5%	0%	0%	88%	88%	56%	88%	58.62%
MCI	12.5%	62.5%	12.5%	12.5%	45%	63%	52%	53%
MAD	0%	50%	50%	0%	43%	50%	67%	46%
MSAD	0%	28.6%	42.9%	28.6%	67%	29%	83%	40%

TABLE IV.

THE CONFUSION MATRIX AND CLASSIFICATION PERFORMANCE OBTAINED BY THE OPTIMAL HYBRID FEATURES.

True Class	Predicted Class				Model Performance
	HC	MCI	MAD	MSAD	Precision	Sensitivity	Specificity	F-Score	Accuracy
HC	100%	0%	0%	0%	89%	100%	63%	94%	79.31%
MCI	12.5%	75%	0%	12.5%	60%	75%	63%	67%
MAD	0%	67.7%	33.3%	0%	100%	33%	91%	50%
MSAD	0%	0%	0%	100%	88%	100%	67%	93%

Apart from the evaluation of the performance of the hybrid EEG-fNIRS model, we also attempted to identify the key regions of interest that are associated with cognitive decline based on the optimal hybrid feature set. Figure 5 shows the brain regions that contributed most to the performance of the classifiers based on the PCCFS-sorted features. To determine the activation sites on the cortical surface, we first projected the 3D coordinates of the EEG and fNIRS channels that yielded the optimal hybrid model features to a template brain model obtained from the MNI305 space (publicly available at: http://surfer.nmr.mgh.harvard.edu/). The size of the clusters (colored circles) is based on the number of top hybrid features derived from each channel, which ranged from one to four features per channel. Interestingly, among the 21 hybrid EEG features, most features were derived from the channels located in the left parietal region (yellow markers), while the 10 hybrid fNIRS features were mainly derived from the channels located in the right frontal region (blue markers).

Fig. 5.

Cortical activation mapping derived from the EEG (yellow) and fNIRS channels (blue) that contributed to the optimal hybrid feature set. The larger the size of the circle, the more features the corresponding channel contributes.

4. Discussion

Multi-modal imaging techniques have been previously reported to enhance classification performance over unimodal methods in binary classification tasks, yet the hybrid approach we presented in this work has received little research focus with multiple classes (Li et al., 2017). In this work, our goal was to evaluate the feasibility of utilizing hybrid EEG-fNIRS data to classify, with a clinically relevant accuracy, between four groups of subjects that included patients with varying degrees of AD. We elected to use a hybrid EEG-fNIRS model that was expected to outperform the EEG and fNIRS unimodal approach. The result suggests that the hybrid model is a superior and effective way to accurately classify and assess AD patients, with LOOCV accuracy approaching 79.31% in the four-class hybrid model. It is important to note that chance in our model is defined as the ratio of the largest group over the total sample size (~27.6%), which we exceed by a significant margin (Combrisson and Jerbi, 2015).

Due to the similar symptomologies shared by mild AD and other neurological pathologies (Kalaria, 2002; Mendez, 2006; Rodrigues et al., 2014), the difficulty in diagnosing and monitoring early-onset AD has been a major problem for clinicians. These diagnostic uncertainties further delay more definitive AD diagnoses until the disease has progressed, thus reducing the potential efficacy of treatment and costing an estimated 7.9 trillion USD in medical and patient care costs (Assoc, 2018). By capitalizing on the advantages of hybrid neuroimaging modalities, we were interested in presenting an ML-based system that was capable of accurately distinguishing between various stages of AD as a source of supplemental information for physicians in clinical settings. Overall, the high classification accuracy achieved in this study through the hybrid EEG-fNIRS feature set suggests that it is (compared to the unimodal approach) beneficial to combine these imaging modalities as an effective means to assess and diagnose AD. Analysis of the classifier performance (Table 4) further supports that the hybrid model was more robust than the EEG and fNIRS models in classifying the four subject groups.

The high precision achieved by the hybrid model indicates that it was more likely to make relevant predictions, while its sensitivity indicates that it was more likely to correctly classify control subjects, MCI subjects, and MSAD patients. To make better sense of the overall meaning of these two metrics, we elected to compute the F-score in order to make a more definitive statement about the performance of the hybrid model. The F-scores of the hybrid model were consistently higher than the EEG and fNIRS models, indicating that it was collectively more precise and/or sensitive than its unimodal counterparts. Importantly, the hybrid model had a higher specificity when classifying mild AD patients, relative to the EEG and fNIRS models. High specificity indicates that the likeliness of falsely identifying subjects as having mild AD was relatively low, which suggests that the hybrid model was less likely to misdiagnose each subject. However, the MCI and mild AD groups revealed relatively low accuracies compared to other groups, even though the hybrid EEG-fNIRS model yielded the optimal performance over EEG and fNIRS unimodal approaches. The diagnosis of patients at the early stage of AD may still stand as a primary challenge even when new methods are developed. Despite this limitation, the findings in the present study suggest that the proposed hybrid EEG-fNIRS model holds great promise to provide physicians with a more definitive and preemptive diagnosis of AD. We also believe that the improved confidence in the diagnosis would thus permit physicians to treat AD more effectively, therefore improving patient outcomes and reducing the cost associated with AD management.

Apart from the evaluation of the performance of the hybrid EEG-fNIRS model, we attempted to expand our investigation with physiologically meaningful information based on the top performing hybrid features. Interestingly, several activated cortical regions, mainly including the right prefrontal area and left parietal area, were identified to be associated with AD-linked cognitive decline (Figure 5) (Fernández et al., 2006; Machulda et al., 2003; McNab and Klingberg, 2008; Salat et al., 2001; Staff et al., 2010; Yap et al., 2017; Zanto et al., 2011). Specifically, as evidenced in previous studies, it is well-accepted that working memory task could induce active neuronal activity in several cortical regions including the frontal, parietal, and temporal lobe (Palva et al., 2010; Prabhakaran et al., 2000). Structural imaging technique also showed that reduced grey matter in the left and right parietal regions of the brain is associated with the progression of AD (Staff et al., 2010). These existing evidences are in line with the cortical regions we identified based on the optimal EEG features (Figure 5). On the other hand, it is important to note that oxygenation abnormalities in the prefrontal cortex have been reported in MCI patients, while volumetric changes in the prefrontal cortex are linked to the progression of AD (Salat et al., 2001; Yap et al., 2017). In addition, the hyper-activation in the frontal gyrus is found in patients with memory deficit during an encoding-retrieval task (Heun et al., 2007), demonstrating the important role of prefrontal cortex in the cognitive processing. These findings, again, are consistent with the identified right prefrontal region that was primarily derived from the optimal fNIRS features in our study (Figure 5). These findings presented here, therefore, provide evidence that the hybrid EEG-fNIRS model could serve as a potential tool to effectively identify brain regions affected by AD-linked cognitive impairment, advancing our understanding of AD progression and treatment.

Although this research has adequately evaluated the feasibility of utilizing the hybrid EEG-fNIRS model to classify patients at various stages of AD, several limitations should be acknowledged. The most apparent limitation of this study lies in the relatively small sample size that we used to evaluate the performance of hybrid feature set, which can result in falsely elevated classifier performance and prevent us from drawing a definitive conclusion with our current findings (Combrisson and Jerbi, 2015). Future work on a larger cohort should be carried out to validate and extend the present findings. Additionally, the fNIRS setup for this study was not able to provide full coverage due to a limited number of optodes, making it impossible to capture the hemodynamic response signals from uncovered areas. It is therefore suggested that future research utilize an improved setup with full coverage when possible.

5. Conclusions

In this study, we presented a hybrid EEG-fNIRS model to effectively classify four subjects’ groups including healthy controls and patients at different stages of AD. We show that the memorization task ML model could be used to inexpensively and rapidly supplement the diagnosis (and assess the degree of) dementia in AD patients. To select the most representative features in the challenging multi-class classification problem, we evaluated and optimized the features derived from EEG and fNIRS signals using a Pearson correlation coefficient-based feature selection (PCCFS) strategy. The superior performance achieved by the hybrid features suggests that hybrid EEG-fNIRS models such as the one proposed in this study may be used in a clinical setting to accurately diagnose and assess the severity of AD.

Highlights.

• Machine learning technique is able to classify patients with Alzheimer’s disease at different stages.

• Hybrid EEG/fNIRS feature set achieves higher classification accuracy by integrating the complementary properties of EEG and fNIRS, compared to using EEG or fNIRS alone.

• The right prefrontal and left parietal regions are associated with the progression of AD.