A Method to Combine Cognitive and Neurophysiological Assessments of the Elderly

Abstract

Background

The development of better treatments for brain diseases of the elderly will necessitate more sensitive and efficient means of repeatedly assessing an individual's neurocognitive status.

Aim

To illustrate the development of an assessment combining episodic memory and working memory tasks with simultaneous electroencephalography and evoked potential (EP) brain function measures.

Methods

Data from matched groups of elderly subjects with mildly impaired episodic verbal memory on neuropsychological tests and those with no objective signs of impairment were used for scale development. An exploratory multivariate divergence analysis selected task performance and neurophysiological variables that best recognized impairment. Discriminant validity was then initially assessed on separate impaired and unimpaired groups.

Results

Decreased response accuracy and parietal late positive component EP amplitude in the episodic memory task best characterized impaired subjects. Sensitivity in recognizing impairment in the validation analysis was 89% with 79% specificity (area under the curve = 0.94). Retest reliability was 0.89 for the unimpaired and 0.74 for the impaired validation groups.

Conclusion

These promising initial results suggest that with further refinement and testing, an assessment combining cognitive task performance with simultaneous neurofunctional measures could eventually provide an important benefit for clinicians and researchers.

Introduction

Current methods for assessing cognitive brain function of elderly people are limited in a number of respects. As improved medications become available for treating brain dysfunction, there will be an increasing need for better measures to guide early treatment [1,2]. Specifically, although a formal neuropsychological assessment remains the gold standard for clinical assessment of cognitive capability, validity of testing is dependent upon astute observation of the patient during the testing to determine the degree to which diminished alertness due to sleepiness or a drug, or extra effort to compensate for a mild deficit, affected cognitive performance. Direct neurofunctional measurements during cognitive testing, such as positron emission tomography, functional MRI or EEG, can provide information about both cognitive performance and brain function, and thus may be helpful in this regard. Combining the different types of information available from neuropsychological testing and direct brain function measures into a single index could provide an important benefit for clinicians and researchers. Here we report methodological progress on this issue and provide an example of how such a method could be derived.

Of the neurofunctional modalities, electroencephalography (EEG) is the least stressful for the patient and the most economical. Although not a three-dimensional imaging modality, EEG provides a unique and sensitive means of monitoring subsecond evoked potential (EP) processes of attention and memory that are not otherwise directly observable. For instance, EEG measures have been shown to be more sensitive than cognitive task performance measures to drug-related changes in functional alertness in the elderly [3].

Here we introduce an assessment that combines computerized attention and memory testing with simultaneous EEG and EP measures. This assessment is not intended as a substitute either for comprehensive neuropsychological testing or for three-dimensional anatomical or functional neuroimaging in diagnosing incipient degenerative brain disorders such as mild cognitive impairment (MCI). Rather, it is possible that a combined assessment using EEG or another neurofunctional assessment modality may prove useful in efficiently quantifying the neurocognitive effects of medications or other treatments on elderly patients.

Pathological age-related cognitive decline first becomes evident as impaired memory, attention and/or other executive functions. Greater episodic memory impairments characterize amnestic MCI with early medial temporal lobe (MTL) damage prodromal to the development of Alzheimer disease (AD), whereas greater executive function impairments may occur in MCI associated with cerebrovascular pathology and other disorders [4,5]. Working memory (WM), the ability to control attention and sustain its focus on active mental representations in the face of distracting influences [6], is an essential executive cognitive function. The capacity to retain and manipulate information in WM decreases with age in the elderly [7]. Whereas WM involves the current manipulation of information, episodic memory refers to the retrieval of events recently experienced, which depends on somewhat different neural systems. Performance on episodic memory tests incorporating a distraction between study and test periods can be disrupted by MTL dysfunction [8]. Given the early onset of atrophy in MTL regions in amnestic MCI and AD [9], it is not surprising that these conditions are accompanied by declines in episodic memory abilities. To assess both of these essential cognitive functions, well-established spatial n-back WM and verbal episodic memory recognition tasks were used for the initial combined assessment.

To illustrate how neuropsychological testing and EEG brain function measures could be combined into a single index, a retrospective analysis was done on a heterogeneous data sample. Healthy elderly adults and patients with memory complaints were recorded at numerous research laboratories and geriatric clinics across the USA. Repeated assessments were made over periods of up to 24 months to assess the retest reliability and sensitivity of the measures. All subjects were tested with the same EEG neurocognitive task battery, but because these recordings were mostly added onto existing studies, the number and frequency of the sessions and the specific neuropsychological tests administered differed across locations. Based on neuropsychological test scores and/or clinical diagnoses, a group of mildly impaired subjects was selected, as was a matched comparison group of cognitively healthy subjects. The performance and neurophysiological variables that best discriminated the data from these two groups were selected by an algorithm and combined into an exploratory equation. The validity of this equation was then tested by applying it to data from independent impaired and comparison validation groups. The tests and measures were not intended to be a definitive characterization of MCI, but rather an illustration of how a metric combining neuropsychological test performance and neurophysiological measures could be developed.

Methods

Test Administration

Physiological signals were recorded with a custom-built EEG headset with disposable solid-hydrogel electrodes placed at standard 10–20 system scalp locations, including over bilateral and midline dorsolateral prefrontal, midline sensorimotor, bilateral superior parietal and midline parietooccipital cortical areas, referenced to digitally linked mastoids. These locations were selected based on cognitive EEG studies with 40 or 100 electrodes [10,11,12]. Vertical and horizontal eye movements were monitored with 4 additional electrodes placed above and lateral to each eye. Signals were sampled at 128 Hz and band-pass filtered from 0.1 to 35 Hz.

The 20-min test battery consisted of 2 blocks of a verbal episodic memory task, each consisting of 3 phases: word presentation, distracter and word recognition [13]. These are merely illustrative of what tasks could be reasonably expected to expose differences between impaired and healthy subjects, and not intended to be definitive choices for assessing amnestic MCI. During the word presentation phase, subjects indicated whether each of 24 words appearing sequentially contained 1 or 2 syllables. (A phonological instead of a semantic task was used in the presentation phase to facilitate the construction of a large number of nonoverlapping word lists, to allow for multiple repeated assessments.) Subjects were told that they would be tested on the words later, so they should try to remember the words as they were making syllable judgments. Following the first word presentation phase, the distracter phase consisted of a simple reaction time task. This was followed by the word recognition phase, in which subjects indicated whether each of 48 words appearing sequentially was one of the words seen 4 min earlier on the presentation list, with half the words being old and half new. The second block of the episodic memory task then started with a presentation phase using the same list of 24 words to be remembered. The distracter phase in the second block was a 1-back WM task, followed by a recognition phase that used the same 24 words from the presentation list and 24 new foil words. The 1-back WM [11] and reaction time tasks each took 4 min and consisted of 50 trials. For the 1-back task, subjects had to decide whether the location of the dot stimulus on each trial was the same as on the immediately preceding trial. The simple reaction time task had the same stimulus characteristics as the 1-back WM task and served as a control. Following the repetition of the episodic memory task, a resting EEG was recorded with eyes open and closed for 5 min. Subjects received instructions and sufficient practice for 1-back WM and episodic memory tasks to stabilize performance prior to the test. The total time to administer the test was approximately 1 h.

Subjects

The test was administered in pilot studies at 5 research laboratories and 2 clinics. The research protocols were approved by each local institutional review board, and all subjects provided informed consent. A total of 191 subjects were recruited from the community and were at least 55 years of age, had no uncontrolled major health problems (including psychiatric disorders) and were not being treated for cognitive complaints. Forty-four patients with mild cognitive complaints were recruited from neurology and psychiatry clinics. So that the exploratory analysis would identify performance and neurophysiological measures reliably associated with impairment per se, groups of elderly people unequivocally with and without consistent objective evidence of episodic memory impairment were formed (table 1a).

Table 1.

Demographic variables for the mildly impaired and comparison groups

aExploratory analysis groups
	Impaired (n = 28)	Comparison (n = 28)	Statistics
Age, years	74 ± 10	74 ± 8	t(54) < 1, N.S.
Education, years	15 ± 3	15 ± 2	t(54) < 1, N.S.
Gender – female, n	10 (36%)	15 (54%)	χ2(1) = 1.81, N.S.
MMSE score at baselinea	28.2 ± 1.6	29.3 ± 1.0	t(40) = 2.49†
Memory test score at baselineb	−2.5 ± 1.0	0.5 ± 0.7	t(54) = 13.31***

bValidation analysis groups
	Impaired (n = 19)	Comparison (n = 16)	Statistics
Age, years	74 ± 11	72 ± 6	t(33) < 1, N.S.
Education, years	15 ± 3	16 ± 2	t(33) = 1.68, N.S.
Gender – female, n	7 (37%)	16 (100%)	χ2(l) = 15.38***
MMSE score at baselinea	26.3 ± 3.7	29.3 ± 1.1	t(27) = 2.46†
Memory test score at baselineb	−1.6 ± 1.1	0.8 ± 0.8	t(25) = 6.52***

Values denote means ± SD unless otherwise indicated. N.S. = Not significant; MMSE = Mini-Mental State Examination.

^†p < 0.05

^***p < 0.0001.

^aMMSE scores were available for 23/28 impaired and 19/28 comparison participants in the exploratory analysis, and for 19/19 impaired and 10/16 comparison participants in the validation analysis.

^bMemory test scores are z-scores.

The mildly impaired group comprised subjects who either demonstrated objective impairment on standard neuropsychological tests at the baseline test session and on 1 or more follow-up tests occurring at least 6 months later, or if follow-up neuropsychological test results were unavailable, they tested as impaired at baseline and were diagnosed as having MCI by a neurologist or psychiatrist specializing in geriatrics. Because the recordings were made during the course of different studies and treatment regimens over the multiple laboratories and clinics, the content and timing of the neuropsychological batteries differed across recording sites. The standard episodic memory tests included the California Verbal Learning Test, Hopkins Verbal Learning Test, Wechsler Logical Memory Test, Rey Auditory Verbal Learning Test, and Free and Cued Selective Reminding Test. The objective impairment criteria for inclusion in the impaired group were: (1) a score at least 2 SD below normal on 1 standard episodic memory test, or (2) a score at least 1.5 SD below normal on 1 standard memory test if accompanied by a score 1 SD below normal on a nonmemory cognitive test, or (3) scores at least 1 SD below normal on 2 or more standard memory tests. Test scores were assessed using published norms.

Out of all the subjects recorded at the research laboratories and clinics, 28 met these criteria (mean age: 74 years). Seven of the 28 were undergoing treatment at a clinic, including 4 with predominant amnestic impairment and 3 with MCI due to vascular disease, head trauma or depression. The other 21 subjects were recruited from the community. The comparison group comprised individuals whose memory or other cognitive test scores were better than 0.75 SD below normal on baseline and 1 or more follow-up testing sessions over a minimum period of 6 months. Forty-four subjects, all from the community, met this definition. The 28 who best matched the impaired subjects on age and education were selected for the cognitively healthy comparison group.

In addition to lower memory tests scores [t(54) = 13.31; p < 0.0001], the impaired group also had lower Mini-Mental State Examination (MMSE) scores than the comparison group [28.2 vs. 29.3; t(40) = 2.49; p < 0.05], although MMSE scores were well within the normal range for both groups [14]. The groups did not differ in age [t(54) = 0.06; p = 0.95] or education [t(54) = 0.26; p = 0.80], and although the comparison group had more female members than did the impaired group, the difference in gender composition was not significant [χ²(1) = 1.81; p = 0.18]. Nonetheless, the gender imbalance was accounted for statistically with linear mixed models that used gender as a covariate. When enrolled, all subjects were scheduled for testing at follow-up intervals 12 and 24 months after their baseline test; many of the subjects were also tested between 1 week and 3 months after baseline, and again at 6 months after baseline. To assess the effects of aging, 72 healthy young adults (18–39 years old, matched with the elderly on education level) were tested in our laboratory 3 times (at baseline, 3 and 6 months).

An impaired elderly validation group was composed of 19 other subjects (mean age: 74 years; 7 females) who were tested at a clinic and received a diagnosis of MCI or a related disorder (mild AD, or Parkinson's disease with MCI or mild dementia) but either had test scores that were not sufficiently poor to meet the criteria for the impaired group used in the exploratory analysis or were not given neuropsychological tests (table 1b). A cognitively healthy validation comparison group was composed of the 16 community members (mean age: 72 years; 16 females) who met the objective criteria for being cognitively healthy but did not match the impaired exploratory analysis subjects on age and education, and therefore were not used in the exploratory analysis. In addition to lower neuropsychological memory test scores [t(25) = 6.52; p < 0.0001], the impaired validation group had lower MMSE scores than the comparison validation group [26.3 vs. 29.3; t(27) = 2.46; p < 0.05]. The groups did not differ in age [t(33) = 0.57; p = 0.57] or education [t(33) = 1.68; p = 0.10]. However, the validation groups were not balanced on gender [χ²(1) = 15.38; p < 0.0001] as all members of the comparison group were female. The remaining 143 subjects comprised a heterogeneous mixture of 121 healthy community members and 22 clinic patients with a variety of neurological or psychiatric issues. These subjects did not have poor enough neuropsychological scores to qualify for inclusion in the impaired exploratory analysis or validation groups, yet did not meet all the criteria for inclusion in the cognitively healthy groups (e.g. 1 or more neuropsychological test scores may have been 0.75 SD below the norm on 1 of multiple test dates, or follow-up neuropsychological test data may have been unavailable).

To illustrate an example application of a combined assessment, data were analyzed from a separate experiment with 12 healthy elderly subjects (62–75 years; 6 male) who took a single oral dose of diphenhydramine, an over-the-counter antihistamine with sedating effects [3]. These subjects were well educated (mean of 18 years of education), high functioning (mean Wechsler Abbreviated Scale of Intelligence score of 125) and had no evidence of dementia (MMSE score of 28 or higher).

Data Analysis

Preliminary EEG Analysis

Automatic detection and removal of artifacts due to eye movements and blinks, scalp muscle activity, head and body movements and bad electrode contacts [15] was followed by visual inspection of all decontaminated and raw data. Power spectral estimates were computed on the decontaminated EEG data by averaging 2-second periodograms over the task or resting condition and combining them into frequency band variables as described below. Averaged EP were computed across the interval from 0.5 s before to 1 s after the task stimulus.

Combining EEG, EP and Task Performance Data into a Score

For the exploratory analysis, a set of task performance, EEG power spectral, and EP amplitude and latency measures known to be affected by aging and conditions that impair memory was selected for consideration [3,12,16,17,18]. Based on examination of the performance, EP and EEG spectra (fig. 1a), this set was reduced to 10 candidate variables that appeared to differ between the impaired and comparison exploratory groups, including 4 performance, 3 EP and 3 EEG measures. The 4 performance variables consisted of percent correct and mean reaction time from the episodic memory recognition phase and the WM task. The 3 EP variables were computed from the episodic memory task: right frontal and right parietal late positive component (LPC) amplitude and parietooccipital P200 amplitude. The LPC amplitude is sensitive to episodic memory manipulations [19,20,21], whereas the P200 amplitude is modulated by manipulations of attention [22,23]. The 3 EEG variables sensitive to aging [24,25] were measured during eyes-closed resting conditions: left frontal theta power, left frontal peak alpha frequency and parietooccipital peak alpha power. To put all variables on the same scale, raw values of each variable were standardized relative to the mean and SD of the exploratory comparison group at each test.

Fig. 1.

Neurophysiological data from mildly impaired and healthy comparison groups from exploratory analysis (data averaged from baseline, 12- and 24-month tests) (a) and validation analysis (data averaged from baseline, 3-, 6- and 12-month tests) (b). Left panels: episodic memory task EP at the right frontal (F4) and right parietal (P4) sites; frontal and parietal LPC and parietal P200 EP amplitudes were candidate variables for distinguishing impaired from comparison subjects in the exploratory analysis, and the parietal LPC was selected by the divergence analysis. Right panels: EEG power spectra at the left frontal (F3) and midline parietooccipital (POz) sites while subjects rested with their eyes closed; frontal alpha peak frequency, frontal theta power and parietooccipital alpha power were candidate variables for distinguishing impaired from comparison subjects in the exploratory analysis, none of which were selected by the divergence analysis.

The values of the candidate variables for each subject were then submitted to an exploratory multivariate divergence analysis, a simple type of discriminant analysis that performed an exhaustive search over all possible subsets of up to 3 of the 10 candidate variables to find the particular subset that – considered together – maximized the separation (divergence) between the 2 groups of subjects [26,27,28,29]. Data from baseline, 12- and 24-month tests were used because nearly all subjects were recorded at those 3 time points. The final set of variables was limited to 3 to avoid overfitting the exploratory data set [27]. Because summary values were used (e.g. single EP or spectral band measurements averaged over an EEG data file), the resulting models were not overly large. The values of each of the chosen variables for each test session were weighted by multiplying each one by the percentage of the total divergence contributed by that variable in the exploratory analysis and dividing by its variance in the exploratory data set. The resulting values were then added together to form a single score characterizing the subject's neurophysiological and neurocognitive brain function at the time of that session. Data from the validation groups and remaining subjects were likewise combined into scores for each test session using the 3 variables, weights and variances calculated in the divergence analysis of the exploratory groups. The same final set of 3 variables chosen by the divergence analysis of the exploratory data were frozen for application to the validation analysis and the remaining subjects; a second divergence was not performed for the validation analysis.

Based on the scores for each subject in the impaired and comparison groups, receiver-operator characteristic (ROC) curves were then generated to calculate area under the curve (AUC), sensitivity (correctly identifying members of the impaired group) and specificity (correctly identifying members of the comparison group) values over all subjects [30]. Significance of each AUC was calculated using the normal distribution from the z-ratio computed as the AUC minus 0.5 divided by the standard error for AUC = 0.5, the null hypothesis. Differences between 2 AUC were assessed by a two-sample Z test [31]. So that an individual's score could easily be interpreted relative to a normative sample, raw scores were standardized relative to the values of the cognitively healthy comparison group at baseline. The mean score of the comparison group at baseline was thus set at 100 with an SD of 15. Retest reliability for the scores at follow-up test sessions was calculated as the Pearson product moment correlation coefficient. Comparison of reliability values across groups was done by applying the Fisher transformation to each correlation coefficient. The difference between the 2 transformed correlations was divided by the standard error to yield the normally distributed variable z, and significance was calculated using the normal distribution. Because not all subjects were tested at all time points, the effect of group membership on the score and individual variables was tested with a linear mixed model analysis to adjust for the correlation due to differing numbers of repeated observations on each subject. ‘Subject ID’ was used as a random factor, ‘session’ as a repeated factor, ‘group’ as a between-subjects factor, and ‘gender’ as a covariate. A model with an unstructured covariance structure was used because it produced the smallest value of Akaike's information criterion.

Results

Valid data were recorded in 99% of the total 1,592 test sessions; the 18 invalid tests resulted from excessive movement artifacts or equipment problems. Forty-two of the 44 patients with cognitive complaints (95%) were able to understand and perform the tasks without apparent difficulty; the other 2 patients initially had some difficulty understanding the instructions but were able to complete the test.

A linear mixed model analysis of the 10 candidate variables in the exploratory data indicated that accuracy was lower (p < 0.0001) and reaction time longer (p < 0.001) in the episodic memory task, and 1-back WM task accuracy was lower (p < 0.01) for the impaired than the comparison group. The impaired group had a lower episodic memory parietal LPC EP amplitude (p < 0.01) and lower parietooccipital P200 EP amplitude (p < 0.05) (fig. 1a, left panels). None of the 3 background EEG measures differed between impaired and comparison groups.

The exploratory divergence analysis selected a combination of 3 measures to discriminate the impaired from comparison groups, all from the episodic memory task (table 2): (1) performance accuracy (weight: 0.43); (2) parietal LPC EP amplitude (weight: 0.35), and (3) reaction time (weight: 0.22). These 3 measures were combined to make a single score, as described above. Impaired subjects had lower scores than comparison subjects across test sessions and at each individual test session (t > 5.10, p < 0.0001 in all cases) (table 3a). Across all tests, the AUC was 0.89 [z(213) = 9.80; p < 0.0001] with 81% sensitivity and 81% specificity. The increase in AUC from 0.85 at baseline to 0.93 at 24 months was not significant [z(85) = 1.02; p = 0.31]. Ten of the 12 impaired subjects (83%) who completed the 24-month follow-up test were still classified as impaired at 24 months, and 15 of the 19 comparison subjects (79%) were still classified in the comparison group. The score was highly reliable at follow-up tests, with a retest correlation of r(81) = 0.83 (p < 0.0001) for comparison subjects and r(74) = 0.68 (p < 0.0001) for impaired subjects across all follow-ups. Scores were more variable across tests for the impaired than the comparison group, but retest reliability did not significantly differ between the 2 groups (z = 1.04; p = 0.30).

Table 2.

Values of episodic memory task variables distinguishing impaired and comparison elderly groups in the exploratory and validation analyses

	Exploratory analysis		Validation analysis
	elderly impaired (n = 28)	elderly comparison (n = 28)	elderly impaired (n = 19)	elderly comparison (n = 16)	young adults (n = 72)
Performance accuracy, %	69 + 13***	84 ± 8	56 ± 17*	84 ± 10	88 ± 7**
Parietal LPC EP amplitude, μV	2.2 ± 1.5*	4.1 ± 2.9	2.0 ± 1.9*	4.5 ± 2.6	4.0 ± 2.5, N.S.
Reaction time, ms	947 ± 130**	839 ± 93	891 ± 150, N.S.	842 ± 95	724 ± 91***

Mean ± SD variable values averaged over all 5 tests for the exploratory elderly groups, over 4 tests (baseline, 3, 6 and 12 months) for the validation elderly groups, and over 3 tests (baseline, 3 and 6 months) for the young adults. Significance results are from linear mixed models using gender as a covariate. For the elderly impaired groups, symbols indicate whether the value differs from that of the elderly comparison group from the same analysis. Values from a healthy young adult group are also shown; symbols indicate whether the value differs from that of the combined elderly comparison groups (n = 44). N.S. = Not significant.

^*p < 0.01

^**p < 0.001

^***p < 0.0001.

Table 3.

Scores, AUC, sensitivity/specificity and retest reliability of exploratory analysis recognizing impaired and comparison subject groups, and of validation analysis on other impaired and comparison subject groups

aExploratory analysis results
	Baseline (n = 28, 28)a	3 months (n = 24, 25)a	6 months (n=19,13)a	12 months (n = 21,26)a	24 months (n=12,19)a	All tests
Score
Impaired	78 ± 17	77 ± 13	73 ± 15	73 ± 20	71 ± 17	75 ± 16
Comparisonb	100 ± 15	100 ± 14	100 ± 14	100 ± 15	100 ± 15	100 ± 14
AUC	0.85*** (0.75–0.95)	0.92*** (0.84–0.99)	0.90** (0.78–1.01)	0.88*** (0.77–0.99)	0.93** (0.82–1.03)	0.89*** (0.84–0.93)
Sensitivity, %	79	83	79	81	83	81
Specificity, %	82	76	85	85	79	81
Retest reliabilityc
Impaired		0.73***	0.64**	0.70**	0.70†	0.68***
Comparison		0.86***	0.90***	0.82***	0.76**	0.83***

bValidation results
	Baseline (n = 19, 16)a	3 months (n = 11, 15)a	6 months (n = 14, 8)a	12 months (n = 12, 13)a	All tests
Score
Impaired	64 ± 17	70 ± 15	71 ± 20	60 ± 20	66 ± 18
Comparisonb	102 ± 16	103 ± 16	106 ± 20	104 ± 20	103 ± 17
AUC	0.96*** (0.91–1.01)	0.95** (0.88–1.03)	0.89* (0.75–1.03)	0.94** (0.85–1.03)	0.94*** (0.89–0.98)
Sensitivity, %	89	91	86	92	89
Specificity, %	81	80	88	69	79
Retest reliabilityc
Impaired		0.64†	0.88***	0.74*	0.74***
Comparison		0.90***	0.90*	0.89***	0.89***

Scores are means ± SD. Values in parentheses denote 95% CI. N.S. = Not significant.

^†p < 0.05

^*p < 0.01

^**p < 0.001

^***p < 0.0001.

^an is the number of subjects in the impaired and comparison groups, respectively.

^bScores are scaled so that the mean of the exploratory comparison group is 100 with an SD of 15 at the baseline session.

^cValues are Pearson's correlation coefficients (r) between baseline and follow-up tests.

The exploratory analysis results were then applied to the impaired and comparison validation groups (table 3b). Scores at baseline averaged 64 (SD = 17) for the impaired validation group and 102 (SD = 16) for the healthy validation group, with an AUC of 0.96 [z(33) = 4.64; p < 0.0001], with 89% sensitivity and 81% specificity. The exploratory analysis results consistently discriminated between the validation groups over the baseline, 3-, 6- and 12-month tests (there were insufficient impaired subjects at 24 months). Across all test sessions, scores averaged 66 (SD = 18) for the impaired validation group and 103 (SD = 17) for the healthy comparison validation group, with an AUC of 0.94 [z(106) = 7.80; p < 0.0001], and with 89% sensitivity and 79% specificity. Eleven of the 12 subjects (92%) in the impaired validation group who completed the 12-month follow-up test were still classified as impaired, and 9 of the 13 (69%) comparison subjects were still classified in the comparison group at 12 months. The score was highly reliable across follow-up tests, with retest correlations of r(34) = 0.89 (p < 0.0001) for comparison validation subjects and r(35) = 0.74 (p < 0.0001) for impaired validation subjects across all follow-ups. The validation groups exhibited effects on the individual performance and EEG variables generally similar to those observed in the exploratory analysis (table 2) in that both episodic memory performance accuracy and the parietal LPC EP amplitude were lower for the impaired group (p < 0.01 in both cases) (fig. 1b, left).

The mean score for the remaining 143 subjects whose neuropsychological test results were neither poor enough for inclusion in the impaired groups nor good enough for inclusion in the cognitively healthy comparison groups was 87 (SD = 18) at baseline and remained largely unchanged across follow-up tests (mean = 85, SD = 17). Linear mixed model analyses confirmed that these remaining subjects scored significantly higher than subjects in the impaired validation group (mean = 66; p < 0.0001), and lower than the subjects in the comparison validation group (mean = 103; p < 0.001) across all tests.

Assessing Individual Differences in Response to Medication

Because the value of such a methodology in future research may well prove to be in evaluating whether a therapeutic treatment has affected the neurocognitive function of an individual patient, the combined assessment method was applied to 12 subjects in a separate double-blind, randomized, cross-over experiment in which they took a single oral dose of 50 mg of diphenhydramine [3]. Healthy elderly adults performed the working and episodic memory tasks at hourly intervals before and up to 3.5 h after having taken either diphenhydramine or a placebo. Diphenhydramine produced neurophysiological changes characteristic of drowsiness, such as increased low-frequency EEG power (p < 0.05) and variability (p < 0.001). Task performance did not decrease after diphenhydramine, however, suggesting that participants ‘fought off’ the drowsiness in an effort to maintain a high level of performance.

The analysis described above was performed to discriminate data from the diphenhydramine and placebo conditions, and used to assess individual differences in response to the drug. The variability in parietal low-frequency (2–6 Hz) EEG power and WM task EP slow-wave amplitude were identified by the divergence analysis and combined to distinguish diphenhydramine from placebo with 100% sensitivity and 92% specificity (AUC = 0.98; p < 0.0001). No task performance variables were selected, as using performance information did not increase discrimination accuracy. The result from the group was then applied to each of the 12 individuals to characterize the magnitude of the drug effect on each person in a single score. Encouragingly, large individual differences in the magnitude of the diphenhydramine response were identified (fig. 2).

Fig. 2.

Drug effect scores for each of 12 healthy elderly subjects at the published peak time (150 min) after having ingested 50 mg of diphenhydramine. The group mean (and standard error) is shown for comparison in the black bar at the left. A multivariate divergence analysis chose variability in parietal low-frequency EEG power and WM task EP slow-wave amplitude to distinguish diphenhydramine from placebo in the group with 100% sensitivity and 92% specificity (AUC = 0.98; p < 0.0001). The result from the group was then applied to each of the 12 individuals to characterize the magnitude of the drug effect on each person. The more negative the score, the stronger the subject's neurophysiological response to diphenhydramine. There is a wide range of scores amongst the individuals, although all scores are in the negative direction.

Comparison with Young Adult Group

To assess the effects of normal aging, the 44 cognitively healthy elderly subjects (from both the exploratory and validation comparison groups; mean age: 73 years; 31 females) were contrasted with a group of 72 healthy young adults (mean age: 24 years; 38 females) (table 2, right column; fig. 3). Data were analyzed from 3 tests for young adults and up to 5 tests for healthy elderly adults. Relative to the healthy elderly, young adults had higher episodic memory task accuracy (p < 0.001) and shorter reaction times (p < 0.0001). They also had lower 1-back WM task accuracy (p < 0.01) with shorter reaction times (p < 0.0001). Although episodic memory task parietal LPC and P200 EP amplitudes did not differ between the young adults and healthy elderly (p = 0.65 and 0.065, respectively), the young adults had smaller frontal LPC amplitudes than healthy elderly (p < 0.0001) (fig. 3, left). In addition, young adults had larger contingent negative variation EP amplitudes (p < 0.05) compared to healthy elderly adults. The young adults had greater resting EEG power in the theta band (p < 0.05) and less power in the beta band (13–18 Hz; p < 0.001) (fig. 3, right). The peak alpha frequency was somewhat slower for the healthy elderly than young adults, but the effect was not statistically significant (p = 0.052).

Fig. 3.

Neurophysiological data from cognitively healthy elderly and young adult groups. Data are averages from baseline, 12- and 24-month tests for 44 elderly adults and from baseline, 3- and 6-month tests for 72 young adults. Left panels: episodic memory task EP at the right frontal (F4) and right parietal (P4) sites; the healthy elderly adults had a larger LPC amplitude than the young adults frontally, but not parietally, whereas the young adults had a larger contingent negative variation amplitude than the elderly adults. Right panels: EEG power spectra at the left frontal (F3) and midline parietooccipital (POz) sites while subjects rested with their eyes closed; young adults had larger theta power and smaller beta power than did healthy elderly adults.

Discussion

We described the development of a methodology for combining performance and neurophysiological data into a single metric characterizing brain function in elderly people in various stages of cognitive decline. It consists of a novel assessment of verbal episodic memory and WM that combines cognitive task performance with simultaneously obtained EEG and EP neurophysiological measures to provide a score reflecting an individual's cognitive brain status. An example of its application was presented in pilot exploratory and validation studies of subjects with and without mildly impaired amnestic function.

The scientific foundation of this combined assessment rests on a series of basic cognitive neurophysiological experiments on WM and verbal episodic memory [11,13,32,33], followed by the development of tasks and EEG measures suitable for routine serial assessments [3,16,17,18,34]. Engineering advances facilitated use of the assessment by researchers and clinicians who were not specialists in cognitive neurophysiology, including a quickly applied EEG headset, automated quality assurance for test administration and data analysis, and multivariate analysis to combine cognitive task performance and neurophysiological measures into a single score.

The combined assessment described herein was apparently successful in the example studies in that: (1) in the exploratory analysis, the mildly impaired group had mean test scores of 75 compared to 100 for the comparison group, and the impaired subjects were recognized with good sensitivity and specificity (AUC = 0.89) and high retest reliability (0.83 for comparison and 0.68 for impaired subjects), (2) the set of 3 measures chosen by the exploratory divergence analysis (2 task performance and 1 EP) are intuitively appealing and easily understood in the context of prior research on early degenerative brain disease, as discussed below, and (3) the exploratory analysis results generalized well in the validation analysis to a new sample of mildly impaired and healthy comparison subjects (AUC = 0.94; p < 0.0001; 89% sensitivity and 79% specificity; retest reliability: 0.89 for comparison and 0.74 for impaired subjects).

Taken at face value, the sensitivity and specificity of the results are within the recommended range for an acceptable clinical biomarker of AD, whose cognitive neurophysiological effects are more severe than the preclinical and early clinical stages of cognitive impairment of the subjects in the current study [35]. However, these initial results should merely be interpreted as encouraging further research since they are based on a relatively small number of subjects (56 for the exploratory analysis and 35 for the validation) taken from a heterogeneous sample, and are subject to other limitations, as discussed below. In fact, the most effective combination of performance and neurophysiological variables to recognize impairment, as well as the actual discriminant validity of the combined assessment, must be determined on a larger subject population including patients with amnestic, executive and mixed types of MCI. It also should be noted that the verbal episodic memory and spatial n-back WM tasks used here do not by any means represent the full spectrum of cognitive functions, nor are they likely to be the best tasks for assessing episodic memory and WM of elderly subjects.

The score from the combined assessment averaged 85 across all tests for the 143 subjects whose scores on standard episodic memory tests did not qualify them for inclusion in the impaired or healthy exploratory or validation groups. This fell in between the scores for the impaired (75 on exploratory and 66 on validation analysis) and comparison (100 on exploratory and 103 on validation analysis) groups. The individuals included a heterogeneous mixture of 121 community members (score = 85, SD = 17) and 22 clinic patients with a variety of neurological and psychiatric issues (score = 89, SD = 19). Although it is possible that those among the 143 subjects with the lowest scores actually had an amnestic MCI, such results are not interpretable without a confirmed clinical diagnosis and other objective evidence substantiating a basis for cognitive health or impairment.

A limitation of the example studies is a degree of circularity between the initial subject selection criteria and the differences found between groups in the example study. As neuropsychological episodic memory tests were used as the gold standard to categorize subjects as impaired or cognitively healthy, it is not surprising that group differences were found in both performance and neurophysiological variables from a different psychometric test of verbal episodic memory. This limitation must be considered in judging the accuracy of the equation in distinguishing between the impaired and control groups in the exploratory analysis. This limitation could, in principle, be overcome if an objective measure, independent of standard neuropsychological test performance, were available that definitively classified subjects as cognitively impaired or healthy (e.g. hypothetical blood or brain imaging tests). Because no such measure exists, however, neuropsychological testing is, and is likely to continue to be, the gold standard in this domain for the foreseeable future, and this type of circularity is likely to affect other studies of this type. This limitation was ameliorated by requiring consistently poor memory test scores on multiple occasions – or a medical specialist's clinical judgment of impairment when only one test was available – for inclusion in the exploratory impaired group, or consistently good test scores for inclusion in the healthy comparison groups.

The task performance measures that best distinguished between the impaired and comparison groups were accuracy and reaction time in the episodic memory task, with relative weights of 0.43 and 0.22 from a total weight of 1.0. The finding that episodic memory recognition performance is a sensitive marker of cognitive impairment in the elderly is consistent with previous findings of verbal episodic memory deficits in patients in the preclinical and later stages of AD [36,37]. Lesion, volumetric and functional imaging studies suggest that such performance deficits in amnestic MCI and AD patients may be associated with cortical atrophy in important regions of the semantic cognition network, including the MTL, anterior temporal lobe, inferior prefrontal cortex and left entorhinal cortex [38,39,40]. Although accuracy in the 1-back WM task was lower for the impaired than comparison exploratory groups (this was true in the validation groups as well), this effect was weaker than what was observed with the episodic memory performance variables, and no 1-back measures were chosen by the multivariate divergence analysis. This result may be partly due to the relatively small WM load imposed by the 1-back task; pilot studies indicated that many elderly patients could not perform more difficult versions of the WM task, such as the 2-back. The choice of tasks will be an important issue for future studies aiming at combining neuropsychological and neurophysiological measures to characterize different types of MCI.

The most important neurophysiological marker of impairment was the episodic memory parietal LPC EP amplitude, with a weight of 0.35. The LPC is an established index of memory processes [41,42] whose generators have been localized to the medial temporal and paralimbic areas [43,44]. Prior studies reported that patients with MCI or mild AD had diminished LPC to the repetition of congruous statements [45], but did not link the finding directly with impaired episodic memory performance as is reported here.

The comparison of the elderly subjects with young adults may provide insight into the nature of the LPC amplitude difference observed between the impaired and healthy elderly groups. Whereas the parietal LPC of healthy elderly subjects was larger than that of impaired elderly, and not different than that of young adults, the frontal LPC amplitude of healthy elderly was larger than either that of impaired elderly or young adults. These topographical differences suggest that the episodic memory task LPC may be differentially modulated in normal and pathological aging, and are consistent with prior research suggesting that such increased frontal activation in healthy elderly subjects may indicate a compensatory recruitment of frontal resources in response to the neural changes of normal aging [46,47].

Compared to healthy elderly controls, individuals with AD tend to exhibit increased EEG power in the delta and theta bands, decreased power in the alpha band, and a slower peak alpha frequency [48,49]. Although similar, albeit less robust, effects have been reported in MCI patients [50,51,52], significant differences in such measures were not observed between the impaired and comparison groups in the current study. This may reflect the relatively mild impairment of these subjects and reinforces the point that further studies with larger populations are required to determine the essential discriminating neurophysiological characteristics of mild impairment.

Because of the small number of subjects who were consistently impaired or unimpaired on standard episodic memory tests, it was not possible to match the groups perfectly on all relevant demographic variables, in particular gender. Linear mixed model analyses indicated that none of the 3 variables selected by the exploratory divergence analysis showed a significant interaction between group and gender (p > 0.05 in all cases). Using gender as a covariate, the score and the 3 component variables still differed significantly between the impaired and comparison groups. Because the groups did not differ in other potentially confounding factors such as age or education level, the most prominent neurophysiological difference between the groups (LPC amplitude) seems likely to be indicative of changes in brain function associated with episodic memory impairment.

The value of a combined assessment in future research may well prove to be in evaluating whether a therapeutic treatment has affected the neurocognitive function of an individual, rather than in diagnosing impairment. The results from the example application of the method to a separate study of elderly adults taking the sedating antihistamine diphenhydramine illustrate this potential. After the method identified the measures most sensitive to diphenhydramine in the group as a whole, the group results were used to generate a score for each of the 12 subjects to characterize the magnitude of the drug effect on each person in a single score. The resulting large individual differences in the magnitude of the diphenhydramine response suggest that following further refinement and new exploratory and confirmatory prospective studies, such a combined assessment might facilitate monitoring the effects of therapeutic drugs and help physicians improve patient care for individuals.

In summary, we have shown how a combined cognitive and neurophysiological assessment for the elderly could be derived, and presented an example of its use in pilot exploratory and validation studies. The example data presented are only illustrative of how such information can be combined, and it should not be inferred that the reported equation can be used to characterize amnestic MCI. Promising results were observed when the exploratory equation was applied to the validation analysis, and further studies using an assessment based on this type of combined methodology as an outcome measure in research thus seem worthwhile, particularly in studies evaluating treatments for early-stage cognitive impairment of the elderly, in which the assessment may prove helpful in determining the optimal dosage for individual patients (i.e. personalized medicine). Future research should apply this methodology to more tightly controlled data sets for both the development of a specific equation to characterize MCI and the assessment of MCI patients’ reaction to treatments. Such a combined assessment methodology also could be applied to subjects impaired in specific domains (e.g. amnestic MCI, executive dysfunction) to assess which cognitive and neurophysiological measures best characterize those particular impairments. Regardless of the specific methods and study samples used to derive the equations, the validity of the results must then be tested in prospective studies. With more refinement and validation on larger, better controlled subject samples, some variation of such a combined method may be of practical use to clinicians and researchers alike. More generally, the present results suggest that assessments of neurocognitive function of the elderly that combine direct neurological function measures and cognitive task performance measures merit further exploration.

Disclosure Statement

This research was sponsored by grants from the National Institute of Aging (R44 AG017397), the National Institute of Neurological Diseases and Strokes (R44 NS048815) and the National Institute of Mental Health (R44 MH60053) (A.G., principal investigator). There are no conflicts of interest for any of the authors.