Long-Term and Within-Day Variability of Working Memory Performance and EEG in Individuals

Abstract

Objective

Assess individual-subject long-term and within-day variability of a combined behavioral and EEG test of working memory.

Methods

EEGs were recorded from 16 adults performing n-back working memory tasks, with 10 tested in morning and afternoon sessions over several years. Participants were also tested after ingesting non-prescription medications or recreational substances. Performance and EEG measures were analyzed to derive an Overall score and three constituent sub-scores characterizing changes in performance, cortical activation, and alertness from each individual’s baseline. Long-term and within-day variability were determined for each score; medication effects were assessed by reference to each individual’s normal day-to-day variability.

Results

Over the several year period, the mean Overall score and sub-scores were approximately zero with standard deviations less than one. Overall scores were lower and their variability higher in afternoon relative to morning sessions. At the group level, alcohol, diphenhydramine and marijuana produced significant effects, but there were large individual differences.

Conclusions

Objective working memory measures incorporating performance and EEG are stable over time and sensitive at the level of individual subjects to interventions that affect neurocognitive function.

Significance

With further research these measures may be suitable for use in individualized medical care by providing a sensitive assessment of incipient illness and response to treatment.

1. Introduction

Working memory (WM), the ability to actively sustain attention to a mental representation, is an essential cognitive brain function that underlies many uniquely human cognitive abilities such as language comprehension, reasoning, and planning (Baddeley, 1992). The neural basis of WM ability has been studied extensively using tasks that require individuals to compare a current stimulus with one presented on a prior trial (n-back tasks) (Gevins and Cutillo, 1993; Gevins et al., 1996; Braver et al., 1997; Cohen et al., 1997; Gevins et al., 1997; McEvoy et al., 1998; Jansma et al., 2000; Ravizza et al., 2005). Functional neuroimaging studies have reliably demonstrated that n-back tasks activate frontal lobe areas critical to the control of attention and the maintenance of representations in mind (Jonides et al., 1993; Cohen et al., 1994; McCarthy et al., 1994; Jansma et al., 2000). The magnitude and extent of this activation is directly related to increasing task load (Braver et al., 1997; Druzgal and D’Esposito, 2001), with greater load defined as a larger number of trials over which the representation must be held in mind (e.g. 2-back versus 1-back).

Performance on working memory tasks is affected by a variety of conditions that affect frontal lobe function, including normal aging (McEvoy et al., 2001; Mattay et al., 2006; Schmiedek et al., 2009), schizophrenia (Carter et al., 1998; Perlstein et al., 2003; Elsabagh et al., 2009), head injury (Levin et al., 2004), and ADHD (McCallister et al., 2001; Shallice et al., 2002; Karatekin et al., 2009). Importantly, abnormalities in frontal lobe activation during n-back task performance have been noted even in cases where performance measures were insensitive (McCallister et al., 2001; Callicott et al., 2003). These findings provide a strong rationale for the use of such tasks for monitoring disease or treatment-related changes in executive ability and its neural correlates.

Due to its relatively low cost and high portability, EEG measures are well-suited for monitoring changes in neurocognitive function. These measures are highly sensitive to load manipulations in n-back tasks. Increasing task load is reliably associated with decreases in alpha-band (8-12 Hz) power, indicative of the allocation of greater neuronal resources to maintain performance in higher load tasks (Gevins et al., 1997; Smith et al., 1999; Gevins and Smith, 2000; Meltzer et al., 2007). In young adults, increased task load is also associated with increased frontal midline theta (5-7 Hz) activity, reflecting increased activation of the frontal attentional network (Gevins et al., 1997; Smith et al., 1999; Gevins and Smith, 2000; Jensen and Tesche, 2002; Deiber et al., 2007; Meltzer et al., 2007). WM task-related and resting EEG measures are sensitive to a variety of conditions that affect cognitive function, including mild cognitive impairment, sleep deprivation, and drug or alcohol use (Ilan and Gevins, 2001; Gevins et al., 2002a; Smith et al., 2002; Ilan et al., 2004; Ilan et al., 2005; Kushida et al., 2006; McEvoy et al., 2006; Missonnier et al., 2006; Smith et al., 2006; Meador et al., 2007; Deiber et al., 2009; Hart et al., 2010; Gevins et al., 2011a; Gevins et al., 2011b; Meador et al., 2011).

We recently described an automated method for assessing cognitive function that combines performance measures on n-back WM tasks with simultaneously-recorded EEG measures (Gevins et al., 2011c). The data analysis algorithm integrates response accuracy and speed measures with task-related and resting EEG measures to arrive at an assessment of how an experimental drug or stressor, or a disease and its treatment, has changed an individual’s neurocognitive functional status relative to a baseline state. The test yields an Overall score that combines task performance measures with task-related and resting EEG measures to provide a global metric of a person’s cognitive status. The test’s constituent sub-scores characterize the individual’s performance ability, cortical activation, and alertness.

Results from several well-controlled laboratory studies demonstrated that the test’s component measures are sensitive to the neurocognitive consequences of acute or chronic medication use (Chung et al., 2002; Gevins et al., 2002a; McEvoy et al., 2006; Smith et al., 2006; Meador et al., 2007; Gevins et al., 2011b; Meador et al., 2011), use of recreational drugs or alcohol (Ilan and Gevins, 2001; Ilan et al., 2004; Ilan et al., 2005; Hart et al., 2010) and overnight sleep loss (Smith et al., 2002). This combined behavioral and neurophysiological test of executive function has been used as a primary outcome measure in a recently completed large scale clinical trial of cognitive consequences of continuous positive airway pressure treatment in obstructive sleep apnea (Kushida et al., 2006; Quan et al., 2011).

To be useful in the context of monitoring changes in an individual’s neurocognitive function due to incipient illnesses or their treatment, information on the stability of these measures over at least several years, as well as their sensitivity to normal daily variations in neurocognitive function, is essential. However, research on the long-term variability of cognitive function is limited, as is research on long-term stability of EEG signals. Recently, we demonstrated that the Overall score derived from performance and EEG measures showed low within-day and across session variability, with average standard deviations below 1 for test sessions administered repeatedly to 127 healthy adult subjects in the context of various placebo-controlled formal experiments each spanning several months (Gevins et al., 2011b). For use in detecting clinically significant change within a single individual, test scores must be stable within an individual over longer periods, as well as relatively insensitive to minor, daily fluctuations in cognitive function.

Here we report the results of a study in which a group of healthy adults performed low and high load versions of a spatial n-back working memory task while having their EEG recorded repeatedly over several years. We investigated the within-day variability and long term stability of the Overall score combining performance, task-related EEG and resting EEG measures, as well as the three constituent sub-scores, for individual subjects, and examined the sensitivity of these measures to several common over the counter (OTC) drugs and recreational substances. Our goal was to gather initial evidence about whether these measures are sufficiently stable within individuals to allow tracking of longitudinal changes in an individual’s neurocognitive function consequent to aging, disease or medical treatment.

2. Methods and Materials

2.1 Participants

The sixteen healthy adult participants were scientists, engineers, research associates and administrative staff in our laboratory. Participants were tested repeatedly during a several year period. Ten participants (21-50 years, mean age 32 years, 7 females) had sufficient data (8 test sessions or more) to be included in the analyses of between-day and within-day variability. Data from two other participants (ages 22 and 23, both males) with sufficient drug intervention data, but with insufficient data for inclusion in the normal variability analyses, were included in the analyses of drug effects. There were insufficient data from the other four participants to contribute to either the variability or drug intervention analyses. Participants were light to moderate users of caffeine and alcohol, and had no self-reported history of neurological or psychiatric disorders. One participant was left-handed, and one was a cigarette smoker. Three participants were taking oral contraceptives and two were taking prescription allergy medications. All participation was fully informed and voluntary, and the study was conducted under appropriate institutional review and guidelines for the protection of human subjects.

2.2 EEG Recording

Although the spatial detail of EEG recordings can be substantially increased with use of 100+ electrodes, co-registration with MRI and correction for blur distortion due to conduction through the skull and other tissues (Gevins 1994; Le and Gevins, 1993), the possible benefits of such increased detail comes at considerable cost and effort. Reduced montages are therefore more appropriate in some applications, e.g. repeated tests from an individual over time to gauge effects of diseases and treatments. Accordingly, EEG signals were recorded with a custom-built, stretchable nylon cap with disposable solid-hydrogel electrodes over bilateral and midline dorsolateral prefrontal locations (F3, F4, Fz), midline sensorimotor cortex (Cz), lateral superior parietal cortex (P3, P4), and midline parieto-occipital cortex (POz), referenced to digitally linked mastoids (Gevins et al., 2011a). These locations were selected for their sensitivity to variations in working memory load on the basis of cognitive EEG studies with 40 or 100 electrodes (Gevins et al., 1996). Vertical and horizontal eye movements were monitored by electrodes above and at the outer canthus of each eye. Signals were sampled at 128 Hz and band-pass filtered from 0.1 to 35 Hz.

2.3 Cognitive Testing

EEG was recorded during a test battery consisting of easier and more difficult versions of a spatial n-back working memory (WM) task (Gevins and Cutillo, 1993), as well as during resting. In the WM task a dot stimulus was displayed for 200 ms in one of six positions on each trial with a mean inter-stimulus interval of 4 s (range 3500 – 4500 ms). In the more difficult (high-load) condition, the participant had to decide whether the current dot was the same as or different than the dot that appeared two trials before (2-back). In the easier (low-load) version of the task, the position of the current dot was compared with the position of the dot on the first trial of each block of 50 trials (0-back). Resting EEG was recorded for 90 s each in eyes-open and eyes-closed conditions, followed by approximately 3.5 min blocks of 50 trials each for the low and high-load WM task. (See Gevins et al., 2011b for a more detailed description.)

2. 4 Subjective Ratings

At the end of each task battery, participants completed a set of three subjective ratings scales pertaining to their state during the battery. On a scale of one to nine (one signifying a high level and nine signifying a low level), participants were asked to rate how alert they felt, their ability to concentrate, and how motivated they were to do the test.

2.5 Procedures

Prior to the study, participants were asked to choose the morning time (10am or 11am) at which they usually felt more alert and the afternoon time (2pm or 3pm) at which they usually felt more tired. Participants were then tested at their selected times, generally on the same days each week throughout the study. Since the purpose of the study was to measure the normal day-to-day variability in WM performance and in WM-associated and resting EEG measures, no restrictions were placed on participants’ daily activities including their usual consumption of alcohol or caffeine.

Table 1 summarizes the schedule of participation for each participant included in the within- and between-day variability and drug intervention analyses. For the first six month period, testing was scheduled to occur twice a week, once in the morning and once in the afternoon. Ten participants completed at least 18 morning and 18 afternoon sessions. During the second six month period, 8 of the 10 participants continued to be tested at a reduced rate of twice per month. During the third six month period, 7 of the 10 participants were tested once per month. Four of the 10 participants were tested once every three months over the following ~2.5 years. Preliminary analyses indicated that performance and EEG measures did not differ between the periods with differing testing frequencies. Two new participants who contributed data to the drug intervention analyses but not to the within and between-day analyses were enrolled during the period (T4 in Table 1) of quarterly testing.

Table 1.

Participant testing schedule for normal variability and drug intervention sessions for the 12 participants whose data were analyzed in the current study. The first 10 participants (CM01 through CM11) contributed data to the within and between-day variability analyses. All 12 contributed data to drug intervention analyses. During the first six-month period (T1), participants were tested twice per week (including one morning and one afternoon session) and participated in one or more drug interventions. Testing continued with two sessions per month in the next six-month period (T2); one session per month in the following six-month period (T3). After that, some participants continued to participate, with one session per quarter for an additional 2.5 years (T4). Two new participants (cm14 and cm15) enrolled in the study during this period.

Participant ID	T1 (2x/week) 6-months	T2 (2x/month) 6-months	T3 (1x/month) 6-months	T4 (1x/quarter) 2.5 years	Alcohol	Diphenhydramine	Caffeine	Marijuana
cm01	x	x	x		x	x	x	x
cm02	x	x			x	x	x
cm03	x	x	x	x	x	x	x	x
cm04	x	x	x		x	x	x
cm05	x				x	x
cm06	x	x	x	x		x	x
cm07	x				x	x	x	x
cm08	x	x	x	x		x	x
cm10*	x	x	x				x	x
cm11	x	x	x	x			x	x
cm14				x	x	x		x
cm15				x	x	x	x

^*A large (> 15) number of sessions from CM10 were excluded due to self reports of not feeling well.

2.5.1 Drug Interventions

Participants were able to choose to participate in one or more drug interventions; or to decline to participate in any drug intervention, without affecting their participation in the overall study. For the ten participants who were tested in the first six-month period according to the twice-a-week schedule, drug interventions started after one month of study participation. Only one drug dose was administered in a one week period. For the two participants who enrolled during the period of quarterly testing, drug interventions started after one year of quarterly tests, with drug interventions occurring once per month thereafter. All drug test sessions occurred in the early afternoon. Timing of drug administration was based on published time of maximum plasma concentration (t-peak times) for each substance such that the first post-drug testing interval occurred at approximately the time of t-peak (Ohlsson et al., 1980; Blanchard and Sawers, 1983; Blyden et al., 1986; Fraser et al., 1995). Drug administration was scheduled such that the first post-drug interval occurred at the same time as the participant’s usual non-drug afternoon test session.

2.5.2 Alcohol

Eight participants, light to moderate users of alcohol (1-12 drinks per week), completed the alcohol intervention. Participants were tested in low and high dose sessions, the former designed to raise Blood-Breath Alcohol Content (BBAC) to 0.04 grams per 210 liters of breath and the latter to 0.08 (the legal driving limit in California). The amount of alcohol was based on each participant’s weight, and was mixed with an 8 oz beverage of their choice. Dose order was randomized across participants; and participants were blind to dose. On each alcohol test day, participants were tested before drinking and at two intervals occurring 1 and 2.5 hours following ingestion.

2.5.3 Diphenhydramine

Ten participants completed the diphenhydramine intervention. Participants were tested in high (50mg) and low (25mg) dose conditions, with order randomized across subjects, and subjects blinded to dose. Participants were tested before ingestion, and at two intervals occurring at 2.5 and 3.5 hours following ingestion.

2.5.4 Caffeine

Ten participants completed the caffeine intervention. Participants were moderate users of caffeine, who reported consuming between 1 and 4 cups of coffee (~50 mg to ~200 mg of caffeine) per day. All participants who normally consumed caffeine in the morning consumed their regular dose of caffeine prior to 10:00 AM on this test day. Following ingestion of a pill containing 300 mg of caffeine, participants were tested three times, at intervals beginning 1, 2 and 3.5 hours post ingestion.

2.5.5 Marijuana

Six participants, all of whom had smoked marijuana previously, completed the marijuana intervention. Participants inhaled cigarettes supplied by the U.S. government (3.45% THC) according to pacing instructions presented on a computer monitor that specified six puffs at one minute intervals with each puff lasting 1.5 s and with breath holding for 8.5 s (Ilan et al., 2004). Although it is not possible to equate drug dose precisely when marijuana is smoked due to the wide variety in Δ⁹-THC absorption rates across individuals, this timed procedure holds important smoking parameters such as breath-hold duration and inter-puff interval relatively constant across conditions and participants. Participants were tested before smoking and at two post-smoking intervals occurring approximately 0.25 and 1.5 hours after smoking.

2.6 Data Analysis

Data from each test session were uploaded over the internet to a central data analysis server. Automated algorithms detected and removed artifacts due to eye movements and blinks, scalp muscle activity, head and body movements, and bad electrode contacts (Du, 1994), and all raw and decontaminated data (and EEG spectra) were visually inspected. Power spectral estimates were computed on the decontaminated EEG data by averaging 2 sec periodograms time-locked to the stimulus during the task or over the resting condition (no stimulus) and combining them into theta, alpha and beta frequency band variables.

2.6.1 Computation of Overall Score and Three Sub-Scores

EEG and task performance parameters were combined into an Overall score, indicating the degree of change in a participant’s state from her or his normal baseline score. Computation of the Overall score and the three sub-scores is described in detail in Gevins et al., 2011b and is briefly summarized here. Each participant’s baseline was set to an average of all their morning and afternoon sessions, excluding those sessions in which they ingested drugs. In addition any session in which a participant reported not feeling well, such as having a headache, was excluded from the baseline. The Overall score was computed as the mean of three sub-scores, one based on WM task performance measures and two based on EEG measures (cortical activation and alertness sub-scores). The Overall score, and the performance, activation, and alertness sub-scores, expressed in standard deviation units, indicated whether the participant’s scores for a particular test session were lower or higher than his or her baseline scores.

The performance, activation, and alertness sub-scores each consisted of a combination of several individual variables, which were expressed in standard deviation units, such that the variables, resulting sub-scores and Overall score were all on the same scale independent of the original units of measurement. The performance sub-score indicated how well the participant performed the WM tasks relative to that participant’s baseline performance. It was computed as the mean of three measures: accuracy versus reaction time in the low load WM task (a measure of the speed-accuracy trade-off in the easier task version); accuracy in the high load task; and a measure of mean reaction time relative to reaction time variability in the high load task. Higher scores on all three measures indicated better performance.

The cortical activation sub-score reflected the difference between the current test and the baseline in the divergence (Smith et al., 2001) between EEG power spectral variables measured during performance of the easier and more difficult WM task. The divergence reflects the degree to which large cortical neuronal populations were recruited to perform the more difficult version of the WM task relative to the easier task. The cortical activation sub-score was computed such that a positive score indicated a larger neuronal population mediating performance of the high load WM task than that mediating high load task performance during the baseline. The divergence metric was calculated by first computing power in the theta, alpha, and beta frequency bands within each 2-sec window for each task for three frontal and three parietal channels. Then, for the frontal and parietal regions separately, a multivariate divergence analysis (Gevins et al., 2011b) selected the subset of four EEG power variables that maximized the multivariate distance between low- and high-load tasks in the baseline dataset, producing the greatest differentiation between the low and high-load tasks. The cortical activation score was then computed as the mean of the resulting frontal and parietal divergence measures.

The candidate divergence variables were limited to those known a priori to be sensitive to task-load modulation and less influenced by drowsiness or drug effects. For example, alpha power and beta power were used only if they were larger in the easier task than in the more difficult task; theta was only used if power was larger in the more difficult task than in the easier task (Gevins et al., 2011b). For each feature meeting these criteria, the magnitude of the difference between task load levels was computed and entered as a candidate variable into the divergence analysis. If no features met criteria (for example, if theta was consistently higher in the low load task than the high load task or alpha was consistently higher in the high load task than in the low load task), the cortical activation index was not computed for that participant. In that case, the Overall Score then reflected the mean of the performance and alertness sub-scores only. In the current analyses, the cortical activation sub-score could be computed on all participants.

The alertness sub-score was a neurophysiological measure that indicated how the participant’s alertness during the current test session differed from his or her baseline. The alertness sub-score was computed as the mean of three well-established neurophysiological markers of alertness (Davis et al., 1937; Matousek and Petersen, 1983; Oken and Salinsky, 1992; Makeig and Jung, 1995): a ratio of alpha band power between eyes closed and eyes open resting conditions, slow horizontal eye movement activity during the eyes-open resting condition, and theta band EEG power during the eyes-open resting condition.

The performance, cortical activation, and alertness sub-scores were averaged to form the Overall score, after application of three rules designed to more accurately reflect the association between measures of brain function and task performance (Gevins et al., 2011b). One rule addressed the possible dissociation between performance and alertness sub-scores: if performance decreased from baseline while alertness increased, the sign of the alertness sub-score was inverted because increased alertness is not beneficial if performance is impaired. The other two rules addressed possible dissociations between performance and cortical activation sub-scores: if performance increased from baseline while cortical activation decreased, the sign of the cortical activation sub-score was inverted, the rationale being that less cortical activation in the presence of better performance reflected less effort to perform better. Alternatively, if performance decreased from baseline while cortical activation increased, the sign of the cortical activation sub-score was inverted, the rationale being that greater cortical activation in the presence of worse performance reflected a state in which the individual was trying harder but performing worse.

2.6.2 Statistical Analysis

The degree to which neurocognitive function changed from day to day over an extended period was assessed by computing the variability in the Overall score and three sub-scores across all morning and afternoon sessions without a drug intervention. Descriptive statistics were used to assess circadian and drug effects. For the former, the mean and variability of scores for morning and afternoon sessions across participants were compared with paired samples t-tests. To examine individual differences in circadian variation, each participant’s morning and afternoon scores were compared with independent sample t-tests. To assess correlation of the Overall Score and each of the three sub-scores to subjective alertness, motivation and concentration levels, Pearson correlations were computed. To assess individual responses to drugs, a participant’s Overall score at the post-drug interval corresponding to the published t-peak was standardized with respect to the non-drug sessions and significance was assessed using the normal distribution.

3.0 Results

3.1 Long-term Variability

Normal between-day variability was computed from all sessions without a drug intervention. The number of sessions per participant ranged from 37 to 69 (mean 52, sd 13); with duration of follow-up ranging from 6 to 60 months (mean 31, sd 22). As expected, the mean of each participant’s Overall scores and her or his performance, activation, and alertness sub-scores were all approximately zero. The standard deviation of each participant’s Overall score was less than 0.6 for all subjects. The standard deviations of the three sub-scores were all less than 1 (Table 2).

Table 2.

Normal Between-Day Variability of Overall Score and the Three Sub-Scores for Individual Participants and Group. Entries are mean (top) and standard deviations (bottom) of scores. Number of test sessions is shown in top row (top table). Group means (M) and standard deviations (SD) are shown in the rightmost two columns.

Mean	CM01	CM02	CM03	CM04	CM05	CM06	CM07	CM08	CM10	CM11	Group (M)	Group (SD)
Number of tests	51	37	69	52	39	68	39	64	40	63	52	13
Overall score	0.00	0.01	-0.05	-0.01	0.00	0.01	0.00	-0.02	0.11	-0.01	0.01	0.04
Subscores:
Performance	0.03	0.05	0.02	0.02	0.02	0.05	0.09	0.02	0.22	0.07	0.06	0.06
Activation	0.00	0.07	0.01	0.00	0.00	0.02	0.00	0.00	0.06	-0.01	0.02	0.03
Alertness	-0.04	-0.03	-0.07	-0.04	-0.07	-0.03	-0.06	-0.02	-0.04	-0.01	-0.04	0.02
SD	CM01	CM02	CM03	CM04	CM05	CM06	CM07	CM08	CM10	CM11	Group (M)	Group (SD)
Overall score	0.32	0.36	0.31	0.41	0.29	0.27	0.22	0.29	0.57	0.46	0.35	0.11
Subscores:
Performance	0.41	0.31	0.40	0.49	0.25	0.43	0.35	0.31	0.84	0.50	0.43	0.17
Activation	0.63	0.80	0.41	0.74	0.74	0.46	0.40	0.63	0.82	0.66	0.63	0.16
Alertness	0.45	0.40	0.57	0.42	0.36	0.33	0.29	0.56	0.38	0.72	0.45	0.13

3.2 Morning vs. Afternoon Variability

Time of day effects were assessed by computing the variability separately for all morning and afternoon sessions without drug interventions. Across the group, mean Overall scores, and performance and alertness sub-scores were lower in the afternoon than in the morning (p’s < 0.05) (Table 3, top, right most column). Participants were also more variable in the afternoon: the standard deviations of the Overall score and activation sub-score were greater in the afternoon than in the morning (p’s < .05) (Table 3, bottom, right most column). Afternoon performance scores were marginally more variable than in the morning (p = .055). Independent samples t-tests comparing each participant’s AM and PM scores (Table 3, top, asterisks within each participant’s column) revealed that morning and afternoon sessions differed for one participant’s Overall score, three participants’ performance sub-scores, and two participants’ activation sub-scores, while alertness scores did not differ for any of the participants. Four of the ten participants’ Overall scores, four participant’s performance sub-scores, and two participants’ alertness sub-scores were more variable in the afternoon than in the morning (Table 3, bottom, asterisks within each participant’s column). Variability of activation scores did not differ between morning and afternoon for any of the individual participants.

Table 3.

Morning (AM) vs. Afternoon (PM) Variability of Overall Score and the Three Sub-Scores for Individual Participants and Group. Top panel shows mean scores and bottom panel shows standard deviations (SD). Group means (M) and standard deviations (SD) and the significance of differences between AM and PM values across the group are shown in the three right columns. Asterisks within a participant’s column show the significance of the difference between his or her AM and PM scores.

		CM01	CM02	CM03	CM04	CM05	CM06	CM07	CM08	CM10	CM11	Group (M)	Group (SD)	Significance
Mean	# Sessions (AM, PM)	25, 26	18, 19	34, 35	29, 23	17, 22	30, 38	17, 22	27, 37	22, 18	31, 32
Overall score	AM	-0.02	0.02	-0.01	0.20***	-0.04	0.03	0.07	0.04	0.22	0.09	0.06	0.09	p<.05
	PM	0.02	0.01	-0.09	-0.19	0.03	0.00	-0.05	-0.06	-0.06	-0.12	-0.05	0.07
Performance sub-score	AM	0.06	0.09	0.15**	0.27***	-0.04	0.07	0.18	0.11	0.47*	0.09	0.14	0.14	p<.05
	PM	0.01	0.02	-0.10	-0.22	0.06	0.03	0.02	-0.04	-0.14	0.05	-0.03	0.09
Activation sub-score	AM	-0.13	0.05	0.04	0.24*	-0.06	0.02	0.07	0.03	0.04	0.18*	0.05	0.11	N.S.
	PM	0.12	0.09	-0.02	-0.23	0.05	0.03	-0.06	-0.02	0.09	-0.22	-0.02	0.12
Alertness sub-score	AM	0.05	-0.06	-0.16	0.03	0.01	0.04	0.00	0.09	0.05	0.15	0.02	0.08	p<.05
	PM	-0.13	-0.01	0.02	-0.11	-0.12	-0.09	-0.10	-0.10	-0.14	-0.18	-0.10	0.06
SD		CM01	CM02	CM03	CM04	CM05	CM06	CM07	CM08	CM10	CM11	Group (M)	Group (SD)	Significance
Overall score	AM	0.25*	0.33	0.27	0.23**	0.26	0.26	0.15*	0.28	0.43*	0.40	0.29	0.08	p<.01
	PM	0.39	0.39	0.34	0.44	0.31	0.29	0.25	0.30	0.73	0.50	0.39	0.14
Performance sub-score	AM	0.27**	0.30	0.29*	0.33*	0.30	0.41	0.29	0.33	0.59*	0.49	0.36	0.10	N.S.
	PM	0.52	0.32	0.45	0.51	0.21	0.45	0.38	0.29	1.04	0.52	0.47	0.22
Activation sub-score	AM	0.57	0.79	0.36	0.62	0.61	0.43	0.38	0.69	0.69	0.59	0.57	0.14	p<.05
	PM	0.66	0.83	0.46	0.77	0.83	0.48	0.41	0.60	1.00	0.67	0.67	0.19
Alertness sub-score	AM	0.43	0.34	0.53	0.33	0.45*	0.31	0.31	0.43*	0.41	0.71	0.43	0.12	N.S.
	PM	0.46	0.45	0.60	0.49	0.26	0.34	0.27	0.64	0.31	0.70	0.45	0.16

^*p<.05;

^**p<.01;

^***p<.001

3.3 Correlation of Overall Score and the Three Sub-scores with Subjective Measures

The top part of Table 4 shows the correlation coefficients for each participant between the Overall score and three subjective rating scales of neurocognitive status during the test (drowsiness, concentration and motivation). There were significant negative correlations between Overall score and one or more subjective measures for four of the ten participants, where a worse Overall score was associated with a poorer self assessment. By contrast, for participant CM06, the significant correlations of the Overall score were positive for concentration and motivation self assessments, meaning that a better overall score was associated with worse perceived concentration and motivation for this participant. The lower panels of Table 4 show correlation coefficients for each participant between each of the three sub-scores and the three subjective rating scales. The performance sub-score was significantly negatively correlated with one or more subjective measures for seven of the ten participants indicating worse objective performance was associated with greater subjective difficulty in ability to function. Again, CM06 was an exception, having significant positive correlations. There was a significant correlation between the cortical activation sub-score and the subjective assessments for only two of the participants. Three participants had significant correlations between the alertness sub-score and their self-assessment of drowsiness (with increased perceived drowsiness associated with decreased physiological alertness), while three participants had significant negative correlations between the alertness sub-score and their perceived level of motivation.

Table 4.

Correlations for Each Participant between Overall Score and Each Sub-score with Three Subjective Measures (Drowsiness, Concentration and Motivation).

Pearson Correlation between	Subjective Scales	CM01	CM02	CM03	CM04	CM05	CM06	CM07	CM08	CM10	CM11
Overall score and	Drowsiness	0.075	-0.264	-0.134	-0.358**	-0.020	0.201	-0.077	-0.361**	-0.346*	-0.410**
	Concentration	-0.118	0.007	-0.211	-0.089	0.270	0.461***	-0.100	-0.405**	-0.331*	-0.101
	Motivation	-0.172	-0.267	-0.233	-0.172	0.138	0.465***	-0.086	-0.335**	-0.562***	-0.014
Performance sub-score and	Drowsiness	-0.083	-0.443**	-0.313**	-0.418**	0.040	0.220	-0.154	-0.264*	-0.339*	-0.247
	Concentration	-0.315*	0.036	-0.447***	-0.259	-0.161	0.333**	-0.331*	-0.425***	-0.271	-0.162
	Motivation	-0.126	-0.234	-0.515***	-0.428**	-0.016	0.394***	-0.219	-0.161	-0.505**	0.065
Activation sub-score and	Drowsiness	0.173	-0.103	0.060	-0.195	-0.232	0.068	0.171	0.119	-0.201	-0.182
	Concentration	0.202	0.064	-0.004	0.029	0.300	0.260*	0.139	0.049	-0.368*	-0.049
	Motivation	0.027	-0.316	0.035	-0.052	0.041	0.166	-0.024	0.087	-0.530**	-0.053
Alertness sub-score and	Drowsiness	-0.083	0.150	0.032	-0.196	0.224	-0.159	0.124	-0.596***	-0.324*	-0.530***
	Concentration	-0.233	-0.042	0.161	0.100	0.095	0.106	-0.004	-0.366**	-0.283	-0.177
	Motivation	-0.403**	0.116	0.264*	0.242	0.169	0.100	0.123	-0.404**	-0.367*	-0.122

^*p<.05;

^**p<.01;

^***p<.001.

3.4 Effects of Drugs on Overall Score and the Three Sub-scores

Sensitivity of the Overall score and the three sub-scores to the effects of common over-the-counter drugs and recreational substances was assessed at each drug’s published t-peak time (Figure 1). For each drug, the first post-drug interval corresponded to the t-peak. In Figure 1, it is evident that the magnitude of the effect differed across the drugs and doses, with the larger dose of alcohol producing a consistent decrease in the Overall score and sub-scores, and with marijuana having the largest negative impact on the performance sub-score. The Overall score and sub-scores on drug were compared via independent samples t-tests to the Overall score and sub-scores from participants’ non-drug sessions. Participants were impaired by the high dose but not the low dose of alcohol, as Overall scores and all three sub-scores on the high dose were lower. Both high and low doses of diphenhydramine negatively affected participants’ Overall scores and performance sub-scores, but did not affect activation or alertness sub-scores. Caffeine did not affect the Overall score or any of the sub-scores. Marijuana did not affect Overall scores, but the performance sub-score was lower and the alertness sub-score increased across participants.

Fig. 1.

Effects of Drugs on Overall Score and the Three Sub-scores. *p<.05, **p<.01, ***p<.001 refer to significance of difference between the drug t-peak and participants’ non-drug sessions as tested with a two-sided t-test. Bars are standard errors.

The magnitude of the Overall neurocognitive response to a drug and dose varied among participants (Figure 2 and Table 5). For instance, although Overall scores were lower after the high dose of alcohol, the decrement was significant for only 6 of the 9 participants. While CM03’s and CM05’s response to alcohol was greater than a standard deviation below their non-drug sessions (p<.001), CM01’s, CM02’s and CM04’s Overall scores were not affected (p’s>.05). There was more variability across participants in response to diphenhydramine. Three scored significantly worse than their average baseline score after ingesting the high dose, (one of these participants also scored worse than baseline following ingestion of the low dose) while the 7 other participants were not affected by either dose. None of the 10 participants showed a significant response to caffeine. Half of the individuals who participated in the marijuana session showed significantly lower overall scores, whereas the others were unaffected.

Fig. 2.

Individual Participant and Group Effects of Drugs on Overall Scores at t-peak of (a) alcohol, (b) diphenhydramine, (c) caffeine, and (d) marijuana. *p<.05, **p<.01, ***p<.001 refer to significance of difference between the drug t-peak and non-drug sessions. Difference between group drug scores and non-drug sessions are tested with a two-sided t-test. Difference between an individual’s drug score and their non-drug sessions is assessed by the normally distributed variable z, and significance is assessed using the normal distribution. Group mean bars are standard errors.

Table 5.

Effect of Drugs at t-peak on an Individual’s Overall score. Entries are Overall scores. Asterisks indicate significance of effect.

Participant	Alcohol 0.04 % BAC	Alcohol 0.08 % BAC	Diphenhydramine 25mg	Diphenhydramine 50mg	Caffeine 300mg	Marijuana (6 puffs)
CM01	0.26	-0.27	-0.18	-0.23	-0.28	0.03
CM02	-0.16	-0.37	0.02	0.11	0.10	--
CM03	-0.77*	-1.17***	-0.61	-0.87**	0.25	-0.85**
CM04	0.11	-0.49	-0.27	0.21	0.65	--
CM05	-0.15	-1.06***	-0.04	-0.03	--	--
CM06	--	--	-0.29	-0.67*	0.28	--
CM07	-0.18	-0.70**	-0.52	-0.01	-0.25	-0.19
CM08	--	--	0.40	0.08	0.01	--
CM10	--	-1.13*	--	--	0.79	-0.71
CM11	-0.53	--	--	--	-0.69	-1.29**
CM14	-0.20	-0.50**	-0.93***	-1.46***	--	-0.64***
CM15	-0.35	-0.72*	0.07	-0.55	-0.53	--

^*p<.05;

^**p<.01;

^***p<.001

4.0 Discussion

The results of this study show that a test combining EEG measures with working memory task performance measures offers promise in the ability to track changes in an individual’s cognitive status over time. The Overall score, a combined metric of performance ability, cortical activation, and alertness, showed high long-term stability within subjects. It was generally robust to within-day fluctuations in cognitive function yet sensitive to effects of acute commonly used psychoactive substances, suggesting that it could be suitable for use in monitoring changes in cognition over time due to chronic disease or long term treatment.

In repeated testing sessions occurring over a several year period, both the Overall score and component sub-scores showed low variability. The Overall score was also generally robust to within-day fluctuations in cognition although variability in this score was greater in the afternoon sessions for all ten subjects. Across the group, performance was better in the morning than in the afternoon session, an effect that was significant in three individuals. In group analysis, alertness was also higher in the morning, but no individual subjects showed a significant difference in alertness scores between morning and afternoon sessions. Given the small number of subjects in this study, caution is warranted in generalizing these time of day findings to the wider population. It is quite possible that opposite time of day effects (e.g. better performance in the afternoon than in the morning) could be observed in other subjects.

In contrast to the relative insensitivity of the Overall score to within-day variation in cognitive function, the Overall score was sensitive to changes in cognition induced by common OTC medications or recreational substances. The group-level effects of these substances observed here are consistent with our prior placebo-controlled studies that showed significant negative effects of alcohol, diphenhydramine, and marijuana (Ilan and Gevins, 2001; Gevins et al., 2002a; Ilan et al., 2004; Ilan et al., 2005; Gevins et al., 2010; Hart et al., 2010). In contrast to our prior findings of a small but significant improvement in overall score after caffeine consumption (Gevins et al., 2010), no significant effects of caffeine were noted for any of the participants in the current study. This may simply reflect the fact that all participants in the current study consumed coffee daily. Additionally, in the prior study, participants were taxed with a longer, more demanding task battery, and this could have increased fatigue. Caffeine effects are more pronounced in fatigued than in alert individuals (Lorist et al., 1994).

In general, subjective assessment of drowsiness, motivation, and level of concentration correlated poorly with the Overall score and with its constituent subscores, although there were large individual differences. Much prior research has shown high correlation between subjective ratings of sleepiness and EEG measures of alertness (Akerstedt and Gillberg, 1990; Kaida et al., 2006; Kaida et al., 2007), however such studies have typically involved drowsiness-inducing manipulations, such as sleep deprivation or lengthy performance of monotonous tasks. Our own prior research with drowsiness-inducing manipulations has shown good correlation between EEG measures of alertness used here and subjective sleepiness (Gevins et al., 2002b; Smith et al., 2002; McEvoy et al., 2006). The lack of correlation between objective and subjective measures of sleepiness observed here may arise from the more subtle degrees of alertness change that occur during a typical day in healthy adults. More consistent associations were observed, however, between the performance sub-score and self assessment, with 7 of the 10 subjects showing significant association between worse performance and worse cognitive state (whether attributed to decreased alertness, less concentration, or less motivation). This suggests that although individuals had insight into how well they were likely to perform the tasks (the subjective assessment scales were administered prior to the task battery) they varied in their perception of the factors underlying their performance.

Although the results presented here provide strong preliminary evidence in support of the potential usefulness of a combined neurophysiological and performance based test of neurocognitive function for long term monitoring of neurocognitive status, it should be noted that neither the tests incorporated in the current battery, nor the analysis procedures described here are necessarily optimal for monitoring change due to age, chronic illness, or treatment. While executive functions such as working memory decline with age (Reuter-Lorenz and Sylvester, 2005), a battery that also encompassed tests of episodic memory (Gevins et al., 2011a) might be preferable to help distinguish normal aging from early Amnestic Mild Cognitive Impairment, the usual precursor to the most common form of dementia, Alzheimer’s disease. Other neuropsychological tests might be more sensitive to other illnesses or chronic medications.

In the analyses presented here, the Overall score was derived in comparison to the average of all non-drug sessions, and reflected the degree to which the current session differed from that subject’s average. Other baselines could also be used, depending on the application. For example, the current session could be compared to an initial session, or set of sessions, to plot trajectories of change over time. Or the current session could be compared to the most recent session or sessions to investigate acute effects of a medication or condition. With the automated analyses employed here, one could efficiently examine scores compared with multiple types of baselines.

Similarly, the neurophysiological signals of interest could also be modified dependent on anticipated effects of a medication or disease condition. In the current study, neurophysiological features were restricted to spectral EEG measures, but latency and amplitudes of event-related potentials could also be included for more sensitive detection of alterations in attention, memory, semantic processing or decision-making (Gevins, 2010; Gevins, 2011a).

Limitations of the current study include the relatively small sample size, and the fact that not all subjects were tested according to an identical testing schedule. Nevertheless, the results showed that the Overall score and the constituent sub-scores derived from EEG signals obtained during performance of working memory tasks are stable over long time periods and robust to minor daily variations in cognitive function. They are, however, sensitive enough to factors that affect cognition to yield useful information at the level of individual people. With further development and validation in patient populations, tests that combine neurophysiological measures of attention and alertness, with task-based performance measures may someday prove useful in individualized medical care.

Highlights.

• An objective test of working memory combining EEG with task performance measures provides a measure of an individual’s neurocognitive function with high stability over several years.

• This test is relatively insensitive to normal fluctuations in neurocognitive function but is sensitive, at the individual level, to alterations in neurocognitive function consequent to drug ingestion.

• With further development, this test may prove useful in individualized medical care by providing a sensitive assessment of incipient illness and response to treatment.