Multicenter validation of a bedside antisaccade task as a measure of executive function

Abstract

Objective:

To create and validate a simple, standardized version of the antisaccade (AS) task that requires no specialized equipment for use as a measure of executive function in multicenter clinical studies.

Methods:

The bedside AS (BAS) task consisted of 40 pseudorandomized AS trials presented on a laptop computer. BAS performance was compared with AS performance measured using an infrared eye tracker in normal elders (NE) and individuals with mild cognitive impairment (MCI) or dementia (n = 33). The neuropsychological domain specificity of the BAS was then determined in a cohort of NE, MCI, and dementia (n = 103) at UCSF, and the BAS was validated as a measure of executive function in a 6-center cohort (n = 397) of normal adults and patients with a variety of brain diseases.

Results:

Performance on the BAS and laboratory AS task was strongly correlated and BAS performance was most strongly associated with neuropsychological measures of executive function. Even after controlling for disease severity and processing speed, BAS performance was associated with multiple assessments of executive function, most strongly the informant-based Frontal Systems Behavior Scale.

Conclusions:

The BAS is a simple, valid measure of executive function in aging and neurologic disease.

The ability to inhibit prepotent or reflexive impulses and biological drives is an important function of the frontal lobes and is impaired in a variety of brain diseases.¹ A commonly used measure of cognitive inhibition is the antisaccade (AS) task, which requires suppression of a visually guided saccade toward a target and generation of voluntary saccade in the opposite direction.^2,3 The percentage of correct AS responses is sensitive to frontal lobe dysfunction in aging,⁴ schizophrenia,^5–8 Huntington disease (HD),^9,10 and a variety of neurodegenerative dementias.^11–13

Despite the potential clinical utility of the AS task in evaluating neurologic function, current methods of evaluation are limited to informal bedside evaluation or the use of expensive eye-tracking equipment.¹⁴ Bedside AS evaluation as part of a clinical neurologic examination makes quantification difficult and does not include accurately controlled stimulus presentation, limiting its use in scientific studies. Oculographic equipment necessitates significant financial resources and expertise and limits assessments to specialized laboratories.

A simple, inexpensive, and well-validated bedside AS (BAS) task requiring only a standard laptop computer would be a useful clinical tool to identify and monitor the progression of disorders involving frontal-subcortical and executive dysfunction.¹⁵ The goals of this study were therefore to 1) validate the BAS relative to AS measured using an infrared eye tracker, 2) determine whether the BAS is more sensitive to neuropsychological measurements of executive function as compared to other neuropsychological domains, and 3) validate the BAS relative to real-world assessments of executive function in a multicenter study.

METHODS

Subjects.

UCSF.

A total of 103 subjects were evaluated at the University of California, San Francisco (UCSF) Memory and Aging Center. Subjects received a neurologic examination and neuropsychological testing within 6 months of BAS testing. Subjects were classified as follows: cognitively normal elder (NE), Alzheimer disease (AD), frontotemporal dementia (FTD), mild cognitive impairment (MCI), or progressive supranuclear palsy (PSP). NE subjects had no neurologic complaints, normal neurologic and neuropsychological examinations, and Clinical Dementia Rating (CDR)¹⁶ scores of 0. Individuals with MCI had a memory or other cognitive complaint corroborated by an informant (CDR ≥ 0.5). Subjects with AD met National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer's Disease and Related Disorders Association criteria for probable AD¹⁷; subjects with FTD met the Neary criteria for frontotemporal dementia, semantic dementia, or progressive nonfluent aphasia¹⁸; and subjects with PSP met National Institute of Neurological Disorders and Stroke Society for Progressive Supranuclear Palsy criteria for probable PSP.¹⁹ Three patients were diagnosed with corticobasal syndrome²⁰ and included in the FTD group. A subset of subjects (14 NE, 6 AD, 8 frontotemporal lobar degeneration, 2 MCI, and 3 PSP subjects) received AS evaluation on a laboratory infrared eye tracker.

Multicenter.

To assess the validity of the BAS as a measure of executive function in a multicenter study, 6 sites (Mayo Clinic, Rochester; UC Berkeley; UC Davis; UCSF; University of Iowa; and University of Texas Southwestern) assessed 397 adults (mean age 58.4, range 18–91) according to standard procedures and diagnosed subjects as either cognitively normal or as having a neurologic disease (table e-1 on the Neurology® Web site at www.neurology.org). The data were acquired as part of a multicenter validation study of a novel, National Institute of Neurological Disorders and Stroke–funded, executive function battery, entitled Executive Abilities: Methods and Instruments for Neurobehavioral Evaluation and Research (EXAMINER; http://examiner.ucsf.edu/index.htm) that included the BAS. The cohort included individuals with MCI and dementia, as listed above, as well as patients with frontal and nonfrontal focal brain lesions. Patients with Huntington disease (HD) were symptomatic gene carriers, patients with multiple sclerosis (MS) met McDonald criteria,²¹ patients with Parkinson disease (PD) met Albanese criteria,²² and patients with traumatic brain injury (TBI) had a history of moderate to severe injury (Glasgow Coma Scale <12 at time of injury). BAS data were not collected on 32 out of 429 subjects (7.4%) from the EXAMINER study due to factors related to subjects' physical or mental capacity.

Standard protocol approvals, registrations, and patient consents.

All procedures were approved by the UCSF or local institutional review board. Written informed consent was obtained from all subjects or from a surrogate if cognitive impairment prevented individuals with dementia from providing informed consent.

Neuropsychological testing.

At UCSF, general intellectual function was assessed using the Mini-Mental State Examination (MMSE).²³ Memory was assessed with a 10-minute recall of the Benson figure. Visuospatial abilities were tested by copying the same Benson figure, the Number Location condition from the Visual Object Spatial Perception (VOSP) battery,²⁴ and the Face Matching and Affect Matching conditions of the Comprehensive Affect Testing System (CATS). Language skill was evaluated by a 15-item Boston Naming Test (BNT),²⁵ phonemic fluency (number of D- or H-words in 1 minute), and category fluency (number of animals in 1 minute). Neuropsychological tests of executive function assessed set-shifting (time to make all correct lines in a modified trail-making test),²⁶ inhibition (the Stroop interference condition),²⁶ generation (design fluency subtest from the Delis-Kaplan Executive Function System; filled dots condition, number correct),²⁷ and working memory (longest correct backward digit span),²⁸ as well as the ability to perform 5 arithmetic calculations and interpret 3 similarities and 3 proverbs (abstraction).

Neuropsychological performance was assessed as part of the EXAMINER battery in the multicenter cohort. Tests included the Digit Symbol Subscale (number of correct responses),²⁸ Flanker task,²⁹ Stroop Interference (number correct),²⁶ 1-back task (number correct), Continuous Performance Task (Go/NoGo task; number correct), verbal fluency (number of correct F- and l-words in 1 minute), category fluency (number of correct animals and vegetables in 1 minute), and generation (design fluency subtest from the Delis-Kaplan Executive Function System; empty and switching dots condition, scaled scores).²⁷ Social abilities were assessed using the Revised Self-Monitoring Scale (RSMS),³⁰ a Social Norms questionnaire (total correct), and The Awareness of Social Inference Test (total score).³¹ Behavioral features were categorized by the examiner using a 9-item Behavioral Rating Scale. The informant components of the Frontal Systems Behavior Scale (FrSBe)³² were also administered.

Bedside saccade tasks.

Prosaccade (PS) and AS tasks were used to measure oculomotor responses and response inhibition, respectively. Stimuli for both PS and AS tasks were presented on a 15-inch laptop computer screen using E-Prime software (version 2.0, Psychology Software Tools Inc., Pittsburgh, PA; software available from authors upon request). Subjects were seated approximately 80 cm from the eye level laptop screen and the rater was positioned behind the laptop screen in clear view of the subjects' eye movements (figure e-1). Raters were provided a script that instructed the subjects to “look at the opposite direction of where the dot moves” for the AS trials. Raters were instructed to record the direction of only the first eye movement after the stimulus and not any subsequent self-corrected errors (see below; e-Instructions).

Prosaccade control condition.

In the PS trials, a white dot appeared on a black background for 1,000 msec, followed by a 200 msec gap prior to appearance of a left or right stimulus dot at 10 degrees horizontal for 900 msec. A blank screen appeared between trials for a randomized 1,000–4,000 msec intertrial interval (ITI).

Bedside antisaccades.

Stimuli for the AS trials were similar to the PS trials. However, the AS trials included a 1,000 msec white central dash following the center fixation dot to remind the subjects that an AS was to be performed. The lateral target appeared for 1,000 msec, followed by a randomized, 1,000–2,000 msec ITI.

Testing began with 1 block of 10 PS trials, followed by 1 practice AS trial, and then a series of 40 AS trials, divided into 2 blocks of 20. Raters documented the direction of the subject's initial eye movements for each trial on a provided scoring sheet. An auditory marker (a recorded number) of the trial number accompanied each center fixation to ensure correct recording of subjects' responses. Since the trials were presented in pseudorandom order, the normative eye deviation (R or L) responses could be indicated on the scoring sheet, allowing raters to calculate the percentage of correct responses for the AS trials ([number of correct responses/40] × 100).

Laboratory eye movement recordings.

The movements of the right eye were measured at UCSF in a darkened laboratory using a Dual Purkinje Image Eye Tracker (Fourward Technologies Generation 6.1, Buena Vista, VA) as described in our previous work.¹³ Responses to at least 10 AS trials were recorded in each direction, and only subjects with at least 20 high-quality trials were entered into the analysis. Self-corrected error responses consisted of a correct AS response (away from the target) within 500 msec after an initial erroneous PS (toward the target). The interval between bedside and laboratory AS testing was a maximum of 6 months.

Data analysis.

Mean group differences for demographics, BAS performance, dementia severity, and neuropsychological performance were determined using analyses of variance with post hoc Tukey tests; a χ² test was used for comparing gender frequency. BAS performance was correlated with laboratory AS performance, MMSE, Clinical Dementia Rating–sum of boxes (CDR-SB), and education using Pearson correlations. Partial correlations controlling for education and CDR-SB were used to evaluate the relationships between bedside AS performance and neuropsychological measures. For the multicenter analyses, 3 models were created for comparison of BAS performance with patient, examiner, and informant measures: model 1: Pearson correlation; model 2: partial correlation controlling for age and education; and model 3: partial correlation controlling for age, education, and Digit Symbol number correct. In models 2 and 3, Flanker Incongruent median reaction time was also controlled for by Flanker Congruent median reaction time and Stroop Interference scores were also controlled for by Stroop Color Naming scores (number correct). Corrections for multiple comparisons were implemented using the Bonferroni procedure for each hypothesis. For the UCSF cohort, separate corrections were calculated for the Pearson correlation on general items (MMSE, CDR-SB, and education; n = 3) and for the partial correlation of nonexecutive (n = 8) and executive (n = 6) neuropsychological tasks. For the multicenter cohort, separate Bonferroni corrections were calculated for models 1, 2, and 3 based on the number of observations in each (n = 19, 17, and 16, respectively). All statistical analyses were performed using PASW for Windows (version 18.0, SPSS, Chicago, IL).

RESULTS

We first validated BAS performance by comparing the percentage of correct responses measured at the bedside using our novel laptop stimuli with performance in the laboratory using an infrared eye tracker. The percentage of correct BAS responses correlated with the percentage of correct AS responses measured using the infrared eye tracker (r = 0.862, p < 0.001; n = 33; figure 1). In addition, correct BAS responses also correlated with the percentage of correct AS plus self-corrected error responses (r = 0.667, p < 0.001).

Figure 1. Bedside antisaccade (AS) performance as a function of laboratory eye movement measurements.

Scatterplots exhibit the relationship between the percentage of correct bedside AS responses and laboratory assessment of the percentage of correct AS responses (A) and the percentage of correct plus self-corrected AS responses (B). Trend lines indicate significant correlations (Pearson; p < 0.05). AD = Alzheimer disease; FTLD = frontotemporal lobar degeneration; MCI = mild cognitive impairment; NC = normal control; PSP = progressive supranuclear palsy.

To determine whether the BAS is more sensitive to executive function than other neuropsychological domains, BAS performance was correlated with performance on a standard neuropsychological battery in our UCSF cohort (n = 103; table 1). The percentage of correct AS responses on the BAS correlated with MMSE (r = 0.423) and CDR-SB (r = −0.605; p < 0.05, Bonferroni corrected for both) but not education (r = 0.221). Since BAS performance strongly correlated with a general measure of disease severity (CDR-SB) and cognition (MMSE), and educational attainment might affect the degree of impairment on BAS, we controlled for disease severity and education in subsequent analyses to determine neuropsychological domain specificity. After controlling for CDR-SB and education, BAS performance correlated most strongly with neuropsychological tests of executive functioning, including time to complete modified Trails, digits backward, Stroop inhibition, and design fluency (p < 0.01, Bonferroni corrected for all; table 2). BAS performance was also correlated with CATS Facial Matching (p < 0.01), letter fluency (p < 0.05), and category fluency (p < 0.01, Bonferroni corrected for all).

Table 1.

Demographics and performance on bedside antisaccade task^a

Abbreviations: AD = Alzheimer disease; ANOVA = analysis of variance; BAS = bedside antisaccade (percent correct responses); CATS = Comprehensive Affect Testing System; CDR = Clinical Dementia Rating; CDR-SB = Clinical Dementia Rating–sum of boxes; FTLD = frontotemporal lobar degeneration; MCI = mild cognitive impairment; MMSE = Mini-Mental State Examination; NA = not applicable; PSP = progressive supranuclear palsy; VOSP = Visual Object Spatial Perception.

^aValues represent group mean ± SD.

^bp ≤ 0.05 vs control.

^cp ≤ 0.005 vs control.

^dp ≤ 0.05 vs MCI.

^ep ≤ 0.005 vs MCI.

^fp ≤ 0.005 vs FTLD.

^gp ≤ 0.05 vs PSP for ANOVA with post hoc Tukey test.

Table 2.

Correlation of cognitive testing with bedside antisaccade performance

Abbreviations: BAS = bedside antisaccade; CATS = Comprehensive Affect Testing System; CDR = Clinical Dementia Rating; CDR-SB = Clinical Dementia Rating–sum of boxes; MMSE = Mini-Mental State Examination; VOSP = Visual Object Spatial Perception.

^aPartial correlation coefficient for individual models that included the percentage of correct responses on the BAS task and each neuropsychological value, with Education and CDR-SB as covariates.

^bp < 0.01 after Bonferroni correction.

^cp <0.05 after Bonferroni correction.

^dr ≥ 0.5 or r ≤ −0.5.

The validity of the BAS as a measure of executive function was investigated using data from a multicenter study of a novel executive function battery (EXAMINER). BAS performance was measured in adults (ages 18 to 91) who were cognitively normal or diagnosed with AD, focal brain lesions, FTD, HD, MCI, MS, PD, PSP, or TBI (table e-1) at 6 performance sites and compared with direct, examiner-based, and informant-based measures of executive function. BAS performance did not correlate with age or education. In bivariate Pearson correlations, all EXAMINER measures correlated with BAS performance (model 1, table 3; p < 0.05, Bonferroni corrected for all). After controlling for age and education (model 2), BAS performance correlated with all EXAMINER measures except for Flanker Incongruent median reaction time (p < 0.05, Bonferroni corrected for all). When controlling for age, education, and processing speed (Digit Symbol Number; model 3), BAS performance remained correlated with all measures except for Flanker Incongruent median reaction time and RSMS total (p < 0.05, Bonferroni corrected for all). Of all EXAMINER measures, BAS performance was most strongly associated with informant-based measures of executive function from the FrSBe scale.

Table 3.

Correlation of bedside antisaccade performance with EXAMINER measures

Abbreviations: EXAMINER = Executive Abilities: Methods and Instruments for Neurobehavioral Evaluation and Research; FrSBe = Frontal Systems Behavior Scale; RSMS = Revised Self-Monitoring Scale; TASIT = The Awareness of Social Inference Test.

^aPearson or partial correlation coefficient for each model.

^bp < 0.05 after Bonferroni correction.

DISCUSSION

This study describes a novel BAS task that requires no specialized equipment. We demonstrate that the BAS is strongly correlated with oculographically measured AS performance, indicating that BAS can be related to the extensive literature on the AS task in neurologic and psychiatric disease, as well as mechanistic studies of the AS task in both humans and nonhuman primates. In a cohort of cognitively normal participants and patients with neurodegenerative disease, BAS performance correlated more strongly with measures of executive function than memory, language, or visuospatial skill (after controlling for disease severity and education), similar to previous reports of oculographically measured AS performance.¹² In a multicenter study, BAS performance was associated with both neuropsychological and informant-based measures of executive function in normal adults and individuals with a variety of neurologic disorders. Together, our results suggest the BAS is a cost-effective tool that can be used to assess both executive motor function and disease severity in a variety of neurologic disorders.

We found that subjects with a variety of dementia diagnoses exhibited an impaired ability to inhibit reflexive PS responses on the BAS. The similarity between this finding and previous results using an infrared eye tracker,^11–13,33 along with the strong correlation of performance on the BAS and our laboratory AS task, further underscores the validity of the task. Impaired AS performance is correlated with structural alterations to the right greater than left premotor regions of the frontal lobes in focal lesion patients,³⁴ neurodegenerative disease,¹² psychiatric illness,³⁵ and normal aging.^4,36 Our results suggest that the BAS may be sensitive to damage to these same brain regions and could be used as a screen for such damage at the bedside.

The same frontoparietal network necessary for AS performance has been previously implicated in executive control in normal aging,⁴ schizophrenia,^5–7,37 and neurodegenerative disease.^12–13 The observed correlation between BAS performance and neuropsychological measures of executive function further supports our hypothesis that BAS relies on the integrity of the same network as other AS paradigms. Because the correlations were strongest between BAS performance and executive function tasks that require greater motor skill in the form of drawing lines (Modified Trails and Design Fluency), it is possible that the BAS is more sensitive to executive motor control tasks as opposed to executive function as a whole. More work will be needed to confirm this hypothesis.

In our multicenter cohort, we also found that BAS performance strongly correlated with several measures of executive function, behavior, social cognition, and an informant measure of executive function: the FrSBe, a reliable measure of damage to the frontal lobes³² that produces a total score as well as subscales for Apathy, Disinhibition, and Executive Dysfunction. Even after controlling for demographics and processing speed, BAS performance was correlated with all 4 informant FrSBe-based scores. The FrSBe is sensitive to frontal lobe function in a variety of neurologic disorders including focal lesions, AD, Parkinson disease, and ALS. Our results suggest that the BAS may be useful in efficiently quantifying frontal lobe dysfunction in individuals for whom no informant is available.

BAS performance was also correlated with disease severity, as measured by MMSE and CDR-SB, in the UCSF cohort. Unlike a previously described bedside AS task,¹⁴ we did not observe a correlation between BAS performance and a test of memory. This finding is likely explained by the more diverse population of neurodegenerative disease subjects in our study and the fact that we controlled for disease severity (CDR-SB) in our analyses. Had we limited our analysis to MCI and AD subjects and not corrected for CDR-SB, our results would have been similar to the previous study (data not shown). Interestingly, BAS performance was different in MCI and dementia. Patients with AD, frontotemporal lobar degeneration, and PSP made more BAS errors than patients with MCI. Since we are unaware of any previously published reports of AS performance measured in MCI, further studies will be necessary to interpret the significance of these findings.

Subjects performed better on the bedside version of the AS task than the laboratory version. Several factors may contribute to this finding, including rater errors and time delay between testing modalities. A likely contributor is the presence of ambient lighting and other visual stimuli in the BAS condition that might partially counteract the effects of the gap stimulus, which increases error rate as a result of an increase in express saccade generation in the absence of fixation stimuli.¹ In this respect, our findings are similar to those reported for a previous BAS task.¹⁴ Of note, our BAS task includes auditory cues to improve raters' data acquisition, which might also influence task performance.^38,39

A limitation of the study was that we did not measure the interrater or intrarater reliability of the BAS. This will be necessary in the future for the BAS to be used in multicenter studies such as clinical trials, although the AS task has been used to measure disease progression in HD¹⁵ and genetic risk in first-degree relatives of patients with schizophrenia in large, multicenter studies.⁴⁰ Such reliability studies are currently under way.

Our findings indicate the BAS is a valid measure of executive motor control in aging and neurologic disease, offering practitioners a rapid, simple clinical tool to evaluate executive functioning and dementia severity in their patients.

GLOSSARY

AD

Alzheimer disease

AS

antisaccade

BAS

bedside antisaccade

BNT

Boston Naming Test

CATS

Comprehensive Affect Testing System

CDR

Clinical Dementia Rating

CDR-SB

Clinical Dementia Rating–sum of boxes

EXAMINER

Executive Abilities: Methods and Instruments for Neurobehavioral Evaluation and Research

FrSBe

Frontal Systems Behavior Scale

FTD

frontotemporal dementia

HD

Huntington disease

ITI

intertrial interval

MCI

mild cognitive impairment

MMSE

Mini-Mental State Examination

MS

multiple sclerosis

NE

normal elder

PD

Parkinson disease

PS

prosaccade

PSP

progressive supranuclear palsy

RSMS

Revised Self-Monitoring Scale

TBI

traumatic brain injury

UCSF

University of California, San Francisco

VOSP

Visual Object Spatial Perception

AUTHOR CONTRIBUTIONS

Dr. Hellmuth, Mr. Mirsky, Dr. Matlin, Mr. Jafari, Dr. Garbutt, Ms. Widmeyer, Ms. Berhel, and Ms. Sinha performed assessments and analyzed data. Dr. Hellmuth, Mr. Mirsky, Dr. Heuer, Dr. Miller, Dr. Kramer, and Dr. Boxer contributed to writing and revising the manuscript. Dr. Kramer and Dr. Boxer designed the study.

DISCLOSURE

Dr. Hellmuth, Mr. Mirsky, Dr. Heuer, Dr. Matlin, Mr. Jafari, Dr. Garbutt, Ms. Widmeyer, Ms. Berhel, and Ms. Sinha report no disclosures. Dr. Miller serves on a scientific advisory board for the Alzheimer's Disease Clinical Study; serves as an Editor for Neurocase and as an Associate Editor of ADAD; receives royalties from the publication of Behavioral Neurology of Dementia (Cambridge, 2009), Handbook of Neurology (Elsevier, 2009), and The Human Frontal Lobes (Guilford, 2008); serves as a consultant for Lundbeck Inc., Elan Corporation, and Allon Therapeutics, Inc.; serves on speakers' bureaus for Novartis and Pfizer Inc.; and receives research support from Novartis and the NIH (NIA P50 AG23501 [PI] and NIA P01 AG19724[PI]) and the State of California Alzheimer's Center. Dr. Kramer receives royalties from Pearson for the publication of commercially available neuropsychological tests. He receives research support from the NIH (2R01AG022983, 1R01AG032289, HHSN271200623661C). Dr. Boxer has been a consultant for Accera, Bristol Myers Squibb, Genentech, Medivation, Phloronol, Registrat-Mapi, and TauRx. He has received research funding from Allon Therapeutics, Avid, Elan, Forest, Genentech, Janssen, Medivation, and Pfizer. He is funded by NIH grants R01AG038791 [PI], R01AG031278 [PI], the Alzheimer's Drug Discovery Foundation, CurePSP, the Hellman Family Foundation, and the Tau Research Consortium. Go to Neurology.org for full disclosures.