Oculomotor and Neuropsychological Effects of Antipsychotic Treatment for Schizophrenia

Abstract

Cognitive enhancement has become an important target for drug therapies in schizophrenia. Treatment development in this area requires assessment approaches that are sensitive to procognitive effects of antipsychotic and adjunctive treatments. Ideally, new treatments will have translational characteristics for parallel human and animal research. Previous studies of antipsychotic effects on cognition have relied primarily on paper-and-pencil neuropsychological testing. No study has directly compared neurophysiological biomarkers and neuropsychological testing as strategies for assessing cognitive effects of antipsychotic treatment early in the course of schizophrenia. Antipsychotic-naive patients with schizophrenia were tested before treatment with risperidone and again 6 weeks later. Matched healthy participants were tested over a similar time period. Test-retest reliability, effect sizes of within-subject change, and multivariate/univariate analysis of variance were used to compare 3 neurophysiological tests (visually guided saccade, memory-guided saccade, and antisaccade) with neuropsychological tests covering 4 cognitive domains (executive function, attention, memory, and manual motor function). While both measurement approaches showed robust neurocognitive impairments in patients prior to risperidone treatment, oculomotor biomarkers were more sensitive to treatment-related effects on neurocognitive function than traditional neuropsychological measures. Further, unlike the pattern of modest generalized cognitive improvement suggested by neuropsychological measures, the oculomotor findings revealed a mixed pattern of beneficial and adverse treatment-related effects. These findings warrant further investigation regarding the utility of neurophysiological biomarkers for assessing cognitive outcomes of antipsychotic treatment in clinical trials and in early-phase drug development.

Neurocognitive deficits are core features of schizophrenia that cause considerable long-term disability.^1–3 Consequently, the amelioration of cognitive deficits has become a major focus of new drug development for schizophrenia.^4,5 Traditional neuropsychological approaches and cognitive neuroscience/neurophysiological methodologies represent 2 different approaches for characterizing cognitive deficits in schizophrenia and the impact of antipsychotic treatment on cognitive processes. The relative benefits and disadvantages of each approach are not well characterized because very few studies evaluating the cognitive efficacy of antipsychotic treatments have included both neurophysiological biomarkers and traditional neuropsychological tests as outcome measures. Comparative evaluation of neurophysiological and neuropsychological assessments is important because of the urgent need for informative and efficient assessment of neurocognitive outcomes in clinical trials.

Most studies of cognitive outcome in schizophrenia have utilized clinical neuropsychological tests for evaluating treatment-related effects.^3–14 The neuropsychological approach typically uses standardized clinical tests with normative data that permit direct test-by-test comparison with population expectations. Neuropsychological tests are often multidimensional, relying on numerous cognitive processes, so as to efficiently identify patients with abnormalities in one or more brain regions. Neuropsychological studies have been instrumental in conceptualizing schizophrenia as a brain disorder and has spurred interest in cognitive aspects of the disorder and associated morbidity. Indeed, much of the emphasis on cognitive enhancement in schizophrenia has resulted from studies demonstrating a significant relationship between neuropsychological and functional deficits.² Neurocognitive impairments are now well recognized to have implications for treatment planning and course of illness.⁴

There are clear advantages to the neuropsychological approach, especially in large multisite clinical trials, such as ease of use, portability, established reliability and normative data, availability of psychologists experienced with such testing, low cost, and few technology requirements. However, against a background of clinical improvement, a somewhat surprising finding across neuropsychological studies of antipsychotic drugs has been the modest and generalized improvement in performance that is similar in overall magnitude to practice effects seen in healthy individuals.^6–9 While this modest level of improvement may be greater with atypical antipsychotics, compared with conventional neuroleptics, the profile of change is similar.^10–14 These findings suggest that the procognitive effects of available antipsychotic treatments are either quite modest or that the clinical neuropsychological “assay” may be a weak approach for detecting change in brain function and cognition over time.

By comparison, neurophysiological biomarker approaches test highly specific cognitive processes that have been linked to specific regional brain function and transmitter systems. One strength of the biomarker approach is greater ease for parametric manipulation of tasks to isolate component cognitive processes. This approach espouses experimental investigation of physiological processes by utilizing theoretically based manipulation of an experimental parameter to produce and evaluate pharmacological effects on those processes. In this manner, the biomarker approach has closer ties to animal models, and, by virtue of closer links to brain physiology, this approach may be more sensitive to pharmacologic manipulations than behavioral approaches. By way of comparison, it is important to note that while neuropsychological studies have shown a generalized picture of cognitive improvement after antipsychotic treatment, animal models typically show a more circumscribed impact of dopaminergic drugs on cognitive and motor abilities.^15–19 Specifically, behavioral pharmacology studies have shown specific negative neurocognitive effects of dopaminergic drugs such as declines in executive abilities following prefrontal dopamine depletion²⁰ and after disruption of thalamocortical circuitry via D2 blockade.²¹

Recent efforts by the NIMH-MATRICS program to develop a standard for industry-sponsored trials evaluating cognition-enhancing treatments in schizophrenia have focused primarily on tests with a long history in clinical neuropsychology.⁵ While the neuropsychological approach has provided much of the impetus for bringing cognition into focus as a treatment target, the utility of neuropsychological methods for detecting treatment-related responses may be constrained by psychometric properties that limit differential sensitivity to subtle dysfunction (ceiling effects) and severe deficits (floor effects) as well as variability in measurement reliability and sensitivity to drug effect.^22–24

The advancement of treatment for cognitive deficits in schizophrenia has come to a critical juncture. Because new drugs are developed for this purpose, efficient and valid tools to evaluate procognitive pharmacotherapies are needed to accelerate the development of effective cognition-enhancing treatments for schizophrenia to reduce the personal and societal burden associated with cognitive deficits. In some settings, neurophysiological biomarkers have been reported to be more sensitive to drug effects than neuropsychological tests.^25,26 Therefore, this study was designed to directly compare the differential sensitivity of oculomotor paradigms and neuropsychological tests with risperidone treatment.

A major strength of the cognitive neuroscience/neurophysiological approach is the foundation in animal models linking discrete cognitive processes to specific brain region and receptor systems. Behavioral pharmacology research has clarified the effects of certain drugs on specific functional brain systems, and these findings can be used to guide predictions and interpretation of drug effects in humans. Oculomotor studies have a rich tradition in nonhuman primate research that has a close homology to cognitive processes and their neurobiological substrates in humans. Whereas our neuropsychological studies have shown little or no cognitive change associated with antipsychotic treatment,^8,27 oculomotor studies with overlapping samples treated with risperidone have revealed both beneficial and adverse treatment-related effects.^28–30 The present report directly compares the sensitivity and reliability of neuropsychological data and oculomotor biomarkers with regard to monitoring cognitive effects of risperidone in a sample of antipsychotic-naive, first-episode schizophrenia patients.

METHODS

Participants

Following evaluation for first-episode psychosis, 29 antipsychotic-naive patients (18 male, 11 female) were recruited at the University of Pittsburgh Medical Center. All patients met criteria for schizophrenia based on the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (SCID).³¹ A sample of 26 healthy individuals (17 male, 9 female) recruited from the community were free from any Axis I diagnosis based on SCID. As shown in table 1, groups were matched on age, sex, parental socioeconomic status, and estimated intellectual abilities (Ammon Quick Test³²). All participants were free of substance abuse within the last 3 months, a lifetime history of substance dependence, and history of neurological disease including head injury with loss of consciousness or systemic disorders known to affect brain function. The study was approved by the University of Pittsburgh Institutional Review Board, and all participants provided written consent.

Table 1.

Group Demographics and Clinical Data for Patients

	Healthy Comparison (HC) n = 26	Schizophrenia (SZ) n = 29		Analysis
Demographics				F/χ2	df	P
Age (y)	23.77 (4.33)	25.97 (7.98)		1.65	1,53	.22
Sex				6.62	1	.71
Male	65.4%	62.1%
Female	34.6%	37.9%
Race				0.59	1	1.0
Caucasian	69.2%	62.1%
African American	26.9%	31.0%
Asian/Latino/Other	3.8%	6.9%
Dominant hand				0.08	1	.78
Right	88.0%	89.7%
Left	12.0%	10.3%
Education	15.04 (2.01)	14.03 (3.29)		1.75	1,53	.18
SES	3.08 (1.06)	3.17 (1.04)		0.12	1,53	.74
Parental SES	2.31 (0.79)	2.76 (1.30)		1.66	1,53	.13
Intelligence
AQT	101.19 (6.68)	98.83 (8.16)		1.78	1,53	.25
Clinical scales		Baseline	Follow-up
BPRS		51.97 (8.09)	39.55 (8.90)	50.39	1,28	<.001
SAPS		10.38 (2.83)	4.83 (2.89)	102.08	1,28	<.001
SANS		13.76 (2.79)	11.86 (2.77)	15.65	1,28	<.01
HDRS (24 items)		22.57 (11.02)	15.43 (7.03)	9.53	1,27	<.01

Note: SES, socioeconomic status; BPRS, Brief Psychiatric Rating Scale; SAPS, Scale for the Assessment of Positive Symptoms; SANS, Scale for the Assessment of Negative Symptoms; HDRS, Hamilton Depression Rating Scale; AQT, Ammons' quick test.

Data from each test has been presented previously in detailed reports.^8,28–30 The current patient and healthy groups differed somewhat from those used in previous reports in that the sample for this report was restricted to participants who had completed all neuropsychological and eye movement tests of interest so that cross-test comparisons could be made with the same subjects. The patient sample was further restricted to those treated with risperidone monotherapy. Thus, a core sample of about two thirds of the data previously reported^8,28–30 were included in this report. Selection of outcome variables was challenging given the wide range of oculomotor and neuropsychological impairments linked to schizophrenia. Outcome variables in the present study were limited to a number of key outcome variables (scored in the same manner as noted in prior reports^8,28–30) that characterize core neuropsychological and oculomotor deficits.

Clinicians blind to neurobehavioral findings assessed clinical symptomatology in the patient group using the Brief Psychiatric Rating Scale (BPRS), the Scales for the Assessment of Negative and Positive Symptoms, and the 24-item Hamilton Depression Rating Scale (see table 1). Medication and dosage decisions were made by the treating psychiatrist based on clinical efficacy and tolerance of side effects. Following all baseline neuropsychological, oculomotor, and clinical assessments, patients began treatment with risperidone (4.13 mg/day ± 1.39). Follow-up assessments were conducted approximately 6 weeks later. Just 4 patients were prescribed low-dose benztropine (1 or 2 mg) at the 6-week follow-up, and ratings of extrapyramidal symptoms³³ were minimal (3.77 ± 4.52).

Neuropsychological Evaluation

The neuropsychological battery included 8 tests characterizing 4 commonly assessed cognitive domains (executive function, attention, memory, and motor skills). Table 2 lists the individual scores within each neuropsychological domain for patients and healthy individuals. With the exception of alternate forms of the California Verbal Learning Test, the same tests were administered at follow-up. There was no discernable relationship between neuropsychological and oculomotor measures for either patients or healthy individuals.

Table 2.

Raw Score Means, Reliability Coefficients, and Percent Change Over Time for Each Neuropsychological and Oculomotor Measure by Group

Healthy Controls (n = 26)	Baseline	6 Wk	ICC	Spearman ρ	Percent Change
Oculomotor measures
VGS—latency (ms)	221 (29.89)	217 (24.53)	.72	.74	+1.81
VGS—gain	0.95 (0.05)	0.94 (0.04)	.85	.85	+0.02
Antisaccade latency (ms)	407 (88.02)	394 (75.43)	.83	.82	+3.19
Antisaccade percent errors	0.19 (0.13)	0.13 (0.09)	.77	.75	+30.31
MGS—gain	0.95 (0.11)	0.97 (0.16)	.69	.76	+2.56
MGS—resting error (degree)	−0.69 (1.48)	−0.48 (1.49)	.59	.56	+30.49
Neuropsychological measures
Stroop color-word score	51.60 (8.50)	54.12 (9.55)	.79	.80	+4.88
Controlled Oral Word Associationa	43.64 (9.59)	44.76 (8.95)	.75	.78	+2.57
Trail-Making Test: part B timeb	51.92 (15.67)	48.12 (21.27)	.38	.51	+6.96
CVLT: learning trials 1–5	54.62 (8.34)	54.77 (6.52)	.38	.26	+0.27
CVLT: long delay free recall	11.88 (2.89)	12.76 (2.26)	.56	.40	+7.41
WMS-R visual reproduction I	37.69 (2.31)	37.12 (2.69)	.53	.46	−1.51
WMS-R visual reproduction II	35.23 (4.69)	36.77 (2.37)	.21	.14	+4.37
Trail-Making Test: part A timeb	23.88 (7.49)	20.54 (5.78)	.60	.63	+13.99
WAIS-R digit span	17.64 (3.76)	17.56 (3.80)	.74	.76	−0.45
WAIS-R digit symbol	67.60 (8.24)	70.36 (7.48)	.70	.72	+4.08
Grooved pegs: dominant hand	63.73 (10.31)	61.42 (12.32)	.88	.89	+3.62
Grooved pegs: nondominant hand	66.88 (6.73)	69.75 (8.82)	.69	.79	−4.29
Schizophrenia patients (n = 29)
Oculomotor measures
VGS—latency (ms)	207 (33.46)	225 (44.06)	.77	.81	+8.70
VGS—gain	0.93 (0.04)	0.91 (0.05)	.73	.71	−2.04
Antisaccade latency (ms)	487 (123.74)	453 (95.83)	.63	.58	+6.98
Antisaccade percent errors	0.41 (0.20)	0.35 (0.21)	.75	.66	+14.72
MGS—gain	0.87 (0.17)	0.81 (0.17)	.81	.79	−4.80
MGS—resting error (degree)	−1.19 (1.56)	−1.98 (1.73)	.62	.62	−66.67
Neuropsychological measures
Stroop color-word score	41.00 (8.49)	42.09 (9.95)	.54	.47	+2.66
Controlled Oral Word Associationa	36.28 (10.27)	41.07 (12.26)	.65	.71	+13.20
Trail-Making Test: part B timeb	74.72 (28.65)	63.90 (29.46)	.74	.69	+14.48
CVLT: learning trials 1–5	46.43 (13.48)	45.74 (11.55)	.56	.55	−1.49
CVLT: long delay free recall	10.61 (3.50)	9.39 (3.71)	.52	.42	−11.50
WMS-R visual reproduction I	32.86 (5.32)	33.69 (4.41)	.41	.53	+2.53
WMS-R visual reproduction II	30.76 (6.54)	32.17 (6.19)	.47	.64	+4.58
Trail-Making Test: part A timeb	32.52 (17.72)	32.38 (14.66)	.74	.66	−0.43
WAIS-R digit span	14.97 (3.98)	14.90 (3.55)	.68	.76	−0.47
WAIS-R digit symbol	50.83 (8.73)	53.38 (12.30)	.42	.50	+5.02
Grooved pegs: dominant hand	71.46 (16.30)	76.93 (21.92)	.67	.71	−7.65
Grooved pegs: nondominant hand	80.75 (19.14)	85.25 (16.35)	.61	.65	−5.57

Note: ICC, intraclass correlation; VGS, visually guided saccades; MGS, memory-guided saccades; CVLT, California Verbal Learning Test;⁵⁹ WMS-R, Wechsler Memory Scale—Revised;⁶⁰ WAIS-R, Wechsler Adult Intelligence Scale—Revised.⁶¹

^aMultilingual Aphasia Examination.⁶²

^bHalstead-Reitan Neuropsychological Battery.⁶³

Eye Movement Studies

Participants were tested alone in a darkened black room free from extraneous stimuli that could interfere with performance on eye movement tasks. Detailed methods for data acquisition and processing have been published previously.^28–30

Visually Guided Saccades

This task is used to evaluate visual orienting and allocation of attention across the visual field by measuring saccadic eye movements to the appearance of unpredictable peripheral targets. Electrooculography recordings of saccades were acquired for 54 trials as targets stepped 10°, 20°, or 30° of visual angle from center fixation. Saccade latencies (milliseconds) and gain (amplitude of saccade divided by target displacement) were selected for analysis.

Antisaccade Task

The aim of this task is to evaluate the ability of participants to suppress the natural tendency to look toward unpredictable peripheral targets when they appear and instead voluntarily shift attention and point of gaze to another location. Targets were presented at one of 6 locations (8°, 16°, or 24° to the left or right of center fixation) while participants maintained center fixation. The instructions were to look to the mirror location of the target on the other side of the visual field. Thirty-six trials were administered. The proportion of successfully performed trials and the response latency for successful trials were obtained.

Memory-Guided Saccade Task

This working memory task is used to evaluate the ability to remember the spatial location of targets over brief periods of time. While participants maintained central fixation, peripheral targets appeared for 100 milliseconds at 9°, 18°, or 27° of visual angle to the right or left of center. Participants were instructed to maintain central fixation during delay periods of 1, 2, 4, or 8 seconds, after which the central light was extinguished, signaling the participant to look to the remembered target location. Two primary measurements of performance accuracy were obtained over 24 trials: (1) gain (amplitude of saccade divided by target displacement) of the initial saccade to the remembered target location and (2) error of the final resting eye position (in degrees of visual angle from target) after any additional saccades were made to shift gaze to the remembered location.

Data Analysis

Performance was pooled over all trials in each oculomotor paradigm to focus statistical analyses on group differences and the differential change over time in the 2 groups. Arcsine transformations for proportional data and natural logarithm transformations for reaction time data were used for computing intraclass correlations (ICCs) and effect sizes. To evaluate the sensitivity of each approach, omnibus 2-way (group by time) repeated measures multivariate analysis of variance (MANOVA) was conducted separately for neuropsychological and oculomotor measures. A significant group by time interaction would indicate a treatment-related effect that differs from practice effects in healthy individuals. To clarify any significant omnibus multivariate interactions, univariate analysis of variance was computed for each dependent variable. To further explore the differential sensitivity of neuropsychological and oculomotor measures, between-group effect sizes were computed to quantify disease effects. It is unclear whether measures with large or small effect sizes for disease-related deficits will be more sensitive to treatment-related effects. Thus, we calculated orthogonal within-group effect sizes to evaluate change over time, independent of magnitude of patient impairment. Within-group effect sizes were averaged for neuropsychological and oculomotor measures to contrast their relative sensitivity to treatment-related effects.

Results

Clinical Features

After treatment, patients showed clinical improvement reflected in reduced BPRS scores, positive symptom ratings, and negative symptom ratings (see table 1). There was no relationship between symptom change and oculomotor or neuropsychological change. Medication dose and change in positive symptoms were not significantly correlated with changes in oculomotor or neuropsychological measures.

Group by Time Effects

Table 2 lists the means and SDs of test performance for both the baseline and 6-week follow-up assessments. Two-way repeated measures MANOVA was completed separately to evaluate the differential sensitivity of neuropsychological and oculomotor measures to treatment with risperidone. Specifically, we reasoned that the group by time interaction and the effect size for this term indicated the relative sensitivity of assessment approaches to treatment-related effects. For oculomotor measures, results indicated significant main effects for group (F(6,46) = 5.18, P < .001) and time (F(6,46) = 5.82, P < .001) as well as a significant group by time interaction (F(6,46) = 4.32, P < .01). Results of the 2-way MANOVA for neuropsychological tests indicated significant main effects for group (F(12,32) = 4.26, P = .001) and time (F(12,32) = 4.07, P = .001) but a nonsignificant group by time interaction (F(12,32) =1.71, P = .11). Thus, level of neuropsychological impairment relative to healthy individuals was consistent over time despite hospitalization and pharmacological intervention. Results of these omnibus repeated measures MANOVA indicated a significant time by group interaction for oculomotor but not neuropsychological measures, suggesting that oculomotor measures were more sensitive to the neurocognitive effects of risperidone treatment. To clarify the significant multivariate interaction term for oculomotor measures, a series of univariate repeated measures ANOVAs were completed for eye movement tests. Results are shown in figure 1 and illustrate the significant group by time interactions in 4 of 6 oculomotor variables, after correcting for multiple comparisons.³⁴ Observed effect sizes for the univariate group by time interactions are an indicator of the relative sensitivity to treatment-related effects and clearly favor the oculomotor measures. Effect sizes for these 2-way ANOVA³⁵ ranged from near zero for antisaccade variables to small for visually and memory-guided saccade variables. On the other hand, the nonsignificant omnibus MANOVA indicated that comparable effect sizes for the neuropsychological measures were near zero.

Fig. 1.

Univariate analyses of variance (ANOVAs) are presented to clarify the significant group by time interaction observed following the omnibus repeated measures multivariate analysis of variance (MANOVA) on oculomotor measures. The first column illustrates the significant univariate group by time interactions for the visually guided saccade task in which the schizophrenia group showed decreased accuracy and slower responses to visual targets following treatment. As seen in the third column, the schizophrenia group also displayed a combination of decreased accuracy and increased error following treatment while the healthy control group showed modest practice effects for spatial working memory. In contrast, the group by time interaction was not significant in the middle column because the rate of change over time did not differ between the 2 groups for antisaccade accuracy or response time. Results were not illustrated for neuropsychological data because the group by time interactions for the omnibus MANOVA was nonsignificant for neuropsychological tests. Therefore, similar to the antisaccade findings, a pattern of parallel changes in performance over time can be inferred for both groups on each neuropsychological test. Response latency is measured in milliseconds; final resting error is measured in degrees of visual angle; gain is measured as the amplitude of saccade divided by target displacement.

Effect Size of Baseline Differences and Treatment-Related Effects

Between-Group Comparisons at Baseline

The relative sensitivity of each performance parameter to illness effects was assessed via between-group effect size comparisons, while comparisons of within-group effect sizes assessed treatment-related effects (Cohen's d³⁸ was used for both within- and between-group comparisons: small effects ≤ 0.20, medium 0.20 to 0.50, and large ≥ 0.80). Figure 2 illustrates the magnitude of patient deficits before and after treatment, relative to healthy participants. Neuropsychological deficits (0.91 ± 0.40) were numerically but not significantly larger than oculomotor deficits (0.60 ± 0.39) at baseline, F(1,16) = 2.44, P = 14. With a few minor exceptions, the between-group effect size estimates prior to treatment revealed a generally flat profile of moderate to large neuropsychological impairments across domains. In contrast, performance deficits for oculomotor measures varied considerably in magnitude at the baseline testing. On the visually guided saccade task, patients showed latency differences that were modest in magnitude and no impairment in saccade accuracy. Antisaccade deficits were large and more consistent with the level of neuropsychological impairment. Patients displayed relatively small effect sizes for impairments in the accuracy of memory-guided saccades.

Fig. 2.

These between-group effect sizes illustrate the magnitude and direction of schizophrenia patient performance relative to a healthy sample before and after treatment. Positive values indicate patient deficits, while negative values indicate that the schizophrenia group outperformed healthy participants. A relatively flat profile of neuropsychological dysfunction is apparent in comparison to the wide range of patient performance on oculomotor tasks. CVLT, California Verbal Learning Test; Fig Mem, figure memory as measured by Wechsler Memory Scale—Revised visual reproduction; D, dominant hand; ND, nondominant hand.

Within-Group Comparison of Change Over Time

A second set of effect size estimates were obtained to assess treatment-related effects by comparing baseline and follow-up performance separately for each group. That is, for patients, within-group effect size estimates reflect a combination of practice effects, drug effects, and general clinical stabilization. In the absence of an active treatment, the same effect size most likely represents practice effects in the healthy group. Thus, the discrepancy between practice effects in healthy individuals and the combination of practice effects with pharmacological effects and clinical stabilization in patients provides a general index of treatment-related effects. As illustrated in figure 3, oculomotor measures were more sensitive to the effects of risperidone treatment than neuropsychological measures. The schizophrenia group showed significantly more change (F(1,16) = 4.65, P < .05) on oculomotor measures (0.41 ± 0.18) compared with neuropsychological test scores (0.22 ± 0.16) in terms of the average absolute value of within-subject effect sizes. In contrast, healthy controls showed similar levels of change across the 2 testing sessions (F(1,16) = 0.01, P = .92) for both oculomotor (0.30 ± 0.24) and neuropsychological variables (0.31 ± 0.16). It is noteworthy that the practice-related neuropsychological change in healthy participants was not greater (F(1,22) = 0.68, P = .42) than the change in neuropsychological performance exhibited by patients based on the combined factors of practice effects, clinical stabilization, and procognitive therapeutic efficacy of risperidone.

Fig. 3.

The withingroup effect size comparisons illustrated in this figure show the direction and magnitude of improvementdecline over time relative to baseline. Whereas percent change (table 2) indicates level of change on the original scale of measurement, these within-group effect sizes take into account SDs and can be directly compared across measures. Similar to percent change, positive within-group effect size values represent improved performance over time whereas negative values indicate performance declines from baseline. Treatment-related facilitation/deterioration can be inferred relative to practice effects in untreated healthy participants also retested at 6 weeks. Relative to pretreatment, the schizophrenia group showed longitudinal increases or declines on 4 of 6 oculomotor variables. In contrast, few within-group effect size changes exceeded 0.5 for patients on any neuropsychological variable. By comparison, healthy participants displayed comparable levels of effect size change over time for oculomotor and neuropsychological variables. CVLT, California Verbal Learning Test; Fig Mem, figure memory as measured by Wechsler Memory Scale—Revised visual reproduction; D, dominant hand; ND, nondominant hand.

Both groups showed similar levels of change over time on the antisaccade task suggesting that effects of acute risperidone treatment were no greater than practice effects seen in healthy individuals on this task. Effects on visually guided saccades were complicated in that patients showed abnormally speeded latencies prior to treatment, whereas a slowing in reaction time was observed, in which patients were slower than healthy participants, following treatment and clinical stabilization. The schizophrenia group also showed a modest (2.04%) but consistent decline following treatment in the gain (or accuracy) of visually guided saccades. Whereas healthy participants showed reduced spatial error of responses on the memory-guided saccade task over time, patients showed less accurate responses following treatment (which were not due to anticholinergic effects²⁹).

Effect size (d) estimates for treatment-related neuropsychological change over time revealed small effects for 10 of 12 variables. Patients showed moderate change (Cohen's d > .50) for just 2 variables including improvements for Verbal Fluency and Trails B. In both cases, mean change for the schizophrenia group (Trails B: 14.48% faster, Verbal Fluency: 13.20% more words) was somewhat greater than the practice effects exhibited by the healthy group (Trails B: 6.96% faster, Verbal Fluency: 2.57% more words). While the 2-way repeated measures MANOVA for neuropsychological tests indicated that these changes were not statistically significant, the Verbal Fluency increase was consistent with clinical trial data showing a significant increase in Verbal Fluency following treatment with risperidone.^36–38 At retest, healthy participants showed medium effect sizes for change at retest (>.50), indicating significant practice effects for the Stroop and speeded manual dexterity (Trails A) tests. Again, MANOVA results indicated that prior test taking did not benefit patients at follow-up, and it is possible that any expected practice effects for patients in these areas could be limited by their psychiatric disorder or attenuated by treatment initiation.

Test-Retest Reliability

One approach for evaluating performance consistency is assessment of reliability such as ICCs. This is a conservative approach to estimating reliability in a treatment context because ICCs are reduced by both shifts in mean group performance over time and measurement error. Despite these limitations, we reasoned that direct comparison of assessment approaches using ICCs was informative as one strategy for comparative evaluation of the 2 cognition assessment strategies (rather than comparing reliability between groups). As can be seen in table 2, performance on oculomotor measures was generally more consistent over time compared with neuropsychological measures, particularly for the healthy comparison group. This may reflect a restricted range on some variables such that the rank-order of distributions lacked consistency over time. For this reason, a Spearman rank-order correlation was used to provide an additional estimate of reliability by assessing the degree to which rank-order of distributions held up over time. Rank-order correlations (table 2) were consistent with ICC estimates of reliability as well as previously published visually guided saccade findings.^39–41 The lower reliability on neuropsychological measures in the healthy group may reflect a tendency for increased practice effects in which learning to perform the tests during the first administration is carried over to the next administration resulting in greater changes in performance at retest. In contrast, the likelihood of learning within or between oculomotor testing sessions may be lower.

Discussion

The current findings represent the first effort to directly compare performance on neuropsychological tests with neurophysiological biomarkers in terms of temporal reliability and sensitivity to neurocognitive effects of clinical stabilization and treatment with an antipsychotic medication in previously untreated schizophrenia patients. Both ANOVA and effect size comparisons indicate that oculomotor measures are more sensitive to select neurocognitive effects of risperidone treatment than neuropsychological tests. Indeed, multivariate and univariate ANOVAs revealed that treatment-related effects in patients differed significantly from practice effects in healthy individuals for both visually guided and memory-guided saccade tasks, whereas change in patient performance did not differ from practice effects in the healthy sample on antisaccade or neuropsychological tests. Furthermore, as can be seen in figure 3, within-subject effect sizes showed that healthy individuals actually improved over time on several neuropsychological tests due to practice effects at a level that was unmatched by the schizophrenia group. Although effect sizes were consistent with those reported in meta-analytic studies and larger multisite studies reporting modest generalized neuropsychological improvements,^12,13,42,43 neuropsychological change in patients after risperidone treatment was not statistically significant.

Against a background of limited neuropsychological change among patients treated with risperidone, more robust neurocognitive changes were observed for oculomotor paradigms. Oculomotor measures showed a pattern of both beneficial and adverse treatment-related effects characterized by (a) slowing of response latency from speeded pretreatment visually guided saccades and modest reductions in the precision of motor control, (b) minimal treatment-related effects on inhibitory behavioral control (as evidenced by similar practice effects for both groups on the antisaccade task), and (c) adverse effects on working memory in terms of reduced ability to remember spatial location information over time. The treatment-related effects observed in this study are not confounded by the potential adverse effects or loss of beneficial effects from prior treatment because the patient sample was antipsychotic naive and unmedicated at baseline.

The greater sensitivity of neurophysiological biomarkers to cognitive changes after treatment with an atypical antipsychotic is consistent with other studies providing growing support for the use of such outcome measures in schizophrenia research.⁴⁴ For example, in a recent study, risperidone treatment was associated with oculomotor changes in a chronic schizophrenia sample, whereas no drug effect was observed on the Cambridge Neuropsychological Test Automated Battery computerized cognitive battery.²⁵

Links to Animals Models

Neuropsychological studies indicated a picture of modest generalized cognitive change after antipsychotic treatment. In contrast, the oculomotor findings presented in this report were more consistent with behavioral pharmacology and animal models showing a mixed pattern of adverse and beneficial cognitive effects for pharmacological treatments targeting specific receptors.^18–24 Indeed, the present findings showed not only slow increased sensitivity of oculomotor variables to risperidone treatment but also a distinct pattern of effects that differentiated effects on frontal lobe systems. That is, voluntary response inhibition, as measured by antisaccade latency and accuracy, was unaffected by treatment in our sample at the 6-week follow-up (yet, antisaccade latency and error rates gradually improved when participants were followed out 1 year),²⁸ while performance on the oculomotor delayed response task showed an adverse response of spatial working memory systems to risperidone treatment. Studies using visually guided saccade paradigms to assess treatment-related effects have reported reduced peak saccade velocity, reduced accuracy, or increased response latencies after treatment with a wide range of atypical antipsychotics.^{29,41,45–48} Effects of typical and atypical antipsychotics on voluntary response inhibition, as measured by antisaccade task performance, were initially characterized as neutral.^47,49 However, more recent studies have shown beneficial treatment-related effects of typical and atypical antipsychotics characterized by reductions in prosaccade error rates and response latencies.^40,50 In contrast to some early reports, there is recent evidence suggesting that atypical antipsychotic treatment may be associated with adverse effects on working memory in schizophrenia.^30,45,51

Potential explanations for the greater heterogeneity of responses in the oculomotor data are complicated because discrete neural systems support each of these behaviors, as is discussed in detail in previous papers focused on effects present with each of the tasks.^28–30 Briefly, the changes associated with visually guided saccades are well replicated and may provide a sensitive measure of psychomotor slowing effects associated with antipsychotic treatment. With regard to the memory-guided saccade effect, which we recently replicated in an independent sample,⁵² this measure has been used by the Yale group to investigate drug effects on spatial working memory systems in nonhuman primates.⁵³ Via reduction of thalamocortical drive, effects on D1 receptor expression, or other mechanisms, antipsychotics reduce sustained firing of prefrontal neurons believed to support the ability to maintain spatial information over time, with consequent adverse effects on working memory performance.⁵³ There are weaker “a priori” predictions about drug effects on antisaccades. On clinical grounds, one might expect improvement on a test of the voluntary control of behavior, but changes were modest relative to those of healthy subjects. Interpretation of this effect remains unclear. The similar change over time might reflect similar practice effects in patients and controls. However, if schizophrenia or acute psychosis were to reduce practice effects, the improvement in patients may also reflect some improvement in cognitive ability. Such questions are difficult to resolve using behavioral data alone.

The current findings support the notion that neurophysiological biomarkers, which are rooted in cognitive neuroscience and more directly linked to animal models, may be more sensitive to cognitive changes after antipsychotic treatment than standard neuropsychological tests. There are several factors that could account for this difference. First, clinical neuropsychological tests often simultaneously evaluate multiple discrete cognitive processes that require integration of a wide range of distributed processing modules. If the integrity of one component is compromised or improved, the remaining components may or may not be similarly effected and such tests may not be as sensitive to drug effects relative to tests which directly assess the neural systems supporting a neurocognitive process that is targeted by treatment. Second, psychometric issues may play a role. Psychological batteries such as intelligence tests are often designed to have greater discriminative power around the normative mean, with progressive reduction in discriminative power further from the population mean. On most neuropsychological measures, patient performance is typically well below the population mean and therefore such tests may have reduced discriminative power to detect cognitive changes at the extremes. This problem is less pronounced with oculomotor measures because these measures do not oversample around the population mean in the same manner; rather distributions are more reliable across a greater range of performance values, and sensitivity to change may therefore be more consistent across a wide range of oculomotor measurements.

The advancement of schizophrenia treatment using cognition-enhancing medications will depend, in part, on effective evaluation of novel agents targeting cognition. Taking full advantage of the insight that animal models offer regarding drug effects on neuronal physiology and functional brain systems is crucial for evaluating new cognition-enhancing pharmacotherapies. By using translational, neurophysiological biomarkers, investigators in this area may be in a better position to design clinical studies, based on known pharmacological effects on specific brain systems, which utilize the same neurobehavioral approach previously used in animal models. For example, nonhuman primate models using oculomotor paradigms to study cognitive processes⁵⁴ offer a promising approach for use in the clinic as indicated by the findings summarized here. Further development of neurophysiological biomarkers such as event-related potentials, oculomotor paradigms, and perhaps functional brain imaging may provide efficient translational approaches for studying drug effects on cognitive systems, especially for early proof of concept studies.

Issues Regarding Practice Effects

The current findings indicate that practice effects in healthy participants are greater than the combination of practice effects and therapeutic benefit in patients on some neuropsychological measures. This suggests larger practice effects in the healthy sample, a treatment-related decline in patients that is compensated by similar benefits of practice, or a modest improvement associated with treatment in the context of practice effects that may be reduced in patients due to their illness. The degree to which changes over time in level of deficit can be attributed to differential practice effects in healthy individuals and patients vs treatment-related deterioration or facilitation is difficult to disentangle with behavioral data. But certainly, deciphering the impact of procognitive drugs will be less complicated when using neurocognitive assays with minimal practice effects. Parallel longitudinal data from matched samples of healthy individuals can help place retest data from patients in context by providing useful estimates of the degree to which change over time might be due to practice effects. This has typically not been done in large industry trials, but does provide a useful conservative upper bound of expected practice effects in patients, because practice effects in schizophrenia patients may well be smaller than in healthy participants. In this context, it is noteworthy that biomarkers may offer an advantage in terms of minimal practice effects. Reliability data for oculomotor measures were similar or greater than those for neuropsychological data.

Limitations of Biomarker Strategies

The neurophysiological biomarker approach has several limitations that warrant discussion. First and foremost, experience with this approach is limited and the current findings represent just one example of the kind of support needed to validate neurophysiological biomarkers as endpoints of treatments. Second, the value of this or any neurocognitive assessment approach needs to be examined across treatments. Different approaches are likely to be optimal for examining effects of different drugs. For example, the current patient sample was restricted to those treated with risperidone, and it is unclear whether these findings will generalize to other atypical antipsychotics and other drug treatments. Third, because studies validate different biomarkers, it will be important to compare various biomarkers such as functional neuroimaging, electroencephalogram, evoked potentials, and oculomotor studies in terms of their cost/benefit for assessing cognitive changes after different therapies. For such studies, comparison of approaches in terms of practice effects and retest consistency need to be systematically explored. Fourth, this was a relatively short treatment interval and it is possible that there was insufficient time for treatment to express its full effect on the complex integrative systems subserving neuropsychological measures. Lengthier studies could evaluate the degree to which sensitivity of neuropsychological measures to treatment-related effects improves over time. However, our 2-year follow-up of neuropsychological performance did not indicate such progressive change in sensitivity to treatment effects during the early course of schizophrenia.⁸ Additionally, the ultimate aim of cognitive enhancement therapy is improved functional status in the community. Recent data are beginning to show that cognitive neuroscience approaches provide data that predict changes in functional status,⁵⁵ but longer studies are needed to assess whether biomarker changes are associated with improved functional status or similarly related to community functioning as neuropsychological data. Finally, there are practical limitations to the use of biomarkers in clinical trials. Relative to the nominal instrumentation, training, and calibration needed for brief^5,56 and computerized^57,58 neuropsychological batteries, multisite studies using biomarker outcomes would require sophisticated instrumentation, labor-intensive calibration of equipment, and data analysis procedures to combine data across sites.

Concluding Remarks

At present, the advantages of biomarkers seem likely to have the greatest potential value during the early stages of drug development (eg, dose ranging studies, proof of concept studies to validate animal models), when small sample studies are completed to guide go/no-go decisions about which of several possible drugs in a class should be prioritized for large-scale clinical trials. Biomarkers may be especially advantageous in addressing questions about whether drug effects on a particular functional brain system correspond with effects on the specific cognitive domain subserved by that system, as previously observed in animal models. As suggested by the oculomotor data described in this report, biomarkers may also have better sensitivity for parsing both adverse and beneficial treatment-related effects that might occur with novel therapies. Thus, the critical question may not be whether biomarkers or neuropsychological measures are better, but how and when to make the best use of these approaches to facilitate the development of effective procognitive agents that are urgently needed in clinical practice.

Funding

National Institute of Mental Health (MH62134, MH45156, MH01433), National Center for Research Resources (M01 RR00056), National Alliance for Research on Schizophrenia and Depression.