Optimization and Validation of a Visual Integration Test for Schizophrenia Research

Abstract

The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia initiative highlighted a contour integration test as a promising index of visual integration impairment because of its well-established psychometric properties; its prior validation in healthy adults, patients, and nonhuman primates; and its potential sensitivity to treatment effects. In this multisite study, our goals were to validate the task on the largest subject sample to date, clarify the task conditions and number of trials that best discriminate patients from controls, and determine whether this discrimination can occur in standard clinical trial settings. For our task, subjects briefly observed a field of disconnected, oriented elements and attempted to decide whether a subset of those elements formed a leftward- or rightward-pointing shape. Difficulty depended on the amount of orientational jitter that was added to the shape’s elements. Two versions of this Jittered Orientation Visual Integration task (JOVI) were examined. Study 1 did not reveal between-group differences in threshold (ie, the jitter magnitude needed to reach a performance level of ∼80%), but this likely owed to the wide sampling distribution of jitter levels and resulting floor/ceiling effects in many conditions. Study 2 incorporated a narrower range of difficulty levels and revealed lower thresholds (worse performance) among patients (p < .001). This group difference remained even when only the first half of the trials was analyzed (p = .001). Thus, the JOVI-2 provides a brief, sensitive measure of visual integration deficits in schizophrenia. Neural implications and potential future applications of the JOVI are discussed.

Introduction

A goal of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) initiative is to develop measures of specific cognitive functions that are grounded in cognitive neuroscience. Ideally, the measures should have known links to neurobiology (as documented through pharmacological manipulations, animal studies, and imaging studies); they should be brief and acceptable to patients (ie, not causing frustration or fatigue); and they should be in a format that can be administered and scored in typical clinical settings.

A cognitive domain designated as high priority by CNTRICS was visual integration. Also known as “perceptual organization,” visual integration is “the process by which the bits and pieces of information that are present in the retinal image are structured into the larger units of perceived objects and their interrelations¹ (p. 723).” It occurs one step beyond the registration of color, orientation, motion, and depth. Visual integration impairments in schizophrenia were first described in clinical reports in the 1950s, and research on the dysfunction goes back to the early 1960s, with a rapid increase since the early 1980s.² According to a recent review,² 55 of 61 studies demonstrated some kind of impairment. These studies suggest that (1) task performance is unimpaired for simple, symmetrical, closed figures (eg, triangles) that can be processed by mechanisms that are present at birth or shortly therafter³; (2) task performance is most impaired for novel, noisy, or highly fragmented forms that require significant contributions from later developing mechanisms⁴ and top-down factors⁵ (eg, memory, expectation, strategy); and (3) performance is most abnormal among patients with disorganized symptoms and poor premorbid social functioning.²

Visual integration has been widely studied using variants of a contour integration (CI) paradigm.^6–8 In a typical experiment, participants are shown a set of spatially separated elements and asked to make judgments about how/whether a subset of those elements forms a single contour. The elements are usually Gabor patches—Gaussian-modulated sinusoidal luminance distributions that model the receptive field structure of cells in primary visual cortex (V1) (see figure 1). Two biological mechanisms are especially relevant for CI. The first is long-range horizontal connections between orientation-tuned spatial frequency detectors in V1 and V2 (especially important for integrating elements within 2° of visual angle^9,10); the second is reentrant feedback from V2 or higher visual areas (most important for grouping more distantly spaced elements).^11–13 Importantly, CI cannot proceed purely by local filters or by orientation-tuned neurons with large receptive fields.¹⁴

Fig. 1.

Task and stimuli for JOVI-1. Top left panel depicts the 2 basic shapes that subjects discriminated. Other panels show examples of stimuli from several of the conditions across the 2 versions of the task. The stimuli on the left are rightward pointing and those on the right are leftward pointing.

Several methods have uncovered the neural mechanisms underlying CI and related processes. Single-cell studies in V1 suggest excitatory (facilitating) effects when contour elements are collinear, but not orthogonal, with a central target.¹⁵ Functional magnetic resonance imaging (fMRI) data in humans^16,17 and monkeys¹⁶ indicate that V1, V2, V3, V4, and the lateral occipital complex are more activated when processing Gabor-defined contours, in contrast to stimuli with randomly oriented elements. In studies of amblyopia, human and nonhuman primates have reduced CI ability^8,18 and reduced visual cortex (blood-oxygen-level dependent) activation (V1 and beyond).^19,20 These reductions are specific to the eye affected, which further highlights the tight relation between early cortical activation and integrated contour perception.

CI has been repeatedly shown to be impaired in schizophrenia.^7,17,21–26 The degree of impairment correlates with level of disorganization, and task improvement during treatment for acute psychotic episodes covaries with disorganized symptom reduction (for reviews, see Silverstein and Keane², Silverstein et al²¹, Uhlhaas et al^22,24, and Uhlhaas and Silverstein²⁷). In addition, abnormal CI in schizophrenia is associated with reduced activity (compared with healthy controls) in V2, V3, and V4, areas crucial for integration.¹⁷ In this and other studies, patients demonstrated increased (compensatory) activation in later regions associated with form processing (eg, fusiform gyrus, temporal gyri) and reduced activity in frontal and parietal areas involved in attention. The frontal findings may reflect reduced object-based attentional activity²⁸ or template matching²⁹ in the face of weakened stimulus assembly processes. The parietal findings may reflect reduced attentional capture by the contour stimulus or problems disengaging and reengaging attention to scan the entire stimulus. All the foregoing differences were observed even when patients and controls were matched on task accuracy.

Here, we report 2 studies that further develop a variant of a CI paradigm,^7,17 which we term the Jittered Orientation Visual Integration task (JOVI). The JOVI was employed for reasons noted above: many studies with human and nonhuman primates have used variants of the task; the neural signatures of CI have been uncovered via fMRI, electroencephalogram, and single-cell recording; schizophrenia patients consistently exhibit CI deficits, and these correlate with important clinical variables; and the task has demonstrated test-retest reliability and minimal practice effects. These and other virtues motivated the CNTRICS initiative to select CI as a viable metric for assessing visual integration problems in schizophrenia.³⁰ Our goal in the present article, therefore, was to develop, optimize, and validate the JOVI for clinical trial research. More specifically, our goals were to (1) validate the task with the largest sample to date; (2) clarify the testing conditions and number of trials that best differentiate patients and controls; and (3) show that this group discrimination can occur in clinical trial settings (eg, across multiple testing sites with standard overhead lighting, no chinrest stabilization, etc.).

Study 1: JOVI With a Broad Range of Jitter Values

Subjects

The study was conducted across 5 sites: University of Medicine and Dentistry of New Jersey (UMDNJ), University of Minnesota (UM), Maryland Psychiatric Research Center (MPRC), University of California-Davis (UCD), and Washington University in St Louis (WU). There were 76 control subjects and 105 schizophrenia outpatients. The total number of subjects (and patients) at each site was UMDNJ-30 (14), UM-30 (19), MPRC-37 (23), UCD-47(24), and WU-37(25). Data on male:female ratio and demographic factors can be found in table 1. The groups differed on expected variables (eg, years of education, socioeconomic status) that are typically associated with schizophrenia. See Henderson et al (this issue) for information on participants, recruitment strategies, inclusion/exclusion criteria, training of clinical raters, clinical assessment of participants, general testing procedures, and order of testing. All subjects reported normal or corrected-to-normal vision. This was verified by acuity testing, using a Snellen chart, in study 2 only.

Table 1.

Demographic Data on the Schizophrenia and Control Groups for Both Studies

	Patient Mean	Patient SD	Control Mean	Control SD	Statistic for Group Difference	df	P Value
Study 1: JOVI-1
Age	38.91	11.80	37.63	12.13	t = 0.71	177	.48
Years of education	13.08	2.24	14.79	2.18	t = −5.08	176	<.001***
Mother’s years of education	13.30a	2.74	13.85	2.85	t = −1.27a	167	.21
Father’s years of education	13.24	3.38	13.32	3.20	t = −0.16	152	.88
SES	25.45	10.87	39.17	10.87	t = −8.33	173	<.001
Parental SES	42.6	15.2	46.1	12.3	t = −2.46	173	.02
Gender (% male)	62.5%		48.0%		Chi-square = 3.72	2	.05
Race (% caucasian)	50.0%		54.7%		Chi-square = 7.72	7	.36
Study 2: JOVI-2
Age	41.35	10.30	37.11	9.82	t = 2.09	98	.04
Years of education	13.57	2.10	14.92	1.83	t = −3.44	100	.001
Mother’s years of education	13.16	2.95	12.67	1.91	t = 1.00	96	.37
Father’s years of education	14.14	3.83	12.39	2.47	t = 2.67	92	.02
SES	26.26	8.11	38.17	9.50	t = −6.61	100	<.001
Parental SES	43.2	15.5	40.3	11.2	t = 1.04	100	.30
Gender (% male)	59.5%		63.3%		Chi-square = 0.152	2	.70
Race (% caucasian)	61.9%		60.0%		Chi square = 7.81	6	.25

Note: SES, socioeconomic status.

^aEqual variances not assumed.

Apparatus

The task was implemented in E-prime 2.0. The stimuli were generated with a Dell computer (3.0 GHz, 6M, 1333 MHz front side bus) running Windows XP. The visual displays were presented on a Samsung 2243BWX LCD monitor with viewable dimensions of 47.5 by 29.8 cm. The viewing distance was approximately 24 inches (609.6 mm). The screen resolution was 1680 × 1050, and therefore, the viewable screen subtended approximately 43° × 27° of visual angle. Spyder 3 Elite software was used to calibrate the monitors across sites at the start of the study and then weekly afterwards. Monitors were set to a gamma value of 2.2 and a white point of 6500 K—standard brightness and contrast settings for PC system monitors. Standard overhead fluorescent lights were used at all 5 sites, with no additional lamp lighting. Overall, our goal in the setup was to both minimize variability between sites while at the same time using only procedures and equipment that would be expected to be available in standard clinical trial settings (no chinrest stabilization, no gamma linearization, etc.).

Stimuli and Procedure

On each trial, a single stimulus, with either a leftward- or a rightward-pointing closed contour, was presented (see figure 1). Subjects pressed 1 of 2 keys to indicate the pointing direction of the stimulus. This so-called “symmetric 1-alternative forced choice” method requires only one stimulus presentation per trial. It therefore allows more trials to be collected per session than a sequential 2-alternative forced choice method; it also does not require the subject to make memory-intensive comparisons across locations, as would be needed in a spatial 2-alternative forced choice task.³¹ Trials were blocked according to the amount of orientational jitter that was added to the contour elements: ±0°, 7°–8°, 11°–12°, 15°–16°, 19°–20°, or 23°–24° (see figure 1). An advantage of having a broad range of jitter values was that we could plot each subject’s complete psychometric function—from floor to ceiling. Blocks were presented in increasing order of difficulty, and this 6-block sequence was repeated 4 times. Each block had 10 trials to produce an experiment of 240 trials.

Prior to the actual task, 4 demonstration stimuli (2 left pointing and 2 right pointing) at the 0° jitter level were sequentially shown on the screen—without a time limit—until the subject reported seeing the contour. Next, four 0° jitter stimuli were shown with a rectangular outline around the location of the contour; these were shown for 2 seconds each in order to clarify the appearance of the shape alternatives. Subsequently, a block of 8 additional timed-practice trials with no additional cues, at 5° jitter, were given to further familiarize subjects with the task. If subjects got 7 of these trials correct, the practice phase ended and they proceeded to the actual task. If fewer than 7 practice trials were answered correctly, they could receive up to 4 additional blocks of 8 practice trials with the same criterion. If a subject answered fewer than 7 trials correct on all practice blocks, it was up to the experimenter to judge whether the subject understood the task sufficiently to continue with the actual task.

During the actual task, stimuli were presented for 2 seconds each, followed by a 1-second interstimulus interval (during which responses were not scored). Only a gray background appeared during the interval. The stimulus appeared on a black screen and subtended 16.9° × 12.8° of visual angle. Each stimulus itself consisted of 207 distractor Gabor elements, 15 target elements, and a gray background. The average distance between adjacent Gabors was 1.4°, which is well within the 2° spatial window typically needed for CI to happen without the aid of high-level feedback.^9,10,32 All elements were identical except for their positions and orientations. The width of each Gabor was 0.4°. The Gabor wavelength was approximately 0.4°. The luminance of the gray background was approximately 23 cd/m². The peak and trough luminance values for a Gabor corresponded to ∼192 and ∼2 cd/m2, respectively. The distance between adjacent distractors divided by the distance between adjacent contour elements (ie, delta or signal-noise ratio) was 0.9 so that density cues could not aid in the task. Note that at delta = 0.9, schizophrenia patients can reliably perceive contours under conditions of zero-to-minimal jitter.²¹

Data Analysis

Prior to carrying out all analyses, every other timed-out trial during the actual task (ie, not during practice) was coded as a correct response, so that chance performance would be 50% regardless of whether a subject preferred to guess or time-out on a trial. Next, we performed simple t test comparisons for each jitter level and also a 2 (group) × 6 (jitter) mixed-model ANOVA. We also fit the data with a psychometric curve and compared threshold between groups. (Threshold corresponds to the amount of jitter needed to reach ∼81% for an observer who performs perfectly at the easiest jitter levels.)

Although curve fitting analyses are less common in clinical research, this method has been well established for over 35 years and is the standard data analytic technique in the field of visual perception (including CI).^33,34 One reason for this is their ability to characterize data that are nonlinear. Perceptual performance in many tasks (including ours) follows a characteristic shape: as task difficulty transitions from very hard to very easy, accuracy increases slowly at first, then increases rapidly, and finally climbs more slowly again toward ceiling. This well-established relation between difficulty and accuracy can be described by a variety of sigmoidal (s-shaped) functions, perhaps the most common of which is the cumulative Weibull.³⁵ Fitting a function typically involves calculating accuracy for each condition and finding the parameters that produce a best-fitting curve (in a least squares sense). There are commonly 3 fitted variables, and each denotes a different aspect of performance.³⁶ Threshold (α) measures how much jitter a subject needs to perform at a certain level of accuracy. In our case, higher threshold corresponds to better performance. The slope parameter (β) determines how much a subject's accuracy changes for a given change in jitter (where steeper slopes correspond to higher sensitivity to jitter changes). The upper asymptote parameter (λ) represents how well that the subject performs at the easiest difficulty levels. Nonzero λ values usually, but not necessarily, reflect errors associated with lapses in attention (as discussed below). In sum, a curve fitting approach provides a well-established means to evaluate 3 different aspects of performance and therefore complements and extends the findings derived via standard accuracy-based ANOVA analyses.

Several steps were performed to prepare the data for curve fitting. We removed subjects whose overall proportion correct was less than or equal to 50%. Next, we fit a psychometric curve to each subject's data and removed subjects who could not be modeled with such a curve (see below).

For the analyses, we compared the fitted parameters across groups and also the R-squared values across groups. The latter index served as a goodness of fit statistic and quantified an observer’s variability (noisiness) around a predicted performance pattern (independent of accuracy); therefore, this can also be viewed as a check on internal consistency or degree of error variance in the data. Between-group comparisons were made with nonparametric (Wilcoxon rank sum) tests because the compared variables were not normally distributed on either task.

Curve fitting was done with Matlab's curve fitting toolbox. To fit a curve, we first recoded the x dimension as being distance from a baseline jitter level of 23° to establish the conventional monotonically increasing relationship between the dependent and independent variables. We chose a baseline of 23° because performance at this level was at chance and thus seemed like a reasonable point from which to measure performance. Next, each subject's accuracy data were fit to a cumulative Weibull distribution function³⁷:

Ψ corresponds to the predicted probability correct, and x denotes the recoded jitter values; α and β determine the threshold and slope of the psychometric function; gamma is the guessing rate and was fixed at .5; and λ refers to the upper asymptote of the fitted curve and traditionally corresponds to the stimulus-independent error rate. (Note that a threshold for a Weibull curve in a test with 2 response options corresponds to the jitter value needed to produce 81.6% accuracy; the accuracy percentage is lower if the subject has a positive λ value.) Six subjects (all patients) were excluded for failing to perform above 50%. An additional 6 subjects (5 patients) were excluded because their data could not be modeled with the above function. Parameter estimates for these subjects did not converge because the proportion correct did not improve with reduced jitter or because data were missing for one or more jitter values.

Results and Discussion

Data from study 1 are presented in figure 2. The main effect of condition (jitter) was significant, indicating that the manipulation produced the intended effect on visual integration: F(2.87,502.09) = 1063.46, P < .001, ${\eta }_p^2=.859$. There was also a significant effect of group, with patients performing worse overall than controls: F(1,175) = 18.32, P < .001, ${\eta }_p^2=.095$. Patients were significantly worse than controls in the 0°, 7°, and 11° conditions. There was an interaction between group and condition such that patients performed more like controls as the task became harder, F(2.87,502.1) = 7.81, P < .001, Greenhouse-Geisser.

Fig. 2.

Data from study 1. Proportion correct is plotted against jitter magnitude for controls (C, in black) and persons with schizophrenia (SZ, in red). Individual points denote group averages at each of the 6 jitter values and include only those subjects who were included in the curve analysis. For illustration, a cumulative Weibull curve is drawn through the 6 averaged data points for each group. Note that difficulty level decreases from left to right. Also, while a full range of difficulty levels is plotted, corresponding to all points along the fitted curves, the actual conditions included in the task were those listed on the x-axis, namely, 0°, 7°–8°, 11°–12°, 15°–16°, 19°–20°, and 23–24° of jitter.

The curve fitting provided additional insights. The median R-squared values exceeded 0.96 in each group indicating that the data conformed well to the classic sigmoidal shape, though the patients’ data were marginally noisier (Z = 1.83, P = .068). More importantly, there was neither a threshold difference between groups (Z = 1.34, P = .18; see figure 2) nor any difference in slopes (Z = 0.20, P > .8), indicating no real between-group difference in the effect of orientation jitter on visual integration. The only clear between-group difference was in the λ values, which correspond to the upper asymptote of the curves (Z = 3.34, P = .001; Mdn(patients) = 0.035; Mdn(controls) = 0.02), indicating that patients made more errors at the easiest conditions.

In this, the largest study to date of CI in schizophrenia, patients were less accurate overall than controls, were similar to controls in terms of threshold and slope, and differed on λ. Three caveats are important to mention, however. First, although λ traditionally reflects error rates that are independent of task difficulty (eg, those due to lapses in attention),³⁷ it is possible that patients really are worse at integrating even nonjittered stimuli. More on this will be said below. Second, our results are inconsistent with the findings of 2 past studies,^7,17 in which patients and controls did not differ on the zero-jitter condition. The discrepancy may owe to sample size differences (with a larger sample in this study) or to the fact that the subjects in previous studies were allowed to practice the task until they performed at 80% accuracy on the easiest condition (which in turn may have provided more training for patients). Therefore, past studies may have underestimated patient-control differences at the easiest jitter levels. Finally, the jitter levels may have been too close to floor and ceiling to sensitively identify threshold differences. Because threshold is most accurately estimated when the difficulty levels are closer to the steepest part of the psychometric function (where lambda estimates are less influential on threshold estimates),³⁶ we selected a narrower range of intermediate jitter levels for the next study and added additional conditions within that range.

Study 2: JOVI With a Narrow Range of Jitter Values

Subjects

Floor and ceiling effects uncovered in study 1 prompted us to modify the JOVI task approximately 2/3 of the way through the planned data collection effort. As a result, for study 2, the sample sizes, while still larger than usual, were smaller than in study 1 and are less matched in terms of group sizes. Study 2 included 60 controls and 43 schizophrenia patients. The total number of subjects (and patients) per each site was as follows: UMDNJ-9(9), UM-30(11), MPRC-27(11), UCD-10(4), and WU-27(8). None of the subjects in study 2 had participated in study 1. Data on male:female ratio and demographic factors can be found in table 1.

Stimuli and Procedure

The JOVI-2 included the following conditions: 7°–8°, 9°–10° (new), 11°–12°, 13°–14° (new), and 15°–16°. Stimuli were blocked by condition, with 12 trials per block. Blocks were presented in increasing order of difficulty, and each was presented 4 times for a total of 240 trials (4 repetitions × 5 blocks × 12 trials).

The stimulus subtended a 13.3 × 13.3 square. The Gabor wavelength and width was 0.2°. The numbers of target and distractor elements were 18 and 298, respectively. Target elements were separated by 1° of visual angle. Other aspects of the experiment, including practice and stimulus exposure duration, were the same as in study 1.

Data Analysis

The data analysis procedure was the same as in study 1. In the curve fitting, 2 subjects (both patients) were removed for performing at or below 50% correct and 2 additional subjects (1 patient) could not be modeled with a curve.

Results and Discussion

Data from study 2 are presented in figure 3. The main effect of condition (jitter) was significant, indicating successful manipulation of visual integration: F(3.37,340.23) = 222.99, P < .001, ${\eta }_p^2=.688$. There was also a significant effect of group, with patients performing more poorly than controls: F(1,101) = 13.44, P < .001, ${\eta }_p^2=.117$. As shown in figure 3, patients were significantly worse than controls in the 7°, 9°, and 11° jitter conditions (see table 2 for estimates of discriminating power in each condition). As in study 1, there was an interaction, F(3.37,340.2) = 5.76, P < .001, Greenhouse-Geisser, such that patients performed more like controls as the task difficulty increased. These accuracy findings did not depend on testing site (P values for effects of site, site × jitter, site × group, and site × group × jitter, all > .10). In addition, although the patients were older than controls, age did not significantly correlate with performance within or across groups (all P values > .6).

Fig. 3.

Data from study 2. Proportion correct is plotted against jitter magnitude for controls (C, in black) and persons with schizophrenia (SZ, in red). As before, the individual data points denote group averages at each jitter value and include only subjects who were included in the curve analysis. For illustration, a cumulative Weibull curve is drawn through the 5 averaged data points for each group. Note that each point on the x-axis corresponds to an actual degree of jitter measured by the task, which included conditions of 7°–8°, 9°–10°, 11°–12°, 13°–14°, and 15°–16° of jitter.

Table 2.

Between-Group Effect Sizes, Discriminating Power for Each Jitter Condition, and the Data Used to Calculate These

	Coefficient Alpha	Coefficient Alpha	SD	SD	Variance	Discriminating Power	Mean	Mean
Condition	Controls	Individuals with schizophrenia	Controls	Individuals with schizophrenia	Controls only		Controls	Individuals with schizophrenia	Pooled SD	Cohen's D
7° Jitter	.698	.876	0.10338	0.17394	0.01068742	0.007459822	0.7965	0.7004	0.14417	0.66657418
9° Jitter	.709	.794	0.10816	0.1434	0.01169859	0.008294297	0.7753	0.6726	0.13328	0.77055822
11° Jitter	.754	.764	0.1311273	0.1424604	0.01719437	0.012964554	0.680556	0.565476	0.1467046	0.78443348
13° Jitter	.544	.789	0.10528	0.15121	0.01108388	0.00602963	0.5271	0.4826	0.12743	0.34921133
15° Jitter	.651	.809	0.11862	0.15573	0.0140707	0.009160029	0.4438	0.4172	0.13504	0.19697867

Note: Only raw (unprocessed) data are shown. Discriminating power was calculated as reliability (expressed as coefficient α for each condition) × variance. The latter was calculated based on control data only. Probability (P) values of “.000” represent values where P < .001. Note that coefficient α is a reliability statistic and should not be confused with the parameter α described in the curve fitting analyses, which refers to threshold value.

Overall, the curves fit the data reasonably well for both groups, with a median R-squared value of .79 in the patient group and .88 in the control group. As before, the R-squared values differed marginally between the groups (Wilcoxon rank sum, Z = 1.92, P = .055). The groups were undifferentiated in the slope parameter, as before (Z = 1.48, P = .14). Because estimates of λ are accurate only if performance is at or near ceiling for some of the conditions (which was not the case for study 2, unlike study 1), we did not analyze this parameter in the curve fitting process for this study. Most importantly, thresholds were reliably higher (better performance) in controls than in patients (Z = 3.60, P < .001; see figure 3). Because we employed a nonparametric test, this outcome cannot be attributed to a small number of outliers. Moreover, our outcomes were robust and did not strongly depend on the specific assumptions in the curve fitting analysis. For example, the same results were obtained if the x-axis was coded as a distance from a different baseline jitter value or if the free parameters were constrained differently.

An important question is whether similar effects can emerge with fewer trials, reducing the time burden on both patients and testers. To examine this issue, we analyzed data from only the first half of the experiment (ie, the first 120 trials). Once again, there was a significant effect of condition: F(3.57,357.20) = 162.97, P < .001, ${\eta }_p^2=.620$. The main effect of group was also significant, with patients performing more poorly than controls: F(1,100) = 12.74, P = .001, ${\eta }_p^2=.113$. The group by condition interaction was also significant, with the group difference decreasing as conditions became more difficult: F(3.57,357.2) = 3.75, P = .007, Greenhouse-Geisser correction applied. For the curve fitting, 4 subjects (all patients) were excluded for chance performance and 2 additional subjects (1 patient) were excluded for poorly fitted curves. Two noteworthy results were obtained: (1) “shallower” slopes for patients than controls (Z = 2.64, P = .008) and (2) “better” (higher) thresholds for controls than patients (Z = 3.26, P = .001). The first result suggests that patients were less sensitive to the jitter manipulation; it should be considered preliminary, however, because slope differences were not found in the other analyses. The threshold difference, on the other hand, is consonant with our other results and shows that patient integration deficits emerge with as few as 120 trials.

General Discussion

Results from the second study—using a revised version of the JOVI—replicate past studies in indicating poorer CI performance among people with schizophrenia. As noted, the null result for the threshold differences of study 1 most likely owes to the selection of jitter levels. In that study, only 1 of the 6 conditions yielded a proportion correct (for each group) between .65 and .85 (ie, between floor and ceiling; NB: a proportion correct of .65 or greater is necessary to statistically exceed chance). In contrast, in study 2, 3 of the 5 data points averaged to be within the .65–.85 range. The group differences in JOVI-2 did not depend on testing site and could be identified with a 120-trial experiment using either a curve fitting or an ANOVA analysis. For such reasons, the JOVI-2 is the preferred version of the task.

At least 2 factors contribute to the generalized deficit in schizophrenia, and each is worth considering in light of our results. One possibility is that patients become more frustrated and lose more motivation (relative to controls) as the task gets harder, causing between-group differences to increase with task difficulty. In this case, patients would perform worse overall relative to controls without any corresponding problem in integration.³⁸ Our results are not consistent with this pattern. The psychometric slopes were no steeper for patients than controls, and between-group differences did not increase as the task became more difficult (in fact, the opposite was found). A second type of generalized deficit confound occurs when subjects commit stimulus-independent errors, either by allocating insufficient attention toward the screen or by pressing the wrong key for a given response. In such cases, subjects will have a lower upper asymptote³⁶ and lower overall accuracy but not necessarily any dysfunction in integrating contours. While patient data did match this trend for study 1, it is important to note that patient errors might not be stimulus independent; they may actually reflect poor contour processing. The upper asymptote of the psychometric function can depend, eg, on the ability of a target to exogenously grab attention.³⁹ One possibility therefore is that less-jittered contours pop-out for healthy controls, and this, in turn, enhances form discrimination.⁴⁰ Integration deficits among patients, on the other hand, would reduce the reflexive transient attentional shift toward the target and therefore further impair form discrimination at the lowest jitter values. In line with this view, others have found reduced activity in parietal regions in the CI task for patients,¹⁷ suggesting that dysfunctional bottom-up processing of contours may weaken the normal orienting response to the integrated shapes.

On the other hand, the relationship between discriminating power (calculated as reliability [ie, coefficient α] × variance for each condition) and between-group effect size in each condition revealed inconsistent evidence for a generalized deficit involving motivation and/or attentional effects. Specifically, as can be seen in table 2 (using study 2 data), the 11° condition, which was associated with the largest between-group effect size (d = 0.78), was also associated with the highest degree of discriminating power. However, the condition with the second highest level of discriminating power (15°) was associated with the smallest effect size (d = 0.20). Therefore, there was not a consistent relationship between discriminating power and effect size across all conditions, arguing against a generalized deficit interpretation of the data.

In our current work, we are continuing to refine the task to ensure that the data reflect CI abilities rather than inattention toward the screen or poor motivation. One way we have done this is to add “catch” trials, which should only be failed by people responding randomly. Two types of catch trials have been added, and both derive from the 0° jitter condition. In one type, a black continuous contour has been drawn in through the Gabor elements composing the target (obviating the need for integration). In the other type, the background elements have been excluded (obviating the need for noise inhibition). If patients and controls perform the same on catch trials but differently on the easier jitter conditions, then that would present strong evidence for a specific deficit in visual integration. In addition, although we found that a 120-trial version of the task provides data equivalent to that of the 240-trial version, it is possible that the task could be shortened even further, thereby providing additional protection against fatigue or impaired sustained attention confounds. For example, a version with only the 7° jitter condition and catch trials may be just as sensitive as the current 120-trial version. We are currently investigating this and will report the results in a future article.

The JOVI may be useful in pharmacological challenge studies and in early-phase clinical trials. As noted, the contour linking process is thought to be implemented both via horizontal connections in V1 or V2^11,12 and reentrant feedback from post-V1 visual regions^41,42 and much is known about the underlying physiology of these processes,^43,44 (see also Shimizu et al⁴⁵ and Gais.⁴⁶) Integration deficits may owe to N-methyl D-aspartate (NMDA) receptor hypoactivity and subsequent reduced input to inhibitory gamma amino butyric acid (GABA)-ergic interneurons,^47,48 suggesting that the JOVI might be useful in studies of medications targeting these systems. Furthermore, because NMDA and GABA-ergic contributions to synchronized oscillations are modulated by cholinergic activity via muscarinic receptors,⁴⁹ the effects of anticholinergic agents on JOVI performance would also be useful to investigate. This is especially true given prior findings linking reduced synchrony to reduced visual integration in schizophrenia.² To date, studies have shown that improvements in CI covary with reductions in disorganized symptoms during short-term inpatient treatment²⁴ and that ketamine (an NMDA antagonist) impairs CI in nonpsychotic subjects.⁵⁰ However, sensitivity of test scores to specific pharmacological agents in patient populations has yet to be explored.

Ongoing studies are extending and further validating the JOVI paradigm. For example, we are examining the test reliability of the full version of the task (including catch trials). This will help clarify whether a version with fewer conditions (eg, only 7° jitter and catch trials) is as reliable as a longer version, whether an abbreviated task would lead to a reduced subject exclusion, and whether between-group differences at easier stimulus levels result from integration deficits. Other issues under investigation include the following: fMRI correlates of normal and abnormal JOVI performance in large subject samples, test-retest reliability of the fMRI findings, and diagnostic specificity of the behavioral findings (among schizophrenia and mood disorder patients). All this work is consistent with the goals of the CNTRICS initiative, which include optimizing and validating promising cognitive tasks for future clinical trial research in schizophrenia. To date, most research has not been focused on refining behavioral paradigms, perhaps because it is seen as laborious, tedious, time consuming, and less groundbreaking than investigation of new hypotheses about mechanisms of psychopathology. However, this type of foundational work is critical in that—by establishing validity and reliability of behavioral and brain imaging findings across large subject samples, multiple sites, and varying testing conditions—we can be sure that the effects are sufficiently robust and that the underlying mechanisms are indeed dysfunctional.

In conclusion, the JOVI is an easy-to-administer measure of visual integration that can be used to detect impairments in schizophrenia. Data from study 2 indicate that reliable data can be obtained with a 10-minute, 120-trial task (including instructions and practice trials; the total for just stimulus presentation and intertrial intervals is 120 trials × 3 sec = 6 min). Earlier versions of the measure suggested that it may be especially sensitive to a more severely ill patient subtype (characterized by poor premorbid social functioning^25,26), as well as state effects (ie, level of disorganization^21,22,24). It is therefore potentially useful as both a cross-sectional measure of the integrity of visual integration processes and of their reconstitution during recovery from psychosis. Its demonstrated ability to reveal visual cortex hypoactivation within an fMRI context¹⁷ also suggests that integration with psychophysiological recording can reveal valuable information about the neurobiological mechanisms involved in psychosis and recovery.

Funding

National Institute of Mental Health via 5 collaborative R01 grants to the S.M.S, D.M.B., C.S.C., J.M.G., A.M..