HAS AN IMPORTANT TEST BEEN OVERLOOKED?: CLOSURE FLEXIBILITY IN SCHIZOPHRENIA

Abstract

Deficits in visual processing are now recognized as a core feature of schizophrenia. In the 1940’s, Louis Thurstone developed a series of tests designed to evaluate specific aspects of visual perceptual processing including the Closure Flexibility Test (CFT), which was designed to measure “the ability to hold a configuration in mind despite distraction.” The present study evaluated patients’ performance on this task and its relationship to other tests of neuropsychological function, particularly to a measure of sustained visual attention. Thirty-nine patients with schizophrenia or schizoaffective disorder and 40 controls participated. The CFT was administered both in its original form (10 min) and also in a briefer form (3 min) in which only a portion of stimuli were given. Patients showed highly significant large effect size deficits on both the original (d=1.6) and brief (d=1.2) CFT. Between-group deficits in performance survived covariation for IQ. In addition, the CFT score was significantly related to performance on the MATRICS measure of attention/vigilance, the Continuous Performance Test-Identical Pairs version (CPT-IP). This correlation remained significant even after controlling for non-specific intercorrelations among neurocognitive measures. Results confirm the severity of early visual processing deficits in schizophrenia. In addition, the CFT is a brief, easy to administer alphabet-independent, paper-and-pencil test with established psychometric properties that may be useful as an index of the sustained visual attention construct in schizophrenia.

1. Introduction

Schizophrenia is a complex disorder associated with information processing deficits across a range of cognitive domains. Deficits in visual processing manifest as reduced ability to maintain sustained visual attention (Cornblatt et al. 1988), as well as reduced ability to decode complex information (Crookes, 1984; Kurachi et al. 1994). In the consensus Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) test battery (Nuechterlein and Green, 2006) sustained visual attention is evaluated using the identical pairs continuous performance test (CPT-IP). Other tests sensitive to visual dysfunction in schizophrenia include the Perceptual Organization Index (POI) and the Processing Speed Index (PSI) of the Wechsler Adult Intelligence Scale-III (WAIS; Wechsler 1997a) which are known to be abnormal in schizophrenia (Dickinson et al. 2004).

Although most tests of visual processing assess relatively high-order aspects of visual dysfunction, recent studies suggest that patients show deficits even in relatively simple processes (Butler et al. 2008, for review; Chen et al. 2005; Keri et al. 2004). The degree to which the various measures interrelate, as well as the identification of paper-and-pencil instruments sensitive to various levels of visual dysfunction in schizophrenia, remain areas of active investigation.

The present study evaluates the sensitivity of the Closure Flexibility-Concealed Figures Task (CFT; Thurstone and Jeffrey 1984a; 1984b) to cognitive dysfunction in schizophrenia and, in particular, to the construct of sustained visual attention. In his pioneering factor analytic study in the 1940s, Thurstone examined sixty perceptual measures, including Gottschaldt Figures which were an early form of the CFT. He identified 11 factors, seven of which were interpretable (Holzman 1972; Thurstone 1944) and named them: A) ability to hold a configuration in mind despite distraction including strength of configuration; B) proneness to classical illusions; C) reaction time; D) speed of alternation of gaze; E) “freedom from Gestaltbindung” involving flexibility in manipulating several more or less irrelevant or conflicting gestalts; F) speed of perception; and J) speed of judgement. The Gottschaldt figures, along with other tests, loaded significantly on factors A and E. These two factors have shared variance, as seen by the correlation of 0.38 that Thurstone (1944) found between them. Thurstone (1951) later found that the Gottshaldt Figures loaded significantly only on his second closure factor which he defined as “ability to hold a configuration in mind despite distraction.”

In the present version of the CFT, participants are shown a series of relatively simple, reference figures and required to state whether or not these figures are embedded within more complex test pictures. Although the test is timed (10 min) scoring is performed based only upon whether evaluations are correct or incorrect for the items completed. Both reference and probe stimuli remain visible throughout the test, so correct performance requires primarily comparison and contrast of images to evaluate simple shapes embedded within more complex figures. The task format prevents both images from being centered in the field of view simultaneously, so the experience of the task is of needing to hold the target shape in mind, even while ignoring the extraneous visual information in the more complex figure, as originally proposed by Thurstone (1944). However, this should not be confused with the construct of “holding and manipulating mental information” subsequently proposed by Baddeley (1992) and others to refer to “central executive” type working memory systems.

Scaled scores for the CFT can be calculated based upon results from normative samples, and used for between-group comparisons. The CFT has been shown to have strong psychometric properties as seen in split-half reliability of 0.78 and 0.94 in two studies (Pemberton, 1951; Thurstone, 1944). To date, however, test-retest reliability and potential learning effects have not been evaluated, nor are alternate forms available.

While the CFT is a paper-and-pencil task with simultaneous display of information, more recent tests of visual attention have tended to use sequential display within the continuous performance task (CPT) format. There are a number of versions of this task including the degraded CPT, the CPT-IP, and the AX-CPT (Cornblatt et al. 1988; Delawalla et al. 2008; Javitt et al. 2007; Rissling et al. 2005). The CPT is widely used in schizophrenia research and is currently considered the “gold standard” measure of visual attention. The CPT-IP was chosen for use in the MATRICS battery (Nuechterlein and Green, 2006).

For the present study, CFT performance was evaluated in schizophrenia patients relative to controls. Further, within patients, CFT performance was evaluated relative to neuropsychological tests sensitive to discrete cognitive domains including CPT-IP. In addition, a brief 3-minute version of the CFT was administered. The goal of this study was to compare the visual attention construct developed by Thurstone (1944) and subsequently adopted by other early-stage visual researchers in schizophrenia (e.g. Holzman 1972) to the construct as currently implemented. In addition, the study evaluates the sensitivity of the CFT to neurocognitive dysfunction in schizophrenia and provides initial assessment of applicability in this population.

2. Experimental methods

2.1. Participants

This study was approved by the Nathan Kline Institute IRB. All participants provided written informed consent following full explanation of experimental procedures. Patients were recruited from outpatient and inpatient units. Diagnoses were obtained using the Structured Clinical Interview for DSM-IV (SCID; First et al. 1997) and available clinical information. Controls with a history of SCID-defined Axis I psychiatric disorder were excluded. Participants were excluded if they had any neurological or ophthalmologic disorders that might affect performance or met criteria for alcohol or substance dependence within the last six months or abuse within the last month. All participants had at least 20/32 corrected visual acuity on the Logarithmic Visual Acuity Chart (Precision Vision, LaSalle, IL).

Participants who received the Standard Version of the CFT included 18 patients meeting DSM-IV criteria for schizophrenia or schizoaffective disorder and 23 comparison subjects (Table 1). Patients did not differ in age (t(39)=0.8, p=0.4), gender ratio (Fisher’s exact test; p=1.0) or parental socioeconomic status (t(33)=1.1, p=0.3), but did differ in IQ ( t(30)=3.1, p=0.004). All patients were taking atypical antipsychotic medication at the time of testing. Chlorpromazine (CPZ) equivalents were calculated using conversion factors described previously (Hyman et al. 1995; Jibson and Tandon, 1998; Peuskens and Link, 1997; Woods, 2003).

Table 1.

Demographic and Clinical Characteristics

Standard CFT	Controls (n=23)	Patients (n=18)
Age	36.5±11.5	38.9±7.2
Gender (M/F)	12/11	10/8
Parental Socioeconomic Status	50.1±10.3 (n=23)	43.3±19.7 (n=12)
Quick Test IQ	110.6 ± 10.3 (n=17)	98.7 ± 11.6* (n=15)
Schizophrenia/Schizoaffective Disorder		14/4
Chlorpromazine daily equivalent, mg		1192.4 ± 552.8
BPRS total score		34.5 ± 10.1 (n=15)
SANS total score (including global scores)		32.7 ± 17.2 (n=15)
Brief CFT	Controls (n=17)	Patients (n=21)
Age	34.6±9.9	34.9±8.4
Gender (M/F)	15/2	20/1
Parental Socioeconomic Status	41.8±12.6 n=17	39.6 ± 13.4 n=13
Quick Test IQ	109.1 ± 7.0 (n=16)	98.7 ± 8.1* (n=19)
Schizophrenia/Schizoaffective Disorder		18/3
Chlorpromazine daily equivalent, mg		1086.7 ± 522.4
BPRS total score		45.4 ± 12.5
SANS total score (including global scores)		44.7 ± 13.5

Values are mean±SD. Numbers of subjects per group are noted when there is missing data. Socioeconomic status was measured by the 4-factor Hollingshead Scale (Hollingshead, 1975). IQ was measured using the Quick Test (Ammons and Ammons, 1962). Abbreviations: CFT, Closure Flexibility Test; M, male; F, Female; BPRS, Brief Psychiatric Rating Scale (Overall and Gorham, 1962); SANS, Schedule for Assessment of Negative Symptoms (Andreasen, 1984).

^*p<0.05

Participants who received the Brief Version of the CFT included 21 patients meeting criteria for schizophrenia or schizoaffective disorder and 17 comparison subjects (Table 1). Separate groups of participants received the brief and standard CFTs. Patients and controls were of similar age (t(36)=0.1, p=0.9) and did not differ on gender ratio (Fisher’s exact test; p=0.6) or parental socioeconomic status (t(28)=0.5, p=0.6), but did differ on IQ (t(33)=4.0, p<0.001).

2.2. CFT

2.2.1. Standard CFT

The standard CFT (Thurstone and Jeffrey, 1984a) was administered according to published guidelines. The CFT is a 49-item test that displays a figure on the left that is embedded within several of the 4 complex drawings that are displayed to its right. Participants are given 10 minutes to do as much as they can and are asked to put a check mark under the complex drawings that contain the figure and a “0” under drawings that do not. A scaled score is derived from total correct minus total incorrect responses (Thurstone and Jeffrey, 1984a; 1984b) Percent correct, false positives, and false negatives can also be calculated.

2.2.2. Brief CFT

The first 14 items of the standard CFT were administered. Participants were given three minutes to complete the task. Scoring, including percent correct, was the same as for the standard CFT, except that scaled scores were not available for this brief administration.

2.3. Other Neuropsychological Tests

Neuropsychological measures were administered to patients who agreed to participate in additional testing as follows: 1) The CPT-IP is a computer-generated and timed measure assessing attentional capacity with increased load during continuous performance on trials of 2, 3 and 4 digits (Cornblatt et al. 1988). CPT-d’ is derived from hits and false alarms and was the variable used in this study; 2) The PSI is a factor score derived from scaled scores for performance on the Digit Symbol and Symbol Search subtests of the WAIS-III (Wechsler, 1997a) and reflects speed of processing; 3) The POI is a factor score derived from scaled scores for performance on the Picture Completion, Block Design, and Matrix Reasoning subtests of the WAIS-III (Wechsler, 1997a) and reflects perceptual organization; 4) The WMI is a factor score derived from scaled scores for performance on Letter Number Sequencing and Spatial Span subtests of the WMS-III (Wechsler, 1997b) and reflects verbal and visual working memory; 5) The BVMT-R is a standardized visual learning and memory task (Benedict et al. 2007). The raw score for the sum of all immediate recall trials (1–3) was the variable used; 6) The WMS-III Logical Memory Test (LM) involves narrative recall of verbal material read aloud (Wechsler, 1997b). The scaled score for total recall on the LM subtest was the variable used; 7) The Quick Test IQ measure is based on the selection of an appropriate visual scene that depicts increasingly difficult vocabulary cues (Ammons and Ammons, 1962). Scaled score is based on total correct items and was the variable used.

2.4. Statistical Analysis

Between-group comparisons were performed by student t-test or analysis of variance (ANOVA) as appropriate. Secondary analyses were carried out using IQ as a covariate. Relationships among CPT-IP and CFT were assessed using Pearson product-moment correlations followed by step-wise regression to determine whether the CFT contributes to a significant amount of the variance in CPT-IP performance after other factors are taken into account. Effect sizes for patients and controls were computed using the standard deviations from patient and control groups, respectively. Neuropsychological test scores, with the exception of the Quick IQ test, were only available for the group who received the Standard CFT and only patients received neuropsychological testing.

3. Results

3.1 Standard CFT

3.1.1. Between-group effects

Patients showed significant, large effect size deficits compared with controls on CFT performance, consistent with the a priori hypothesis (Table 2). Patients showed significant deficits in percent correct, with differences driven largely by the number incorrect rather than correct responses. Patients’ impaired performance was not due to a lack of task engagement as they completed as many or more items than controls. However, many more of their responses were incorrect, which included false positives and false negatives. Patients and controls showed similar patterns of false negatives and false positives as seen by a non-significant Group×Error type interaction (F(1,39)=0.7, p=0.41). This suggests that patients were able to perform the task without a response bias for responding with all false positives or all false negatives. The magnitude of the deficits for both the scaled score and percent correct were 1.6 SD units, which is large according to the criteria of Cohen (1988).

Table 2.

Mean (sd) Closure Flexibility Test (CFT) performance in controls (n=23) and patients (n=18) for Standard Administration

Measure	Controls	Patients	T	P	d
Total items completed	96.3 (22.9)	117.6 (44)	1.9	0.07	0.7
# Correct responses	80.5 (23.2)	70.3 (21.4)	1.4	0.16	0.5
Percent Correct	83.1 (10.9)	62.8 (14.5)	5.1	<0.001	1.6
# Incorrect responses	15.8 (10.4)	47.3 (29.1)	4.4	<0.001	1.7
- # False negatives	12.3 (8.9)	31.1 (25.4)	3.0	0.007	1.2
- # False positives	3.5 (2.8)	16.3 (17.5)	3.1	0.007	1.4
Correct –incorrect	64.7 (27.7)	22.9 (25.9)	4.9	<0.001	1.6
Scaled Scores	50.3 (9.8)	34.6 (10.2)	5.0	<0.001	1.6

In a secondary analysis using ANCOVA with percent correct performance on the CFT as the dependent measure, between group factor of cohort, and IQ as covariate, the main effect of cohort remained strongly significant (F_1,29=10.0, p=0.004). The effect of IQ as covariate was also nearly significant (F_1,29=3.09, p=0.09) but obviously did not account for most of the between group variance.

3.1.2. Within-patients effects

In order to identify the relationship between CFT deficits and sustained visual attention, a correlation was performed followed by multi-step regression analyses (Table 3). Only patients were included in this analysis (n=14 participated in neuropsychological testing). There was a significant correlation between percent correct on the CFT and d’ on the CPT-IP (r=0.67, p=0.009; Table 3). In separate regression analyses, neuropsychological tests variables PSI, POI, WMI, BVMT-R, or LM were loaded into the regression versus d’ CPT-IP performance in block 1, and then CFT (percent correct) was added in block 2. Table 3 shows that the CFT accounted for a significant component of the variance in the CPT-IP, even after PSI, POI, WMI, BVMT-R or LM were entered into the regression. In contrast, when CFT was entered first, PSI did not account for an additional significant percentage of variance in CPT performance. This supports a preferential relationship between CFT and CPT-IP.

Table 3.

Correlation between Closure Flexibility Task (CFT) performance and Continuous Performance Task-Identical Pairs (CPT-IP) prior to and following control for specific cognitive task performance in patients.

First Step	Second Step	R change	F change	Sig (p)
None	CFT	.67	9.5	.009
PSI	CFT	.56	6.3	.029
POI	CFT	.74	16.0	.002
WMI	CFT	.74	13.8	.003
BVMT-R	CFT	.65	8.8	.013
LM	CFT	.68	8.5	.015
None	PSI	.35	1.7	.216
CFT	PSI	.00	0.01	.926

Analyses were performed using step-wise regression, with indicated variables forced into the regression in the first and second steps and Identical Pairs Continuous Performance Task (CPT-IP) as the dependent variable. CFT vs. CPT-IP correlations remained significant even following correction for non-specific inter-correlation among neuropsychological measures using processing speed index (PSI), perceptual organization index (POI), working memory index (WMI), Brief Visuospatial Memory Test-Revised (BVMT-R), and the Logical Memory Test (LM). Note that R change is the square root of R² change. In the instance when CFT is entered alone vs CPT-IP, with no previous variable entered, R change is the correlation between CFT and CPT-IP.

3.2. Brief CFT

On the Brief CFT, patients also showed significant deficits compared to controls (Table 4). Patients had fewer correct responses than controls and, as in the Standard CFT, showed significantly increased incorrect responses. Also like the Standard CFT, patients and controls showed similar patterns of false negatives and false positives as seen by a non-significant Group×Error type interaction (F(1,36)=0.6, p=0.5). While the brief version is not standardized, as in the standard version the deficit in percent correct (1.2 SD units) was large. Neuropsychological data were not available for this sample.

Table 4.

Mean (sd) Closure Flexibility Test (CFT) performance in controls (n=17) and patients (n=21) for Short Administration

Measure	Controls	Patients	T	P	d
Total items completed	32.1 (9.7)	28.8 (11.5)	0.9	0.4	0.3
# Correct responses	27.1 (9.4)	20.4 (9.4)	2.2	0.04	0.7
Percent Correct	84.0 (11.5)	69.9 (11.9)	3.7	0.001	1.2
# Incorrect responses	5.0 (3.7)	8.4 (4.5)	2.5	0.02	0.8
- # False negatives	4.1 (3.4)	6.3 (3.5)	1.9	0.06	0.6
- # False positives	0.9 (0.9)	2.1 (2.4)	2.2	0.04	0.7
Correct – incorrect	22.1 (10.6)	11.9 (9.3)	3.1	0.003	1.0

In a secondary analysis using ANCOVA with percent correct performance on the CFT as the dependent measure, between group factor of cohort, and Quick Test IQ as covariate, the main effect of cohort remained strongly significant (F_1,32=5.25, p=0.03). The effect of IQ as a covariate was not significant (F_1,32=1.74, p=0.2).

3.3 Medication Effects and Symptom Ratings

There were no significant correlations between medication dose and percent correct on the Standard CFT (r=−0.16, p=0.5) or the Brief CFT (r=−0.4, p=0.07).

There were no significant correlations between Brief Psychiatric Rating Scale total score (BPRS) or the Schedule for Assessment of Negative Symptoms (SANS) total score including global scores and percent correct on the Standard CFT (BPRS: r=−0.46, p=0.08; SANS: r=0.002, p=0.9) or Brief CFT (BPRS: r=−0.37, p=0.1; SANS: r=0.05, p=0.8).

4. Discussion

Cognitive dysfunction is a major current concern in schizophrenia and a primary target for therapeutic intervention. Recent consensus batteries propose multidimensional assessments with anywhere from 6 to 12 domains (Kern et al. 2004). Although the majority of domains in neuropsychological assessment batteries can be assessed through paper-and-pencil tests, sustained visual attention is currently assessed primarily through the use of computerized tests requiring specialized programs and equipment. Although such tests are considered ideal in specialized research settings, they may be difficult to implement in less established clinical environments. The CFT is a visual processing task originally designed to assess visual attentional functioning among normal individuals.

The primary findings of the present study are first, that schizophrenia subjects show large effect-size deficits in CFT performance, and, second, that performance on this task explains a significant amount of the variation in sustained visual attention assessed with the CPT-IP even after other variables such as processing speed have been entered into the equation. The present findings thus raise the possibility that the CFT may serve as a proxy for the CPT-IP and takes less time to administer.

Scaled scores are based upon the difference between correct and incorrect responses rather than total items completed. Patients completed a similar number of items as controls in this pilot study, but made far more incorrect assessments. Incorrect assessments, moreover, included both false positives in which patients incorrectly reported embedded figures were present when they were not, as well as false negatives, in which no response was made. Importantly, because there is potential for both omission and commission errors, total non-response or chance responding does not produce extreme scores.

The CFT was originally developed to assess the “second closure factor” of Thurstone (1944; 1951), which he defined as “the ability to hold a configuration in mind despite distraction.” As compared to the CPT-IP, reference and test stimuli are present simultaneously. However, because of the complexity of the figures, frequent cross-referencing between figures is required, necessitating sequential processing.

As hypothesized, the CFT task was significantly related (r=0.67, p=0.009) to sustained visual attention assessed with the CPT-IP, accounting for ~ 45% of the variance. Furthermore, the relationship remained significant even when non-specific correlations among neuropsychological measures was controlled for using step-wise regression analyses (Table 3). In particular, although the CFT is timed, controlling for general reductions in processing speed using the PSI did not eliminate the CFT vs. CPT-IP correlation. Similarly, the correlation remained significant even after controlling for working memory using the WMI, suggesting that the CFT and CPT-IP both measure a shared underlying construct (i.e. visual attention).

Despite the venerability of the CFT task, this is the first study of which we are aware to apply this test to schizophrenia. Although its psychometric properties in schizophrenia remain largely unknown, the fact that it is well validated and standardized in normative populations, shows large effect size (d>1.0) between-group differences, is easy-to-administer and was well tolerated by our patients makes it worthy of further investigation. Interestingly, patients with autism show superior performance to controls on similar tasks (Dakin and Frith, 2005), suggesting that deficits may be relatively selective to schizophrenia over other neuropsychiatric populations.

The standard version of the CFT task requires 10 min to administer, which is shorter than the CPT-IP as presently implemented, and does not require specialized equipment. The CFT is also language-, alphabet- and culture-independent, which may make it useful for cross-national investigations. In this study, however, we also evaluated whether between-group differences could be obtained using a briefer version incorporating only 14 of the 49 stimuli and 3 min of testing time (similar to other MATRICS tasks). Significant between-group differences were obtained even with this briefer version, suggesting the possibility that subtests and alternate forms of the test can be developed. A further beneficial psychometric aspect of the CFT is that controls score between 80 and 85% correct rather than at ceiling. Psychometric tests are most discriminating when the mean response of controls is midway between chance and ceiling.

The CFT was specifically developed to test the visual attention construct, and results from the present paper support its use in schizophrenia. Furthermore, at present, the CFT is used as part of a test battery for occupational functioning (Thurstone and Jeffrey, 1984b). A major issue in schizophrenia is outcome and ability to engage in supported employment. Thus, studies looking at relationships between CFT performance and occupational functioning in schizophrenia may be useful.

In summary, ideal tests for assessment of visual dysfunction in schizophrenia are still being developed. Results from the present study suggest that a classic test of sustained visual attention - Thurstones’s CFT - is sensitive to psychopathology in schizophrenia, and that impairments in visual attention may contribute to overall neurocognitive dysfunction.