Abstract
The magnitude of the overlap among dimensions of neuropsychological test performance in schizophrenia has been the subject of perennial controversy. This issue has taken on renewed importance with the recent focus on cognition as a treatment target in schizophrenia. A substantial body of factor analytic literature indicates that dimensions are separable in schizophrenia. However, this literature is generally uninformative as to whether the separable dimensions are independent, weakly correlated, or strongly correlated. Factor analyses have often used methods (ie, principal components analysis with orthogonal rotation) that preclude this determination, and correlations among factor-based domain composites and underlying measures have been reported infrequently in these studies. Current meta-analyses of reported “between-dimension” correlations for individual neuropsychological measures and for cognitive domain composite variables indicate that cognition variables in schizophrenia are correlated, on average, at a “medium” level of r = 0.37 for individual measures from different cognitive dimensions and r = 0.45 for domain composites. Because these are mean bivariate correlations, the multiple correlation of an individual measure with all the other measures in a cognitive battery is likely to be higher. Measure reliabilities of 0.80 or less also imply greater commonality among traditional neuropsychological measures. In short, there are underappreciated constraints on the amount of reliable cognitive performance variance in traditional neuropsychological test batteries that is free to vary independently. The ability of such batteries to reveal cognitive domain–specific treatment effects in schizophrenia may be much more limited than is generally assumed.
Keywords: schizophrenia, cognition, correlation, neuropsychological assessment, clinical trials
Introduction
The Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative has catalyzed the identification and testing of promising new cognitive treatments, which could lead to important advances in schizophrenia therapeutics. The current spotlight on cognition in schizophrenia creates an opportunity, and a reason, to revisit an issue that has long nagged the schizophrenia research community: the degree to which the different neuropsychological performance domains in schizophrenia (and underlying measures) are independent of one another, weakly correlated, or substantially overlapping. A growing body of factor analytic literature supports a multifactor structure of neuropsychological test performance in schizophrenia.1,2 Some in the field have argued that these factors, and the separate groups of traditional neuropsychological variables used to measure them, are sufficiently independent of one another to be suitable for assaying discrete neural systems (eg, Egan et al3). Others have highlighted the multidimensional nature of most neuropsychological tests and the empirical relationships among cognitive domains and underlying measures and have questioned the directness of factor connections to discrete neural substrates.4,5 This discussion continues. For example, an important part of the MATRICS initiative was the development of a battery of neuropsychological tests that could provide a set of consistent outcome measures for clinical trials of cognition-enhancing drugs for schizophrenia.6 The broad sample of participating experts was unambiguous that “reliable coverage of the major separable cognitive deficits in schizophrenia” would be an “essential” feature of the battery.
Ideally, separable in this context would refer to cognitive deficits with distinct causes or neural substrates that may therefore respond to different types of pharmacological interventions. However, at a minimum, separable should refer to cognitive deficits that are distinguishable at the statistical or analytic level, such as in the results of factor analytic studies (Nuechterlein et al7(p30)).
At the same time, this requirement “raised a number of points of consideration primarily concerning the ability to differentiate distinct cognitive factors. Many experts argued that it is important to assess effects on separable cognitive constructs but that we are not able to separate them as clearly as we would like” (Kern et al8(p16), see Carter9).
Numerous trials of cognition-enhancing agents for schizophrenia are planned or under way. Some of these trials will test pharmacological agents that are thought to have discrete neural targets and novel mechanisms of action. However, it is unclear that the most commonly used assessment approach—ie, batteries of mainly traditional neuropsychological instruments—will be able to discriminate domain-specific cognitive treatment effects. There have been reports in the literature of somewhat specific effects of certain agents on particular cognitive measures or performance domains. Many of these reports are from acute challenge studies rather than clinical trials (eg, using amphetamine,10,11 glucose,12,13 or nicotine14–16). Some have used narrowly targeted batteries 11,14,16 and/or nontraditional measures.10 Meanwhile, in schizophrenia clinical trials that have reported cognitive benefits using standard neuropsychological batteries (mostly, but not exclusively, antipsychotic trials), these effects have been broadly generalized across domains of cognitive performance,17–21 small in magnitude,22,23 or both.24 Thus, there is still insufficient information for investigators and sponsors of clinical trials to know whether typical batteries of neuropsychological tests can reliably identify treatment effects that are specific to particular brain systems or mechanisms.
Below, we first address an important conceptual issue concerning the factor analytic literature that is related to the debate about the magnitude of the overlap among cognitive dimensions and measures in schizophrenia. We then present meta-analyses of the schizophrenia literature from the past 10 years reporting correlations among individual and composite measures of neuropsychological performance. The analyses summarize correlation findings for traditional neuropsychological variables that are the same as or similar to those included in the MATRICS battery. This battery was developed through a broad-based consensus process, has been approved for use in schizophrenia clinical trials by the National Mental Health Advisory Council6 of the National Institute of Mental Health and is intended to be the default neuropsychological test battery for this purpose. Although the debate over the magnitude of interrelationships among cognitive domains and measures long predates the MATRICS, we use the example of this battery for illustrative purposes because of the important role it is expected to have in trials of cognition-enhancing agents in the future.
Factor Analysis and the Independence of Cognitive Dimensions
The field has looked to the schizophrenia factor analytic literature to identify separable dimensions of cognitive performance.7 Obtained factors differ somewhat among studies depending on the batteries used. Nevertheless, almost all studies have found that cognitive performance in schizophrenia, as measured by traditional neuropsychological tests, is characterized by multiple factors. It was this literature that guided the MATRICS in identifying cognitive dimensions for representation in the consensus neuropsychological battery.7 The main conclusion of that section of the MATRICS report was that:
the evidence, across all available studies, favored sufficient separation of the selected cognitive dimensions to include them as different cognitive dimensions in a battery intended to examine treatment effects in clinical trials (Nuechterlein et al7(p36)).
It bears emphasis, though, that limitations of the underlying literature precluded conclusions about the magnitude of overlap among these cognitive dimensions. Most of the studies reviewed used principal components analysis (PCA) accompanied by orthogonal rotation methods (eg, Varimax).7 Technically speaking, PCA yields components, not factors. The distinction is not trivial, although it is often overlooked. PCA segments all the variance in a dataset into components, including variance that might be caused by some set of underlying, common latent factors and variance that is unique to individual measures.25 In contrast, the various methods of factor analysis (eg, principal axis, maximum likelihood) only segment the variance attributable to the common factors of the dataset. PCA is perfectly acceptable when the goal is simple data reduction. However, the nonseparation of unique measure variance in PCA means that it is technically inaccurate and potentially misleading to describe the components as “factors” or “dimensions” representing underlying latent constructs (eg, attention/working memory, executive functioning).25
Another overlooked point is that orthogonal rotation methods do not establish the independence of cognitive dimensions. Rotation of factors is undertaken to improve the resolution and interpretability of the derived factors. As the name suggests, orthogonal rotations (such as Varimax) yield independent, orthogonal factors. They have a number of desirable mathematical properties and are widely used.26 However, orthogonal rotation does not discover independent, uncorrelated dimensions in a dataset. Rather, this type of rotation groups and weights individual variables subject to the requirement that the resulting factors be orthogonal, whether or not this is consistent with the actual structure of the data. Findings from a recent schizophrenia clinical trial27 illustrate this possible difference between relationships among orthogonal factors and relationships among observed variables. There, baseline cognitive data were subjected to PCA, and 3 orthogonal factors were derived. Yet, when composite cluster scores were constructed on the basis of the orthogonal components, they nevertheless showed high intercorrelations (r = 0.44–0.63). The fact that the first unrotated component in that analysis accounted for 45% of the total variance was taken to indicate “the presence of a large, reliable general factor” and led the investigators to use a single global cognitive composite as the primary cognitive outcome variable (Green et al27(p975)). Thus, while orthogonal rotations necessarily result in factors (or components) that are mathematically independent, such solutions may not always provide the best model of the actual interrelationships in the data. Indeed, where other methods have been used in research on schizophrenia cognitive performance, derived factors tend to show strong correlations.2,28
The main points of this discussion of factor analytic methodology are as follows. The factor analytic literature provides strong support for the conclusion that there are multiple separable cognitive factors in schizophrenia. However, the existence of a good-fitting, multiple factor solution does not imply weak associations among factors, measures loading on different factors, or the latent cognitive dimensions that the factors represent. As noted, it is entirely possible to derive a good-fitting multiple factor solution in which the different derived factors are strongly correlated.2,28 Composite scores based on such factors may show strong correlations, as well.1 In short, “factors,” “dimensions,” and “domains” may be separable in statistical terms while still showing substantial intercorrelations. Thus, the schizophrenia factor analytic literature leaves largely unresolved the extent of the overlap among neuropsychological tests and factor-based cognitive composites. And it remains unclear whether cognitive batteries composed of traditional neuropsychological measures will be able to identify domain-specific effects in upcoming clinical trials. The following meta-analyses of correlations among individual measures and among domain composite scores were undertaken to provide additional information about this issue.
Meta-analysis of Neuropsychological Correlations in Schizophrenia
Study Selection
The MATRICS Consensus Cognitive Battery was created through a painstaking and broad-based consensus process including the academic community, the National Institutes of Health, the U.S. Food and Drug Administration (FDA), and the pharmaceutical industry.6 Because of this, it is fair to think of the battery as reflecting the accumulated experience of the entire field in using traditional neuropsychological measures to efficiently assess cognitive performance and treatment-related change in schizophrenia. Given this pedigree, and the likely use of the battery in upcoming schizophrenia clinical trials, we used it as a starting point for our analysis. The MATRICS battery consists mainly of well-known measures that have been time-tested in schizophrenia research (eg, the Hopkins Verbal Learning Test [HVLT] and Trail Making Test) or updated analogues of such measures (eg, Brief Assessment of Cognition in Schizophrenia Symbol Digit Coding). Nine of the cognitive tests included in the MATRICS battery address 6 much researched dimensions of performance in schizophrenia: speed of processing, attention/vigilance, working memory, verbal learning and memory, visual learning and memory, and reasoning and problem solving. An additional test addresses “social cognition.” Although the empirical base for this dimension is more limited than it is for the first 6, it was included in recognition of fast-growing interest in social cognition impairment in this illness.7 The final MATRICS battery is reflected in table 1.
Table 1.
Cognitive Dimensions and Tests in MATRICS Consensus Cognitive Battery
Speed of processing | Brief Assessment of Cognition in Schizophrenia: Symbol Digit Coding | Timed paper-and-pencil test; respondent uses a key to write digits that correspond to nonsense symbols |
Category fluency: animal naming | Oral test in which respondent names as many animals as she/he can in 1 min | |
Trail Making Test: Part A | Timed paper-and-pencil test; respondent draws a line through consecutively numbered circles | |
Attention/vigilance | Continuous Performance Test—Identical Pairs | Computer-administered measure of sustained attention; respondent presses a response button to consecutive matching numbers |
Working memory (nonverbal) | Wechsler Memory Scale-III: Spatial Span | Board with 10 irregularly spaced cubes; respondent taps cubes in same (or reverse) sequence |
(verbal) | Letter-Number Span | Orally administered test requires mental reordering of strings of number and letters |
Verbal learning | Hopkins Verbal Learning Test—Revised | Orally administered test using a list of 12 words; 3 learning trials; tests recall of words |
Visual learning | Brief Visuospatial Memory Test—Revised | Requires reproduction of 6 geometric figures from memory |
Reasoning and problem solving | Neuropsychological Assessment Battery: Mazes | Seven timed paper-and-pencil mazes |
Social cognition | Mayer-Salovey-Caruso Emotional Intelligence Test: Managing Emotions | Paper-and-pencil multiple-choice test that assesses how people manage their emotions |
Examination of bivariate correlations reported in the literature directly addresses the magnitude of the overlap among cognitive measures and dimensions in schizophrenia. We assembled reports of correlations among individual neuropsychological variables from schizophrenia samples that are the same as those included in the MATRICS battery, close analogues of those measures (eg, other list learning tasks in addition to the HVLT), or widely used measures that tap the same dimensions specified by the MATRICS (eg, Wechsler Logical Memory for verbal learning, and the Wisconsin Card Sorting Test for reasoning and problem solving). (Specific citations for the measures referred to in this article are available in standard neuropsychological reference works29 and cited studies.) In view of the limited literature addressing social cognition and underlying measures, we left this dimension out of the analysis. Our search was restricted to the past 10 years because all the individual MATRICS battery measures, or close analogues, were issued and in general use during this period. Two approaches were used to locate articles for the analysis. First, articles were identified through a series of searches of the PubMed and PsycINFO electronic databases with combinations of key words schizophreni*, correlat* cogniti*, neuropsychologi*, memory, attention, executive, and processing speed. Second, we searched the reference lists from these articles to identify additional publications.
The following criteria were used to select studies for review. (1) The study must have reported correlations among neuropsychological measures and/or domain composites the same as or similar to those represented in the MATRICS cognitive battery in a sample (or samples) of individuals with schizophrenia or schizoaffective disorder (correlations for social cognition measures and composites were not sought). (2) Current analyses target “between-dimension” associations, so we searched for studies that reported correlations between measures from the different cognitive dimensions defined by the MATRICS and excluded studies if they only reported “within-dimension” correlations. (3) The study must have been published between 1997 and June 2006. (4) The study must have based diagnoses on contemporary diagnostic criteria (eg, Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition; Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; International Classification of Diseases, Ninth Revision, or later). (5) Results must have been reported with sufficient detail to allow calculation of effect sizes. (6) Finally, the study must have been reported in English. More than 500 studies were identified through the overlapping database searches. Initially, abstracts and titles were reviewed to eliminate studies that obviously failed to meet criteria (eg, no correlations were reported). Thirty-three articles reporting schizophrenia correlations on relevant neuropsychological measures were retained for examination in greater detail. Of these, 2 included only “within-dimension” and no “between-dimension” correlations. Ten studies reported correlations of relevant variables or analogues only with other variables that are outside of our current focus (eg, verbal IQ, motor speed). Twenty-one studies met inclusion criteria for the meta-analysis. Of these, 12 studies reported multiple, between-dimension, bivariate correlations between individual MATRICS battery variables or analogues.28,30–40 Seven studies reported bivariate correlations between composite scores that matched or were similar to certain MATRICS-designated cognitive dimensions.1,27,41–45 Two studies reported both individual measure and composite correlations.46,47
Data Extraction and Analysis
We conducted 2 parallel analyses, 1 for bivariate correlations between individual measures and 1 for bivariate correlations between composite measures. All analyses were performed using the Comprehensive Meta-Analysis software package.48 For each study, we entered the specified correlations and sample sizes into the database. These values were averaged study-by-study using appropriate transformations and a random-effects model, rather than the more commonly employed fixed-effects model.49,50 This conservative estimation method yields more generalizable parameter estimates than would be derived using a fixed-effects model, most clearly evident in larger estimated standard errors and confidence intervals (CIs).49 Study-by-study mean correlations were weighted by sample size and combined, again using a random-effects model, to obtain a grand mean correlation. We recorded information about a number of potential moderator variables, including age, education, treatment setting, and chronicity of the patient samples. We used meta-regression (ie, regressing study mean correlations against study moderator variable values) to test the influence of the moderator variables.
Results
Main meta-analysis results are presented in table 2 for individual variable correlations and in table 3 for correlations between composite variables. Across 14 studies of schizophrenia samples and 2405 patients, the mean between-dimension correlation reported between individual neuropsychological variables of the sort included in the MATRICS battery was r = 0.37, with study means ranging from r = 0.24 to r = 0.49. Summarizing information from 9 studies and 1860 patients, the mean correlation of cognitive composite variables was higher, r = 0.45, with a wider range of study means, from r = 0.16 to r = 0.61. Cohen's convention is that correlations of approximately 0.1, 0.3, and 0.5 are “small,” “medium,” and “large” effect sizes, respectively.51 Thus, the grand mean correlations derived here for individual measures and domain composites fall between Cohen's medium and large effect size designations.
Table 2.
Summary of Bivariate Correlations Reported Between Individual Neuropsychological Measures in Different MATRICS Cognitive Dimensions
Study | Sample Size and Characteristics | MATRICS Measures (or analogues) for Which Bivariate Correlations Are Reported | Reported “Within-Dimension” Correlations That Are Excluded From Analysis (Specific MATRICS Dimension) | Number of Included “Between-Dimension” Correlations | Mean of “Between-Dimension” Correlations [95% CI] (Full Range) |
Allen et al30 | 169 male inpatients; age 36.2 (7.9), education 12.3 (1.9), duration 12.7 (7.3) | WAIS-R measures: Digit Span, Arithmetic, Block Design, Digit Symbol | Arithmetic/Digit Span (WM) | 5 | .32 [.25–39] (.26–.43) |
Chen et al31 | 204 (122 male) inpatients; age 40.4 (12.2), education 8 (3.4) | Logical Memory (immediate), Digit Span, Category Fluency, Visual Reproduction, Perseverative Errors (WCST) | None | 10 | .24 [.18–.30] (.07–.37) |
Dickinson et al28 | 120 (91 male) outpatients; age 39.3 (8.4), education ∼12, duration ∼18 | WAIS-III measures: Block Design, Matrix Reasoning, Arithmetic, Digit Span, Letter-Number Sequencing, Digit Symbol, Symbol Search | Block Design/Matrix Reasoning (RP), Arithmetic/Digit Span & Digit Span/Letter-Number & Arithmetic/Letter-Number (WM), Digit Symbol/Symbol Search (PS) | 16 | .49 [.46–.53] (.31–.61) |
Friis et al46 | 219 (125 male) first episode; age 27.9 (9.4), education 12.1 (2.7) | CVLT (immediate), Perseverative Errors (WCST), Letter Fluency, Trails A and B (combined), Identical Pairs CPT | Letter Fluency/Trail Making (PS) | 13 | .31 [.27–.34] (.21–.45) |
Gold et al32 | 36 inpatients; age 34.4 (7.8), education 12.8 (1.6) | Correlations of Letter-Number Span and Perseverative Errors (WCST) with Trails B, Digit Span, Gordon's CPT, and Letter Fluency | Digit Span/Letter-Number (WM), Perseverative Errors/Trails B (RP) | 7 | .44 [.33–.54] (.16–.56) |
Holthausen et al33 | 118 (87 male) first or second episode; age 23.3 (5.3), duration < 1 | CVLT (immediate), Rey Complex Figure Memory, Category Fluency, Stroop (color and word conditions combined), Gordon's CPT | Category Fluency/Stroop color and word (PS) | 9 | .30 [.24–.36] (.15–.44) |
Keefe et al47 | 1123 to 1332 (74% male) outpatients; age 40.6 (11.1), education 12.1 (2.2), duration 14.4 (10.7) | Category and Letter Fluency (combined), Digit Symbol, WCST variables (combined), WISC Mazes, HVLT, Visuospatial Working Memory variables (combined), Letter-Number Sequencing, Identical Pairs CPT variables (combined) | Category and Letter Fluency/Digit Symbol (PS), WCST/WISC Mazes (RP), Visuospatial Working Memory/Letter-Number (WM) | 25 | .40 [.37–.44] (.26–.56) |
Keefe et al34 | ∼140 (79% male) inpatients and outpatients; age 34.7 (11.3), education 12.0 (2.3) | Brief Assessment of Cognition in Schizophrenia subtest correlations: Verbal List Learning, Digit Sequencing (span), Category and Letter Fluency (combined), Symbol Digit Coding, Tower of London | Category and Letter Fluency/Symbol Digit (PS) | 9 | .33 [.24–.41] (.08–.50) |
Kurtz et al35 | 32 (23 male) outpatients; age 39.9 (9.0), education 11.9 (2.0), duration 18.9 (10.7) | Correlations of Perseverative Errors (WCST) and Penn Conditional Exclusion Test with Letter Fluency, Trails A, Trails B, Digit Symbol, HVLT (immediate) | Perseverative Errors/Penn CET and Perseverative Errors/Trails B and Penn CET/Trails B (RP) | 8 | .40 [.28–.50] (.22–.61) |
Mahurin et al36 | 53 (45 male) inpatients; age 33.4 (10.1), education 11.2 (2.5), duration 12.8 (7.5) | Letter Fluency, Verbal Series CPT, Trails B, Symbol Digit, HVLT (immediate), Digit Span, WCST, Stroop (color-word) | Letter Fluency/Symbol Digit (PS), Trails B/WCST and Trails B/Stroop and WCST/Stroop (RP) | 24 | .43 [.38–.47] (.17–.59) |
Morrens et al37 | 30 (22 male) inpatients; age 27.5 (7.0) | Correlations of Symbol Digit with Rey AVLT, WCST, Identical Pairs CPT, Letter-Number Sequencing, Trails A and B | Symbol Digit/Trails A (PS) | 5 | .30 [.18–.41] (.07–.42) |
Silver et al38 | 27 male inpatients; age 38 (13.5), education 11.2 (3.0), duration 12.7 (10.5) | Correlations of WM tasks, Digit Span (backward), and Dot Working Memory Test with Penn CPT, Benton Visual Retention Test, Visual Object Learning Test, Penn Face Memory, Penn Abstraction, Inhibition and Working Memory Test | None | 10 | .46 [.29–.61] (.09–.81) |
Stratta et al39 | 30 (21 male) inpatients; age 36.9 (8.5), education 12.8 (3.6), duration 14.9 (8.8) | Visual-Spatial Working Memory Test, Digit Span (backward), Digit Symbol, Perseverative Errors (WCST) | Visual-Spatial Working Memory/Digits backward (WM) | 4 | .24 [.01–.46] (.01–.46) |
Strauss and Summerfelt40 | 27 patients | Dot Working Memory Test, Letter-Number Sequencing, Spatial Working Memory Test, Perseverative Errors (WCST), HVLT (immediate) | Dot Test/Letter-Number and Dot Test/Spatial Working Memory and Letter-Number/Spatial Working Memory (WM) | 7 | .27 [.12–.40] (.06–.42) |
Sample-weighted grand mean (14 studies, ∼2405 patients) [95% CI] | .37 [.33–.40] z = 18.73 P < .001 |
Note: Specific citations for the neuropsychological measures referred to in this article are available in standard neuropsychological reference works29 and in the specific studies cited. Cognitive dimensions specified by the MATRICS are abbreviated as follows: . MATRICS, Measurement and Treatment Research to Improve Cognition in Schizophrenia; CI, confidence interval; WAIS-R, Wechsler Adult Intelligence Scale-Revised; WM, working memory; WCST, Wisconsin Card Sorting Test; WAIS-III, Wechsler Adult Intelligence Scale-Third Edition; RP, reasoning and problem solving; PS, processing speed; CVLT, California Verbal Learning Test; CPT, continuous performance test; WISC, Wechsler Intelligence Scale for Children; HVLT=Hopkins Verbal Learning Test; AVLT=Auditory Verbal Learning Test.
Table 3.
Summary of Bivariate Correlations Reported Between Cognitive Composite Scores Representing Different MATRICS Cognitive Dimensions
Study | Sample Size and Characteristics | Description of Cognitive Composite Scores for Which Bivariate Correlations Are Reported | Number of Included Correlations | Mean of Correlations [95% CI] (Full Range) |
Binder et al41 | 40 (20 male) first episode; age ∼31.5 | Composites based on a priori variable groupings for Verbal Learning—Logical Memory, Paired Associates, and CVLT variables; Visual Learning—Visual Reproduction variables; Working Memory—Digit Span and Reading Span; Processing Speed—Trails, Digit Symbol, and Stroop variables | 6 | .35 [.21–.47] (.12–.54) |
Dickinson et al1 | 148 (97 male) outpatients; age 33.5 (7.5), education 13.2 (2.2), duration ∼11 | Composites based on a priori variable groupings for Verbal Learning—CVLT and Logical Memory variables; Visual Learning—Visual Reproduction variables; Processing Speed—Symbol Cancellation, Trails A, Category Fluency; Executive Function—Trails B, WCST variables, Digit Span | 6 | .61 [.56–.66] (.50–.69) |
Friis et al46 | 219 (125 male) first episode; age 27.9 (9.4), education 12.1 (2.7) | Factor analysis–based composites for Verbal Learning—CVLT variables; Working Memory—Letter Fluency, Digit Span Distractibility variables, CPT hits; Executive Function—WCST variables | 3 | .34 [.27–.40] (.31–.37) |
Green et al27 | 62 (57 male) outpatients; age ∼43.3 (∼8.5), education ∼12.7 (∼1.2), duration ∼25 (∼5.5) | Factor analysis–based composites for Attention/Perception—CPT variables and Span of Apprehension; Memory and Fluency—CVLT, Fluency, and Digit Span Distractibility variables; Executive Function—WCST variables | 3 | .54a [.33–.70] (.44–.63) |
Gur et al41 | 53 (34 male) outpatients; age 34.1 (11.1), education 13.4 (2.3), duration 10.4 (9.5) | Composites of traditional measures correlated with composites of computerized battery measures for Verbal Learning: Traditional—Logical Memory and CVLT variables and Computerized—Penn Word Memory Test variables; Executive: Traditional—WCST variables and Computerized—Computerized Ravens, Penn Inhibition Test, Penn Abstraction, Inhibition, and Working Memory Test | 2 | .54 [.40–.66] (.48–.60) |
Hill et al43 | 45 (28 male) first episode; age 26.1 (8.1), education 13.7 (3.3) | Composites based on a priori variable groupings for Verbal Learning—CVLT variables; Visual Learning—Visual Reproduction variables; Executive Function—WCST and Stroop variables, Trails B, and Letter Fluency; Attention—Trails A, Digit Symbol, Digit Span, Cancellation variables | 6 | .52 [.38–.64] (.35–.71) |
Keefe et al47 | 1123 to 1332 (74% male) outpatients; age 40.6 (11.1), education 12.1 (2.2), duration 14.4 (10.7) | Composites based on a priori variable groupings for Verbal Learning—HVLT variables; Processing Speed—Letter Fluency, Category Fluency, Digit Symbol, Grooved Pegboard; Reasoning—WCST variables, WISC Mazes; Working Memory—Visuospatial Working Memory, Letter-Number Sequencing; Vigilance—Identical Pairs CPT | 10 | .50a [.47–.53] (.36–.62) |
Velligan et al44 | 40 (34 male) patients tested on hospital release; age 35.7 (9.7), education 11.4 (2.5), duration ∼12 | Composites based on a priori variable groupings for Verbal Memory—HVLT variables and Digit Span; Executive Function—WCST and Complex Reaction Time variables; Sustained Attention—Identical Pairs CPT variables; Visual Memory—Rey Complex Figure Memory | 6 | .39 [.28–.50] (.24–.58) |
Villalta-Gil et al45 | 113 (77 male) outpatients; age 41.6 (12.8), duration 18.9 (11.2) | Composites based on a priori variable groupings for attention—Conners CPT variables, Trails A, Digit Span, and Digit Symbol; Abstraction and Flexibility—WCST and Stroop variables, Trails B; Verbal Learning—Spanish Verbal Learning Test, “Operative Memory”—Verbal Fluency variables, Trails B | 6 | .16 [.08–.23] (.01–.3) |
Sample-weighted grand mean (9 studies, ∼1860 patients) [95% CI] | .45 [.35–.54] z = 8.2 P < .001 |
Note: Specific citations for the neuropsychological measures referred to in this article are available in standard neuropsychological reference works29 and in the specific studies cited. MATRICS, Measurement and Treatment Research to Improve Cognition in Schizophrenia; CI, confidence interval; CVLT, California Verbal Learning Test; CPT, continuous performance test; WCST, Wisconsin Card Sorting Test; HVLT=Hopkins Verbal Learning Test.
While r = .37 represents a combined correlation across all the analyzed studies, it is likely that the mean between-dimension correlation for measures specific to the MATRICS cognitive battery would be higher. This conclusion follows from an analysis of the published data from the largest study included in the current review, which reports baseline cognitive findings from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE).47 Of all the studies included in this review, the neuropsychological battery used in CATIE has the most specific measure overlap with the MATRICS battery (as well as the largest subject sample), employing 6 measures that are the same as or similar to measures specified for the MATRICS battery. These 6 measures yielded 14 between-dimension correlations out of the 25 between-dimension correlations averaged and reported in table 2. Combining just these correlations, in the same manner as in the foregoing analyses, yields a mean correlation of r = 0.45 (95% CI = 0.41–0.49). This compares to a mean correlation of r = 0.34 (95% CI = 0.30–0.38) for the CATIE battery correlations outside the MATRICS battery subset.
To be clear, the derived correlations address the traditional neuropsychological measures and domains exemplified by the MATRICS cognitive battery. Given limited schizophrenia research on the MATRICS social cognition measure, its relations with more traditional measures were not included in these analyses. Thus, the points raised in the following paragraphs may not apply to this dimension of cognitive performance (but see Sergi et al52).
Meta-regression of mean study correlations on age was not significant, either for individual neuropsychological measures or for cognitive composites. Moderator variable information for education, treatment setting, and duration of illness was insufficient (ie, either missing or nonnormally distributed) to allow meta-regression on these variables.
Discussion
There are 2 main conclusions of this review. First, the schizophrenia factor analytic literature provides strong evidence that dimensions of cognitive performance in schizophrenia are separable but does not reveal the degree of separateness among those dimensions or among the neuropsychological measures used to measure them. Many of the studies in this literature have used PCA with orthogonal rotational methods that make independence among factors a precondition of the factor analytic solution. Such solutions may not optimally identify true “factors” or accurately reflect variable interrelationships in the underlying data. Second, the current meta-analyses of reported “between-dimension” correlations for individual cognitive measures and for cognitive domain composite variables indicate that, in schizophrenia, pairs of cognitive variables from different domains correlate significantly. The basis for the first conclusion was presented earlier, so this discussion focuses on the second conclusion and its implications.
We assembled studies of schizophrenia patients that reported bivariate correlations between individual neuropsychological measures and between cognitive domain composites. Given the consensus development of the MATRICS cognitive battery and its likely prominence in schizophrenia research in coming years, we took this battery as a starting point. We focused on correlations reported for measures and domains that are the same as or similar to those specified for the MATRICS cognitive battery. In order to highlight the magnitude of relationships across cognitive dimension boundaries, we examined only “between-dimension” correlations (eg, between an individual verbal list learning measure, such as the HVLT, and an individual working memory measure, such as Letter-Number Sequencing) and excluded “within-dimension” correlations (eg, between 2 processing speed measures, such as Digit Symbol and Trails A). Our meta-analysis averaged relevant correlations reported in the literature, first within studies and then between studies, to derive representative grand mean correlations. Results across numerous studies and schizophrenia subjects indicated that the typical “between-dimension” correlation reported in the literature between traditional neuropsychological measures is r = 0.37. Adding weight to this finding, “between-dimension” correlations among individual measures in a MATRICS battery subset from the CATIE study were higher (mean r = 0.45). The typical correlation between domain composites is comparable (r = 0.45).
For reasons discussed subsequently, we believe that bivariate correlations of this magnitude place an important constraint on the ability of traditional neuropsychological batteries to signal independent, domain-specific treatment effects. Likely objections to this conclusion include the following: (1) that these correlations are not large from the standpoint of the squared correlation (ie, “r2”), leaving much of performance on 1 neuropsychological measure free to vary independently of performance on other measures, and (2) that even large correlations among neuropsychological measures at baseline do not preclude independent dimensions of change with treatment.
Regarding the first objection, we believe that squared correlation calculations understate the implications of the derived correlations for 3 principal reasons. First, our position is that squared correlations are simply the wrong metric by which to evaluate the likelihood that individual neuropsychological measures can signal unique treatment effects in the context of other such measures. The squared correlation is relevant in regression as “the amount of variance in 1 variable that is predictable from knowledge of the other variable” (Jensen53(p198); see also Cohen J and Cohen P54). In contrast, the correlation coefficient directly characterizes the nature and degree of relation between 2 variables because “the proportion of their total variance that [two variables] have in common” (Jensen53(p198). In this sense, it is analogous to the “genetic correlation,” which directly indexes (ie, in its unsquared form) the proportion of genetic influence on 2 variables that is common to both.55 As Jensen states further,
when we wish to express the proportion of the total variance in one variable that is predictable from another variable, we use r2. When we speak of the proportion of the total variance that the two variables have in common, we use r (Jensen53(p255)).
Second, it bears emphasis that the average correlation values derived in these meta-analyses are bivariate correlations. In other words, a given variable from a typical battery of traditional neuropsychological tests is likely to have an observed correlation approximating r = 0.37 with each of the “other domain” variables in the battery. (Correlations with other variables within a domain will be higher, by definition.) Ideally, we would like to know the multiple correlation of the individual variable with the entire group of “other domain” variables (ie, excluding “within-domain” correlations). This information is too seldom available for meta-analysis. However, a specific example, also from the CATIE report, illustrates the point. In that study, the mean correlation of the HVLT with individual neuropsychological measures from other domains was r = 0.40 (95% CI = 0.32–0.46; range = 0.27–0.53). At the same time, the correlation of the HVLT with a composite of all the “other domain” CATIE neuropsychological measures (excluding the HVLT) was r = 0.55 (Keefe et al47(p2039)). In general, with such neuropsychological measures, it is fair to assume that multiple correlations (even excluding within-domain correlations) will increase relative to the mean bivariate correlation as the number of tests in a battery increases. Consequently, the ability of a given test battery variable to signal a unique effect, above and beyond the entire group of “other domain” test battery variables, would be less than indicated by the mean bivariate association.
Third, even multiple correlations would leave a misimpression about the amount of potentially unique variance in the current context—ie, where the focus is on correlations between psychological measures known to have adequate, but not exceptional, reliability. Reliabilities set an upper limit on correlations between measures and are the basis for the “correction for attenuation due to unreliability” from classical psychometric theory: the “disattenuated” correlation equals the observed correlation divided by the square root of the product of the measures’ reliabilities.26,56 Reported reliabilities for individual and composite cognitive measures in schizophrenia are seldom above (and often below) the 0.8 level.34,57–61 Applying the disattenuation correction to the correlations derived in this meta-analysis, using a generous reliability estimate of 0.80, yields an estimated bivariate correlation of r = 0.46 between individual measures from different cognitive dimensions in schizophrenia (corrected upwards from r = 0.37) and an estimated correlation of r = 0.56 between typical domain composite scores (corrected from r = 0.45). An estimated correlation of r = 0.56 would likewise result if the correction were applied to correlations between variables within the MATRICS cognitive battery subset of measures reported in the CATIE baseline data. Obviously, using as a starting point, the multiple correlation values reported in the CATIE study (Keefe et al47(p2039)), or lower reliability values, would result in considerably higher disattenuated multiple correlations. Of course, these adjustments are hypothetical. However, application of the disattenuation formula illustrates that, given typical reliabilities and bivariate correlations for traditional neuropsychological measures in schizophrenia, considerably less reliable cognitive performance variance is likely to be free to vary independently of other measures than it might first appear.
A second likely objection to the significance of the typical between-dimension correlations reported here is that even strong baseline correlations do not necessarily preclude independent change in cognitive performance with treatment. It is true that relatively specific cognitive treatment effects have sometimes been reported with pharmacological manipulations. It is also true that most of these reports have been based on narrow or specialized assessments in the context of acute challenge studies.10,11,13–16 The relevance of these studies for extended pharmacological treatment trials is uncertain. In actual clinical trials reporting cognitive benefits for schizophrenia patients, and using typical neuropsychological batteries, these effects tend to be broadly generalized.17,19–24,62 Further, 2 recent studies have used factor analytic techniques to examine the factor structure of cognitive change with antipsychotic treatment in schizophrenia. Analyses of data from the CAFÉ clinical trial (Comparison of Atypicals in First Episode schizophrenia), which included many traditional neuropsychological measures, found that a single, generalized factor characterized both baseline neuropsychological performance and change in performance after treatment.63 In a similar factor analysis of neuropsychological change scores, Harvey et al64 identified 2 broad dimensions of neuropsychological change (“verbal/learning” and “attention”). The points raised in this article leave ample room for debate. However, in combination, they raise the possibility that more than half of the reliable overall variance in a typical battery of traditional neuropsychological measures, perhaps substantially more, is common variance (see also Dickinson et al65). This hypothesis would help to explain why, at least from a psychometric standpoint, clear independent signals have been rare in schizophrenia clinical trials that have used batteries of traditional neuropsychological tasks to measure cognitive outcome.
In conclusion, current evidence suggests that cognitive dimensions assayed by typical neuropsychological test batteries, and the underlying measures, share substantial common variance. The lack of a widely accepted cognitive test battery was a clear obstacle to progress in testing and securing FDA approval for cognition-enhancing agents for schizophrenia.6 The MATRICS cognitive battery, developed with the broad support of the schizophrenia research community, provided a starting point in addressing this obstacle. However, while such traditional neuropsychological test batteries are well suited to document broad changes in overall cognitive performance, they remain somewhat dubious tools for unraveling the complex neurobiology of schizophrenia and are unlikely to allow precise testing of novel pharmacological agents targeting specific brain mechanisms.5,9
The concerns raised here are well known. Indeed, the MATRICS emphasized the field's need for new measurement technologies more tightly linked with discrete neural systems in its recommendations to the FDA.66 The CNTRICS project (Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia), a recently-funded extension of the MATRICS, will play a central role in this regard, adapting methods and conceptual frameworks developed in the basic cognitive neuroscience literature for use in clinical research. Ideally, these more molecular cognitive measures will complement the molar clinical measurement approaches adopted and validated as part of the MATRICS effort. Even assuming such advances, however, confident resolution of the debate about the extent of the overlap among cognitive dimensions in schizophrenia ultimately will depend on improved understanding of the neurobiological and genetic underpinnings of cognition and pharmacological response in this illness.
References
- 1.Dickinson D, Ragland JD, Calkins ME, Gold JM, Gur RC. A comparison of cognitive structure in schizophrenia patients and healthy controls using confirmatory factor analysis. Schizophr Res. 2006;85:20–29. doi: 10.1016/j.schres.2006.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gladsjo JA, McAdams LA, Palmer BW, Moore DJ, Jeste DV, Heaton R. A six-factor model of cognition in schizophrenia and related psychotic disorders: relationships with clinical symptoms and functional capacity. Schizophr Bull. 2004;30:739–754. doi: 10.1093/oxfordjournals.schbul.a007127. [DOI] [PubMed] [Google Scholar]
- 3.Egan MF, Goldberg TE, Gscheidle T, et al. Relative risk for cognitive impairments in siblings of patients with schizophrenia. Biol Psychiatry. 2001;50:98–107. doi: 10.1016/s0006-3223(01)01133-7. [DOI] [PubMed] [Google Scholar]
- 4.Keefe RS. The contribution of neuropsychology to psychiatry. Am J Psychiatry. 1995;152:6–15. doi: 10.1176/ajp.152.1.6. [DOI] [PubMed] [Google Scholar]
- 5.MacDonald AW, 3rd, Carter CS. Cognitive experimental approaches to investigating impaired cognition in schizophrenia: a paradigm shift. J Clin Exp Neuropsychol. 2002;24:873–882. doi: 10.1076/jcen.24.7.873.8386. [DOI] [PubMed] [Google Scholar]
- 6.Nuechterlein KH, Green MF. MATRICS Consensus Cognitive Battery. Los Angeles, Calif: MATRICS Assessment, Inc; 2006. [Google Scholar]
- 7.Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, Green MF, Heaton RK. Identification of separable cognitive factors in schizophrenia. Schizophr Res. 2004;72:29–39. doi: 10.1016/j.schres.2004.09.007. [DOI] [PubMed] [Google Scholar]
- 8.Kern RS, Green MF, Nuechterlein KH, Deng BH. NIMH-MATRICS survey on assessment of neurocognition in schizophrenia. Schizophr Res. 2004;72:11–19. doi: 10.1016/j.schres.2004.09.004. [DOI] [PubMed] [Google Scholar]
- 9.Carter CS. Applying new approaches from cognitive neuroscience to enhance drug development for the treatment of impaired cognition in schizophrenia. Schizophr Bull. 2005;31:810–815. doi: 10.1093/schbul/sbi046. [DOI] [PubMed] [Google Scholar]
- 10.Barch DM, Carter CS. Amphetamine improves cognitive function in medicated individuals with schizophrenia and in healthy volunteers. Schizophr Res. 2005;77:43–58. doi: 10.1016/j.schres.2004.12.019. [DOI] [PubMed] [Google Scholar]
- 11.Siegel BV, Jr, Trestman RL, O'Flaithbheartaigh S, et al. D-amphetamine challenge effects on Wisconsin Card Sort Test. Performance in schizotypal personality disorder. Schizophr Res. 1996;20:29–32. doi: 10.1016/0920-9964(95)00002-x. [DOI] [PubMed] [Google Scholar]
- 12.Newcomer JW, Craft S, Fucetola R, et al. Glucose-induced increase in memory performance in patients with schizophrenia. Schizophr Bull. 1999;25:321–335. doi: 10.1093/oxfordjournals.schbul.a033381. [DOI] [PubMed] [Google Scholar]
- 13.Stone WS, Seidman LJ, Wojcik JD, Green AI. Glucose effects on cognition in schizophrenia. Schizophr Res. 2003;62:93–103. doi: 10.1016/s0920-9964(02)00406-1. [DOI] [PubMed] [Google Scholar]
- 14.Barr RS, Culhane MA, Jubelt LE, et al. The effects of transdermal nicotine on cognition in nonsmokers with schizophrenia and nonpsychiatric controls. Neuropsychopharmacology. doi: 10.1038/sj.npp.1301423. advance online publication, April 18, 2007. doi: 10.1038/sj.npp.1301423. [DOI] [PubMed] [Google Scholar]
- 15.Harris JG, Kongs S, Allensworth D, et al. Effects of nicotine on cognitive deficits in schizophrenia. Neuropsychopharmacology. 2004;29:1378–1385. doi: 10.1038/sj.npp.1300450. [DOI] [PubMed] [Google Scholar]
- 16.Smith RC, Warner-Cohen J, Matute M, et al. Effects of nicotine nasal spray on cognitive function in schizophrenia. Neuropsychopharmacology. 2006;31:637–643. doi: 10.1038/sj.npp.1300881. [DOI] [PubMed] [Google Scholar]
- 17.Harvey PD, Green MF, McGurk SR, Meltzer HY. Changes in cognitive functioning with risperidone and olanzapine treatment: a large-scale, double-blind, randomized study. Psychopharmacology (Berl) 2003;169:404–411. doi: 10.1007/s00213-002-1342-5. [DOI] [PubMed] [Google Scholar]
- 18.Keefe RS, Seidman LJ, Christensen BK, et al. Long-term neurocognitive effects of olanzapine or low-dose haloperidol in first-episode psychosis. Biol Psychiatry. 2006;59:97–105. doi: 10.1016/j.biopsych.2005.06.022. [DOI] [PubMed] [Google Scholar]
- 19.Purdon SE, Jones BD, Stip E, et al. Neuropsychological change in early phase schizophrenia during 12 months of treatment with olanzapine, risperidone, or haloperidol. The Canadian Collaborative Group for research in schizophrenia. Arch Gen Psychiatry. 2000;57:249–258. doi: 10.1001/archpsyc.57.3.249. [DOI] [PubMed] [Google Scholar]
- 20.Velligan DI, Prihoda TJ, Sui D, Ritch JL, Maples N, Miller AL. The effectiveness of quetiapine versus conventional antipsychotics in improving cognitive and functional outcomes in standard treatment settings. J Clin Psychiatry. 2003;64:524–531. doi: 10.4088/jcp.v64n0505. [DOI] [PubMed] [Google Scholar]
- 21.Weickert TW, Goldberg TE, Marenco S, Bigelow LB, Egan MF, Weinberger DR. Comparison of cognitive performances during a placebo period and an atypical antipsychotic treatment period in schizophrenia: critical examination of confounds. Neuropsychopharmacology. 2003;28:1491–1500. doi: 10.1038/sj.npp.1300216. [DOI] [PubMed] [Google Scholar]
- 22.Buchanan RW, Conley RR, Dickinson D, Nelson MW, Gold JM, McMahon RP. Galantamine for the treatment of cognitive impairments in patients with schizophrenia [abstract] Schizophr Bull. 2007;33:424. [Google Scholar]
- 23.Turner DC, Clark L, Pomarol-Clotet E, McKenna P, Robbins TW, Sahakian BJ. Modafinil improves cognition and attentional set shifting in patients with chronic schizophrenia. Neuropsychopharmacology. 2004;29:1363–1373. doi: 10.1038/sj.npp.1300457. [DOI] [PubMed] [Google Scholar]
- 24.Olincy A, Harris JG, Johnson LL, et al. Proof-of-concept trial of an alpha7 nicotinic agonist in schizophrenia. Arch Gen Psychiatry. 2006;63:630–638. doi: 10.1001/archpsyc.63.6.630. [DOI] [PubMed] [Google Scholar]
- 25.Costello AB, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10:1–9. [Google Scholar]
- 26.Nunnally JC. Psychometric Theory. 2nd ed. New York, NY: McGraw-Hill; 1978. [Google Scholar]
- 27.Green MF, Marder SR, Glynn SM, et al. The neurocognitive effects of low-dose haloperidol: a two-year comparison with risperidone. Biol Psychiatry. 2002;51:972–978. doi: 10.1016/s0006-3223(02)01370-7. [DOI] [PubMed] [Google Scholar]
- 28.Dickinson D, Iannone VN, Gold JM. Factor structure of the Wechsler Adult Intelligence Scale-III in schizophrenia. Assessment. 2002;9:171–180. doi: 10.1177/10791102009002008. [DOI] [PubMed] [Google Scholar]
- 29.Lezak MD. Neuropsychological Assessment. 3rd ed. New York, NY: Oxford University Press; 1995. [Google Scholar]
- 30.Allen DN, Huegel SG, Seaton BE, Goldstein G, Gurklis JA, Jr, van Kammen DP. Confirmatory factor analysis of the WAIS-R in patients with schizophrenia. Schizophr Res. 1998;34:87–94. doi: 10.1016/s0920-9964(98)00090-5. [DOI] [PubMed] [Google Scholar]
- 31.Chen EY, Lam LC, Chen RY, Nguyen DG, Chan CK, Wilkins AJ. Neuropsychological correlates of sustained attention in schizophrenia. Schizophr Res. 1997;24:299–310. doi: 10.1016/s0920-9964(96)00120-x. [DOI] [PubMed] [Google Scholar]
- 32.Gold JM, Carpenter C, Randolph C, Goldberg TE, Weinberger DR. Auditory working memory and Wisconsin Card Sorting Test performance in schizophrenia. Arch Gen Psychiatry. 1997;54:159–165. doi: 10.1001/archpsyc.1997.01830140071013. [DOI] [PubMed] [Google Scholar]
- 33.Holthausen EA, Wiersma D, Sitskoorn MM, Dingemans PM, Schene AH, van den Bosch RJ. Long-term memory deficits in schizophrenia: primary or secondary dysfunction? Neuropsychology. 2003;17:539–547. doi: 10.1037/0894-4105.17.4.539. [DOI] [PubMed] [Google Scholar]
- 34.Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, Coughenour L. The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr Res. 2004;68:283–297. doi: 10.1016/j.schres.2003.09.011. [DOI] [PubMed] [Google Scholar]
- 35.Kurtz MM, Wexler BE, Bell MD. The Penn Conditional Exclusion Test (PCET): relationship to the Wisconsin Card Sorting Test and work function in patients with schizophrenia. Schizophr Res. 2004;68:95–102. doi: 10.1016/S0920-9964(03)00179-8. [DOI] [PubMed] [Google Scholar]
- 36.Mahurin RK, Velligan DI, Miller AL. Executive-frontal lobe cognitive dysfunction in schizophrenia: a symptom subtype analysis. Psychiatry Res. 1998;79:139–149. doi: 10.1016/s0165-1781(98)00031-6. [DOI] [PubMed] [Google Scholar]
- 37.Morrens M, Hulstijn W, Van Hecke J, Peuskens J, Sabbe BG. Sensorimotor and cognitive slowing in schizophrenia as measured by the Symbol Digit Substitution Test. J Psychiatr Res. 2006;40:200–206. doi: 10.1016/j.jpsychires.2005.04.014. [DOI] [PubMed] [Google Scholar]
- 38.Silver H, Feldman P, Bilker W, Gur RC. Working memory deficit as a core neuropsychological dysfunction in schizophrenia. Am J Psychiatry. 2003;160:1809–1816. doi: 10.1176/appi.ajp.160.10.1809. [DOI] [PubMed] [Google Scholar]
- 39.Stratta P, Daneluzzo E, Prosperini P, Bustini M, Mattei P, Rossi A. Is Wisconsin Card Sorting Test performance related to ‘working memory’ capacity? Schizophr Res. 1997;27:11–19. doi: 10.1016/S0920-9964(97)00090-X. [DOI] [PubMed] [Google Scholar]
- 40.Strauss ME, Summerfelt A. The neuropsychological study of schizophrenia: a methodological perspective. In: Lenzenweger MF, Hooley JM, editors. Principles of Experimental Psychology: Essays in Honor of Brendan A. Maher. Washington, DC: American Psychological Association; 2003. pp. 119–134. [Google Scholar]
- 41.Binder J, Albus M, Hubmann W, et al. Neuropsychological impairment and psychopathology in first-episode schizophrenic patients related to the early course of illness. Eur Arch Psychiatry Clin Neurosci. 1998;248:70–77. doi: 10.1007/s004060050020. [DOI] [PubMed] [Google Scholar]
- 42.Gur RC, Ragland JD, Moberg PJ, et al. Computerized neurocognitive scanning: II. The profile of schizophrenia. Neuropsychopharmacology. 2001;25:777–788. doi: 10.1016/S0893-133X(01)00279-2. [DOI] [PubMed] [Google Scholar]
- 43.Hill SK, Schuepbach D, Herbener ES, Keshavan MS, Sweeney JA. Pretreatment and longitudinal studies of neuropsychological deficits in antipsychotic-naive patients with schizophrenia. Schizophr Res. 2004;68:49–63. doi: 10.1016/S0920-9964(03)00213-5. [DOI] [PubMed] [Google Scholar]
- 44.Velligan DI, Bow-Thomas CC, Mahurin RK, Miller AL, Halgunseth LC. Do specific neurocognitive deficits predict specific domains of community function in schizophrenia? J Nerv Ment Dis. 2000;188:518–524. doi: 10.1097/00005053-200008000-00007. [DOI] [PubMed] [Google Scholar]
- 45.Villalta-Gil V, Vilaplana M, Ochoa S, et al. Neurocognitive performance and negative symptoms: are they equal in explaining disability in schizophrenia outpatients? Schizophr Res. 2006;87:246–253. doi: 10.1016/j.schres.2006.06.013. [DOI] [PubMed] [Google Scholar]
- 46.Friis S, Sundet K, Rund BR, Vaglum P, McGlashan TH. Neurocognitive dimensions characterising patients with first-episode psychosis. Br J Psychiatry Suppl. 2002;43:s85–s90. doi: 10.1192/bjp.181.43.s85. [DOI] [PubMed] [Google Scholar]
- 47.Keefe RS, Bilder RM, Harvey PD, et al. Baseline neurocognitive deficits in the CATIE schizophrenia trial. Neuropsychopharmacology. 2006;31:2033–2046. doi: 10.1038/sj.npp.1301072. [DOI] [PubMed] [Google Scholar]
- 48.Borenstein M, Hedges LV, Higgins J, Rothstein H. Comprehensive Meta-analysis. Englewood, NJ: Biostat; 2000. [Google Scholar]
- 49.Raudenbush SW. Random effects models. In: Cooper H, Hedges LV, editors. The Handbook of Research Synthesis. New York, NY: Sage Publications; 1994. pp. 301–321. [Google Scholar]
- 50.Shaddish WR, Haddock CK. Combining estimates of effect size. In: Cooper H, Hedges LV, editors. The Handbook of Research Synthesis. New York, NY: Sage Publications; 1994. pp. 261–285. [Google Scholar]
- 51.Cohen JD. Statistical Power for the Behavioral Sciences. Hillsdale, NJ: Lawrence Earlbaum; 1988. [Google Scholar]
- 52.Sergi MJ, Rassovsky Y, Widmark C, et al. Social cognition in schizophrenia: relationships with neurocognition and negative symptoms. Schizophr Res. 2007;90:316–324. doi: 10.1016/j.schres.2006.09.028. [DOI] [PubMed] [Google Scholar]
- 53.Jensen AR. Bias in Mental Testing. New York, NY: Free Press; 1980. [Google Scholar]
- 54.Cohen J, Cohen P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum; 1983. [Google Scholar]
- 55.Pennington BF, Filipek PA, Lefly D, et al. A twin MRI study of size variations in human brain. J Cogn Neurosci. 2000;12:223–232. doi: 10.1162/089892900561850. [DOI] [PubMed] [Google Scholar]
- 56.Schmidt FL, Hunter JE. Measurement error in psychological research: lessons from 26 research scenarios. Psychol Methods. 1996;1:199–223. [Google Scholar]
- 57.Cutler NR, Veroff AE, Frackiewicz EJ, Welke TL, Kurtz NM, Sramek JJ. Assessing the neuropsychological profile of stable schizophrenic outpatients. J Neuropsychiatry Clin Neurosci. 1996;8:423–428. doi: 10.1176/jnp.8.4.423. [DOI] [PubMed] [Google Scholar]
- 58.Harvey PD, Palmer BW, Heaton RK, Mohamed S, Kennedy J, Brickman A. Stability of cognitive performance in older patients with schizophrenia: an 8-week test-retest study. Am J Psychiatry. 2005;162:110–117. doi: 10.1176/appi.ajp.162.1.110. [DOI] [PubMed] [Google Scholar]
- 59.Heaton RK, Gladsjo JA, Palmer BW, Kuck J, Marcotte TD, Jeste DV. Stability and course of neuropsychological deficits in schizophrenia. Arch Gen Psychiatry. 2001;58:24–32. doi: 10.1001/archpsyc.58.1.24. [DOI] [PubMed] [Google Scholar]
- 60.Velligan DI, DiCocco M, Bow-Thomas CC, et al. A brief cognitive assessment for use with schizophrenia patients in community clinics. Schizophr Res. 2004;71:273–283. doi: 10.1016/j.schres.2004.02.027. [DOI] [PubMed] [Google Scholar]
- 61.Wilk CM, Gold JM, Bartko JJ, et al. Test-retest stability of the repeatable battery for the assessment of neuropsychological status in schizophrenia. Am J Psychiatry. 2002;159:838–844. doi: 10.1176/appi.ajp.159.5.838. [DOI] [PubMed] [Google Scholar]
- 62.Keefe RS, Seidman LJ, Christensen BK, et al. Comparative effect of atypical and conventional antipsychotic drugs on neurocognition in first-episode psychosis: a randomized, double-blind trial of olanzapine versus low doses of haloperidol. Am J Psychiatry. 2004;161:985–995. doi: 10.1176/appi.ajp.161.6.985. [DOI] [PubMed] [Google Scholar]
- 63.Hill SK, Sweeney JA, Hamer RA, et al. Efficiency of the BACS and CATIE neuropsychological batteries in assessing cognition and antipsychotic related change in cognition during the CAFE clinical trial [abstract] Schizophr Bull. 2007;33:561. [Google Scholar]
- 64.Harvey PD, Green MF, Bowie C, Loebel A. The dimensions of clinical and cognitive change in schizophrenia: evidence for independence of improvements. Psychopharmacology (Berl) 2006;187:356–363. doi: 10.1007/s00213-006-0432-1. [DOI] [PubMed] [Google Scholar]
- 65.Dickinson D, Iannone VN, Wilk CM, Gold JM. General and specific cognitive deficits in schizophrenia. Biol Psychiatry. 2004;55:826–833. doi: 10.1016/j.biopsych.2003.12.010. [DOI] [PubMed] [Google Scholar]
- 66.MATRICS. Recommendations to the U.S. Food and Drug Administration, Division of Neuropharmacological Drug Products, for Evaluation of Efficacy of Pharmacological Treatments for Cognition in Schizophrenia. Los Angeles, Calif: University of California Los Angeles; 2005. [Google Scholar]