Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jan 11.
Published in final edited form as: Schizophr Res. 2015 Nov 30;170(1):156–161. doi: 10.1016/j.schres.2015.11.022

Unitary construct of generalized cognitive ability underlying BACS performance across psychotic disorders and in their first-degree relatives

WC Hochberger 1, S K Hill 1,*, CLM Nelson 1, JL Reilly 2, RSE Keefe 3, GD Pearlson 4, MS Keshavan 5, CA Tamminga 6, BA Clementz 7, JA Sweeney 6
PMCID: PMC5226655  NIHMSID: NIHMS745490  PMID: 26645510

Abstract

Despite robust evidence of neurocognitive dysfunction in psychotic patients, the degree of similarity in cognitive architecture across psychotic disorders and among their respective first-degree relatives is not well delineated. The present study examined the latent factor structure of the Brief Assessment of Cognition in Schizophrenia (BACS) neuropsychological battery. Analyses were conducted on 783 psychosis spectrum probands (schizophrenia, schizoaffective, psychotic bipolar), 887 of their first-degree relatives, and 396 non-psychiatric controls from the Bipolar-Schizophrenia Network on Intermediate Phenotypes (B-SNIP) consortium. Exploratory factor analysis of BACS subtest scores indicated a single-factor solution that was similar across all groups and provided the best overall data fit in confirmatory analyses. Correlations between the standard BACS composite score and the sum of subscale scores weighted by their loadings on this unitary factor was very high in all groups (r ≥.99). Thus, the BACS assesses a similar unitary cognitive construct in probands with different psychotic disorders, in their first-degree relatives, and in healthy controls, and this factor is well measured by the test’s standard composite score.

Keywords: psychosis, brief assessment of cognition in schizophrenia, first-degree relatives, factor analysis

1. Introduction

Cognitive deficits have been firmly established as a common debilitating feature of schizophrenia spectrum disorders (Bilder et al., 2000; Dickinson et al., 2008; Hill et al., 2004; Keefe et al., 2006; Reilly & Sweeney, 2014). These deficits are present at illness onset, stable, and minimally affected by antipsychotic treatment (Bilder et al., 2000; Hill et al., 2004; Hoff et al., 1999), and predict functional outcome (Bowie & Harvey, 2006; Green, 1996). A less severe pattern of generalized deficits has been reported in affective psychotic disorders and in first-degree relatives of patients with both affective and nonaffective psychotic disorders (Hill et al., 2014). However, the factor structure of deficits across disorders and in family members has not been systematically examined in a single study.

The Brief Assessment of Cognition in Schizophrenia (BACS) neuropsychological battery was designed to be easily and quickly administered (in <35 minutes), and sensitive to the profile of generalized impairment seen in schizophrenia (Keefe et al., 2004; Keefe et al., 2008). This scale has been widely used in schizophrenia research, especially in clinical trials as a cognitive outcome measure. Factor analytic research with schizophrenia patients has indicated that a single generalized factor accounts for a high percentage of the variance in scores on both the BACS (Hill et al., 2008b) and larger neuropsychological batteries (Dickinson et al., 2006; Keefe et al., 2006), though additional factors have been identified in some studies. However, few studies have examined the degree to which the cognitive architecture of performance across a battery of neuropsychological measures is consistent across psychotic disorders, whether these latent constructs in patients are similar to those of their first-degree relatives (Sitskoorn et al., 2004; Snitz et al., 2006), and whether those structures are similar to that seen in healthy controls. These issues are important for diagnostic differentiation and to support the use of neuropsychological batteries as outcome measures across disorders.

The BACS was used by the Bipolar-Schizophrenia Network on Intermediate Phenotypes (B-SNIP) consortium to address questions about diagnostic boundaries and familiarity of intermediate phenotypes in schizophrenia, schizoaffective disorder, and bipolar disorder. The initial report of cognitive deficits showed significant familiality and differences across disorders in the severity of deficit (Hill et al., 2014), but the composition of the deficit across disorders and in family members was not formally addressed previously and is the focus of this investigation. To date, no other studies have examined such a range of diagnostic and family groups in a large study of this nature. Clarifying the latent variable structure across disorders and any differences across groups is important to establish the utility of the BACS and other measures of general intellectual ability in assessing cognition across a broad range of psychiatric populations in clinical trials, and in tracking cognitive phenotypes in family genetic research.

A two-step, split-half, cross-validation method using complimentary exploratory and confirmatory factor analytic techniques (Gorsuch, 1983) was first applied to each subject group separately. Then, exploratory factor analysis (EFA) was conducted on half of each group (randomly selected) to determine the number of latent factors underlying BACS subscales in a data driven manner. The remaining half of each group was then separately examined using a confirmatory factor analysis (CFA) to validate the findings. The primary scientific questions pertained to the homogeneity of factor structure across proband groups, between proband groups and their respective family members, and between these groups and healthy controls.

2. Methods

2.1. Participants

Recruitment strategy and patient characteristics of the BSNIP study sample have been reported previously (Tamminga et al., 2014). Patients were recruited from the community if they had a history of psychotic symptoms and at least one first-degree relative between the ages of 15–65 also willing to participate in the study. Probands were required to have a DSM-IV diagnosis of schizophrenia, schizoaffective disorder, or bipolar disorder with a history of psychotic symptoms determined using the Structured Clinical Interview for DSM Disorders (SCID) (First et al., 1995).

All participants had 1) no history of seizures or head injury with loss of consciousness (>10 minutes), 2) no diagnosis of substance abuse in the preceding 30 days or substance dependence in the preceding 6 months, 3) negative urine drug screen for common drugs of abuse on the day of testing, 4) no change in medication and generally clinically stable over the past month, 5) no history of systemic medical or neurological disorder known to affect cognitive abilities, 6) age-corrected Wide Range Achievement Test-IV Reading standard score (SS) >65, and 7) adequate fluency in English to complete testing.

2.2. Procedures

The Brief Assessment of Cognition in Schizophrenia (BACS) is a neuropsychological battery designed to evaluate global neuropsychological function in individuals with schizophrenia that has been demonstrated to be reliable and valid (Keefe et al., 2004; Keefe et al., 2008). The BACS consists of six subtests covering four domains (Verbal Memory, Processing Speed, Reasoning, and Problem Solving, and Working Memory). Subtest scores were converted to z-scores using published norms (Keefe et al. 2008). To limit the impact of extreme values on group means, outliers were Winsorized (Tabachnick & Fidell, 2007) to a maximum absolute value of 4.0 for subtest z-scores before BACS composite scores were computed to reduce outlier effects on data analyses.

2.3. Data analytic plan

A two-step, split-half, cross validation method was utilized with complimentary exploratory and confirmatory approaches (Cudeck & Browne, 1983; Loehlin, 1992). Exploratory factor analysis (EFA) is an empirically guided technique that was used with a randomly selected half of each participant group to determine the number of latent factors underlying the BACS subtest scores and the loadings of subtests on the factor(s). The split-half groups within each diagnostic category were demographically similar and there were no significant differences between EFA and CFA groups in demographic or cognitive parameters after correcting for multiple comparisons (see Table 1).

Table 1.

Demographic characteristics of split-half groups used for exploratory factor analysis and cross-validation (confirmatory factor analysis) groups. No significant differences between EFA and CFA groups were observed after correcting for multiple comparisons.

HC SZP SZR BPP BPR SzAffP SzAffR
EFA CFA P EFA CFA P EFA CFA P EFA CFA P EFA CFA P EFA CFA P EFA CFA P
Age 37 37 .85 36 35 .54 41 44 .09 35 37 .17 40 40 .99 37 37 .73 37 43 .01
Sex
  Male 45% 45% .87 66% 68% .74 34% 26% .11 37% 35% .74 37% 33% .43 42% 39% .63 36% 23% .03
55% 55% 44% 42% 66% 74% 63% 65% 63% 67% 58% 61% 64% 77%
Female
Race - - .35 - - .93 - - .87 - - .37 - - .24 - - .70 - - .19
WRAT 104 102 .05 94 93 .53 97 97 .60 100 102 .32 103 103 .70 96 97 .79 99 100 .54
Open in a new tab

Principal axis factoring was conducted with the first split-half of each group. To avoid over extracting factors (see Costello & Osborne, 1983), the Kaiser criterion (factors with eigen values > 1.0) (Yeomans & Golder, 1982) was used to determine the maximum number of factors eligible for extraction and scree plots were used to optimize the final solution. In step two, a confirmatory factor analysis (CFA) was performed using the remaining half of each group to validate the derived model. As a secondary aim given the limited number of tests in the BACS battery, we also compared the single factor solution determined with the EFA to a model with more factors based on reports from some previous investigations about the cognitive structure of psychotic disorders (Lam et al., 2014; Keefe et al., 2004; McCleery et al., 2015). Model fit was evaluated using the following measures: Tucker-Lewis Index (TLI) (Bollen, 1989), Comparative Fit Index (CFI) (Bentler, 1990), Root Mean Square Approximation (RMSEA) (Browne & Cudeck, 1993), Akaike information criterion (AIC), Bayesian information criterion (BIC), and a Chi-square test of goodness of fit.

3. Results

3.1. Exploratory factor analyses

As illustrated in Figure 1, findings clearly indicated a one-factor solution in all groups (see Table 2 for the amount of variance explained for each group). Because a single factor was extracted in all groups, there was no rotation or evaluation of factor correlations. Overall, these findings indicate a single, generalized cognitive factor underlying the BACS in all diagnostic groups, their relatives, and in healthy controls. As can be seen in Table 2, all subtests loaded meaningfully, and factor loadings of tests on the generalized factor were similar across all groups.

Figure 1.

Figure 1

Figure 1

Open in a new tab

a. Scree plot for exploratory factor analyses (EFA) indicates a single factor solution underlying the BACS in each group.

b. Scree plot for exploratory factor analyses (EFA) for healthy controls, all first-degree relatives, and all psychotic probands.

Table 2.

BACS subtest factor loadings and percent variance in subtest scores explained by the first (and only) factor from an exploratory factor analysis for each diagnostic group.

HC SZ Proband BP Proband SzAff Proband SZ Relative BP Relative SzAff Relative
Sample size 396 323 260 200 349 301 237
Verbal Memory .563 .666 .685 .723 .682 .674 .747
Digit Sequencing .610 .673 .606 .724 .653 .614 .722
Token Motor .326 .538 .414 .352 .436 .420 .388
Verbal Fluency .587 .675 .625 .547 .620 .512 .690
Symbol Coding .672 .791 .716 .790 .641 .700 .772
Tower .656 .637 .576 .611 .567 .615 .569
Variance Explained (%) 44.17 53.49 47.36 49.93 46.88 45.89 52.31
Open in a new tab

3.1.1. Factor loadings and relation to the standard BACS composite score

Using a norm-based approach, factor scores were computed (for all groups) based on the BACS subtest factor loadings observed in the healthy control group, with demographic variables used as covariates as determined by their relation to BACS performance in controls. This was done to provide a relatively unbiased and consistent approach for examining the similarity of a total test scores created by weighting subtest scores by their loading on the single factor from the exploratory factor analysis with the standard composite score used in previous BACS research which sums raw subtest scores (Keefe et al, 2004). These two measures were correlated separately for each group. The factor loading weighted sum of test scores and the standard BACS composite score were correlated very highly in each group (r > .99, p<.001). Thus, the standard BACS composite score and a score assessing the primary cognitive factor determined by factor analyses of the BACS subtests were essentially identical. It should be noted that for both scores we followed the standard procedure of first correcting for performance differences related to demographic factors.

3.2. Confirmatory factor analysis

3.2.1. Model evaluation

Cross-validation of the factor analysis solution provided by the exploratory factor analysis was performed using the second split-half of each participant group and then comparing the results to those from the exploratory analysis with the first half of the samples and more complex models. All CFA models were hierarchical and factor-correlated, with each model converging across all groups. Table 3 presents the results of the CFA. Although the three-factor model was a marginally better fit in bipolar and schizoaffective relatives, the unitary latent factor had significantly greater support overall as in the exploratory analysis, and the single factor solution was the more parsimonious model, particularly for healthy controls and probands with psychotic disorders. Furthermore, factorial invariance was assessed using a metric invariance method. When loadings were constrained to be equal there were no significant differences between models for proband and relative groups [χ2(25) = 32.02, p = 0.16]. Thus, confirmatory factor analysis and factorial invariance both validated the EFA derived single factor model, and there was no evidence of appreciable divergence in the structure of that factor across groups, or any benefit to using more complex models to characterize the underlying cognitive architecture of the BACS battery.

Table 3.

Model fit statistics for one- and three-factor models for BACS subtest structure. CFI and TLI scores at 0.90 or above, lower AIC and BIC, and RMSEA scores less than 0.05, indicate a good fit.

CFI TLI χ2 (p-value) RMSEA (90% CI) AIC BIC
One-Factor
  HC 0.977* 0.961* 14.81 (0.10)* 0.056 (0.000–0.105)* 3565.90* 3625.72*
  SZ Proband 0.994^ 0.989^ 10.83 (0.29)^ 0.035 (0.000–0.097)* 2959.00* 3015.34*
  BP Proband 0.978* 0.963* 12.00 (0.21)* 0.054 (0.000–0.126)* 1961.02* 2010.11*
  SzAff Proband 0.917* 0.862* 18.81 (0.027)* 0.111 (0.036–0.182)* 1660.18* 1704.77*
  SZ Relative 0.993* 0.988* 10.67 (0.30)* 0.032 (0.000–0.094)* 3014.21* 3071.28*
  BP Relative 0.969 0.949 14.25 (0.11) 0.063 (0.000–0.122) 2543.56 2597.39*
  SzAff Relative 0.954 0.923 19.40 (0.02) 0.097 (0.035–0.156) 2061.44 2112.21
Three-Factor
  HC 0.969 0.934 14.67 (0.041) 0.073 (0.015–0.126) 3569.76 3636.22
  SZ Proband 0.995 0.989 8.49 (0.29) 0.036 (0.000–0.105) 2960.67 3023.26
  BP Proband 0.970 0.936 11.11 (0.13) 0.072 (0.000–0.148) 1964.12 2018.67
  SzAff Proband 0.909 0.806 17.78 (0.01) 0.132 (0.057–0.210) 1663.15 1712.69
  SZ Relative 0.990 0.978 9.41 (0.22) 0.044 (0.000–0.109) 3016.95 3080.36
  BP Relative 0.997* 0.994* 7.52 (0.38)* 0.022 (0.00–0.106)* 2540.82* 2600.63
  SzAff Relative 0.964* 0.926* 14.77 (0.04)* 0.095 (0.020–0.162)* 2060.81* 2117.22*
Open in a new tab
*

Indicates best model fit (across one- and three-factor models)

^

Indicates equivalent model fit (across one- and three-factor models).

4. Discussion

This study was the first to examine the cognitive architecture underlying the BACS battery and the structure of generalized cognitive impairment across psychotic disorders (schizophrenia, schizoaffective, and psychotic bipolar) and their first-degree relatives. A split-half, cross-validation analysis starting with a data-driven exploratory factor analytic technique was conducted separately for each group. Results indicated a single-factor solution underlying BACS performance that was similar in nature across all groups of patients and relatives as well as healthy control participants. This was complimented by a confirmatory factor analysis evaluating the single factor model derived in the exploratory analyses and more complex models. The cross-validation supported the exploratory findings of single-factor solution of generalized cognitive deficit and provided minimal support for a higher order models in which cognitive impairment was comprised of discrete deficits. Thus, the BACS battery seems to primarily assess a similar and unidimensional cognitive construct across patients with different psychotic disorders, in their first-degree relatives, and in controls alike.

The generalized factor resulting from the exploratory factor analysis was highly correlated with, and well characterized by, the standard BACS composite score. Thus, the standard composite score and a composite score derived by weighting individual tests by their loading on the generalized factor provide comparable indices of overall cognitive ability. Further, it is important that the structure of cognitive deficit was similar in form across psychotic disorders and in their first-degree relatives.

These findings have implications for psychopathology of cognition across disorders, and for the utility of the BACS across patient groups both as a treatment outcome measure and in family research. First, with regard to the cognitive deficits in psychotic disorders, the findings provide additional support to a growing body of literature suggesting that a global generalized deficit accounts for much of the cognitive impairment seen across psychotic disorders (Hill et al., 2013; Reilly & Sweeney, 2014; Reichenberg et al., 2009). Second, the findings provide novel evidence that the BACS battery assesses a similar cognitive deficit in schizophrenia, schizoaffective disorder, and bipolar disorder with psychotic features. While the severity of generalized cognitive deficit differs across disorders (Hill et al., 2013), the composition of this deficit is very similar and provides a basis for using the BACS to assess general cognitive deficit across psychotic disorders.

Third, the similarity of the cognitive architecture in family members and patients indicates that the BACS battery may have utility in assessing a comparable dimension of generalized cognitive deficits across disorders in a variety of research designs (e.g., clinical trials, family studies). As is the case with comparisons of the severity of deficit across disorders, this would be much more challenging if there were significant differences in the latent cognitive structure of illness-related or familial cognitive performance. The findings of a similar structure to cognitive deficit in probands and relatives provides support for interpreting the general deficit assessed by the BACS as reflecting a similar cognitive deficit as an endophenotype associated with familial risk for illness across psychotic disorders. The similar cognitive structure of generalized deficit also suggests that the BACS test may provide a useful outcome measure in clinical trials targeting cognitive deficit across psychotic disorders, rather than only in schizophrenia where it has been most frequently used.

4.1. Factor Structure of the BACS

Factor analytic approaches require large sample sizes and a high ratio of indicators to factors. The B-SNIP study was sufficiently large to accommodate a conservative analytic approach using both data driven exploratory and top-down confirmatory factor analytic techniques that yielded similar solutions and provide internal replication for the primary findings. However, while the BACS test has the advantage of being a brief and efficient approach for assessing generalized deficit, this benefit is a disadvantage for addressing the question of potential higher order structures to cognitive deficit in psychotic disorders because it does not provide a high ratio of indicators to factors or coverage of all cognitive processes.

The literature regarding the higher order factor structure underlying neuropsychological batteries and cognitive deficit in psychotic disorders has been mixed, perhaps related to methodological issues and/or approaches to factor extraction. Multifactor models have been reported in schizophrenia samples when evaluating intelligence tests (Allen et al., 1998) as well as brief (Keefe et al., 2004) and larger neuropsychological batteries (Gladsjo et al., 2004; Green et al., 2002; Hobart et al., 1999). Some of these studies may have over-extracted factors as some studies extracted factors with eigenvalues less than 1.0 or strictly adhered to the Kaiser criterion without incorporating scree plots when determining the number of factors to retain (Green et al., 2002; Hobart et al., 1999; Keefe et al., 2004). Some studies recently reported multiple factors underlying neuropsychological batteries (Lam et al., 2014; McCleery et al., 2015) and provided the basis for the more complicated model in the confirmatory factor analysis. Yet, in contrast, the present findings clearly favored a unitary cognitive dimension of the BACS battery. This finding provides broader support for a unitary dimension of generalized cognitive deficit in the literature in step with stronger methodology and more sophisticated statistical analyses (Strauss & Summerfelt, 2003). Additionally, Dickinson and colleagues (2004) reported that a single common factor accounted for the majority of intellectual and memory deficits in schizophrenia patients (compared to controls) and higher order models accounted for little unique betweengroup variance. Despite extracting a three-factor solution from a lengthy neuropsychological battery, Green and colleagues argued that a large reliable general factor (accounting for 45% of total variance) justified combining all variables into a single composite to evaluate pharmacological treatment effects (Green et al., 2002).

Exploratory factor analysis of both the BACS and CATIE batteries in a large sample of first episode psychosis patients indicated a general cognitive factor underlying both batteries regardless of whether the batteries were analyzed separately or together (Hill et al., 2008b). Unitary and multi-factor models were directly compared using confirmatory factor analysis of the CATIE neuropsychological battery and a single-factor model provided a better fit than a more complicated model (Keefe et al., 2006). Furthermore, principal components analysis of the CATIE data resulted in a single component exceeding 1.0 eigens (Keefe et al., 2006). Finally, in a comparison of a hierarchical model representing one broad cognitive dimension and a multifactor model consisting of separate cognitive dimensions, the unitary cognitive factor was a better fit for performance in chronic schizophrenia (Dickinson et al., 2006). Overall, findings from a variety of methodologies across a wide range of neuropsychological measures in chronic and early course schizophrenia samples were consistent with the present findings of a general cognitive factor underlying the cognitive deficit associated with psychotic disorders.

However, given that the BACS is a brief battery with a limited number of tests to define discrete higher order factors, the present findings do not provide a strong basis for drawing inferences about whether different cognitive factors might be differentiated in a more extensive battery covering broader range of neurocognitive dimensions. Thus, while the BACS captures generalized cognitive deficits efficiently, addressing the broader question of the complexity of the cognitive architecture of neurocognitive deficits in psychotic disorders almost certainly requires a much larger test battery.

4.2. Generalized versus Specific Deficits

Generalized cognitive deficits are characteristic of psychotic disorders with some variability in level of severity. Brief batteries can capture this broad factor in a useful way for clinical trials and potentially for family studies. There are two advantages of assessing generalized cognitive deficits for these purposes. First, neuropsychological assessment of generalized impairment can be done quickly in an efficient manner (Gold & Harvey, 1993; Hill et al., 2008b). Second, available evidence indicates that generalized impairment has broad clinical relevance more than specific cognitive measures, being more consistently related to important functional outcomes in the domains of interpersonal functioning, personal care skills, and work skills (Bowie & Harvey, 2006). Generalized cognitive impairment thus seems to have stronger generalization to real-world competencies than measures of specific cognitive domains, at least in so far as this issue has been addressed to date (Bowie et al., 2014). The present findings support the BACS as an efficient measure that can be used to assess this deficit across a wide range of cognitive investigations for studies of diverse psychotic disorders and family studies of affected individuals.

4.3. Limitations

While showing a unidimensional nature of BACS subtest scores that was similar across disorders and relative groups, the present data cannot demonstrate that there is a unidimensional nature to cognitive impairments in psychotic disorders, only that the deficit assessed by the BACS battery appears to be unidimensional. More extensive neurocognitive batteries incorporating biomarkers or neurocognitive measures (i.e., eye movement, EEG, translational) may have the breadth in terms of both multiple measures of domains and broader coverage of potentially relevant domains contributing to complex neurocognitive architectures. While the findings with the BACS battery in the present study show it to be a useful way for tracking a similar generalized deficit across disorders and family groups, its comparative utility vs. other approaches in terms of optimal characterization and efficiency of testing remains a question for future research.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Addington J, Brooks BL, Addington D. Cognitive functioning in first episode psychosis: Initial presentation. Schizophr. Res. 2003;62(1):59–64. doi: 10.1016/s0920-9964(02)00340-7. [DOI] [PubMed] [Google Scholar]
  2. Allen DN, Huegel SG, Seaton BE, Goldstein G, Gurklis JA, van Kammen DP. Confirmatory factor analysis of the WAIS-R in patients with schizophrenia. Schizophr. Res. 1998;34:87–94. doi: 10.1016/s0920-9964(98)00090-5. [DOI] [PubMed] [Google Scholar]
  3. Bentler PM. Comparative fit indexes in structural models. Psychological Bull. 1990;107:238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
  4. Bilder RM, Goldman RS, Robinson D, Lieberman JA. Neuropsychology of first-episode schizophrenia: Initial characterization and clinical correlates. Am. J. Psychiatry. 2000;157:549–559. doi: 10.1176/appi.ajp.157.4.549. [DOI] [PubMed] [Google Scholar]
  5. Bollen KA. A new incremental fit index for general structural equation models. Sociological Methods Res. 1989;17:303–316. [Google Scholar]
  6. Bora E, Yucel M, Pantelis C. Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: Meta-analytic study. The Br. J. Psychiatry. 2009;195(6):475–482. doi: 10.1192/bjp.bp.108.055731. [DOI] [PubMed] [Google Scholar]
  7. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing Structural Models. Newbury Park: Sage Publications; 1993. pp. 136–162. [Google Scholar]
  8. Browne MW, Cudeck R. Single sample cross-validation indices for covariance structures. Multivar. Behavioral Res. 1989;24(4):445–455. doi: 10.1207/s15327906mbr2404_4. [DOI] [PubMed] [Google Scholar]
  9. Bowie CR, Harvey PD. Cognitive deficits and functional outcome in schizophrenia. Neuropsychiatric Dis. and Treat. 2000;2(4):531. doi: 10.2147/nedt.2006.2.4.531. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bowie CR, McGurk SR, Mausbach B, Patterson TL, Harvey PD. Combined cognitive remediation and functional skills training for schizophrenia: Effects on cognition, functional competence, and real-world behavior. Am. J. Psychiatry. 2014;169(7):710–718. doi: 10.1176/appi.ajp.2012.11091337. [DOI] [PubMed] [Google Scholar]
  11. Cardno AG, Marshall EJ, Coid B, Macdonald AM, Ribchester TR, Davies NJ, Murray RM. Heritability estimates for psychotic disorders: The Maudsley twin psychosis series. Arch. Gen. Psychiatry. 1999;56(2):162–168. doi: 10.1001/archpsyc.56.2.162. [DOI] [PubMed] [Google Scholar]
  12. Costello AB, Osboren JW. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assess. Res. & Eval. 2005;10(7):1–9. [Google Scholar]
  13. Davies G, Tenesa A, Payton A, Yang J, Harris SE, Liewald D, Deary IJ. Genome-wide association studies establish that human intelligence is highly heritable and polygenic. Molecular Psychiatry. 2011;16(10):996–1005. doi: 10.1038/mp.2011.85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Devlin B, Daniels M, Roeder K. The heritability of IQ. Nature. 1997;388(6641):468–471. doi: 10.1038/41319. [DOI] [PubMed] [Google Scholar]
  15. Dickinson D, Ragland JD, Calkins ME, Gold JM, Gur RC. A comparison of cognitive structure in schizophrenia patients and healthy controls using confirmatory factor analysis. Schizophr. Res. 2006;85(1):20–29. doi: 10.1016/j.schres.2006.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dickinson D, Ragland JD, Gold JM, Gur RC. General and specific cognitive deficits in schizophrenia: Goliath defeats David? Biological Psychiatry. 2008;64(9):823–827. doi: 10.1016/j.biopsych.2008.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gladsjo JA, McAdams LA, Palmer BW, Moore DJ, Jeste DV, Heaton RK. A six-factor model of cognition in schizophrenia and related psychotic disorders: Relationships with clinical symptoms and functional capacity. Schizophr. Bull. 2004;30:739–754. doi: 10.1093/oxfordjournals.schbul.a007127. [DOI] [PubMed] [Google Scholar]
  18. Glahn DC, Almasy L, Blangero J, Burk GM, Estrada J, Peralta JM, Escamilla MA. Adjudicating neurocognitive endophenotypes for schizophrenia. Am. J. of Med. Genetics Part B: Neuropsychiatric Genetics. 2007;4(2):242–249. doi: 10.1002/ajmg.b.30446. [DOI] [PubMed] [Google Scholar]
  19. Gold JM, Harvey PD. Cognitive deficits in schizophrenia. Psychiatric Clin. of N. Am. 1993;16(2):295–312. [PubMed] [Google Scholar]
  20. Green MF, Marder SR, Glynn SM, McGurk SR, Wirshing WC, Wirshing DA, Liberman RP, Mintz J. The neurocognitive effects of low-dose haloperidol: A two-year comparison with risperidone. Biological Psychiatry. 2002;51:972–978. doi: 10.1016/s0006-3223(02)01370-7. [DOI] [PubMed] [Google Scholar]
  21. Green MF, Nuechterlein KH. The MATRICS initiative: Developing a consensus clinical battery for clinical trials. Schizophr. Res. 2004;72(1):1–3. doi: 10.1016/j.schres.2004.09.006. [DOI] [PubMed] [Google Scholar]
  22. Harvey PD, Keefe RS. Cognitive impairment in schizophrenia and implications of atypical neuroleptic treatment. CNS Spectrums. 1997;2(8):41–55. [Google Scholar]
  23. Heinrichs RW, Zakzanis KK. Neurocognitive deficit in schizophrenia: A quantitative review of the evidence. Neuropsychology. 1998;12(3):426. doi: 10.1037//0894-4105.12.3.426. [DOI] [PubMed] [Google Scholar]
  24. Hill SK, Schuepbach D, Herbener ES, Keshavan MS, Sweeney JA. Pretreatment and Longitudinal Studies of Neuropsychological Deficits in Antipsychotic-Naïve Patients with Schizophrenia. Schizophr. Res. 2004;68:49–63. doi: 10.1016/S0920-9964(03)00213-5. [DOI] [PubMed] [Google Scholar]
  25. Hill SK, Harris MS, Herbener ES, Pavuluri M, Sweeney JA. Neurocognitive allied phenotypes for schizophrenia and bipolar disorder. Schizophr. Bull. 2008a;34(4):743–759. doi: 10.1093/schbul/sbn027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Hill SK, Sweeney JA, Hamer RM, Keefe RS, Perkins DO, Gu H, Lieberman JA. Efficiency of the CATIE and BACS neuropsychological batteries in assessing cognitive effects of antipsychotic treatments in schizophrenia. J. Int. Neuropsychological Society. 2008b;14(2):209–221. doi: 10.1017/S1355617708080570. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hill SK, Reilly JL, Keefe RS, Gold JM, Bishop JR, Gershon ES, Sweeney JA. Neuropsychological impairments in schizophrenia and psychotic bipolar disorder: Findings from the Bipolar-Schizophrenia Network on Intermediate Phenotypes (B-SNIP) study. Am. J. Psychiatry. 2014;170(11):1275–1284. doi: 10.1176/appi.ajp.2013.12101298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hill SK, Bucholtz A, Amsbaugh H, Reilly JL, Rubin L, Gold JM, Keefe RSE, Pearlson GD, Keshavan MS, Tamminga CA, Sweeny JA. Working memory impairments in probands with schizoaffective disorder and first-degree relatives of schizophrenia probands extend beyond deficits predicted by generalized neuropsychological impairment. Schizophr. Res. doi: 10.1016/j.schres.2015.05.018. (IN PRESS). [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hobart MP, Goldberg R, Bartko JJ, Gold JM. Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia: II. Convergent/discriminant validity and diagnostic group comparisons. Am. J. Psychiatry. 1999;156:1951–1957. doi: 10.1176/ajp.156.12.1951. [DOI] [PubMed] [Google Scholar]
  30. Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, Coughenour L. The Brief Assessment of Cognition in Schizophrenia: Reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr. Res. 2004;68(2):283–297. doi: 10.1016/j.schres.2003.09.011. [DOI] [PubMed] [Google Scholar]
  31. Keefe RS, Bilder RM, Harvey PD, Davis SM, Palmer BW, Gold JM, Lieberman JA. Baseline neurocognitive deficits in the CATIE schizophrenia trial. Neuropsychopharmacol. 2006;31(9):2033–2046. doi: 10.1038/sj.npp.1301072. [DOI] [PubMed] [Google Scholar]
  32. Keefe RS, Harvey PD, Goldberg TE, Gold JM, Walker TM, Kennel C, Hawkins K. Norms and standardization of the Brief Assessment of Cognition in Schizophrenia (BACS) Schizophr. Res. 2008;102(1–3):108–115. doi: 10.1016/j.schres.2008.03.024. [DOI] [PubMed] [Google Scholar]
  33. Keefe RS, Harvey PD. Cognitive impairment in schizophrenia. Springer, Berlin: Novel Antischizophrenia Treatments; 2012. pp. 11–37. [DOI] [PubMed] [Google Scholar]
  34. Keshavan MS, Tandon R, Boutros NN, Nasrallah HA. Schizophrenia,“just the facts”: What we know in 2008: Part 3: Neurobiology. Schizophr. Res. 2008;106(2):89–107. doi: 10.1016/j.schres.2008.07.020. [DOI] [PubMed] [Google Scholar]
  35. Lam M, Collinson SL, Eng GK, Rapisarda A, Kraus M, Lee J, Keefe RSE. Refining the latent structure of neuropsychological performance in schizophrenia. Psychological Med. 2014;44:1–14. doi: 10.1017/S0033291714001020. [DOI] [PubMed] [Google Scholar]
  36. Loehlin JC. Latent variable models: An introduction to factor, path, and structural equation analysis. Psychology Press; 2004. [Google Scholar]
  37. McCleery A, Green MF, Hellemann GS, Baade LE, Gold JM, Keefe RSE, Nuechterlein KH. Latent structure of cognition in schizophrenia: a confirmatory factor analysis of the MATRICS Consensus Cognitive Battery (MCCB) Psychological Med. 2015:1–10. doi: 10.1017/S0033291715000641. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Mitrushina M, editor. Handbook of normative data for neuropsychological assessment. Oxford University Press; 2005. [Google Scholar]
  39. Muthén LK, Muthén BO. Mplus statistical analysis with latent variables: User’s guide. Los Angeles: Muthén & Muthén; 2001. [Google Scholar]
  40. Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, Green MF, Heaton RK. Identification of separable cognitive factors in schizophrenia. Schizophr. Res. 2004;72(1):29–39. doi: 10.1016/j.schres.2004.09.007. [DOI] [PubMed] [Google Scholar]
  41. Osborne JW, Costello AB. Sample size and subject to item ratio in principal components analysis. Practical Assess, Res. & Eval. 2004;9(11):8. [Google Scholar]
  42. Prasad KM, Keshavan MS. Structural cerebral variations as useful endophenotypes in schizophrenia: do they help construct “extended endophenotypes”? Schizophr. Bull. 2008;34(4):774–790. doi: 10.1093/schbul/sbn017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Reichenberg A, Harvey PD, Bowie CR, Mojtabai R, Rabinowitz J, Heaton RK, Bromet E. Neuropsychological function and dysfunction in schizophrenia and psychotic affective disorders. Schizophr. Bull. 2009;35(5):1022–1029. doi: 10.1093/schbul/sbn044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Reilly JL, Sweeney JA. Generalized and specific neurocognitive deficits in psychotic disorders: Utility for evaluating pharmacological treatment effects and as intermediate phenotypes for gene discovery. Schizophr. Bull. 2014;40(3):516–522. doi: 10.1093/schbul/sbu013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Ruocco AC, Reilly JL, Rubin LH, Daros AR, Gershon ES, Tamminga CA, Sweeney JA. Emotion recognition deficits in schizophrenia-spectrum disorders and psychotic bipolar disorder: Findings from the Bipolar- Schizophrenia Network on Intermediate Phenotypes (B-SNIP) study. Schizophr. Res. 2014;158(1):105–112. doi: 10.1016/j.schres.2014.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Saykin AJ, Shtasel DL, Gur RE, Kester DB, Gur RC. Neuropsychological deficits in neuroleptic naive patients with first-episode schizophrenia. Arch. Gen. Psychiatry. 1994;51(2):124–131. doi: 10.1001/archpsyc.1994.03950020048005. [DOI] [PubMed] [Google Scholar]
  47. Sitskoorn MM, Aleman A, Ebisch SJ, Appels M, Kahn RS. Cognitive deficits in relatives of patients with schizophrenia: a meta-analysis. Schizophr. Res. 2004;71(2):285–295. doi: 10.1016/j.schres.2004.03.007. [DOI] [PubMed] [Google Scholar]
  48. Snitz BE, MacDonald AW, Carter CS. Cognitive deficits in unaffected first-degree relatives of schizophrenia patients: A meta-analytic review of putative endophenotypes. Schizophr. Bull. 2006;32(1):179–194. doi: 10.1093/schbul/sbi048. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Strauss ME, Summerfelt A. The neuropsychological study of schizophrenia: A methodological perspective. In: Lenzenweger MF, Hooley JM, editors. Principles of experimental psychopathology: Essays in honor of Brendan A. Maher. Washington DC: American Psychological Association; 2003. pp. 119–134. [Google Scholar]
  50. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 5th. Boston: Pearson; 2007. [Google Scholar]
  51. Tamminga CA, Pearlson G, Keshavan M, Sweeney JA, Clementz B, Thaker G. Bipolar and schizophrenia network for intermediate phenotypes: outcomes across the psychosis continuum. Schizophr. Bull. 2014;40(Suppl 2):S131–S137. doi: 10.1093/schbul/sbt179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Yeomans KA, Golder PA. The Guttman-Kaiser criterion as a predictor of the number of common factors. The Statistician. 1982:221–229. [Google Scholar]

RESOURCES