Confirmatory factor analysis reveals a latent cognitive structure common to bipolar disorder, schizophrenia, and healthy adults

Abstract

Objective

We sought to determine whether a single hypothesized latent factor structure would characterize cognitive functioning in three distinct groups.

Methods

We assessed 576 adults (340 community controls, 126 with bipolar disorder, and 110 with schizophrenia) using 15 measures derived from nine cognitive tests. Confirmatory factor analysis (CFA) was conducted to examine the fit of a hypothesized six-factor model. The hypothesized factors included attention, psychomotor speed, verbal memory, visual memory, ideational fluency, and executive functioning.

Results

The six-factor model provided an excellent fit for all three groups [community controls root mean square error of approximation (RMSEA) < 0.048 and comparative fit index (CFI) = 0.99; adults with bipolar disorder RMSEA = 0.071 and CFI = 0.99; and adults with schizophrenia RMSEA = 0.06 and CFI = 0.98]. Alternate models that combined fluency with processing speed or verbal and visual memory reduced the goodness-of-fit. Multi-group CFA results supported factor invariance across the three groups.

Conclusions

Confirmatory factor analysis supported a single six-factor structure of cognitive functioning among patients with schizophrenia or bipolar disorder and community controls. While the three groups clearly differ in level of performance, they share a common underlying architecture of information processing abilities. These cognitive factors could provide useful targets for clinical trials of treatments that aim to enhance information processing in persons with neurological and neuropsychiatric disorders.

Cognitive dysfunction is an established feature of schizophrenia (SZ) (1, 2). Persons with bipolar disorder (BD) also show cognitive deficits, even during periods of stable mood (3), but their deficits tend to be less severe than those shown by patients with SZ (4). Because cognitive dysfunction likely moderates work and other functional outcomes, it is important to understand the essential dimensions of human cognitive functioning (5–7).

Numerous attempts to characterize the structure of cognitive architecture have been reported. In healthy adults, numerous studies provide well-founded, compelling evidence that human cognitive functioning reflects a hierarchical, multi-factorial architecture (8, 9). In SZ research, despite a few reports to the contrary (10), most previous studies have found that multi-factorial models best describe cognitive functioning as well. Nuechterlein and colleagues (1) reviewed 13 factor analytic studies based primarily on samples of persons with SZ. Their aim was to abstract from the literature a set of separable factors to inform the selection of tests for the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative. They concluded that six factors capture the fundamental domains of cognitive functioning typically disrupted in SZ. These include processing speed, attention, visual learning/memory, verbal learning/memory, working memory, and reasoning/problem solving. Dickinson et al. (11) also found that a six-factor model of cognitive functioning best fit the test performance of 157 healthy adults (NC) and 148 patients with SZ. The six factors assess verbal comprehension, perceptual organization, verbal learning/memory, visual learning/memory, information processing speed (including verbal fluency), and working memory/executive functioning. Keefe et al. (12) conducted both exploratory factor analyses (EFA) and confirmatory factor analyses (CFA) of 11 cognitive tests administered to 1,332 persons with SZ in the Clinical Antipsychotic Trials of Intervention Effectiveness. These were thought to assess five cognitive domains (processing speed, reasoning, verbal memory, working memory, and vigilance) and social cognition. However, EFA yielded only one factor with an eigenvalue greater than 1.0, and this factor accounted for just 45% of the test variance. While CFA confirmed that a five-factor model fit the data, a single-factor model based on the five domain scores proved a better fit. Thus, there is still no consensus on how to best represent the latent structure of cognitive functioning in SZ.

Compared to the extensive literature on cognition in SZ, less is known about the cognitive architecture in BD. While an increasing number of research groups have examined aspects of cognitive functioning among patients with BD, we know of only one factor analytic study (13). These investigators administered 14 cognitive measures to 155 adult outpatients with BD. Using both EFA and CFA, they found that a six-factor structure—consisting of attention, working memory, learning, verbal knowledge, non-verbal functions, and ideational fluency—best represented the underlying cognitive structure in BD. The same factor structure based on the same cognitive measures also emerged from an analysis of patients with SZ (14), suggesting that these illnesses are associated with fundamentally similar latent cognitive organizations. However, since this is the only factor analytic study in BD thus far, replication is necessary.

It is important to determine whether the underlying organization of human cognition is similar in persons with SZ or BD and NC. Finding a common structure would provide a useful foundation for efforts designed to elucidate cognitive endophenotypes that might cross diagnostic categories and identify more homogenous subgroups of patients with affective versus non-affective psychoses (15–17). Despite group differences in the severity of cognitive dysfunction, such studies assume a common underlying organization of cognition (18–20). However, we are not aware of clear empirical evidence to support this assumption across SZ, BD, and NC. If the cognitive architecture varies across groups (i.e., if there are group-specific differences in how variables relate to one another and what they represent), the same neuropsychological tests might assess different abilities in patients with SZ or BD and cognitively intact persons (18, 21). Thus, it becomes difficult to detect and interpret potential differences in neurocognitive processes (18–20, 22). Dowling et al. (20) argued that investigations of construct comparability are needed to ensure that observed differences in cognitive test performances represent true differences between groups rather than artifacts caused by nonequivalent constructs. Finding a common factor structure across pathological and healthy populations also would enable us to extract a few meaningful dimensions from the large number of variables that comprise most comprehensive test batteries, and thereby Type I errors from multiple comparisons (14).

Several statistical methods are available to investigate dimensions of cognitive functioning. While EFA is frequently used to elucidate the underlying latent structure of cognitive functioning, CFA is preferred for testing a priori hypotheses (23). One advantage of CFA is that variables are grouped into factors a priori based on either a conceptual model or previous findings. In addition, CFA enables one to evaluate how well a hypothesized model fits the observed data, and to compare it with specific alternative models (24). A common approach is to examine nested models, wherein each subsequent model is created by combining one or more factors from a preceding model into a single factor. In comparing nested models, the fit indices for each competing model are tested against all others to identify the best-fitting model. Finally, the use of multi-group CFAs has been suggested as a method to compare factorial invariance across groups (18, 20, 22, 25). This approach examines the significance of potential differences in factor structures across groups and enables the detection of the specific aspects of a factor solution by which groups differ (20).

In summary, notwithstanding many previous studies of cognitive organization in SZ and the growing literature on cognition in BD, the fundamental dimensions of human cognition remain unclear. Most importantly, whether a single underlying organizational structure characterizes patients with SZ or BD and NC is still unknown.

In this study, we examined the adequacy of a six-factor cognitive architecture in NC, persons with BD, and persons with SZ. The underlying factor structure was based on the assignment of 15 cognitive measures to one of six domains (psychomotor speed, attention/vigilance, ideational fluency, executive functioning, verbal learning/memory, and visual learning/memory). In a previous publication, we reported that patients with BD and SZ performed worse than NC in all six domains, and that patients with BD outperformed those with SZ in every domain except attention/vigilance (4). However, that study did not examine construct validity for the hypothesized six-factor model of cognitive architecture. Nor did it show whether the underlying structural model applies as well to individuals with normal brain functioning as it does to those with BD or SZ. Finally, it did not address the question of whether alternative models might better capture the underlying architecture of information processing. In the present study, we therefore tested the hypotheses that our six-factor model would represent the constituent measures better than single-factor, four-factor, and five-factor models in three participant samples separately, and in the pooled sample of all three groups.

Methods

Participants and procedures

Altogether, 236 patients were recruited for studies of cognitive functioning in BD or SZ. The patients were recruited mostly from outpatient clinics and via fliers posted at The Johns Hopkins Hospital. A few were recruited from psychiatric inpatient units or a day hospital at the Johns Hopkins Hospital. One patient sample included 126 adults with BD type I. Most (75%) of these patients had a history of psychotic symptoms (i.e., hallucinations or delusions) when acutely manic or depressed. The other patient sample included 110 adults with SZ (but not schizoaffective or schizophreniform disorder). Diagnoses in both samples were made by a study psychiatrist based on DSM-IV criteria (26). In order to minimize the effects of illness acuity on clinician ratings and cognitive test performance, most patients were assessed as outpatients. However, four persons with BD and 11 with SZ were tested immediately prior to discharge from the hospital, when their attending physicians determined that they were stable. Potential study participants were excluded if they had any history of mental retardation, dementia, stroke, traumatic brain injury with more than one hour loss of consciousness, or any type of substance dependence within the preceding 12 months.

In addition, 394 adults (NC) representative of the community were recruited from the Baltimore, MD, and Hartford, CT, metropolitan areas via random digit dialing or calling randomly selected listings from residential telephone directories for participation in the Aging, Brain Imaging, and Cognition (ABC) study (27). Fifty-four participants with a history of substance dependence within the preceding 12 months, dementia, stroke, mental retardation, traumatic brain injury with greater than one hour loss of consciousness, or other medical or neurological conditions commonly associated with cognitive impairment (e.g., multiple sclerosis, Parkinson's disease), SZ, BD, or current major depression, or who scored below 24 out of 30 on the Mini-Mental State Exam (MMSE) (28) were excluded from the NC sample. This left a total of 340 NC. The Johns Hopkins Medicine and Hartford Hospital Institutional Review Boards approved all the studies from which subjects were drawn, and each person gave written informed consent to participate.

Clinical diagnoses of patients were based on the Diagnostic Interview for Genetic Studies (DIGS) (29), Structured Clinical Interview for DSM-IV Axis I Disorders-Clinician Version (SCID-IV) (30), or Mini International Neuropsychiatric Interview (MINI) (31), which were administered by a board-certified psychiatrist or clinical psychologist. Symptom severity was rated by the clinician based on the Scales for the Assessment of Positive (SAPS) and Negative Symptoms (SANS) (32). Prior to making a diagnosis, the study clinician reviewed all available psychiatric records. Healthy controls also underwent a diagnostic and clinical assessment by a study psychiatrist or psychologist. This included a structured psychiatric interview, the Schedule for Clinical Assessment in Neuropsychiatry (SCAN) (33), review of medical history, and physical and neurological examinations.

Cognitive measures

Each participant was administered nine neuropsychological tests from which 15 measures were derived for analysis. The tests were administered and scored according to standard instructions by a trained research assistant. The specific measures selected from each test and the cognitive processes they assess are summarized in Table 1.

Table 1.

Summary of tests administered

Cognitive function	Test	Measures selected	NC (n = 340)	SZ (n = 110)	BD (n = 126)
Fine manual speed and dexterity	Grooved Pegboard Test (60)	Mean completion time (sec) over two trials of each hand	84.01 (28.97)	95.52 (24.63)	86.02 (24.73)
Visual scanning and sequencing	Trail Making Test (TMT) (61)	Part A (TMT-A) completion time (sec)	34.23 (16.21)	46.43 (20.12)	35.46 (16.38)
		Part B (TMT-B) completion time (sec)	91.18 (63.41)	145.36 (93.99)	103.76 (68.63)
Auditory divided attention	Brief Test of Attention (BTA) (62)	Total correct for Letters (BTA-L)	7.83 (2.07)	5.84 (2.33)	6.96 (2.21)
		Total correct for Numbers (BTA-N)	7.77 (1.97)	6.21 (2.21)	6.95 (2.15)
Visual sustained attention	Conners' Continuous Performance Test (CPT) (63)	Hit reaction time standard error (CPT RTse)	6.41 (2.39)	9.26 (4.56)	9.43 (8.88)
Verbal learning/memory	Hopkins Verbal Learning Test-Revised (HVLT) (64)	Total learning over three trials (HVLT 1–3)	24.68 (4.71)	18.90 (5.12)	23.57 (5.90)
		Delayed free recall (HVLT delay)	8.75 (2.68)	5.72 (2.65)	7.77 (3.09)
Visual learning/memory	Brief Visuospatial Memory Test-Revised (BVMT) (65)	Total learning over three trials (BVMT 1–3)	22.50 (7.27)	17.64 (7.51)	21.27 (7.01)
		Delayed free recall (BVMT delay)	8.82 (2.58)	6.98 (3.09)	8.06 (2.93)
Ideational fluency	Calibrated Ideational Fluency Test Battery (66)	Letter-cued word fluency	28.39 (9.02)	21.89 (9.32)	26.31 (7.87)
		Category-cued word fluency	44.91 (11.52)	34.42 (10.53)	42.50 (10.65)
		Design fluency	14.28 (6.99)	7.79 (5.26)	10.38 (5.86)
Executive functioning	Modified Wisconsin Card Sorting Test (67)	Numbers of category sorts	35.00 (5.25)	29.10 (10.05)	34.33 (5.54)
		Numbers of perseverative errors	7.95 (7.58)	17.08 (11.87)	9.16 (8.06)

NC = healthy adults; SZ = schizophrenia; BD = bipolar disorder.

Statistical analyses

CFA

CFA was used to examine the relationship between the observed variables and the hypothesized underlying constructs. The a priori six-factor model and the five alternative models are specified in Table 2. The hypothesized six-factor model includes psychomotor speed [Trail Making Test–part A (TMT-A), Trail Making Test–part B (TMT-B), and Grooved Pegboard (GPT)], attention [Brief Test of Attention–Letter (BTA-L), Brief Test of Attention–Number (BTA-N), and Conners' Continuous Performance Test hit reaction time standard error (CPT RTse)], verbal learning/memory [Hopkins Verbal Learning Test-Revised–Learning (HVLT Lrn) and Hopkins Verbal Learning Test–Delayed Recall (HVLT Del)], visual learning/memory [Brief Visuospatial Memory Test-Revised–Learning (BVMT Lrn) and Brief Visuospatial Memory Test-Revised–Delayed Recall (BVMT Del)], ideational fluency (Letter, Category, and Design), and executive functioning [Modified Wisconsin Card Sorting Test–category sorts (M-WCST cat) and Modified Wisconsin Card Sorting Test–errors (M-WCST err)]. This model was compared to a one-factor model, a four-factor model, two different five-factor models, and a different six-factor model in which TMT-B was assigned to the executive functioning dimension.

Table 2.

Model specifications for confirmatory analyses

Model	Factors	Measures
Six-factor model	Psychomotor Speed	TMT-A, TMT-B, GPT
	Attention	BTA-L, BTA-N, CPT RTse
	Ideational Fluency	Letter, Category, Design
	Verbal Memory	HVLT Lrn, HVLT Del
	Visual Memory	BVMT Lrn, BVMT Del
	Executive Function	M-WCST cat, M-WCST err
Five-factor speed model	Psychomotor Speed	TMT-A, TMT-B, GPT, Letter, Category, Design
	Attention	BTA-L, BTA-N, CPT RTse
	Verbal Memory	HVLT Lrn, HVLT Del
	Visual Memory	BVMT Lrn, BVMT Del
	Executive Function	M-WCST cat, M-WCST err
Five-factor memory model	Psychomotor Speed	TMT-A, TMT-B, GPT
	Attention	BTA-L, BTA-N, CPT RTse
	Ideational Fluency	Letter, Category, Design
	Memory	HVLT Lrn, HVLT Del, BVMT Lrn, BVMT Del
	Executive Function	M-WCST cat, M-WCST err
Four-factor model	Psychomotor Speed	TMT-A, TMT-B, GPT, Letter, Category, Design
	Attention	BTA-L, BTA-N, CPT RTse
	Memory	HVLT Lrn, HVLT Del, BVMT Lrn, BVMT Del
	Executive Function	M-WCST cat, M-WCST err
One-factor model	General Cognition	All measures

TMT-A = Trail Making Test–Part A; TMT-B = Trail Making Test Part–B; GPT = Grooved Pegboard; BTA-L = Brief Test of Attention–Letter; BTA-N = Brief Test of Attention–Number trials; CPT RTse = Continuous Performance Test hit reaction time standard error; HVLT Lrn = Hopkins Verbal Learning Test-Revised–Learning; HVLT Del = Hopkins Verbal Learning Test-Revised–Delayed recall; BVMT Lrn = Brief Visuospatial Memory Test-Revised–Learning; BVMT Del = Brief Visuospatial Memory Test-Revised–Delayed recall; M-WCST cat = Modified Wisconsin Card Sorting Test–category sorts; M-WCST err = Modified Wisconsin Card Sorting Test–errors.

In the five-factor speed model, we grouped fluency tests in the psychomotor speed factor as suggested by Nuechterlein et al. (1), but left other dimensions of the six-factor model unchanged. In the five-factor memory model, we grouped verbal and visual memory measures together as a general memory factor, but left the other factors unchanged. In the four-factor model, we used the original executive functioning and attention factors, but included fluency measures in the psychomotor speed factor and grouped verbal and visual memory tests in a general memory factor. Finally, we defined a one-factor model, in which all the measures were grouped together, to test the hypothesis that a single general cognitive ability factor would adequately account for all the measures used.

Analyses were conducted using LISREL 8.80 (Chicago, IL, USA) (34) with maximum likelihood estimation. Seven different goodness-of-fit statistics were used to assess the fit between the hypothesized models and the actual data: the χ², the χ²/degrees of freedom (df) ratio, the comparative fit index (CFI), the root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the Akaike Information Criterion (AIC), and the non-normed fit index (NNFI). The χ² statistic is used to evaluate the fit between the hypothesized statistical model and the actual data set, while the χ²/df ratio is less dependent on sample size (35). The RMSEA indicates the fit of the model to the covariance matrix and represents the square root of the average amount by which sample covariances differ from their estimates derived on the basis of the posited factor model. The SRMR is defined as the standardized difference between the observed correlation and the predicted correlation. The CFI compares the final model with an independence null model that assumes all variables are uncorrelated with dependent variable. RMSEA and SRMR values below 0.05 indicate a very close fit, while values below 0.08 reflect a reasonable error of approximation (36, 37). NNFI and CFI at 0.90 or greater indicate a good fit, and 0.95 or greater indicate a close fit (34). The AIC is a comparative measure of fit and so it is meaningful only when two different models are estimated. Lower values indicate a better fit and so the model with the lowest AIC is the best fitting model (38). Finally, a model that has a χ²/df ratio < 3 is considered to fit well the observed data (39).

We conducted a series of multi-group CFAs to test three levels of factorial invariance: dimensional, configural, and metric. Dimensional invariance requires the finding that the same number of factors provides the best fit for each group. If this requirement is met, then configural invariance will be tested. Configural invariance requires that identical corresponding items (i.e., the same cognitive measures) define each factor of the common model. If this requirement is met, then we will test for metric invariance, which is the most rigorous form of factorial invariance appropriate to the present study. Metric invariance requires equivalent factor loadings across all three groups. Unlike dimensional and configural invariance, it is tested in two steps. First, the data from both groups are modeled simultaneously with equality constraints on the corresponding factors. Then the configural and metric invariance models are compared. If the fit indices do not change significantly, this suggests that the model has metric invariance, meaning that factor loadings are consistent across the groups (40). Previous studies have concluded that using χ² difference for testing invariance results in a number of problems. For example, it is more sensitive to minor departures from multivariate normality and it is usually significant with complex models and large samples (41, 42), and might lead researchers to contradictory conclusions (43). Therefore, alternative goodness-of-fit indices have been recommended. CFI, SRMR, and RMSEA statistics are less sensitive to sample size and they are superior to χ² for testing invariance in large groups (41, 44). As suggested by Chen (41), when sample sizes are unequal, as it is the case in this study, the following cutoff criteria were used for testing loading invariance: a change of ≤ 0.005 in CFI, supplemented by a change of ≤ 0.010 in RMSEA, or a change of ≤ 0.025 in SRMR would indicate invariance. Hence, although reported, the χ² statistics were not further discussed.

Results

Sample characteristics

Table 3 presents the demographic and clinical characteristics of the samples. MANOVA and chi-squared analyses showed that the three groups differed significantly in age, years of education, and estimated premorbid IQ based on the Hopkins Adult Reading Test (HART) (45). Planned contrasts showed that the SZ and BD groups were younger than NCs. Patients with SZ, but not BD, completed fewer years of schooling and produced lower premorbid IQ estimates than NCs. Bonferroni-adjusted post-hoc comparisons revealed that BD patients completed more years of schooling and produced higher premorbid IQ estimates than patients with SZ, although the two groups did not differ in age. Chi-squared analyses showed that the diagnostic groups differed in sex and race, with one exception: the BD and NC groups did not differ in sex. Independent samples t-tests and chi-squared analyses revealed that SZ and BD patients did not differ in age at illness onset, number of prior psychiatric hospitalizations, or illness duration. The SZ patients manifested more severe negative (SANS) and positive (SAPS) symptoms than BD patients. Also, more SZ patients than BD patients were receiving antipsychotic medications; whereas more BD than SZ patients were receiving lithium, antidepressant, and anticonvulsant medications.

Table 3.

Demographic characteristics of the patients and healthy control groups^a,b

	NC (n = 340)	SZ (n = 110)	BD (n = 126)	Statistic	p-value
Age, years	53.5 ± 18.5d	39.5 ± 11.2e	42.4 ± 10.8e	F(2,571) = 44.1	0.001
Sex, male:female, (%)	43.5:56.5d	70:30e	39:61d	χ2(2) = 28.2	0.001
Race, White:Black:Other, (%)	79:18:3d	39:55:6e	55:40:5f	χ2(4) = 68.9	0.001
Education, years	14.1 ± 3.0d	12.1 ± 2.3e	14.0 ± 3.2d	F(2,571) = 19.5	0.001
Estimated premorbid IQc	104.6 ± 10.2d	96.6 ± 10.5e	102.9 ± 11.6d	F(2,571) = 23.3	0.001
Age at onset, years	—	22.8 ± 7.4	24.9 ± 9.0	t(212) = −1.8	0.064
Illness duration, years	—	16.7 ± 10.6	17.6 ± 10.7	t(212) = −0.6	0.519
No. of hospitalizations	—	5.0 ± 5.6	3.7 ± 5.1	t(210) = 1 .8	0.066
SANS (sum)	—	8.9 ± 5.5	1.8 ± 2.4	t(193) = 8.6	0.001
SAPS (sum)	—	4.7 ± 3.8	1.0 ± 1.8	t(191) = 11.9	0.001
Typical antipsychotic (%)	—	34	5	χ2(1) = 14.7	0.001
Atypical antipsychotic (%)	—	74	47	χ2(1) = 13.9	0.001
Any antidepressant (%)	—	23	48	χ2(1) = 12.0	0.002
Lithium (%)	—	4	56	χ2(1) = 58.6	0.001
Any anticonvulsant (%)	—	12	44	χ2(1) = 23.7	0.001

NC = healthy adults; SZ = schizophrenia; BD = bipolar disorder; SANS = Scale for the Assessment of Negative Symptoms; SAPS = Scale for the Assessment of Positive Symptoms.

^aValues shown as mean ± standard deviation for continuous variables and percentages for categorical variables, denoted by (%) following variable name.

^bFor three-group analyses, different subscripts denote significant (p < 0.05) differences.

^cEstimated premorbid IQ based on the Hopkins Adult Reading Test (45).

CFA

The data showed multivariate normality in the three samples (Mardia's test of multivariate kurtosis for: patients with BD = 1.28; patients with SZ = 1.02; NC = 1.31). Mardia's test values of < 3 assume the assumption of multivariate normality is met (46). Table 4 shows the goodness-of-fit statistics for each hypothesized model in all three groups. The one-factor model proved to fit the data poorly in every sample. The RMSEA values ranged from 0.165 for patients with SZ to 0.181 for NCs. Other goodness-of-fit statistics yielded similar results (NNFI range = 0.70 to 0.78; CFI range = 0.74 to 0.81). Thus, the one-factor model clearly does not represent a latent structure that fits the observed data. The five-factor speed model, which groups fluency measures with psychomotor speed, yielded more promising results. However, the χ²/df ratio and RMSEA statistics indicate that this model does not adequately fit the data for NC (χ²/df ratio = 3.38 and RMSEA = 0.084) and patients with BD (RMSEA = 0.082), even though it does fit the data for patients with SZ (χ²/df ratio = 1.54 and RMSEA = 0.071). However, the six-factor model represents a significant improvement over the five-factor speed model for the SZ group (χ² difference = 17.65, df = 5, p < 0.005). Both models in which the verbal and visual memory scores were grouped together as a single factor (five-factor memory model and four-factor model) yielded poor goodness-of-fit statistics for all three groups. Similarly, the six-factor model in which we grouped the TMT-B with measures comprising the executive function factor did not fit the data for any group. Overall, our hypothesized six-factor cognitive model showed the best fit to the observed data for all three subgroups (χ²/df range = 1.4 to 1.7; RMSEA range = 0.05 to 0.06, SRMR range = 0.03 to 0.05; NNFI range = 0.96 to 0.98; CFI range = 0.97 to 0.99). The modification indices did not suggest adjustments in the model that could be plausibly interpreted in a substantive manner. Table 5 presents the inter-factor correlations across groups.

Table 4.

Goodness-of-fit statistics for confirmatory factor analyses

Model	Group	χ 2	df	χ2/df	RMSEA	NNFI	CFI	SRMR	AIC
Six-factor model	NC	134.60	75	1.79	0.048	0.983	0.988	0.038	225.42
	BD	122.28	75	1.63	0.071	0.960	0.972	0.059	212.28
	SZ	105.07	75	1.40	0.060	0.975	0.982	0.052	195.07
Six-factor model with TMT-B in EF factor	NC	258.03	75	3.44	0.085	0.936	0.954	0.100	348.02
	BD	145.38	75	1.93	0.087	0.941	0.957	0.090	2335.38
	SZ	152.31	75	2.03	0.097	0.923	0.945	0.103	242.31
Five-factor speed model	NC	270.84	80	3.38	0.084	0.955	0.965	0.055	350.84
	BD	145.86	80	1.82	0.081	0.951	0.963	0.061	225.85
	SZ	123.35	80	1.54	0.071	0.967	0.975	0.060	203.35
Five-factor memory model	NC	353.08	80	4.41	0.100	0.916	0.936	0.075	433.08
	BD	207.42	80	2.59	0.112	0.879	0.907	0.071	287.42
	SZ	214.44	80	2.68	0.124	0.892	0.918	0.072	294.44
Four-factor model	NC	478.64	84	5.69	0.117	0.895	0.916	0.082	550.65
	BD	231.57	84	2.75	0.118	0.873	0.899	0.072	303.57
	SZ	232.57	84	2.76	0.127	0.889	0.911	0.080	304.58
One-factor model	NC	1094.01	90	12.15	0.181	0.700	0.743	0.104	1154.01
	BD	355.96	90	3.95	0.165	0.782	0.813	0.100	478.73
	SZ	418.74	90	4.65	0.171	0.724	0.763	0.093	415.96

df = degrees of freedom; RMSEA = root mean square error of approximation; NNFI = non-normed fit index; CFI = comparative fit index; SRMR = standardized root mean square residual; AIC = Akaike Information Criterion; TMT-B = Trail Making Test–Part B; EF = executive functioning; NC = healthy adults; BD = bipolar disorder; SZ = schizophrenia.

Table 5.

Inter-factor correlations among factors across groups^a

	Processing Speed	Attention	Ideational Fluency	Verbal Memory	Visual Memory	Executive Function
Processing Speed	1.00/1.00/1.00
Attention	−0.73/0.72/0.72	1.00/1.00/1.00
Ideational Fluency	0.64/0.70/0.78	0.56/0.53/0.75	1.00/1.00/1.00
Verbal Memory	−0.48/0.66/0.82	0.43/0.64/0.68	−0.33/0.49/0.58	1.00/1.00/1.00
Visual Memory	−0.65/0.54/0.70	0.46/0.47/0.42	−0.38/0.42/0.41	0.55/0.66/0.45	1.00/1.00/1.00
Executive Function	0.51/0.61/0.64	−0.43/0.38/0.57	−0.28/0.42/0.42	0.50/0.56/0.69	0.43/0.44/0.53	1.00/1.00/1.00

^aInter-factor loadings for each sample shown as follows: Healthy adults/Bipolar disorder/Schizophrenia.

Standardized factor loadings are shown in Figure 1. All of the loadings were statistically significant (p < 0.05). This indicates that the measures loaded on the factors they were hypothesized to assess. Most measures showed high loadings on each factor. The mean factor loadings were very high for each group (NC = 0.79, BD = 0.78, and SZ = 0.77). The CPT hit reaction time standard errors showed the weakest factor loadings across samples. Design fluency showed factor loadings that were intermediate in magnitude between those of letter and category word fluency in all three groups.

Fig. 1.

Results of a confirmatory factor analysis for the hypothesized six-factor cognitive model. Standardized factor loadings for each sample shown as follows: Healthy adults; Bipolar disorder; Schizophrenia. Error terms were not included in the figure. BTA-L = Brief Test of Attention–Letter; BTA-N = Brief Test of Attention–Number; CPT-RTse = Continuous Performance Test hit reaction time standard error; TMT-A = Trail Making Test–Part A; TMT-B = Trail Making Test–Part B; GPT = Grooved Pegboard; BVMT Lrn = Brief Visuospatial Memory Test-Revised–Learning; BVMT Del = Brief Visuospatial Memory Test-Revised–Delayed recall; HVLT Lrn = Hopkins Verbal Learning Test-Revised–Learning; HVLT Del = Hopkins Verbal Learning Test-Revised–Delayed recall; M-WCST cat = Modified Wisconsin Card Sorting Test–category sorts; M-WCST err = Modified Wisconsin Card Sorting Test–errors.

Factorial invariance

Single group analyses supported the six-factor model for each of the three groups, suggesting dimensional invariance. Similarly, all individual indicators were associated with the six factors in a consistent pattern across groups, indicating configural invariance. Finally, the equivalence of the factorial loadings across groups (metric invariance) was tested (Table 6). The changes in fit indices between the configural and metric invariant model were less than the cut-offs provided and, thus, not substantial for any of the group comparisons (BD versus SZ; BD versus NC; and SZ versus NC). All values of ΔCFI, ΔRMSEA, and ΔSRMR were < 0.005, < 0.010, and 0.025, respectively. These results show that factor loadings were equivalent across groups, thereby demonstrating metric invariance.

Table 6.

Goodness-of-fit statistics for configural and metric Invariance

Groups and model	df	χ 2	RMSEA	RMSEA 90% CI	ΔRMSEA	CFI	ΔCFI	SRMR	ΔSRMR
BD versus SZ					0.003		0.001		0.0086
Configural invariance	150	169.42	0.033	0.000–0.056		0.993		0.0571
Metric invariance	159	176.11	0.030	0.000–0.054		0.994		0.0657
BD versus NC					0.005		0.004		0.0032
Configural invariance	150	240.17	0.053	0.040–0.065		0.985		0.0368
Metric invariance	159	274.32	0.058	0.046–0.070		0.981		0.0400
SZ versus NC					0.007		0.003		0.0027
Configural invariance	150	189.53	0.034	0.016–0.048		0.994		0.0368
Metric invariance	159	219.17	0.041	0.027–0.054		0.991		0.0395

BD = bipolar disorder; NC = healthy adults; SZ = schizophrenia; df = degrees of freedom; RMSEA = root mean square error of approximation; RMSEA 90% CI = RMSEA 90% confidence interval; CFI = comparative fit index; SRMR= standardized root mean square residual.

Discussion

As far as we know, this is the first study to compare alternative factor structures derived from a common set of neuropsychological instruments in patients with SZ or BD and NC. Our findings confirm the existence of a common latent cognitive structure that consists of six factors: attention, psychomotor speed, ideational fluency, verbal memory, visual memory, and executive functioning. This model fit the observed data better and more consistently than every alternative that we tested, and it was invariant across the three groups.

Unlike the factor solutions proposed by previous factor analytic studies (13, 14, 47, 48) we did not find a separate factor for working memory. However, the factor we labeled attention is also partially measuring working memory. Both the BTA and CPT measures, which were found to load on this factor, assess not only the ability to resist distraction and sustain attention, but also to hold and manipulate information currently in mind. In fact, BTA performance correlates highly with classical measures of working memory (49). Moreover, in an EFA, the BTA and CPT together also defined a single factor in patients with first-episode psychosis and NC (50). Finally, our proposed model is consistent with factor analytic studies in NC, which identify a common factor for attention and working memory (51).

The cognitive structure model we obtained resembles that hypothesized (though not shown) to underlie the MATRICS test battery (1). The major difference is that we found a separate factor composed of letter word, category word, and design fluency. This is consistent with findings of hierarchical multiple regression and an EFA, suggesting that ideational fluency represents a fundamental dimension of human cognition (52). This capacity is not material-specific, as it involves both verbal (word) and nonverbal (non-nameable design) production. Fluency measures have also been reported to load on factors that primarily assess working memory (53), verbal memory (54), or executive functioning (14). However, most previous analyses relied exclusively on tests of word list generation to assess fluency. Finally, three other studies exploring the cognitive factor structure in patients with SZ (13, 14, 55) and one study in patients with BD (13) further support the conceptualization of ideational fluency as a separate factor. Among the six-factor solutions that emerged in these studies, measures of ideational fluency comprised a distinct domain of cognition.

In this study we explicitly compared the comparability of cognitive constructs across psychiatric groups and NC using multi-group CFA to test different forms of factorial invariance. As others have noted, while the comparability of cognitive factor structures across different diagnostic groups is a prerequisite for meaningful group comparisons (56), it is routinely ignored in neuropsychological studies (18, 20, 57). In fact, among the vast number of studies comparing cognitive test performance of groups with various psychiatric disorders to healthy adults, we have yet to find a single study using the methods reported here to assess factorial invariance. After we evaluated the relative fit of six competing hypothetical models using CFA, we examined the stability of the optimal model across distinct homogeneous groups with invariance analysis.

Our finding of a common latent cognitive structure invariant across the three groups has important implications. We previously found that patients with BD or SZ performed more poorly than NCs on all six of the latent cognitive factors described in the present study (4). The same cognitive factor scores also differentiated SZ and BD patient groups. Together with the current findings, these results support the inference that patients with SZ or BD differ in the severity of their neurocognitive impairment, but not in the organization of latent cognitive structure. Thus, any differences observed are quantitative and not qualitative in nature. Of note, there were substantial differences between the NC, SZ, and BD groups on age, sex, and race. As demographic factors are correlated with cognitive performances (58, 59), we cannot exclude the possibility that potential effects of these demographic differences may have confounded our findings, resulting in greater variability of performances among the groups. However, the common factor structure obtained in this study was robust against demographic and also clinical differences among the groups. Neither the greater incidence of positive symptoms in patients with SZ, nor the greater presence of depressive symptoms in patients with BD, altered the cognitive structure of these groups. Likewise, neither group differences in medication exposure, nor putative differences in the neurobiological substrates of BD and SZ altered the factorial grouping of cognitive measures. Consequently, the observed factor structure represents core cognitive functions that could be useful targets for clinical trials of medications and behavioral interventions designed to enhance cognition in patients with BD or SZ. Finally, finding the same factor structure in NC suggests that it might also generalize to other populations not included in this study, but verification of this will require replication in different groups.

In summary, in this study a common cognitive architecture emerged from analyses of patients with BD, patients with SZ, and NC. This establishes the comparability of underlying cognitive functions across samples, which is an essential prerequisite to compare their cognitive profiles. One limitation of the present study, like all studies using CFA, is that the results depend on the measures used and different factor structures obtained can be attributed to different tasks administered. Moreover, tests may share method variance and it is unlikely that cognitive measures or the obtained factors are truly independent (8). Our selection of measures was based on earlier work, is clinically meaningful, and was intended to capture functions often compromised by neuropsychiatric disorders. However, we acknowledge that our findings need to be interpreted with caution, and replication of the proposed factor structure with larger neuropsychological batteries using multiple different neuropsychological measures to assess/represent each cognitive factor is necessary. A final limitation of this study is that the findings are based solely on cross-sectional data. Future studies will be required to determine whether the common structure identified here remains stable over time.