The MATRICS Consensus Cognitive Battery (MCCB): Clinical and Cognitive Correlates

Abstract

The purpose of this study was to examine the cognitive and clinical correlates of the MATRICS Consensus Cognitive Battery (MCCB) which was originally developed to be an endpoint for cognitive enhancement clinical trials. In a sample of 117 people with schizophrenia and 77 healthy control participants we found the following: a) the MCCB was highly sensitive to the type and level of impairment typically observed in schizophrenia, b) the MCCB composite score was highly correlated with WASI Estimated Full Scale IQ score, c) that the MCCB domain scores were generally moderately-highly intercorrelated, d) that MCCB performance was minimally related to clinical symptom type and severity, and e) the MCCB is sensitive to employment status with better performance in employed vs. unemployed patients. These data support the validity of the MCCB as a sensitive measure of cognitive impairment in schizophrenia and suggest that MCCB performance is relevant for functional outcome. The data also suggest that the MCCB domain scores may offer limited resolution on discrete cognitive functions.

1 Introduction

The MATRICS Consensus Cognitive Battery (MCCB) was developed to be an endpoint for clinical trials aiming to enhance cognition in schizophrenia. (Nuechterlein et al., 2008, Kern et al., 2008). To date, there is limited data concerning the clinical and cognitive correlates of the battery. Three recent studies demonstrate the sensitivity of the battery to the general severity and pattern of cognitive impairments expected in people with schizophrenia—a pattern of general impairment on the order of 1–2 standard deviations below control levels with some variation in the extent of impairment across domains (Kern et al., 2011; Shamsi et al., 2011; Keefe et al., 2011). Of note, in each of these publications the overall level of impairment on the MCCB composite score was approximately 2–2.5 standard deviations below that of controls, a greater level or impairment than expected based on meta-analytic findings (Dickinson et al., 2007; Heinrich and Zakzanis, 1998). Such magnification of impairment is likely explained by the use of the MCCB standardized composite score measure, which is sensitive to the extent of impairment across seven domains, versus meta-analytic results that are typically performed on a single task or type of task. In addition, there is recent evidence that MCCB performance is related to important aspects of functional outcome. Kern et al. (2011) reported that employment status is related to performance on measures of processing speed, visual learning, and attention/vigilance. Shamsi et al. (2011) reported that working memory performance was related to work/educational status, verbal memory with residential status, and social cognition related to social functioning. Thus, there is initial evidence that the MCCB provides a clinically useful signal revealing the extent of cognitive impairment and the possible functional significance of this impairment in patients with schizophrenia.

One large gap in this clinical validity literature is that it is unknown how the performance on the MCCB relates to more general measures of cognitive ability such as intelligence or how the MCCB domain scores relate to one another. In an attempt to fill this gap, we investigated the relationship between MCCB performance and general cognitive ability. In addition, we present data on the distributions of MCCB scores in order to enhance clinical interpretation of performance and report on the relationship to symptomatic state, and the key outcome dimensions of work and social functioning.

2 Methods

2.1 Participants

Participants included a total of 117 people with a DSM-IV diagnosis of schizophrenia or schizoaffective disorder (SZ) and 77 healthy controls (HC) (see Table 1 for demographic information). Patient participants were recruited from community clinics and the Maryland Psychiatric Research Center (MPRC) and were being treated with antipsychotics for at least four weeks on stable medication. Patient diagnoses were established using with the Structured Clinical Interview for DSM-IV (SCID) (First et al., 1997), with interview and other historical information being presented at a best-estimate diagnostic conference chaired by one of us (JG). Participants were between the ages of 18 and 55 years, clinically stable (as evidenced by no change in antipsychotic medication type or dose for at least four weeks prior to study entry), with no substance abuse in past month and no dependence within six months.

Table 1.

Demographic Characteristics of Sample

	Schizophrenia	Healthy Controls	Test Statistic
N	117	77
% Male	69%	66%	X2 = 0.19
Mean Age (SD)	41.49 (10.1)	41.00 (10.2)	F = 0.11
Mean Ed Level (SD)	12.59 (2.3)	14.74 (2.1)	F= 43.63*
Maternal Ed Level (SD)	12.93 (3.2)	13.45 (2.4)	F=1.39
Paternal Ed Level (SD)	13.83 (3.8)	13.28 (3.0)	F=1.08
Race (%W, AA, A, Other)	59/33/3/4	61/38/0/1	X2 = 4.59
WTAR	96.34 (16.80)	107.00 (13.37)	F = 21.86*
WASI 2 - Subtest IQ	95.77 (14.92)	113.08 (12.26)	F=71.69*

^*p=.000

Note: W-White; AA- African American; A-Asian; WTAR- Wechsler Test of Adult Reading ; WASI- Wechsler Abbreviated Scale of Intelligence

Healthy control participants were recruited from the community, primarily by random-digit dialing and word of mouth, as well as through community advertisements and letters of invitation to elicit interest in research participation. All controls were screened using the Structured Clinical Interview for DSM-IV (First et al., 1997) and the Structured Interview for DSM-IV Personality (SIDP-IV: Pfohl et. al, 1997). Healthy controls were free of a current Axis I disorder, had no lifetime history of psychosis, no current substance abuse/dependence, no Axis II schizophrenia spectrum personality disorder, and denied a family history of psychosis. All participants denied a history of neurological disorders or significant head injury.

The study was approved by both the University of Maryland Baltimore and Department of Health and Mental Hygiene institutional review boards. All participants gave written informed consent after study procedures and risks had been fully explained. Ability to provide valid consent was ensured by the use of an evaluation to sign consent form.

3 Assessments

3.1 MCCB and Other Cognitive Measures

The participants performed the MCCB (Nuechterlein and Green, 2006) as part of a behavioral research protocol at the MPRC. Participants were also assessed with the Wechsler Test of Adult Reading (WTAR; 2001) and a Wechsler Adult Scale of Intelligence (WASI; 1999) two-subtest estimate of IQ using the Vocabulary and Matrix Reasoning subtests.

3.2 Symptom Evaluation

Clinical ratings including the Brief Psychiatric Rating Scale (BPRS; Overall and Goram, 1962), the Scale for the Assessment of Negative Symptoms (SANS; Andreasen, 1984), and the Calgary Depression Scale (CDS; Addington et al., 1992) were assessed in patient participants.

3.3 Functional Outcome

The Level of Functioning scale (LOF; Hawk et al., 1975) was used to assess social and occupational outcomes in the SZ group. To examine functional outcome, we classified patient participants as having either good (score ≥ 3) or poor (score < 3) vocational and social function based on LOF averaged ratings of the employment/work quality and social contacts/quality of relations items. For the work items this criteria meant that patients were working at least half time and appeared to be functioning at work without difficulty. Unfortunately, job titles were not recorded in a uniform fashion, but in order to receive these ratings, patients would need to have held competitive employment (a few with supports) rather than any type of sheltered work. The higher social functioning group was considered to have a good score if patients maintained at least one close relationship with interpersonal contacts, ranging in frequency from at least once a month to once a week.

3.4 Statistical Analyses

For the HC versus SZ comparison, we fitted the mixed model for repeated measures model: T-score = domain + diagnostic group + domain × diagnostic group, where “domain,” the within-subject factor, is a set of categorical markers of which MCCB domain a given T-score comes from. An unstructured covariance matrix was used to model within-subject correlations among domain T-scores. The domain × diagnostic group interaction estimates the variation among domains in the mean HC - SZ difference in T-scores. If this term was statistically significant, we then did pairwise tests comparing the magnitude of the differences among all 21 possible pairs of the 7 MCCB domains. To control the number of false positive findings among these post hoc tests, we used Benjamini and Hochberg’s (1995) False Discovery Rate (FDR) procedure. P-values for pairwise hypotheses were sorted from largest to smallest, and compared in sequence by rank, with the critical value at the k-th step given by c_k =0.05 × (21 − rank + 1)/21. As soon as a p-value < c_k was encountered, the corresponding null hypothesis and all null hypotheses with smaller p-values were rejected.

Within the patients with schizophrenia, we compared the MCCB domain score profile among SZ with good versus poor vocational performance and good versus poor social functioning, using similar statistical modeling methods. We used Spearman’s Rho correlations to examine the symptom-cognition relationships due to the non-normal distributions of symptom rating values.

4 Results

4.1 The demographic features of the two study groups are shown in Table 1

The groups were similar in age, sex distribution, race distributions, and maternal and paternal education years. The controls had significantly more years of personal education than patient participants as is typically the case in schizophrenia research. The controls scored higher than the patient group on the WTAR and the WASI, but note that the patient performance on these measures is within 1/3 of a standard deviation of the standardization sample means (see Table 1). Thus our patient population does not demonstrate evidence of marked premorbid or general intellectual compromise, enhancing confidence that our findings on the MCCB are not the result of generalized, developmentally based impairment

4.2 What does the MCCB measure?

To address this question we examined: 1) the correlations of the MCCB composite T score and WASI Estimated Full Scale IQ, and 2) the inter-correlations among the MCCB domain scores. The MCCB overall composite score was highly correlated with the WASI Estimated Full Scale IQ score (r_patients = 0.73, p < 0.001; r_controls = 0.69, p < 0.001) (see Figures 1A and 1B for scatterplots), suggesting that the MCCB composite score substantially co-varies with general level of intellectual ability.

Figure 1.

Figures 1a and 1b. Relationship of Estimated Full Scale IQ and MCCB Overall Composite Score

In order for the WASI, a putative measure of general intelligence derived from sampling two broad cognitive factors, and the MCCB, a putative measure of 7 different cognitive domains, to demonstrate such a high level of correlation at the composite score level it would be necessary for the 7 MCCB domain scores to be substantially inter-correlated with each other, and each with WASI Estimated IQ. As seen in Table 2, in both healthy controls and patients, the domain scores are typically highly correlated with one another (less true of social cognition than other domains), with this effect being somewhat more pronounced in patients than in controls. In patients the mean Pearson inter-correlation of the domain scores amongst each other was 0.454, whereas it was 0.416 in controls. Thus, in both groups, good performance in one domain predicts good performance in others. Similarly, the mean Pearson correlation of the 7 domain scores with WASI IQ was 0.536 in patients (range 0.216 to 0.819) and 0.485 in controls (range 0.151 to 0.816), demonstrating a similar level of overlap across the more discrete MCCB domains and general intellectual ability. These initial analyses support two broad conclusions: 1) the MCCB overall composite score is strongly correlated with general intellectual ability, and 2) that the magnitude of the correlations among the MCCB domain scores and general ability, while impressive, still allows for the possibility that different domain scores may have different clinical correlates. To examine this issue, we next examined the role of the overall composite score, and the domain scores, in sensitivity to clinical diagnosis and important clinical features of the illness.

Table 2.

Correlations between MCCB Cognitive Domain T Scores and Wechsler Abbreviated Scale of Intelligence

	ProcSpeed	AttnVig	WorkMem	VerbLrn	VisLrn	ProbSolv	SocCog	Composite	WASI IQ
ProcSpeed	1	.577**	.625**	.650**	.390**	.516**	.291**	.816**	.540**
AttnVig	.579**	1	.610**	.581**	.343**	.292**	.285**	.731**	.427**
WorkMem	.659**	.490**	1	.611**	.517**	.390**	.201**	.797**	.590**
VerbLrn	.595**	.482**	.478**	1	.394**	.445**	.242**	.796**	.587**
VisLrn	.510**	.326**	.517**	.529**	1	.458**	.151	.676**	.567**
ProbSolv	.544**	.501**	.529**	.426**	.554**	1	.168	.668**	.491**
SocCog	.313**	.340**	.370**	.294**	.291**	.216*	1	.456**	.194
Composite	.819**	.731**	.784**	.729**	.736**	.726**	.576**	1	.695**
WASI IQ	.640**	.491**	.705**	.498**	.495**	.469**	.456**	.733**	1

Correlations for healthy controls are above the table diagonal and correlations for people with schizophrenia are below the diagonal. Note: ProcSpeed- Processing Speed; AttnVig- Attention Vigilance; WorkMem – Working Memory; VerbLrn – Verbal Learning; VisLrn – Visual Learning; ProbSolv – Reasoning and Problem Solving; SocCog – Social Cognition; Composite – Overall Composite Score; WASI- Wechsler Abbreviated Scale of Intelligence. No p-values were between 0.05 and 0.01.

^*Correlation is significant at p <0.01 level (2 tailed);

^**Correlation is significant at p < 0.001 level (2 tailed);

4.3 Sensitivity to diagnosis

An independent sample t-test comparing the overall composite MCCB score between the patient and control groups was highly significant, t(192)=10.94, p<0.001. As seen in Figure 2, the patient and control distributions are dramatically shifted apart. In the MPRC sample, 56% of healthy controls had an overall composite MCCB of 50 (the mean T score for the MATRICS normal standardization sample) or above. In contrast, only 7% of patient participants scored above 50. Thus, the MCCB overall score appears to be a very reliable indicator of the type of broad intellectual impairment that is characteristic of the illness.

Figure 2. MCCB Overall Composite Score Distributions.

MATRICS overall composite score distributions for healthy controls (black bars) and patients (white bars)

4.4 Extent of impairment across domains

We next turned to the question of whether consideration of the individual domain scores might qualify or inform the evidence that the composite scores was a powerful measure of between group differences using a mixed model for repeated measures. For the HC versus SZ comparison, both the average across domains of the HC - SZ difference (main effect of diagnostic group, F(1,192)=119.34, p<0.001)) and the diagnostic group × domain interaction (F(6,187) = 7.34, p<0.001) were statistically significant (See Figure 3). Post hoc testing from the mixed model found that 9 of 21 pairwise comparison among domains on the magnitude of the HC-SZ difference remained significant after application of the Benjamini-Hochberg (1995) method to keep the False Discovery Rate <0.05: 1) the HC - SZ difference on processing speed was significantly larger than the HC - SZ differences for all other domains except social cognition (unadjusted p=0.060), with unadjusted p-values ranging from p<0.006 to p < 0.001 (between-domain differences in size of HC-SZ contrast in T-scores ranging 4.2±1.4 to 8.9±1.6); 2) the HC - SZ difference on working memory was significantly larger than the HC - SZ differences on verbal learning (3.5±1.5, unadjusted p=0.018) and reasoning/problem-solving (4.7±1.6, unadjusted p=0.004); 3) the HC - SZ difference on reasoning/problem-solving was significantly smaller than the HC - SZ differences on attention/vigilance (−4.12±1,7, unadjusted p<0.015) and social cognition (−5.02±2.06, unadjusted p<0.016).. Thus, measures of processing speed, working memory, and social cognition appear to be differentially sensitive to diagnostic status.

Figure 3. MCCB Domain Mean T Score Comparisons.

Note: ProcSpeed- Processing Speed; AttnVig- Attention Vigilance; WorkMem – Working Memory; VerbLern – Verbal Learning; VisLearn – Visual Learning; ProbSolv – Reasoning and Problem Solving; SocCog – Social Cognition; Composite – Overall Composite Score

In order to facilitate interpretation of results at the domain level, we present the observed patient sample percentile distributions for MCCB domain scores in Table 3. These data may be of use in clinical interpretation of test performance among patients, acknowledging the potential limits of generalizability from our sample given potential ascertainment bias issues and relatively modest size. These data may facilitate answering the question of “how impaired is this individual patient relative to other patients with this illness on this cognitive domain” rather than in comparison to healthy controls.

Table 3.

Percentile Distributions for MCCB Domain scores for the total patient sample (n = 117)

Patients	Composite Score	Speed of Processing	Attention/Vigilance	Working Memory	Verbal Learning	Visual Learning	Reasoning and Problem Solving	Social Cognition
Mean	30	35	38	38	38	34	42	38
Percentile
1	7	9	11	16	24	7	25	15
5	8	18	17	22	26	15	28	18
10	14	21	23	25	27	18	30	22
20	19	25	29	28	30	24	33	26
30	22	27	31	33	32	27	36	30
40	26	30	34	36	34	31	38	32
50	28	32	40	39	37	33	40	37
60	31	38	42	41	39	36	42	41
70	35	41	45	43	41	39	48	44
80	40	44	49	46	44	45	51	48
90	49	50	52	51	52	50	55	54

4.5 MCCB: Symptom Correlates

In light of this evidence that the MCCB total and domain scores are highly sensitive to diagnostic status, we next turned to the question of whether specific illness features might be implicated. Does MCCB performance reflect symptom status or aspects of functional outcome? To address these issues we first examined correlation coefficients between the MCCB domain and composite scores and SANS total scores, BPRS total score and the positive, negative, and disorganization BPRS factor scores as well as the Calgary Depression scale. No significant correlations were observed with Calgary Depression Scale ratings. The SANS total scores were significantly correlated with processing speed (r_s = −0.31, p < 0.001), working memory (r_s = −0.30, p <0.001) and overall composite (r_s = −0.22, p = 0.021). The BPRS disorganized syndrome was negatively correlated with the overall MCCB overall composite score (r_s = −0.20, p = 0.037) and several individual domain scores as well: processing speed (r_s = −0.19, p = 0.049), verbal learning (r_s = −0.19, p = 0.042), visual learning (r_s = −0.21, p = 0.027) and most robustly with social cognition (r_s = −0.25, p = 0.007). Thus, there is a relatively modest degree of overlap between the MCCB and measures of clinical symptom severity.

4.6 MCCB: Cognitive Domains Associated with Vocational Outcome and Social Functioning

As seen in Figure 4, the Good vocational outcome group performed at higher levels than the Poor vocational outcome group. Statistical analysis revealed evidence for an overall effect of group (where the average difference across domains for the Good - Poor work groups was significant (mean difference ± s.e. 4.8 ± 1.8, F(1,112)=7.35, p=0.008). Further the work group × domain interaction were statistically significant (F(6,107)=4.47, p<0.001. Good- Poor work group differences in domain means (± s.e.) on attention/vigilance (5.9 ± 2.6, p=0.026), processing speed (10.6 ± 2.5, p<0.001) and working memory (7.5 ± 2.3, p=0.002) were statistically significant. In post hoc comparisons of the magnitude of mixed model estimates of Good - Poor work differences among domains, using the FDR multiple comparison method, the good - poor work differences was significantly greater for processing speed than for verbal learning, visual learning, or reasoning/problem-solving, and for working memory than for visual learning.

Figure 4. MCCB Cognitive Domain Scores vs. Vocational Performance.

Note: ProcSpeed- Processing Speed; AttnVig- Attention Vigilance; WorkMem – Working Memory; VerbLern – Verbal Learning; VisLearn – Visual Learning; ProbSolv – Reasoning and Problem Solving; SocCog – Social Cognition; Composite – Overall Composite Score

For the Good versus Poor social total groups within SZ comparison, neither the average across domains of the Good - Poor difference (mean difference across domains ± s.e. = 2.9 ± 1.6, p=0.069, favoring the good social functioning group) nor the social group × domain interaction were statistically significant (p=0.27). Since the social group × domain interaction was not statistically significant, no post hoc tests were performed in the overall modeling analysis. We did, however, examine performance on the social cognition domain score and found that the mean score of the good social functioning group score was 5.13 ± 2.41 points higher than in the poor social functioning group, a significant difference, t(112)= −2.13, p<0.05.

5 Discussion

The MCCB is sensitive to the cognitive impairments observed in schizophrenia. The reported frequency distribution of the overall composite MCCB score is strikingly different in SZ and HC populations, demonstrating the sensitivity of the battery to the type of impairment observed in schizophrenia. Note that the composite score of the current study sample (29.6) very closely resembles the scores of 28.6 and 24.7 reported by Kern et al. (2011), and Keefe et al. (2001) respectively, suggesting that the sample we studied is fairly representative of the types of patients who get included in research studies across multiple sites. Patients demonstrated clear impairments, relative to controls, on each of the MCCB domain scores. Inspection of the patient percentile distribution offers a somewhat more nuanced perspective. For example, the 50^th percentile of patient performance is at a T score of 39–40 on the attention/vigilance, working memory, and reasoning and problem solving domain scores but is at a T score of 32–33 on the speed of processing and visual learning domains, suggesting that these latter two areas may be more frequently severely impaired in patients. Concern might be raised that the apparent greater impairment in processing speed needs is an artifact of greater reliability for this domain score, which is based on the performance of three different tasks, relative to the other domain scores. However, in a study of 323 participants with schizophrenia who had performed the MCCB on two occasions approximately 2 weeks apart, Keefe et al (2011), found that the ICCs of the speed of processing, attention/vigilance, working memory domain, reasoning/problem solving and social cognition domains scores were all between 0.77 and 0.79; only the T-scores for verbal learning (0.58) and visual learning (0.65) had potentially meaningfully lower reliability than processing speed. In the meta-analysis of Dickinson, Ramsey and Gold that found that digit symbol, as a single measure, appears to be the most reliably impaired commonly used clinical neuropsychological task

The MCCB overall score was significantly correlated with WASI IQ score in both groups. Thus, the mix of abilities assessed on the MCCB and combined in the summary T score substantially overlaps with more general intellectual ability. This may have implications for the use of the MCCB composite score as a measure of cognitive change. In clinical neuropsychological assessment, measures of Full Scale IQ are often considered to be more resistant to the impact of many types of brain injury than some of the subtests that contribute to the Full Scale IQ score (Lezak et al., 2004). If this logic may be extended in the opposite direction, in the consideration of cognitive enhancement, then it is possible that some of the individual MCCB domain scores may provide a more sensitive measure than the overall T score. This theoretical possibility needs to be tempered by the recognition that the reliability of domain scores is substantially lower than that of composite measures. Clearly, this is speculative and the hypothetical advantage of domain over composite scores needs to be tested in the context of clinical trials where cognitive change has been observed.

Of note, the MCCB domain scores are substantially inter-correlated with one another, with the cross domain correlations involving social cognition tending to be somewhat lower. Our cross domain correlations, (0.536 in patients and 0.485 in controls), are substantially higher than the mean cross domain correlation of 0.31 reported among patients by Shamsi et al (2010). We cannot explain this difference. However, the fact that our data are consistent across two samples, and are in close agreement with the meta-analysis presented by Dickinson and Gold, increase confidence that our results are likely to be reliable. From a “glass half full” perspective, the magnitude of many of the correlations suggests that a positive treatment effect observed on one domain is likely to be observed in other domains as well. From a “glass half empty” perspective, these data suggest that the assessment of specific cognitive domains may be limited on the MCCB. That is, if there was a compound that was hypothesized to have a specific, differential effect on episodic memory (or any of the domains) in particular, the measures available on the MCCB do not offer a “construct pure” assessment given the substantial correlations with other domains. It remains to be seen if measures coming from the cognitive neuroscience literature as part of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; Carter and Barch, 2007) initiative can provide for these types of “construct pure” assessments that may be useful in the context of cognitive clinical trials.

Our data also suggest that MCCB performance is relevant for functional outcome. We saw clear differences in performance between patients as a function of employment status, with better overall performance in the employed group of patients (see Figure 4). The post-hoc tests suggest that processing speed is the single domain most powerfully associated with vocational status, consistent with the results reported by Kern et al. (2011), while we also found evidence for a working memory effect as reported by Shamsi et al. (2011) using a very different analytic approach. When we looked at social functioning, the overall pattern of group differences the trend for group differences to favor the good social functioning group was not statistically significant. The fact that the social cognition domain score was related to actual social functioning is encouraging evidence for the utility and validity of this measure.

In summary, our data suggest that the MCCB is highly sensitive to the types of impairments observed in schizophrenia. The composite score appears to be a measure of broad intellectual ability and the individual domain scores demonstrate significant inter-correlations. MCCB performance is related to functional outcome in the area of vocational performance.