Abstract
The MATRICS Consensus Cognitive Battery (MCCB) fills a significant need for a standardized battery of cognitive tests to use in clinical trials for schizophrenia in adults aged 20-59. A need remains, however, to develop norms for younger individuals, who also show elevated risks for schizophrenia. Toward this end, we assessed performance in healthy adolescents. Baseline MCCB, reading and IQ data were obtained from healthy controls (ages 12-19) participating in two concurrent NIMH-funded studies: North American Prodromal Longitudinal Study phase 2 (NAPLS-2; n=126) and Boston Center for Intervention Development and Applied Research (CIDAR; n=13). All MCCB tests were administered except the Managing Emotions subtest from the Mayer-Salovey-Caruso Emotional Intelligence Test. Data were collected from 8 sites across North America. MCCB scores were presented in four 2-year age cohorts as T-scores for each test and cognitive domain, and analyzed for effects of age and sex. Due to IQ differences between age-grouped subsamples, IQ served as a covariate in analyses. Overall and sex-based raw scores for individual MCCB tests are presented for each age-based cohort. Adolescents generally showed improvement with age in most MCCB cognitive domains, with the clearest linear trends in Speed of Processing, Attention/Vigilance, and Working Memory. These control data show that healthy adolescence is a dynamic period for cognitive development that is marked by substantial improvement in MCCB performance through the 12-19 age range. They also provide healthy comparison raw scores to facilitate clinical evaluations of adolescents, including those at risk for developing psychiatric disorders such as schizophrenia-related conditions.
Keywords: Healthy Controls, Adolescents, MATRICS Consensus Cognitive Battery, Psychosis, clinical high risk, schizophrenia, cognition
Neurocognitive impairments are a central problem for individuals with schizophrenia, beginning in childhood during the premorbid period and continuing throughout life. The development of a pathway for regulatory approval for new treatment strategies to reduce cognitive deficits in schizophrenia created the need for a standardized battery of neuropsychological tests to assess the effectiveness of proposed treatments. A National Institute of Mental Health (NIMH) initiative to encourage the development of novel interventions to attenuate cognitive deficits in schizophrenia, called Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS), addressed this need. This led to the development of a reliable, valid ―MATRICS Consensus Cognitive Battery‖ (MCCB) for use in clinical trials and for other purposes (Kern, et al., 2008; Nuechterlein, et al., 2008). The MCCB is comprised of 9 tests that reflect six distinct, replicated dimensions of neurocognitive dysfunction in schizophrenia, including: Speed of Processing, Attention/Vigilance, Working Memory, Verbal Learning, Visual Learning, and Reasoning and Problem Solving (Nuechterlein et al., 2004). A 10th test, the Managing Emotions Branch of the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT), and the additional dimension, Social Cognition, was added because of growing interest in this component of function, and because of its relevance to real world function (Nuechterlein and Green, 2006).
Thus far, investigations into the psychometric properties of the MCCB have been positive. The MCCB demonstrates excellent reliability, minimal practice effects, a low rate of missing data and significant correlations with functional capacity measures (Green, et al., 2011; Nuechterlein, et al., 2008). Subsequent large, industry-sponsored, multi-site clinical trial studies have affirmed these positive psychometric characteristics (Buchanan, et al., 2011; Keefe, et al., 2011). The MCCB has also shown strong sensitivity to change in adults with schizophrenia who participated in an intensive cognitive remediation program (Fisher et al., 2009).
Validation studies involving the MCCB have also been encouraging. Progress has been made in demonstrating that improvement on neuropsychological tests such as those in the MCCB is related to improvement in =real-world‘ tasks (Buchanan, et al., 2005; Buchanan, et al., 2011). Methods for evaluating performance-based and interview-based measures were developed as part of the MATRICS Psychometric and Standardization Study (PASS) (Green, et al., 2008), and supported by the Validation of Intermediate Measures study (VIM) (Green, et al., 2011). The success of the MCCB has also been facilitated by its translation into many languages for use in international trials. Indeed, the MATRICS website (http://www.matricsinc.org) listed 21 languages by 2015.
The initial success of the MCCB reliability and validity studies has spurred efforts to characterize the battery further and expand its utility. Recent investigations focused on efforts to refine its factor structure (Burton, et al., 2013; McCleery, et al., 2015), assess its performance in other psychiatric conditions such as bipolar disorder (Burdick, et al., 2011; Sperry, et al., 2015), and evaluate its performance in older individuals (Rajji, et al., 2013). One area in particular need of further study is the performance of psychiatrically healthy adolescents on the MCCB. While the MCCB norms are stratified for age and sex, and cover the adult age ranges of 20-59 in five year intervals, a substantial proportion of people with schizophrenia experience the onset of their illness (Cannon, et al., 1999; Juuhl-Langseth et al., 2014; Thomsen, 1996) or associated, prodromal symptoms of their illness before age 20. The interpretation of the MCCB scores is thus limited in pharmacological and/or psychosocial clinical trials involving youth. Individuals with early-onset schizophrenia (EOS; onset before age 18), including childhood onset (COS; onset by age 13) and adolescent-onset schizophrenia (AOS; onset between ages 13-18), constitute an important clinical population, particularly for early intervention.
There are additional reasons to study cognitive functions in this younger age group. First, the identification of cognitive problems is significant in its own right, regardless of etiology. Second, it is often prognostic, especially in clinical high risk (CHR) adolescent samples (Fusar-Poli, Borgwardt, et al., 2012; Fusar-Poli, Deste, et al., 2012; Giuliano, et al., 2012; Seidman, et al., 2010; Woodberry et al., 2008). Moreover, cognitive weaknesses often predict current function in high risk youth (Lin, et al., 2011), rates of premorbid abnormalities, familial schizophrenia-spectrum disorders (Nicolson, et al., 2003), subsequent exacerbations of cognitive problems (Rajji et al., 2009), greater social disability (Eggers and Bunk, 1997) and functional outcomes over time (Amminger, et al., 2011; Ballageer et al., 2005; Hollis, 2000; Insel, 2007, 2010).
It is thus important to clearly interpret MCCB performance in healthy individuals younger than 20, and particularly in adolescents. Among the few published studies using the MCCB in patients with schizophrenia-spectrum disorders and healthy control subjects below the age of 18, adolescents aged 11-13 years old who showed symptoms of psychosis performed more poorly on several MCCB tasks than adolescents who did not show psychotic symptoms (Kelleher et al., 2013). Similarly, adolescents 12 to 18 years old with schizophrenia-spectrum disorders (mean=15.8 years) showed stable performance deficits compared to age-matched healthy controls on all MCCB domains except social cognition (Holmen et al., 2010; Juuhl-Langseth, et al., 2014).
Still needed, however, are studies of MCCB performance across normal development, both to identify developmental trajectories, and to establish useful comparison data for researchers and clinicians. A Norwegian standardization project reported progress towards this goal by measuring MCCB performance for 5 groups ranging from 12 to 59 years of age, including a 12 to 19 year-old group (n=50) (Mohn et al., 2012). Similarly, a recent study assessed MCCB performance in 5 groups ranging from 8 to 23 years of age and narrowed the age bands to 4-year bins (e.g. 8 to 11 years; n‘s=21 to 54 for the 3 adolescent bins) (Nitzburg, et al., 2014). This study provided raw scores and T-scores, the latter of which were derived across all age groups. While both of these studies assessed adolescents, they covered this dynamically complex and heterogeneous stage of development as either a single eight-year period (Mohn, et al., 2012), or as 3 four-year periods that also included younger children in the earlier period and adults older than 19 in the later period (Nitzburg, et al., 2014). The current study extends the analysis of normal MCCB performance by focusing solely on a teenage sample to assess age-related differences within adolescence, with data presented in 4 two-year age periods that range from ages 12 to 19, further refining healthy comparison data for this period.
We present adolescent control data and examine the influence of age and sex on test performance, comparing our findings with those described in the original adult (aged 20-59) version. Our goal is to expand the utility of the MCCB as a standard battery for adolescents by providing both developmental trajectories for each of its cognitive domains (except social cognition), and control comparison data as raw scores to facilitate clinical evaluations of adolescents who may be at risk for developing psychiatric disorders.
METHODS
Participants
Participants included 139 adolescents and young adults between the ages of 12-19 who were recruited as healthy control (HC) volunteers from two concurrent NIMH-funded studies: the second phase of the multi-site North American Prodromal Longitudinal Study (NAPLS-2; n=126; 91% of the overall sample, which included eight sites from four geographic regions), and the single-site Boston Center for Intervention Development and Applied Research (CIDAR) study, ―Vulnerability to Progression in Schizophrenia‖ (n=13; 9% of the overall sample). The studies evaluated the same age ranges and comparison groups (i.e., CHR youth), during the same time period (2007-2012), and it thus seemed justified to combine them. Study inclusion and exclusion criteria were similar for the two studies, and are summarized in Table 1.
Table1.
Variable | NAPLS-2 | CIDAR |
---|---|---|
Adolescent Age Range | 12-19 | 13-19 |
Screening for Axis I DSM-IV- TR Disorders |
Structured Clinical Interview (SCID) for DSM-IV-TR, Non- patient edition (First, Spitzer, Gibbon, & Williams, 2002) for ages ≥18, or the Kid-SCID (Hien, et al., 1994) for subjects 13- 17 years of age, |
|
Assessment of Prodromal Symptoms and Syndromes |
Structured Interview for Prodromal Syndromes; (Miller, et al., 2003) |
|
Exclusion Criteria | Current or past major DSM-IV-TR Axis I disorder, prodromal syndrome, schizotypal or other Cluster A personality disorders, significant traumatic brain injury or other central nervous system disorder, sensory motor handicaps interfering with test taking, psychiatric hospitalizations, history of electroconvulsive therapy, a first degree relative with psychosis, current or past use of antipsychotic medications, other psychotropic medication use within 6 months of study participation (except sleep medications or anxiolytic agents), IQ < 70, lack of fluency in English, DSM-IV-TR substance abuse in the past month or dependence in the last 3 months (excluding nicotine), current suicidality. |
All study participants (or legal guardians for those under 18) gave written informed consent prior to study participation. Participants under 18 provided assent. The NAPLS-2 study was approved by institutional review boards (IRBs) at all NAPLS-2 sites (Emory University, Harvard University/Beth Israel Deaconess Medical Center, University of California Los Angeles, University of California San Diego, University of Calgary, University of North Carolina, Yale University, and Zucker Hillside Hospital). The CIDAR study was approved by local Boston IRB committees at Harvard Medical School, Beth Israel Deaconess Medical Center, Massachusetts General Hospital, and Brigham and Women‘s Hospital.
Neuropsychological Measures
Intelligence was estimated with the Vocabulary and Block Design subtests of the Wechsler Abbreviated Scale of Intelligence-I (WASI-I) (Wechsler, 1999), and word reading skills were estimated with the Word Reading subtest of the Wide Range Achievement Test-4 (Wilkinson and Robertson, 2006). The MCCB (Kern, et al., 2008; Nuechterlein and Green, 2006) was administered as part of larger neuropsychological assessments in the NAPLS and CIDAR studies. Nine of 10 MCCB tests were administered to all participants. The Managing Emotions subtest of the MSCEIT (Mayer et al, 2002) was excluded because its validity for subjects as young as age 12 was not established and alternative measures of social cognition were used in the studies. The six cognitive domains and nine tests are shown in Table 2.
Table 2.
COGNITIVE DOMAIN | TEST |
---|---|
Speed of Processing | Category fluency/verbal fluency for animals (CF) (Blair & Spreen, 1989) |
Symbol coding subtest from the Brief Assessment of Cognition in Schizophrenia (SC-BACS) (Keefe, et al., 2004) | |
Trail Making Test- Part A (TMT-A) (United States War Department, 1944) | |
Attention/Vigilance | Continuous Performance Test-Identical Pairs (CPT-IP) (Cornblatt, Risch, Faris, Friedman, & Erlenmeyer-Kimling, 1988) |
Working Memory | University of Maryland—Letter-Number Span (LNS) (Gold, Carpernter, Randolph, Goldberg, & Weinberger, 1997) |
Spatial Span subtest from the Wechsler Memory Scale-III (WMS-III SS) (Wechsler, 1997b) | |
Verbal Learning | Hopkins Verbal Learning Test- Revised (HVLT-R) immediate recall (Brandt & Benedict, 2001) |
Visual Learning | Brief Visuospatial Memory Test-Revised (BVMT-R) (Benedict, 1997) |
Reasoning and Problem Solving | Mazes subtest from the Neuropsychological Assessment Battery (NAB Mazes) (White & Stern, 2003) |
Data Analyses
Statistical analyses were performed with IBM SPSS Statistics, Version 20 (IBM Corporation, Somers, NY). To support direct comparison with the original MCCB co-norming and standardization study (Kern, et al., 2008), we largely adopted the same data analytic procedures, with some exceptions. We were not able to compute a social cognition domain score and overall composite score given that the MSCEIT subtest was not administered. Moreover, due to significant IQ differences between age groups (described below), we employed IQ as a covariate in subsequent analyses. This modification from the procedure used to norm the MCCB in adults allowed us to address our central goal of assessing its domain-specific cognitive effects beyond the level of generalized cognitive effects (i.e. IQ) more clearly. This approach is consistent with evidence that generalized and more specific cognitive effects are separable in schizophrenia (Mohn et al., 2014b; Nuechterlein, et al., 2004). It may be particularly useful more generally in developmental periods characterized by larger cognitive performance differences than are typically present in young and middle adulthood, such as adolescence and aging (Rentz, et al., 2004; Rentz, et al., 2007; Steinberg, Smith et al., 2005; Steinberg, Bieliauskas et al., 2005).
As with Kern et al., raw scores for each of the 9 MCCB tests administered were examined for normality of distribution (Kern, et al., 2008). Any distributions that were notably skewed were logarithmically transformed. Raw or transformed scores were standardized to T scores (mean = 50, SD = 10) based on the overall sample of 139 participants. Summary scores for cognitive domains that included more than one measure were computed by summing the T scores of the tests in those domains and standardizing the sums to T scores. The T scores were analyzed to examine age, sex, and education effects. Differences in performance across the six cognitive domains were assessed using analyses of covariance (ANCOVAs). Follow-up contrasts were conducted for significant ANCOVAs, with a p value <0.05 considered statistically significant.
RESULTS
Several distributions of the nine MCCB tasks were skewed and thus transformed as described above. Logarithmically transformed variables included Trail Making Test Part A (following which the direction of the log scores was reversed before T-score conversion), HVLT-R, BVMT-R, and NAB Mazes. Sociodemographic and other characteristics of the overall sample and age cohorts are provided in Table 3. The age range from 12 to 19 provides coverage of the adolescent period of development up to the lower boundary of the MCCB standardization sample. Because age effects are typically substantial for most cognitive tests during the rapid neurodevelopment of the childhood and adolescent periods, age was used as the primary stratification variable. However, because this healthy control sample was ascertained to match CHR or first episode (FE) psychosis clinical subjects within their respective studies, and was not collected for purely normative purposes, the sample sizes varied across age cohorts with ns ranging from 10 to 22. Due to relatively low sample sizes within some individual one-year age bands, participants were grouped into two-year age increments. Participants‘ age ranged between 12.1 and 19.9 years, with a mean of 16.3 years (SD = 2.2). Given the significant difference in estimated WASI IQ between the four age groups (F (3, 135) = 5.02, p = .002), as noted above, IQ was used as a covariate in all analyses of MCCB T-score differences. Tables 4 and 5 provide cognitive domain T-score and test raw score means and standard deviations by 2-year age cohort, respectively. Figure 1 displays age-referenced performance levels for each of the six cognitive domains. Furthermore, in line with procedures detailed previously for the MCCB scoring program (Kern, et al., 2008), regression-based T-scores are presented for all cognitive domains in Supplemental Table 3.
Table 3.
Age Cohort/
Characteristics |
12-13
(n = 24)* |
14-15
(n = 37)* |
16-17
(n = 39)* |
18-19
(n = 39)* |
12-19
combined (n = 139)* |
---|---|---|---|---|---|
Age M (SD) | 13.1 (0.6) | 14.9 (0.6) | 17.0 (0.5) | 19.1 (0.6) | 16.3 (2.2) |
Gender, Males N (%) | 18 (75%) | 24 (65%) | 21 (54%) | 20 (51%) | 83 (60%) |
Participant Years of Educationa M (SD) |
6.8 (0.9) | 8.4 (0.9) | 10.4 (0.8) | 12.5 (0.6) | 9.8 (2.2) |
Median Parental Education Levelb |
7.0 | 7.0 | 7.0 | 7.0 | 7.0 |
Median Household Incomec | 6.0 | 5.0 | 6.0 | 5.0 | 5.5 |
Race: Caucasian N (%) | 15 (63%) | 23 (62%) | 18 (46%) | 23 (59%) | 79 (57%) |
Ethnicity: Non- Hispanic/Latino N (%) |
22 (92%) | 29 (78%) | 32 (82%) | 33 (85%) | 116 (84%) |
Handedness Right N (%) | 24 (100%) | 34 (92%) | 31 (80%) | 33 (85%) | 122 (88%) |
WASI FSIQa,d M (SD) | 109.5 (15.9) |
107.0 (11.9) |
104.2 (18.3) |
116.6 (12.6) |
109.3 (15.4) range 72- 139 |
GAF, M (SD) | 85.3 (11.5) | 83.8 (9.6) | 79.7 (13.1) | 83.6 (9.6) | 82.8 (11.1) |
Global Functioning Scale, M (SD) Social Role |
8.9 (1.1) 8.6 (1.6) |
9.0 (0.7) 8.5 (1.0) |
8.5 (1.2) 8.1 (1.4) |
8.7 (0.8) 8.7 (0.8) |
8.8 (1.0) 8.5 (1.2) |
Note: All sites contributed to each 2-year age band except for the 12-13 year olds for whom none came from the
University of North Carolina.
ns vary by variable; actual n indicated in notes for superscript letters b and c below.
Significant age group differences were found for WASI FSIQ (p =0.002) and education (p < .001); Participant age and education level are highly correlated (r = 0.96).
Highest level of education achieved by participant’s mother or father. 6: some college/technical school/undergraduate education; 7: completed college/technical school/ undergraduate education; 8: some graduate/professional school; ns: 23, 35, 38, 38, and 134 left to right across columns.
Median Household Income: 5: $60,000-99,999, 6: $100,000 and above. ns: 24, 33, 37, 32, and 126 left to right across columns.
Wechsler Abbreviated Scale of Intelligence (WASI) Full Scale IQ estimate is based on administration of two subtests: Vocabulary and Block Design.
Table 4.
Age Cohort/
Cognitive Domain |
12-13
(n = 24) M (SD) |
14-15
(n = 37)* M (SD) |
16-17
(n = 39)* M (SD) |
18-19
(n = 39)* M (SD) |
12-19
combined (n = 139) M (SD) |
---|---|---|---|---|---|
Speed of Processing |
43.4 (9.5) | 48.0 (9.2) | 51.5 (9.6) | 54.4 (9.1) | 50 (10) |
Attention/Vigilance | 42.1 (8.7) | 47.7 (9.6)a | 51.5 (8.9)b | 55.8 (8.3)c | 50 (10)d |
Working Memory | 44.8 (8.6) | 47.2(8.6) | 50.8 (10.3) | 55.0 (9.5) | 50 (10) |
Verbal Learning | 49.4 (11.4) | 48.9 (9.5) | 47.5 (10.4) | 53.9 (8.3) | 50 (10) |
Visual Learning | 47.4 (11.1) | 49.7 (7.5) | 51.1 (11.0) | 50.8 (10.3) | 50 (10) |
Reasoning & Problem-Solving† |
45.6 (10.1) | 49.7 (8.8) | 51.6 (10.6) | 51.4 (10.0) | 50 (10) |
ns vary by cognitive domain; actual n indicated by letter superscript:
= 36;
= 37;
= 37;
= 134.
Significant overall group sex differences were found for this cognitive domain (p < .001).
Table 5.
Age Cohort/
MCCB Test |
12-13
(n = 24)* M (SD) |
14-15
(n = 37)* M (SD) |
16-17
(n = 39)* M (SD) |
18-19
(n = 39)* M (SD) |
12-19 combined
(n = 139)* M (SD) |
---|---|---|---|---|---|
TMT: Part A | 33 6(14.7) | 28.2 (10.3) | 26.8 (10.1) | 23.5 (10.0) | 27.4 (11.4) |
BACS Symbol Coding | 52.9 (10.1) | 58.5 (10.3) | 64.7 (10.9) | 66.6 (13.0) | 61.6 (12.2) |
HVLT-R† | 26.7 (4.8) | 26.7 (4.3) | 25.9 (4.9) | 29.0 (3.1) | 27.1 (4.4) |
WMS-II Spatial Span | 15.8 (2.8) | 17.3 (3.2) | 17.8 (3.4) | 18.5 (3.2) | 17.5 (3.3) |
Letter-Number Span | 14.5 (3.1) | 14.3 (2.8) | 15.7 (3.4) | 17.3 (3.1) | 15.6 (3.3) |
NAB Mazes† | 18.4 (5.4) | 20.7 (3.8) | 21.0 (5.3) | 21.2 (4.2) | 20.5 (4.7) |
BVMT-R | 25.3 (6.9) | 27.5 (4.1) | 27.5 (5.5) | 27.5 (5.6) | 27.1 (5.5) |
Category Fluency: Animal Naming |
22.3 (5.5) | 23.5 (5.7) | 24.5 (5.3) | 25.2 (4.8) | 24.0 (5.3) |
CPT-IP d1 | 2.00 (0.64) | 2.42 (0.70)a |
2.70 (0.65)b |
3.01 (0.60)c |
2.58 (0.73)d |
TMT = Trail Making Test; BACS = Brief Assessment of Cognition in Schizophrenia; HVLT-R = Hopkins Verbal Learning Test-Revised; WMS: Wechsler Memory Scale; NAB = Neuropsychological Assessment Battery; BVMT-R = Brief Visuospatial Memory Test-Revised; CPT-IP = Continuous Performance Test – Independent Pairs.
d1 (d prime): = target/nontarget discrimination during vigilance
ns vary by test; actual n indicated by letter superscript:
= 36;
= 37;
= 37;
= 134.
Significant overall group gender difference
Performance level significantly increased with age in all cognitive domains except for Verbal and Visual learning (Speed of Processing: F (3, 130) = 9.34, p < .0001; Attention/Vigilance: F (3, 125) = 10.61, p < .0001; Working Memory: F (3, 130) = 7.44, p < .0001; Verbal Learning: F (3, 130) = 1.17, p = .32; Visual Learning: F (3, 130) = 2.37, p = .07; Reasoning and Problem-Solving: F (3, 130) = 4.47, p = .005). Follow-up polynomial contrasts for the domains showing significant age effects revealed significant quadratic trends for Speed of Processing (p = .004) and Reasoning and Problem-Solving (p = .005), and significant linear trends for Attention/Vigilance (p < .0001) and Working Memory (p < .0001). These results suggest that while there were generally significant incremental improvements in performance with age across these domains, there was some leveling off in performance with increased age for Speed of Processing and Reasoning and Problem-Solving. For the regression-based T-scores, there were similar significant quadratic trends for Speed of Processing (p = .003) and Reasoning and Problem-Solving (p = .004), but there were no significant trends observed for Attention/Vigilance and Working Memory.
Sex and educational level were also considered as stratification variables. The majority of the overall sample was male (n = 83; 60%), ascertained to be comparable on sex ratios to CHR and FE case samples which tend to have more males than females. This contrasts with the MCCB adult standardization sample, which was 53% female. For the overall sample (n = 139), significant sex effects were present only for the Reasoning and Problem-Solving domain (t (142) = 3.24, p = .002), with males outperforming females. Thus, Reasoning and Problem-Solving Domain scores are stratified by age and sex (see Tables 2 & 3). Overall performance on cognitive domains (T-scores) and individual tests (raw scores) are provided along with sex-specific mean scores in Supplemental Tables 1 and 2, respectively.
As expected, given our school age sample, age and education were highly correlated (r = .96). Thus, in contrast to adult samples that benefit from stratification by education to control for education effects on cognitive test performance, we present our data by age group only. As displayed in Table 3, the average level of education for the overall sample was 9.8 years (SD = 2.2), with a range of 5-14 years.
A majority of the overall sample was right hand dominant (n = 122, 87.8%). Notably, the overall sample demonstrated an estimated WASI Full Scale IQ which falls on the cusp of the average and high average ranges (M = 109.3, SD = 15.4). This pattern is largely consistent with the IQ levels of many healthy control samples ascertained in clinical research studies, and is likely due to the selection biases that influence study participation (in contrast to census-referenced or population-based standardization studies). Moreover, the IQs of the CHR and FE samples were in this range, and controls were selected to be in the same IQ range on average.
DISCUSSION
These findings show that performance in MCCB cognitive domains follows a dynamic trajectory of improvement in adolescent NAPLS and CIDAR control subjects that is at least partially independent of overall cognitive ability (i.e. IQ). They also show means and standard deviations for individual test scores throughout adolescence. These data add to a small literature describing the performance of healthy individuals below the age of 20 on the MCCB. It is the first study to focus on adolescents specifically, with performance data provided in the relatively narrow 2-year age bins between the ages of 12 and 19. The use of standardized T-scores in the 8-year period under study increases the sensitivity to detect normal, age-related cognitive performance differences in adolescence. The current study also includes the most geographically diverse sample of MCCB performance in adolescents collected to date, with data obtained from eight sites across North America.
The results show that after controlling for IQ in all analyses, performance in each cognitive domain, except verbal and visual learning, significantly increased with age, as expected (see Fig. 1). Consistent with Nitzburg et al (2014), who also showed general cognitive improvement with age, most domains (4 out of 6) demonstrated larger improvements at the younger end of the age range (i.e. between the 12-13 and the 14-15 year-old groups). Continued improvements were also evident for most domains (also 4 out of 6), into the oldest age group (18-19 years old). Both of these trends underscore the point that normal adolescence is a dynamic period of cognitive development that may be delineated most accurately by multiple assessment points at different ages, as some tests may not demonstrate maturational plateaus until late adolescence or early adulthood. Consistent with Nitzburg et al, sex effects were quite modest. In both studies, males (collapsed across age groups) showed significantly better performance on the Reasoning and Problem-Solving task (NAB Mazes), though Mohn and colleagues did not observe significant gender differences in their 12-19 year old subjects (Mohn, et al., 2012; Mohn et al., 2014a). Our significant sex effects for Reasoning and Problem-Solving were observed within each age band except for the 14-15 year old group. It should be noted that our sample and that of Nitzburg et al differed demographically from that of Mohn et al, particularly in showing higher levels of racial and ethnic heterogeneity, though it is unclear whether those factors interacted with gender to produce different cognitive performance profiles. It is also noteworthy that the Reasoning and Problem Solving task in the MCCB is a visuospatial one, which might favor males (Quaiser-Pohl and Lehmann, 2002).
The T-scores in the current study were derived from the current sample for the purpose of showing normal developmental trajectories in adolescents using the same method as the MCCB (Kern, et al., 2008). Mean performance T-scores could thus not serve as normative values for other samples in educational, clinical or other applied contexts, as they were not standardized within each age group, but were instead standardized across gender and all age groups. For normative purposes, mean raw scores for all MATRICS‘ tests are presented for each age of the 4 adolescent age ranges in Table 3. Raw scores are also presented in Supplemental Table 2, with sex specific norms for each age band to allow clinicians or researchers to identify the most appropriate comparison reference data. Dividing the sample by sex lowered the subject numbers in the 3 older groups to between 12 and 24 subjects, with the lower cell sizes comprising the female subjects. The overall number of subjects in the youngest group (12-13 year-olds) was smaller than the other groups (n=24, compared to 37, 39 and 39), however, and included only 6 females. Thus, while this Table may still be useful for estimates of sex-related differences in each age band, the relatively low number of subjects in each cell, particularly in the 12-13 year old group, renders interpretations of performance values for each sex much more tentative than healthy comparison values based on age alone.
As Table 3 demonstrated, most demographic measures assessed in this study did not differ by age. The WASI IQ score was an important exception, and was thus used as a covariate in subsequent analyses. Notably, adolescent, age-related differences in MCCB performance remained robust, which underscores the point that while general cognitive ability (e.g. IQ) and other cognitive domains are typically related to each other in healthy samples, they are also separable (e.g. Heyanka et al., 2013; Mohn, et al., 2014b; Wechsler, 1997). The generally progressive improvement in MCCB performance during development in healthy adolescents, which was at least partially independent of general cognitive abilities, supports the view established in adult MCCB studies that test performance is influenced differentially by multiple, at least partially separable factors (Nuechterlein, et al., 2004; Nuechterlein, et al., 2008).
Several recent studies that employed the MCCB in schizophrenia or related conditions in adolescents showed evidence that cognitive dysfunctions are also subject to at least partially separable factors. Among these, Holmen et al (2010) showed evidence for both a generalized cognitive deficit (all MCCB test scores were significantly lower in a schizophrenia-spectrum group aged 12 to 18 years old than in a control group, except for the MSCEIT, which did not differ between groups) and for relatively more severe or milder deficits in individual cognitive domains. The same group also assessed a sample of schizophrenia-spectrum patients aged 12 to 18 at baseline and then 1 and 2 years later, and showed significant impairment on all MCCB cognitive domains at each time point, compared to healthy controls (Juuhl-Langseth, et al., 2014). Kelleher et al (2013), studying an extended schizophrenia phenotype in a sample of community-dwelling schoolchildren aged 11 to 13 years old who reported psychotic symptoms, showed MCCB performance deficits (versus a group of students who did not report psychotic symptoms) that were limited to processing speed (TMT-A and SC-BACS) and working memory (WMS-III SS). Of note, Juuhl-Langseth et al reported a mean age of 15.6 years for their schizophrenia spectrum subjects, and Holmen et al. reported a mean age of 15.8. The mean age of the total current sample was 16.3, which is similar. Inspection of the patient MCCB scores in both patient groups shows poorer performance on all MCCB measures than the healthy subjects in the current study (Table 5).
Thus, at this point, the available literature suggests that the MCCB may be sensitive to the development of both generalized and more specific cognitive deficits in schizophrenia-related conditions in adolescence, as it is in adults. Both because schizophrenia or related conditions may onset in adolescence, and also because cognitive deficits often begin in childhood or adolescence and precede the development of psychosis, it is essential to extend the normative foundation of MCCB performance to cover this critical developmental period marked by escalating vulnerability to first-episode psychiatric disorders. Such efforts will facilitate the development or refinement of treatment efforts, and early intervention and preventive efforts as well. In this context, the current findings add to the literature establishing the utility of the MCCB as a standard battery for clinical trials of cognitive-enhancing treatments for adolescent-onset schizophrenia, and also for schizophrenia-related conditions that manifest themselves in this period either before the onset of psychosis, or in its absence. Similarly, these healthy comparison data in adolescence contribute to the validation of the MCCB as a clinical battery to facilitate both assessment and intervention efforts. Finally, our findings also underscore the dynamic nature of adolescence, and the need for larger, population-based studies to establish normative performance values at multiple ages and in the context of multiple demographic influences on cognitive function.
Supplementary Material
ACKNOWLEDGMENTS
We acknowledge and thank the study participants and their family members. We also thank the clinical, research assistant, and data management staff from the Boston CIDAR and NAPLS-2 studies, including Caitlin Bryant, Ann Cousins, Grace Francis, Molly Franz, Michelle Friedman-Yakoobian, Lauren Gibson, Andréa Gnong-Granato, Maria Hiraldo, Sarah Hornbach, Matcheri Keshavan, Kristy Klein, Grace Min, Elena Molokotos, Keira O‘Donovan, Corin Pilo, Janine Rodenhiser-Hill, Julia Schutt, Rachael Serur, Shannon Sorenson, Reka Szent-Imry, Alison Thomas, Chelsea Wakeham, and Joanne Wojcik. We are further grateful for the hard work of many research volunteers, including Devin Donohoe, Zach Feder, Max Feit, Elizabeth Haxton, Sylvia Khromina, Alexandra Oldershaw, Katharine O‘Neal, Elizabeth Piazza, Julia Reading, Olivia Schanz, and Naomi Stapleton. Finally, we thank the data management staff at the University of Calgary, including Jean Addington and Lu Liu.
ROLE OF FUNDING SOURCE
This project was a cooperative agreement between the investigator sites and the National Institutes of Health.
FUNDING
National Institute of Mental Health grants (U01MH0818902 to T.D.C., U01MH081984 to J.M.A., P50MH066286 to C.E.B., U01MH082022 to K.S.C., U01MH081857 to B.A.C., U01MH082004 to D.O.P., U01MH081928 to L.J.S., U01MH081988 to E.F.W., U01MH066160 to S.W.W., P50MH080272 to R.W. M.) and the Commonwealth Research Center of the Massachusetts Department of Mental Health (SCDMH82101008006 to L.J.S.). This work was also conducted with support from Harvard Catalyst | The Harvard Clinical and Translational Science Center Harvard Catalyst Clinical Research Center at Beth Israel Deaconess Medical Center, A Harvard University Clinical and Translational Science Center Research Unit (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health Award 1UL1 TR001102-01 financial contributions from Harvard University and its affiliated academic health care centers). The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic health care centers, or the National Institutes of Health.
Role of Funding Source
This project was a cooperative agreement between the investigator sites and the National Institutes of Health
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
CONTRIBUTORS
Dr. Stone takes responsibility for all statistical analyses and the general integrity of the data. Dr. Seidman reviewed all versions of the manuscript and contributed to statistical analyses and conceptualization. All authors were responsible for reviewing and approving the final manuscript.
CONFLICT OF INTEREST
Dr. Green has been a consultant to AbbVie, Biogen, DSP, EnVivo/Forum and Roche, and he is on the scientific advisory board of Mnemosyne. Dr. Nuechterlein has received unrelated research support from Janssen Scientific Affairs, Genentech, and Brain Plasticity, Inc., and has consulted to Genentech, Otsuka, Janssen, and Brain Plasticity, Inc. All other authors declare that they have no actual or potential conflict of interest including any financial, personal or other relationships with other people or organizations within three (3) years of beginning the work submitted that could inappropriately influence, or be perceived to influence, their work.
Contributors
Dr. Stone takes responsibility for all statistical analyses and the general integrity of the data. Dr. Seidman reviewed all versions of the manuscript and contributed to statistical analyses and conceptualization. All authors were responsible for reviewing and approving the final manuscript.
Conflict of Interest
Dr. Green has been a consultant to AbbVie, Biogen, DSP, EnVivo/Forum and Roche, and he is on the scientific advisory board of Mnemosyne. Dr. Nuechterlein has received unrelated research support from Janssen Scientific Affairs, Genentech, and Brain Plasticity, Inc., and has consulted to Genentech, Otsuka, Janssen, and Brain Plasticity, Inc. All other authors declare that they have no actual or potential conflict of interest including any financial, personal or other relationships with other people or organizations within three (3) years of beginning the work submitted that could inappropriately influence, or be perceived to influence, their work.
REFERENCES
- Amminger GP, Henry LP, Harrigan SM, Harris MG, Alvarez-Jimenez M, Herrman H, Jackson JJ, McGorry PD. Outcome in early-onset schizophrenia revisited: findings from the Early Psychosis Prevention and Intervention Centre long-term follow-up study. Schizophr. Res. 2011;131:112–119. doi: 10.1016/j.schres.2011.06.009. [DOI] [PubMed] [Google Scholar]
- Ballageer T, Malla A, Manchanda R, Takhar J, Haricharan R. Is adolescent-onset first-episode psychosis different from adult onset? J. Am. Acad. Child Adolescent Psychiatry. 2005;44:782–789. doi: 10.1097/01.chi.0000164591.55942.ea. [DOI] [PubMed] [Google Scholar]
- Buchanan RW, Davis M, Goff D, Green MF, Keefe RS, Leon AC, Nuechterlein KH, Laughren T, Levin R, Stover E, Fenton W, Marder SR. A summary of the FDA-NIMH-MATRICS workshop on clinical trial design for neurocognitive drugs for schizophrenia. Schizophr. Bull. 2005;31:5–19. doi: 10.1093/schbul/sbi020. [DOI] [PubMed] [Google Scholar]
- Buchanan RW, Keefe RS, Umbricht D, Green MF, Laughren T, Marder SR. The FDA-NIMH-MATRICS Guidelines for Clinical Trial Design of Cognitive-Enhancing Drugs: what do we know 5 years later? Schizophr. Bull. 2011;37:1209–1217. doi: 10.1093/schbul/sbq038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burdick KE, Goldberg TE, Cornblatt B, Keefe RS, Gopin CB, Derosse P, Braga RJ, Malhotra AK. The MATRICS consensus cognitive battery in patients with bipolar I disorder. Neuropsychopharmacology. 2011;36:1587–1592. doi: 10.1038/npp.2011.36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burton CZ, Vella L, Harvey PD, Patterson TL, Heaton RK, Twamley EW. factor structure of the MATRICS Consensus Cognitive Battery in schizophrenia. Schizophr. Res. 2013;146:244–248. doi: 10.1016/j.schres.2013.02.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cannon M, Jones P, Huttunen MO, Tanskanen A, Huttunen T, Rabe-Hesketh S, Murray RM. School performance in Finnish children and later development of schizophrenia: A population-based longitudinal study. Arch. Gen. Psychiatry. 1999;56:457–463. doi: 10.1001/archpsyc.56.5.457. [DOI] [PubMed] [Google Scholar]
- Eggers C, Bunk D. The long-term course of childhood-onset schizophrenia: a 42-year followup. Schiz. Bull. 1997;23:105–117. doi: 10.1093/schbul/23.1.105. [DOI] [PubMed] [Google Scholar]
- Fisher M, Holland C, Merzenich MM, Vinogradov S. Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. Am. J. Psychiatry. 2009;166:805–811. doi: 10.1176/appi.ajp.2009.08050757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riech-Rossler A, Schultze-Lutter F, Keshavan MS, Wood SJ, Ruhrmann S, Seidman LJ, Valmaggia L, Cannon TD, Velthorst E, De Haan L, Cornblatt B, Bonoldi I, Birchwood M, McGlashan TH, Carpenter WT, McGorry PD, Klosterkotter J, McGuire P, Yung A. The psychosis high risk state: A comprehensive state-of-the-art review. JAMA Psychiatry. 2012;70:107–120. doi: 10.1001/jamapsychiatry.2013.269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fusar-Poli P, Deste G, Smieskova R, Barlati S, Yung AR, Howes O, Stieglitz RD, Vita A, McGuire P, Borgwardt S. Cognitive functioning in prodromal psychosis: a meta analysis. Arch. Gen. Psychiatry. 2012;69:562–571. doi: 10.1001/archgenpsychiatry.2011.1592. [DOI] [PubMed] [Google Scholar]
- Giuliano AJ, Li H, Mesholam-Gately R, Sorenson SM, Woodberry KA, Seidman LJ. Neurocognition in psychosis risk syndrome: a quantitative and qualitative review. Curr. Pharmaceutical Design. 2012;18:399–415. doi: 10.2174/138161212799316019. [DOI] [PubMed] [Google Scholar]
- Green MF, Nuechterlein KH, Kern RS, Baade LE, Fenton W, Gold JM, Keefe RS, Mesholam-Gately R, Seidman LJ, Stover E, Marder SR. Functional co-primary measures for clinical trials in schizophrenia: results from the MATRICS Psychometric and Standardization Study. Am. J. Psychiatry. 2008;165:221–228. doi: 10.1176/appi.ajp.2007.07010089. [DOI] [PubMed] [Google Scholar]
- Green MF, Schooler NR, Kern RS, Frese F, Granberry W, Harvey PD, Karson CN, Peters N, Stewart M, Seidman LJ, Sonnenberg J, Stone WS, Walling D, Stover E, Marder SR. Evaluation of co-primary measures for clinical trials of cognition enhancement in schizophrenia. Am. J Psychiatry. 2011;168:400–407. doi: 10.1176/appi.ajp.2010.10030414. [DOI] [PubMed] [Google Scholar]
- Heyanka DJ, Holster JL, Golden CJ. Intraindividual neuropsychological test variability in healthy individuals with high average intelligence and educational attainment. Int. J Neuroscience. 2013;123:526–531. doi: 10.3109/00207454.2013.771261. [DOI] [PubMed] [Google Scholar]
- Hollis C. Adolescent schizophrenia. Adv. Psychiatr. Treat. 2000;6:83–92. [Google Scholar]
- Holmen A, Juuhl-Langseth M, Thormodsen R, Melle I, Rund BR. Neuropsychological profile in early-onset schizophrenia-spectrum disorders measured with the MATRICS battery. Schizophr. Bull. 2010;36:852–859. doi: 10.1093/schbul/sbn174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Insel TR. The arrival of preemptive psychiatry. Early Intervention in Psychiatry. 2007;1:5–6. doi: 10.1111/j.1751-7893.2007.00017.x. [DOI] [PubMed] [Google Scholar]
- Insel TR. Rethinking schizophrenia. Nature. 2010;468:187–193. doi: 10.1038/nature09552. [DOI] [PubMed] [Google Scholar]
- Juuhl-Langseth M, Holmen A, Thormodsen R, Oie M, Rund BR. Relative stability of neurocognitive deficits in early onset schizophrenia spectrum patients. Schizophr. Res. 2014;156:241–247. doi: 10.1016/j.schres.2014.04.014. [DOI] [PubMed] [Google Scholar]
- Keefe RS, Fox KH, Harvey PD, Cucchiaro J, Siu C, Loebel A. Characteristics of the MATRICS Consensus Cognitive Battery in a 29-site antipsychotic schizophrenia clinical trial. Schizophr. Res. 2011;125:161–168. doi: 10.1016/j.schres.2010.09.015. [DOI] [PubMed] [Google Scholar]
- Kelleher I, Clarke MC, Rawdon C, Murphy J, Cannon M. Neurocognition in the extended psychosis phenotype: Performance of a community sample of adolescents with psychotic symptoms on the MATRICS neurocognitive battery. Schizophr. Bull. 2013;39:1018–1026. doi: 10.1093/schbul/sbs086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kern RS, Nuechterlein KH, Green MF, Baade LE, Fenton WS, Gold JM, Keefe RS, Mesholam-Gately R, Mintz J, Seidman LJ, Stover E, Marder SR. The MATRICS Consensus Cognitive Battery, part 2: co-norming and standardization. Am. J. Psychiatry. 2008;165:214–220. doi: 10.1176/appi.ajp.2007.07010043. [DOI] [PubMed] [Google Scholar]
- Lin A, Wood SJ, Nelson B, Brewer WJ, Spiliotacopoulos D, Bruxner A, Broussard C, Pantelis C, Yung AR. Neurocognitive predictors of functional outcome two to 13 years after identification as ultra high risk for psychosis. Schizophr. Res. 2011;132:1–7. doi: 10.1016/j.schres.2011.06.014. [DOI] [PubMed] [Google Scholar]
- Mayer JD, Salovey P, Caruso DR. Mayer-Salovey-Caruso Emotional Intelligence Test. MHS Publishers; Toronto, Canada: 2002. [Google Scholar]
- McCleery A, Green MF, Hellemann GS, Baade LE, Gold JM, Keefe RS, Kern RS, Mesholam-Gately RI, Seidman LJ, Subotnik KL, Ventura J, Nuechterlein KH. Latent structure of cognition in schizophrenia: a confirmatory factor analysis of the MATRICS Consensus Cognitive Battery (MCCB) Psychol. Med. 2015 doi: 10.1017/S0033291715000641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mohn C, Sundet K, Rund BR. The Norwegian standardization of the MATRICS (Measurement and Treatment Research to Improve Cognition in Schizophrenia) Cognitive Consensus Battery. J Clin. Exp. Neuropsychology. 2012;34:667–677. doi: 10.1080/13803395.2012.667792. [DOI] [PubMed] [Google Scholar]
- Mohn C, Sundet K, Rund BR. The Norwegian standardization of the MATRICS (Measurement and Treatment Research to Improve Cognition in Schizophrenia. J Clin. Exp. Neuropsychology. 2014a;34:667–677. doi: 10.1080/13803395.2012.667792. [DOI] [PubMed] [Google Scholar]
- Mohn C, Sundet K, Rund BR. The relationship between IQ and performance on the MATRICS consensus cognitive battery. Schizophr. Res.: Cogn. 2014b;1:96–100. doi: 10.1016/j.scog.2014.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nicolson R, Brookner FB, Lenane M, Gochman P, Ingraham LJ, Egan MF, Kendler KS, Pickar D, Weinberger DR, Rapoport JL. Parental schizophrenia spectrum disorders in childhood-onsetand adult-onset schizophrenia. Am. J. Psychiatry. 2003;160:490–495. doi: 10.1176/appi.ajp.160.3.490. [DOI] [PubMed] [Google Scholar]
- Nitzburg GC, DeRosse P, Burdick KE, Peters BD, Gopin CB, Malhotra AK. MATRICS cognitive consensus battery (MCCB) performance in children, adolescents and young adults. Schizophr. Res. 2014;152:223–228. doi: 10.1016/j.schres.2013.11.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, Green MF, Heaton RK. Identification of separable cognitive factors in schizophrenia. Schizophr. Res. 2004;72:29–39. doi: 10.1016/j.schres.2004.09.007. [DOI] [PubMed] [Google Scholar]
- Nuechterlein KH, Green MF. MATRICS Consensus Cognitive Battery. MATRICS Assessment, Inc.; Los Angeles: 2006. [Google Scholar]
- Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, Essock S, Fenton WS, Frese F, Gold JM, Goldberg T, Heaton RK, Keefe RS, Kraemer H, Mesholam-Gately R, Seidman LJ, Stover E, Weinberger DR, Young AS, Zalcman S, Marder SR. The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am. J. Psychiatry. 2008;165:203–213. doi: 10.1176/appi.ajp.2007.07010042. [DOI] [PubMed] [Google Scholar]
- Quaiser-Pohl C, Lehmann W. Girl's spatial abilities: Charting the contributions of experiences and attitudes in different academic groups. Br. J. Ed. Psychology. 2002;72:245–260. doi: 10.1348/000709902158874. [DOI] [PubMed] [Google Scholar]
- Rajji TK, Ismail Z, Mulsant BH. Age at onset and cognition in schizophrenia: meta-analysis. Br. J. Psychiatry. 2009;195:286–293. doi: 10.1192/bjp.bp.108.060723. [DOI] [PubMed] [Google Scholar]
- Rajji TK, Voineskos AN, Butters MA, Miranda D, Arenovich T, Menon M, Ismail Z, Kern RS, Mulsant BH. Cognitive performance of individuals with schizophrenia across seven decades: a study using the MATRICS Consensus Cognitive Battery. Am. J. Ger. Psychiatry. 2013;21:108–118. doi: 10.1016/j.jagp.2012.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rentz DM, Huh TJ, Fausr RR, Budson AE, Scinto LF, Sperling RA, Daffner KR. Use of IQ-adjusted norms to predict progressive cognitive decline in highly intelligent older individuals. Neuropsychology. 2004;18:38–49. doi: 10.1037/0894-4105.18.1.38. [DOI] [PubMed] [Google Scholar]
- Rentz DM, Huh TJ, Sardinha LM, Moran EK, Becker JA, Daffner KR, Sperling RA, Johnson KA. Intelligence quotient-adjusted memory impairment is associated with abnormal single photon emission computed tomography perfusion. J. Int. Neuropsychological Soc. 2007;13:821–831. doi: 10.1017/S1355617707071056. [DOI] [PubMed] [Google Scholar]
- Seidman LJ, Giuliano AJ, Meyer EC, Addington J, Cadenhead KS, Cannon TD, McGlashan TH, Perkins DO, Tsuang MT, Walker EF, Woods SW, Bearden CE, Christensen BK, Hawkins K, Heaton R, Keefe RS, Heinssen R, Cornblatt B. Neuropsychology of the prodrome to psychosis in the NAPLS Consortium. Arch. Gen. Psychiatry. 2010;67:578–588. doi: 10.1001/archgenpsychiatry.2010.66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sperry SH, O'Connor LK, Ongur D, Cohen BM, Keshavan MS, Lewandowski KE. Measuring cognition in bipolar disorder with psychosis using the MATRICS Consensus Cognitive Battery. J. Int. Neuropsychological Soc. 2015;21:468–472. doi: 10.1017/S1355617715000442. [DOI] [PubMed] [Google Scholar]
- Steinberg BA, Bieliauskas LA, Smith GE, Ivnik RJ. Mayo's older Americans normative studies: age- and IQ adjusted norms for the Wechsler Memory Scale-Revised. The Clin. Neuropsychologist. 2005;19:378–463. doi: 10.1080/13854040590945201. [DOI] [PubMed] [Google Scholar]
- Steinberg BA, Bieliauskas LA, Smith GE, Ivnik RJ, Malec JF. Mayo's older Americans normative studies: age- and IQ-adjusted norms for the auditory verbal learning test and the visual spatial learning test. The Clin. Neuropsychologist. 2005;19:464–523. doi: 10.1080/13854040590945193. [DOI] [PubMed] [Google Scholar]
- Thomsen PH. Schizophrenia with childhood and adolescent onset - a nationwide register-based study. Acta Pscyhiatr. Scand. 1996;94:187–193. doi: 10.1111/j.1600-0447.1996.tb09847.x. [DOI] [PubMed] [Google Scholar]
- Wechsler D. WAIS-III WMS-III Technical Manual. The Psychological Corporation: Harcourt Brace and Company; San Antonio: 1997. [Google Scholar]
- Wechsler D. WASI Manual. The Psychological Corporation, Harcourt Brace & Company; San Antonio: 1999. [Google Scholar]
- Wilkinson GS, Robertson GJ. Wide Range Achievement Test Manual. 4 Vol. 4. Psychological Assessment Resources, Inc.; Lutz, Florida: 2006. [Google Scholar]
- Woodberry KA, Giuliano AJ, Seidman LJ. Premorbid IQ in schizophrenia: a meta-analytic review. Am. J. Psychiatry. 2008;165:579–587. doi: 10.1176/appi.ajp.2008.07081242. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.