Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jun 1.
Published in final edited form as: J Psychiatr Res. 2019 Mar 21;113:117–124. doi: 10.1016/j.jpsychires.2019.03.012

Cognitive Functioning as a Predictor of Response to Comprehensive Cognitive Remediation

Nicole R DeTore a, Kim T Mueser a, Jessica A Byrd a, Susan R McGurk a
PMCID: PMC6486446  NIHMSID: NIHMS1526132  PMID: 30933830

Abstract

Cognitive remediation is aimed at reducing cognitive impairments in severe mental illnesses such as schizophrenia, but little is known about whether severity of cognitive impairment predicts benefit from this intervention. To address this question, this study aggregated data from five randomized controlled trials (N=300) of a standardized comprehensive, multimodal outpatient cognitive remediation program, the Thinking Skills for Work program, and evaluated whether baseline level of cognitive impairment differentially predicted improvement in cognitive functioning following cognitive remediation vs. usual services. Using standardized scores of neuropsychological functioning to designate “low average,” “moderate,” and “severe” levels of cognitive impairment, participants with greater cognitive impairment were found to benefit differentially more from cognitive remediation than usual services compared to less cognitive impaired participants. The findings were unaffected by statistically controlling for participant demographic and clinical characteristics. The findings suggest that individuals with the greatest cognitive impairment, for whom cognitive remediation was developed, are also most likely to benefit from this intervention.

Keywords: Severe mental illness, Thinking skills for work, cognition, cognitive remediation, neuropsychological functioning

1. Introduction

Impaired cognitive functioning is a common feature of severe mental illnesses such as schizophrenia, bipolar disorder, and treatment refractory major depression (Depp, et al., 2012; Heinrichs, 2001; Porter et al., 2015). Lower cognitive functioning in areas such as attention, working memory, executive functions, and speed of processing is strongly associated with poorer psychosocial functioning, including quality of social relationships, role functioning self-care skills, and independent living (Green, 2006; Green et al., 2015; Wykes, 1994). Furthermore, reduced cognitive abilities are a potent predictor of attenuated response to psychosocial treatments for severe mental illness, such as supported employment and social skills training (Kurtz, 2011).

In response to the abundant evidence about the importance of cognitive functioning to psychosocial adjustment in people with severe mental illness, cognitive remediation methods have been developed and evaluated. The most common of these methods include drill-and-practice of cognitive exercises, strategy coaching to improve performance on cognitive exercises, and teaching cognitive compensatory (or self-management) strategies to reduce the effects of impaired cognitive functioning in everyday life (McGurk et al., 2013). Over 50 randomized controlled trials of cognitive remediation for persons with severe mental illness have been conducted over the past several decades. Meta-analyses show that cognitive remediation programs are effective at improving both cognitive and psychosocial functioning, with the impact on psychosocial functioning stronger in programs that provide both cognitive remediation and psychosocial treatment (compared to psychosocial treatment alone) than stand-alone cognitive remediation programs (McGurk et al., 2007; Revell et al., 2015; Wykes et al., 2011).

Despite the strong evidence supporting the effectiveness of cognitive remediation, limited research has examined which individuals are most likely to improve with treatment. Of particular relevance is whether an individual’s baseline level of cognitive functioning is related to the extent to which they benefit from cognitive remediation. Greater cognitive impairment could make it more difficult to learn how to practice computer cognitive exercises, resulting in less practice, and contribute to greater frustration and reduced motivation to engage in cognitive remediation (Medalia & Richardson, 2005).

Several studies have explored whether initial levels of cognitive functioning predict cognitive improvement following cognitive remediation, with mixed results. Some studies have found that people with greater cognitive impairment benefit less from cognitive remediation (Lindenmayer et al., 2017; Wiedl et al., 2001), while others have reported the opposite, with lower cognitive functioning predicting greater benefit from cognitive remediation (Twamley et al., 2011). However, the majority of studies have found few cognitive predictors of benefit from treatment (Medalia & Richardson, 2005; Rodewald et al., 2014; Vita et al., 2013).

The lack of consistent findings across studies that evaluated cognitive functioning as a predictor of response to cognitive remediation is not surprising considering the methodological limitations of the research conducted thus far. Most of the studies have had modest sample sizes of under 60 participants receiving a particular type of cognitive remediation (Medalia & Richardson, 2005; Rodewald et al., 2014; Twamley et al., 2011; Vita et al., 2013; Wiedl et al., 2001), and two studies that combined participants receiving different types of cognitive remediation (Lindenmayer et al., 2017; Vita et al., 2013). Three studies evaluated cognitive remediation interventions with as few as one (Wiedl et al., 2001) to ten sessions (Medalia & Richardson, 2005; Rodewald et al., 2014). Finally, none of the studies evaluated whether cognitive functioning predicted response to cognitive remediation compared to a control group. Determining the extent to which cognitive functioning differentially predicts benefit from cognitive remediation vs. usual treatment is crucial to making informed decisions as to which individuals can benefit and should be referred to this treatment.

The present analysis was conducted to overcome the methodological limitations of previous research on cognitive functioning as a predictor of response to cognitive remediation. To address this question, data from five randomized controlled trials evaluating a standardized comprehensive, multi-modal cognitive remediation intervention for persons with severe mental illness were combined and a meta-regression analysis was performed. The pooled sample size included more participants in the cognitive remediation program than in the largest of the prior studies (Lindenmayer et al., 2017), with approximately an equal number of control participants as well, and therefore had sufficient statistical power to address the research question.

2. Materials and methods

Data for the meta-regression were drawn from five randomized controlled trials of the Thinking Skills for Work program, a comprehensive, multi-modal cognitive remediation intervention that is integrated with vocational rehabilitation services (McGurk & Mueser, 2015; McGurk et al., 2009; McGurk et al., 2005; McGurk et al., 2016; McGurk et al., 2015). All five of these studies were implemented in outpatient community mental health treatment settings with baseline and post treatment assessments of cognition and symptoms conducted by raters who were blind to treatment group. Improvement in neuropsychological test performance from baseline to post-treatment was the primary outcome of the analyses. All studies were approved by their respective IRB committees and all study participants provided written informed consent.

2.1. Participants

A total of 300 participants across the five studies were included in the analyses. All participants met the following inclusion criteria: a) 18 years of age or older; b) diagnosis of severe mental illness, which was defined as an Axis I diagnosis (with the exception of substance use disorder as the primary disorder) associated with severe functional impairments in work, social functioning, or self-care; c) not currently employed; d) a stated desire to work; and e) a history of an unsuccessful ending of a job that lasted less than three months (being fired from a job or quitting a job without another one in place), or recent failure to benefit from vocational rehabilitation services (not obtaining work for three months or more, or being fired from a job or quitting without another in place despite receiving services). Participant characteristics are presented for each study and the overall sample in Table 1.

Table 1.

Study Details, and Demographic and Diagnostic Characteristics of Participants by Study (N = 300)

Study Study 1 McGurk et al. (2005) Study 2 McGurk et al. (2009) Study 3 McGurk et al. (2015a) Study 4 McGurk et al. (2015b) Study 5 McGurk et al. (2016)
Study Details
Location Brooklyn New York Brooklyn Manchester, NH/Chicago Brooklyn
Study design RCT1 RCT RCT RCT RCT
 Experimental group TSW2+SE3 TSW+VR-I4 TSW+VR-DP5 TSW+SE TSW+IMR6 +VR-DP
 Control group SE Only VR-I Only VR-DP
Only		 SE Only VR-DP
Only
Participant Characteristics Total
N 44 34 61 107 54 300
Age M (sd) 35.6 (12.4) 44.1 (9.1) 37.3 (10.7) 44.2 (11.1) 37.7 (9.4) 39.8 (10.5)
Hispanic N (%) 7 (15.9) 5 (14.7) 12 (19.7) 12 (11.2) 10 (18.5) 46 (15.3)
Race
 White N (%) 14 (31.8) 12 (35.3) 16 (26.2) 92 (86.0) 13 (24.1) 147 (49.0)
 Black N (%) 29 (65.9) 21 (61.8) 43 (70.5) 11 (10.3) 33 (61.1) 137 (45.7)
 Other N (%) 1 (2.3) 1 (2.9) 2 (3.3) 4 (3.7) 8 (14.8) 16 (5.3)
Never married N (%) 38 (86.4) 26 (76.5) 51 (83.6) 69 (64.5) 44 (81.5) 228 (76.0)
Completed high school N (%) 15 (34.1) 21 (23.5) 37 (60.7) 77 (72.0) 36 (66.7) 186 (62.0)
Schizophrenia spectrum N (%) 34 (77.3) 22 (61.8) 44 (72.1) 49 (45.8) 45 (83.3) 194 (64.7)
Exposed to treatment N (%) 21 (47.7) 18 (52.9) 29 (47.5) 40 (37.4) 26 (48.1) 134 (44.7)
BPRS7 M (sd) 2.1 (0.4)
PANSS8 M (sd) 2.4 (0.6) 2.2 (0.5) 1.9 (0.4) 2.3 (0.5) 2.1 (0.5)
WRAT9 M (sd) 80.6 (18.7) 89.1 (13.6) 86.4 (15.5) 82.7 (10.2) 90.8 (16.3) 85.9 (14.9)
Cognitive composite M (sd) 27.6 (8.1) 29.5 (7.4) 30.7 (6.2) 24.0 (8.0) 30.5 (9.8) 28.5 (7.9)
Cognitive domains M (sd)
 Psychomotor Speed 27.7 (16.2) 29.9 (14.3) 34.0 (12.0) 27.1 (11.3) 35.7 (14.4) 30.8 (13.6)
 Speed of Processing 26.0 (14.2) 34.8 (11.5) 35.2 (13.9) 27.1 (13.2) 34.2 (13.3) 31.5 (13.2)
 Attention/ Vigilance 39.1 (8.1) 38.4 (8.1) 38.2 (6.0) 31.8 (12.2) 33.2 (12.6) 36.1 (9.4)
 Verbal Working Memory 38.4 (9.5) 41.2 (10.2) 41.4 (7.1) 29.5 (12.4) 37.5 (14.0) 37.6 (10.6)
 Verbal Learning 24.7 (14.6) 36.2 (12.7) 35.1 (13.7) 32.8 (7.5) 36.8 (9.0) 33.1 (11.5)
 Reasoning & Problem Solving 30.9 (4.1) 30.5 (7.6) 31.0 (7.7) 33.8 (8.1) 38.8 (9.1) 33.0 (7.3)
 Cognitive Flexibility −2.5 (14.8) −6.1 (15.3) −2.8 (13.6) −16.0 (23.6) −5.3 (22.5) −6.5 (17.9)
Open in a new tab
1

RCT = randomized controlled trial

2

TSW = Thinking Skills for Work program

3

SE = Supported Employment program

4

VR-I = Vocational Rehabilitation Internship program

5

VR-DP = Vocational Rehabilitation Diversified Placement program

6

IMR = Illness Management and Recovery program

7

BPRS - Brief Psychiatric Rating Scale

8

PANSS = Positive and Negative Syndrome Scale

9

WRAT = Wide Range Achievement Test—III

The rates of participant attrition from research at the post-treatment assessment were 73%, 94%, 83%, 74%, and 94% for Study 1 through Study 5, respectively. Reasons for attrition included inability to contact participant, moved away from the treatment setting, psychiatric hospitalization, no longer interested in the study, and involvement in the criminal justice system. “Exposure” to the Thinking Skills for Work program was defined as completion of a minimum of 6 (out of 24) computer cognitive training sessions. Rates of exposure to the program were 91%, 100%, 70%, 77%, and 93% for Study 1 through Study 5, respectively.

2.2. Study Interventions

All five studies compared vocational rehabilitation alone to vocational rehabilitation enhanced with the Thinking Skills for Work program. Participants continued to receive the usual care available to them at their community treatment site, including pharmacological treatment, case management, vocational rehabilitation, and access to other psychosocial interventions. No restrictions were placed on the treatment services that study participants could receive during the study.

2.2.1. Thinking Skills for Work program

Only participants who were randomized to the Thinking Skills for Work program received cognitive remediation. This program is implemented by a trained clinician (a “cognitive specialist”) who provides the intervention and integrates it with the vocational rehabilitation services the participant is receiving. The program generally requires 6–9 months to implement, and is divided into five core components, including: 1) assessment of cognitive contributions to work attainment/performance difficulties; 2) 24 sessions of cognitive training using selected computerized cognitive exercises drawn from COGPACK software (Marker, 2014); 3) teaching cognitive self-management strategies to help participants “work around” persisting difficulties; 4) job search consultation with the participant and employment specialist to address problems obtaining jobs; and 5) job support consultation to address job performance difficulties or recent job losses.

2.2.2. Vocational Rehabilitation programs

All participants were enrolled in the vocational rehabilitation program available at their center and continued to receive the same vocational services throughout the study, regardless of whether they were randomized to also receive the Thinking Skills for Work program. In Studies 1 and 4 (McGurk et al., 2005; McGurk et al., 2015), the vocational rehabilitation program was supported employment (Becker & Drake, 2003); in Study 2 (McGurk et al., 2009) it was an internship-based vocational program; and in Studies 3 and 5 (McGurk & Mueser, 2015; McGurk et al., 2016) it was a diversified placement vocational program (Bond et al., 2007). In Study 3, participants in the Thinking Skills for Work program also received the illness management and recovery program (Gingerich & Mueser, 2011). The details of each of the five study designs are summarized in Table 1.

2.3. Assessment

2.3.1. Premorbid intellectual functioning

Premorbid intellectual functioning was measured at baseline only with the Wide Range Achievement Test-III – Reading subtest (Wilkinson, 1993).

2.3.2. Cognitive functioning

A comprehensive battery of established neurocognitive tests was administered in all the studies to evaluate cognitive functioning across the broad range of domains, including attention, working memory, verbal learning and memory, speed of processing, and executive functioning. Studies 3 and 4 used the MATRICS Consensus Cognitive Battery (MCCB) (Nuechterlein et al., 2008). The other three studies were initiated before the MCCB was established, and employed a somewhat less extensive, but nevertheless comprehensive battery of measures, several of which overlapped with the MCCB. The cognitive tests for the two batteries were organized into the domains of cognitive functioning they tap, based on the organization of cognitive domains reported in the McGurk et al. (2007) and Wykes et al. (2011) meta-analyses of cognitive remediation, which are summarized in Table 2.

Table 2.

Cognitive Tests by Cognitive Domain

Cognitive domain Study 1 McGurk et al. (2005) Study 2 McGurk et al. (2009) Study 3 McGurk et al. (2015a) Study 4 McGurk et al. (2015b) Study 5 McGurk et al. (2016)
Psychomotor Speed
 Trail Making-A X X X X X
Speed of Processing
 Wechsler Adult Intelligence Scale-Revised-Digit Symbol X X
 Brief Assessment of Cognition in Schizophrenia-Symbol Coding X X X
Attention/ Vigilance
 Wechsler Adult Intelligence Scale-Revised-Digit Span Forwards X X
 Continuous Performance Test-Identical Pairs X X X
Verbal Working Memory
 Wechsler Adult Intelligence Scale-Revised-Digit Span Backwards X X
 Letter Number Span X X X
Verbal Learning
 California Verbal Learning Test X X
 Hopkin’s Verbal Learning Test-Revised X X X
Reasoning and Problem Solving
 Wisconsin Card Sort X X
 Neuropsychological Assessment Battery- Mazes X X X
Cognitive Flexibility
 Trail Making-B X X X X X
Open in a new tab
Attention/ vigilance.

Digit Span Forward (Wechsler Adult Intelligence Scale-Revised) (Wechsler, 1981) (total correct); Continuous Performance Test- Identical Pairs (CPT-IP) (Cornblatt, Risch, Faris, Friedman, & Erlenmeyer-Kimling, 1988) (D prime).

Psychomotor speed.

The Trail Making Test, Part A (Radford et al., 1978) was used in all 5 studies (seconds to complete).

Speed of processing.

Digit Symbol Substitution Test, Wechsler Adult Intelligence Scale- Revised) (Wechsler, 1981) (total number correct); Symbol Coding Test, Brief Assessment of Cognition in Schizophrenia (Keefe et al., 2004) (total number correct).

Learning:

California Verbal Learning Test (Delis et al., 1987) (total words recalled in learning trials 1–5); Hopkins Verbal Learning Test (Brandt, 1991) (total words recalled in learning trials 1–3).

Executive functioning:

The Wisconsin Card Sorting Test (Berg, 1948) (total categories obtained); Neuropsychological Assessment Battery-Mazes (Stern & White, 2003) (total correct).

Cognitive flexibility.

The Trail Making Test, Part B (Radford et al., 1978) was used in all five studies (completion time in seconds).

2.3.3. Psychiatric diagnosis and symptoms

Psychiatric diagnosis was ascertained by interviewers at baseline with the Structured Clinical Interview for DSM-IV (First et al., 1996).

Psychiatric symptoms were assessed with the Brief Psychiatric Rating Scale-Expanded (Lukoff et al., 1986) in Study 4, and the Positive and Negative Syndrome Scale (Kay et al., 1987) in the other four studies. Both of these measures include behaviorally anchored ratings on seven-point scales, and share 16 items: anxiety, delusions/ unusual thought content, suspiciousness/ persecutory, hallucinations, somatic concern, depression, guilt feelings, grandiosity, disorientation, excitement, hostility, blunted affect, emotional withdrawal, tension, mannerisms and posturing, motor retardation, uncooperativeness, and conceptual disorganization. In order to develop a common index of overall psychopathology across all studies, the average rating across these items was computed.

2.4. Statistical analyses

Cognitive composite scores were formed for the primary data analyses. After reversing the scores on Trails A and B so that higher scores reflect better performance, all the individual measure raw scores were computed to T-scores for each of the tests in included in the cognitive domains listed in Table 2. Then, the composite cognitive measure was created by computing the mean T-score across the seven cognitive domains.

To evaluate whether participants with lower levels of cognitive functioning at baseline differed in their benefit from the Thinking Skills for Work program compared to those with higher baseline cognitive functioning, we used three different cut-off scores for the cognitive composite index to dichotomize the sample into low vs. high groups. The three cognition cut-off scores were chosen to reflect three levels of impairment: low average impairment or worse (T-score ≤ 40), moderate impairment or worse (T-score ≤ 23), and severe impairment (T-score ≤ 19) (Wechsler, 1981). Chi-square and t-test analyses were used to examine demographic and clinical differences within the low and high cognition groups for each of the three cut-offs. Change scores in cognitive functioning over the treatment period were then computed by subtracting the baseline cognitive composite score from the post-treatment composite cognitive score.

Three multiple regressions were conducted to examine whether cognitive functioning at baseline (low vs. high using each of the three cut-off scores), treatment group, or the treatment group by cognitive functioning interaction predicted improvement in cognitive functioning at post-treatment. Site was also included as a covariate in these analyses, with the largest site set as the reference site (McGurk et al., 2015). For these analyses, a main effect for baseline cognitive functioning group indicates that participants in either the high (or low) cognition group at baseline improved significantly more in cognitive functioning over the treatment period than those in the other group, and a main effect for treatment group indicates that participants in the Thinking Skills for Work program (or control condition) improved more over the treatment period than those in the other group. A significant baseline cognition group by treatment group interaction indicates that participants in one of the cognition groups (high or low) improved significantly more from cognitive remediation (the Thinking Skills for Work program) than participants in the other cognition group.

Sensitivity analyses were conducted to determine whether the findings were influenced by inclusion of psychiatric diagnosis, age, or symptom severity in the analyses. This was accomplished by including psychiatric diagnosis (schizophrenia or schizoaffective disorder vs. other) as a main effect and the diagnosis by treatment group interaction in the multiple regression analyses, and (separately) by including average psychiatric symptom severity as a main effect and the symptom by treatment group interaction in the multiple regressions.

3. Results

The low average cognition group (T-score ≤ 40) had a cognitive score M = 26.9 (N = 271) and the high group had a cognitive score M = 44.1 (N = 23). The low moderate impairment group (T-score ≤ 23) had a cognitive score M = 21.7 (N = 156), and the high group had a cognitive score M = 36.6 (N = 132). The low severe cognitive impairment group (T-score ≤ 19) had a cognitive score M = 14.5 (N = 42) while the high group had a cognitive score M = 31.2 (N = 248).

Across each of the three baseline cognitive functioning cut-off scores used to divide the sample into low vs. high cognition groups, participants in the low cognition group were significantly more likely to have schizophrenia or schizoaffective disorder (vs. other diagnoses), have lower WRAT scores, and be of non-white race than those in the high cognition group. In addition, participants with above low average cognitive scores (T > 40) were significantly more likely to have completed high school than those with below low average scores; and participants with above moderate cognitive impairment (T > 23) were significantly more likely to be male than those with below moderate impairment.

The three multiple regression analyses predicting cognitive change included the following variables: site, treatment group, baseline cognition (T-score ≤ 40 for low average; T-score ≤ 23 for moderate impairment; T-score ≤ 19 for severe impairment), and the treatment group by baseline cognition interaction. All three overall regression models were statistically significant. In the regression examining the low average cognition cut-point (T ≤ 40), neither the cognition group nor the cognition group by treatment group interaction effects were statistically significant, while the effects for site and treatment group were significant. In the two regressions with the lower cognition cut-points (T ≤ 23 and T ≤ 19), the effects for baseline cognition group, treatment group, the cognition by treatment interaction, and site were statistically significant (see Table 3). In all three models, participants in the lower baseline cognition group improved more following cognitive remediation than participants in the higher cognition group, whereas participants in both cognition groups who received usual treatment tended not to change over time in cognitive functioning (see Figure 1a, 1b, and 1c).

Table 3.

Multiple Regressions Predicting Cognitive Change

β t p df F R2 p
Model 1- Low average cognition1
 Treatment group −.257 −4.217 <.001
 Cognition group −.296 −1.652 .100
 Cognition x treatment interaction .185 1.035 .302
 Site 1 .055 .819 .414
 Site 2 −.166 −2.551 .011
 Site 3 −.220 −3.368 .001
 Site 4 .070 1.052 .294
 Overall model 251 7.114 .169 <.001
Model 2- Minimal cognitive impairment2
 Treatment group −.383 −4.893 <.001
 Cognition group −.617 −3.439 .001
 Cognition x treatment interaction .499 2.696 .008
 Site 1 .084 1.279 .202
 Site 2 −.147 −2.230 .021
 Site 3 −.196 −3.064 .002
 Site 4 .072 1.086 .278
 Overall model 243 8.490 .201 <.001
Model 3- Severe cognitive impairment3
 Treatment group −.634 −4.254 <.001
 Cognition group −.630 −3.358 .001
 Cognition x treatment interaction .648 2.888 .004
 Site 1 .100 1.523 .129
 Site 2 −.141 −2.209 .028
 Site 3 −.198 −3.060 .002
 Site 4 .055 .841 .401
 Overall model 247 8.099 .191 <.001
Open in a new tab

Note: Site 5 was used as the reference for site comparisons.

1

Low average cognition includes T-scores ≤ 40.

2

Moderate cognitive impairment includes T-scores ≤ 23.

3

Severe cognitive impairment includes T-scores ≤ 19.

Figure 1a.

Figure 1a.

Open in a new tab

Mean Change of Low Average Cognition Group vs. Average (T > 40)

Figure 1b.

Figure 1b.

Open in a new tab

Mean Change of Mildly Impaired Cognition Group vs. Above Mildly Impaired (T > 23)

Figure 1c.

Figure 1c.

Open in a new tab

Mean Change of Severely Impaired Cognition Group vs. Above Severely Impaired (T > 19)

The multiple regression analyses were repeated separately controlling for diagnosis, age, and baseline severity of psychiatric symptoms, but did not alter the main findings. Similarly, controlling for age and education in the multiple regressions had no effect on the results.

4. Discussion

The present study evaluated whether severity of cognitive impairment was related to cognitive improvement in persons with schizophrenia and other severe mental illnesses who were participants in five randomized controlled trials of a well standardized, comprehensive cognitive remediation program, the Thinking Skills for Work program. The findings indicated that participants in the cognitive remediation program improved more in neuropsychological test performance than those who received usual care, with those participants who had greater levels of cognitive impairment benefitting significantly more than those with less severe impairment. Furthermore, the findings were replicated across two levels of cognitive impairment (moderate and severe impairment), and were unaffected by controlling for demographic or diagnostic characteristics, or severity of psychopathology.

In contrast to the analyses comparing participants with low vs. high cognitive functioning at baseline using the cut-points for moderate or severe impairment, the analysis using the cut-point for low average cognition or worse (T ≤ 40) did not find a significant cognition group by treatment group interaction. Thus, while participants with moderate or severe levels of cognitive impairment at baseline who received cognitive remediation tended to improve more in cognitive functioning than less impaired participants, individuals whose cognitive functioning was relatively intact (i.e., above the low-average cut-off score) benefitted just as much from cognitive remediation as those with low average or worse cognitive functioning. It should be noted, however, that this analysis was statistically underpowered due to the relatively small number of participants whose baseline cognitive functioning was above the low-average cut-off score (N = 23).

Some prior research on cognitive predictors of response to cognitive remediation has produced null (Vita et al., 2013) or inconsistent results (Medalia & Richardson, 2005), although the findings of the present study are partially supported by the results of two small studies. In a sample of 23 participants who received 12 sessions of cognitive remediation based on teaching compensatory strategies, individuals with lower baseline cognitive functioning improved more at post-treatment than less impaired persons (Twamley et al., 2011). In a randomized controlled trial involving 77 participants assigned to either ten sessions of computer-based training in basic areas of cognitive functioning or problem solving, Rodewald et al. (2014) reported that lower baseline planning ability predicted greater improvement in planning for participants who received problem solving training but not those who received basic cognitive training, whereas neither area of cognitive functioning at baseline predicted improvement in basic cognitive functioning for either group.

In comparison to the present findings, two other studies reported that lower levels of cognitive functioning predicted less improvement following cognitive remediation. Wiedl et al. (2001) reported that worse performance on Degraded Stimulus Performance Test (Nuechterlein & Asarnow, 1986) predicted less learning on the Wisconsin Card Sorting Test (Berg, 1948) after a single session of special instructions and feedback on the test. However, it is unclear to what extent a single session of training cognitive performance on a neuropsychological test can be characterized as cognitive remediation, as most other programs provide multiple sessions of training across different cognitive areas.

In a study of 137 psychiatric inpatients who received approximately 36 sessions of cognitive remediation using Cogpack software based on the Thinking Skills for Work program and “bridging groups” aimed at fostering generalization, Lindenmayer and colleagues (2017) found that participants with lower baseline cognitive functioning improved less on the MCCB than those with higher levels of initial neurocognitive functioning. Two related features of the Lindenmayer study may have implications for reconciling the opposite findings to those reported in the present study, despite similarities in the cognitive remediation programs used. First, the participants in Lindenmayer et al. were inpatients with significantly lower levels of baseline cognitive functioning than the outpatients included in the five studies reported here, as indicated by the lower overall T-scores between the samples (T = 16.30 vs. 28.58, respectively). Second, cognitive training was conducted in 40 minute groups of six to nine participants, with each person working at his or her own pace, and two (or more) staff members providing individual instruction or coaching as needed. In contrast, across the five studies reported here, cognitive training in the Thinking Skills for Work program was typically provided in 60–90 minute sessions, either individually or in small groups of two or three participants. The combination of greater cognitive impairment and less intensive and individualized cognitive remediation in Lindenmayer et al. compared to this study suggests that participants with more severe cognitive impairment may require more intensive and personally tailored cognitive remediation to benefit as much or more than those less cognitively impaired. More work is needed to better understand how to provide cognitive remediation to address the more severe cognitive impairment often found in longer term psychiatric inpatients.

The present study has several methodological advantages over past research on this topic. First, prior research has explored cognitive predictors of response to cognitive remediation among participants who received the intervention, but has not evaluated the prediction of differential benefit from cognitive remediation compared to usual care, as in this study. The prediction of differential benefit from cognitive remediation provides more useful information than prediction within treatment recipients because it addresses the question of whether or not providing an intervention (cognitive remediation) is more likely to benefit some participants than others, a question with important clinical and resource implications.

Second, combining the data from five randomized controlled trials conducted in four different treatment settings and utilizing somewhat different neuropsychological batteries both permitted the formation of relatively large sample size for statistical analyses, and increased the generalizability of the results across participant characteristics and measures of cognitive functioning. Third, the cognitive remediation program studied (the Thinking Skills for Work program) was well standardized in a manual and employed selected cognitive exercises from an established cognitive training software package (Cogpack). In addition, high rates of participation in the Thinking Skills for Work program were present throughout all five studies, ensuring that participants had sufficient exposure to the cognitive remediation program. This high level of standardization of cognitive remediation increased the ability to obtain meaningful results by maximizing the replicability of the cognitive remediation methods across the studies.

The present findings suggest that those individuals with the greatest cognitive impairment stand to benefit the most from cognitive remediation in improving their cognitive functioning. These results confirm the most basic rational for developing cognitive remediation methods in the first place—to reduce the cognitive impairment present in many individuals with schizophrenia and other severe mental illnesses (Spaulding et al., 1999; Wykes et al., 1999). The findings suggest that even outpatients with relatively severe cognitive impairment can benefit from a well-standardized, sufficiently intensive, facilitated cognitive remediation program.

Several limitations for the generalizability of these findings should be noted. First, although the symptom scales (Brief Psychiatric Rating Scale and Positive and Negative Syndrome Scale) overlap considerably, their anchors differ somewhat. The impreciseness of these ratings may have contributed to the lack of significant effects of psychiatric symptoms. Second, because the long-term focus of the Thinking Skills for Work program is on improving employment outcomes (McGurk et al., 2005), participants in all five studies had competitive employment as a personal goal and were also enrolled in vocational rehabilitation programs. Thus, it is unknown whether the observed associations between baseline cognitive impairment and improvement in cognitive functioning during cognitive remediation would also be found if participants were receiving a different type of psychosocial rehabilitation, or no rehabilitation.

Third, the Thinking Skills for Work program is a comprehensive approach to cognitive remediation in which participants are engaged in a standardized curriculum of cognitive exercises drawn from Cogpack software, facilitated by a trained cognitive specialist who provides individualized coaching in the use of cognitive strategies during the exercises, and supplemented by teaching self-management strategies for improving day-to-day cognitive functioning. The incorporation of three major approaches to cognitive remediation into this program (i.e., cognitive exercises, strategy coaching on cognitive exercises, the teaching of cognitive self-management strategies) means that different associations could be found between cognitive impairment and response to intervention in cognitive remediation programs that include some but not all three of these cognitive remediation methods. Further research is needed to address these questions.

Acknowledgements

The authors express their appreciation to the following people for their assistance in this project: Sarah Margolin, Ivy Tran, & Rosemarie Wolfe and everyone from Thresholds Inc., Chicago, IL, Brooklyn Community Services, Mental Health Center of Manchester, and Department of Vocational Services, Mount Sinai School of Medicine.

Funding Sources

This project was conducted with grant funding from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR Grant #90AR5018). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). Results do not necessarily represent the policy of NIDILRR, ACL, HHS, and you should not assume endorsement by the Federal Government. Funding for the original studies includes NIDILRR grants: #H133G050230 and #H133G090206; NIDILRR/SAMHSA grant #9ORT5029, NIMH grant R01 MH077210 and funding from the New York State Office of Mental Health and the Center for Rehabilitation and Recovery, Coalition of Behavioral Health Agencies Incorporated, New York.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflicts

None of the authors have any conflicts of interest to report.

References

  1. Becker DR, Drake RE, 2003. A Working Life for People with Severe Mental Illness. New York: Oxford University Press. [Google Scholar]
  2. Berg EA, 1948. A simple objective test for measuring flexibility in thinking. J. Gen. Psychol 39, 15–22. [DOI] [PubMed] [Google Scholar]
  3. Bond GR, Salyers MP, Dincin J, R.E. D, Becker DR, Fraser VV, Haines M, 2007. A randomized controlled trial comparing two vocational models for persons with severe mental illness. J. Consult. Clin. Psychol 968–82. [DOI] [PubMed] [Google Scholar]
  4. Brandt J, 1991. The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. Clin. Neurophysiol 5, 125–42. [Google Scholar]
  5. Cornblatt BA, Risch NS, Faris G, Friedman D, Erlenmeyer-Kimling L, 1988. The Continuous Performance Test, Identical Pairs Version (CPT-IP): I. New findings about sustained attention in normal families. Psychiatry Res. 26, 223–38. [DOI] [PubMed] [Google Scholar]
  6. Delis DC, Kramer JH, Kaplan E, Ober BA, 1987. California Verbal Learning and Memory Test (Manual). San Antonio, TX: Psychological Corporation. [Google Scholar]
  7. Depp CA, Mausbach BT, Harmell AL, Savla GN, Bowie CR, Harvey PD, Patterson TL, 2012. Meta-analysis of the association between cognitive abilities and everyday functioning in bipolar disorder. Bipolar Disord. 14, 217–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. First MB, Spitzer RL, Gibbon M, Williams JBW, 1996. Structured Clinical Interview for DSM-IV Axis-I Disorders - Patient Edition (SCID-I/P, Version 2.0). New York: Biometrics Research Department, New York State Psychiatric Institute. [Google Scholar]
  9. Gingerich S, Mueser KT, 2011. Illness Management and Recovery: Personalized Skills and Strategies for Those with Mental Illness (3rd ed.). Center City, MN: Hazelden. [Google Scholar]
  10. Green MF, 2006. Cognitive impairment and functional outcome in schizophrenia and bipolar disorder. J. Clin. Psychiatry 67 Suppl 9, 3–8. [PubMed] [Google Scholar]
  11. Green MF, Llerena K, Kern RS, 2015. The “right stuff” revisited: What have we learned about the determinants of daily functioning in schizophrenia? Schizophr. Bull 41, 781–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Heinrichs RW, 2001. In Search of Madness: Schizophrenia and Neuroscience. New York: Oxford University Press. [Google Scholar]
  13. Kay SR, Opler LA, Fiszbein A, 1987. The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr. Bull 13, 261–76. [DOI] [PubMed] [Google Scholar]
  14. Keefe RSE, Goldberg TE, Harvey PD, Gold JM, Poe MP, Coughenour L, 2004. The Brief Assessment of Cognition in Schizophrenia: Reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr. Res 68, 283–97. [DOI] [PubMed] [Google Scholar]
  15. Kurtz MM, 2011. Neurocognition as a predictor of response to evidence-based psychosocial interventions in schizophrenia: What is the state of the evidence? Clin. Psychol. Rev 31, 663–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Lindenmayer JP, Ozog VA, Khan A, Ljuri I, Fregenti S, McGurk SR, 2017. Predictors of response to cognitive remediation in service recipients with severe mental illness. Psychiatr. Rehabil. J 40, 61–69. [DOI] [PubMed] [Google Scholar]
  17. Lukoff D, Nuechterlein KH, Ventura J, 1986. Manual for the Expanded Brief Psychiatric Rating Scale (BPRS). Schizophr. Bull 12, 594–602. [Google Scholar]
  18. Marker KR, 2014. COGPACK: The Cognitive Training Package Manual. In (8.8 ed.). Heigelberg, Germany: Marker Software. [Google Scholar]
  19. McGurk SR, Mueser KT, 2015. Cognitive enhancement and illness self-management in vocational rehabilitation. Schizophr. Bull 41, S183. [Google Scholar]
  20. McGurk SR, Mueser KT, Covell NH, Cicerone KD, Drake RE, Silverstein SM, Medalia A, Myers R, Bellack AS, Bell MD, Essock SM, 2013. Mental health system funding of cognitive enhancement interventions for schizophrenia: Summary and update of the New York Office of Mental Health Expert Panel and Stakeholder Meeting. Psychiatr. Rehabil. J 36, 133–45. [DOI] [PubMed] [Google Scholar]
  21. McGurk SR, Mueser KT, DeRosa T, Wolfe R, 2009. Work, recovery, and comorbidity in schizophrenia: A randomized controlled trial of cognitive remediation. Schizophr. Bull 35, 319–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. McGurk SR, Mueser KT, Pascaris A, 2005. Cognitive training and supported employment for persons with severe mental illness: One year results from a randomized controlled trial. Schizophr. Bull 31, 898–909. [DOI] [PubMed] [Google Scholar]
  23. McGurk SR, Mueser KT, Xie H, Feldman K, Shay Y, Klein L, Wolfe R, 2016. Cognitive remediation for vocational rehabilitation nonresponders. Schizophr.Res. 175, 48–56. [DOI] [PubMed] [Google Scholar]
  24. McGurk SR, Mueser KT, Xie H, Welsh J, Bailey E, Guarino S, Kaiser S, Frasier V, Drake RE, Becker DR, Wolfe R, McHugo GJ, 2015. Cognitive enhancement treatment for people with mental illness who do not respond to supported employment: A randomized controlled trial. Am. J. Psychiatry 172, 852–61. [DOI] [PubMed] [Google Scholar]
  25. McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT, 2007. A meta-analysis of cognitive remediation in schizophrenia. Am. J. Psychiatry 164, 1791–802. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Medalia A, Richardson R, 2005. What predicts a good response to cognitive remediation interventions? Schizophr. Bull 31, 942–53. [DOI] [PubMed] [Google Scholar]
  27. Nuechterlein KH, Asarnow RF, 1986. Continuous Performance Test (CPT) Program for IBM-Compatible Microcomputers, Version 7.10 for the Degraded Stimulus CPT and Conventional “0” CPT. Los Angeles: Authors. [Google Scholar]
  28. Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, Essock S, Fenton WS, Frese F.J.r., Gold JM, Goldberg T, Heaton RK, Keefe RS, Kraemer H, Mesholam-Gately R, Seidman LJ, Stover E, Weinberger DR, Young AS, Zalcman S, Marder SR, 2008. The MATRICS Consensus Cognitive Battery, part 1: Test selection, reliability, and validity. Am. J.Psychiatry. 165, 203–13. [DOI] [PubMed] [Google Scholar]
  29. Porter RJ, Robinson LJ, Malhi GS, Gallagher P, 2015. The neurocognitive profile of mood disorders - a review of the evidence and methodological issues. Bipolar Disord. 17, 21–40. [DOI] [PubMed] [Google Scholar]
  30. Radford LM, Chaney EF, O’Leary MR, 1978. Screening for cognitive impairment among inpatients. J. Clin. Psychiatry 39, 712. [PubMed] [Google Scholar]
  31. Revell ER, Neill JC, Harte M, Khan Z, Drake RJ, 2015. A systematic review and meta-analysis of cognitive remediation in early schizophrenia. Schizophr. Res 168, 213–22. [DOI] [PubMed] [Google Scholar]
  32. Rodewald K, Holt DV, Rentrop M, Roesch-Ely D, Liebrenz M, Funke J, Weisbrod M, Kaiser S, 2014. Predictors of improvement of problem-solving during cognitive remediation for patients with schizophrenia. J. Int. Neuropsychol. Soc 20, 1–6. [DOI] [PubMed] [Google Scholar]
  33. Spaulding WD, Fleming SK, Reed D, Sullivan M, Storzbach D, Lam M, 1999. Cognitive functioning in schizophrenia: Implications for psychiatric rehabilitation. Schizophr. Bull 25, 275–89. [DOI] [PubMed] [Google Scholar]
  34. Stern RA, & White T (2003). Neuropsychological Assessment Battery. Wood Dale, IL: Stoelting Co. [Google Scholar]
  35. Twamley EW, Burton CZ, Vella L, 2011. Compensatory cognitive training for psychosis: who benefits? Who stays in treatment? Schizophr. Bull 37, S55–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Vita A, Deste G, De Peri L, Bariati S, Poli R, Cesana BM, Sacchetti E, 2013. Predictors of cognitive and functional improvement and normalization after cognitive remediation in patients with schizophrenia. Schizophr. Res 150, 51–7. [DOI] [PubMed] [Google Scholar]
  37. Wechsler D, 1981. Wechsler Adult Intelligence Scale-Revised. New York: Harcourt, Brace, Jovanovich. [Google Scholar]
  38. Wiedl KH, Wienobst J, Schottke HH, Green MF, Nuechterlein KH, 2001. Attentional characteristics of schizphrenia patients differing in learning proficiency on the Wisconsin Card Sorting Test. Schizophr. Bull 27, 687–95. [DOI] [PubMed] [Google Scholar]
  39. Wilkinson G, 1993. Wide Range Achievement Test 3 (Manual). Wilmington, DE: Wide Range Inc. [Google Scholar]
  40. Wykes T, 1994. Predicting symptomatic and behavioural outcomes of community care. Brit. J. Psychiatry 165, 486–92. [DOI] [PubMed] [Google Scholar]
  41. Wykes T, Huddy V, Cellard C, McGurk SR, Czobar P, 2011. A meta-analysis of cognitive remediation for schizophrenia: Methodology and effect sizes. Am. J. Psychiatry 168, 472–85. [DOI] [PubMed] [Google Scholar]
  42. Wykes T, Reeder C, Corner J, Williams CA, Everitt B,1999. The effects of neurocognitive remediation on executive processing in patients with schizophrenia. Schizophr. Bull 25, 291–307. [DOI] [PubMed] [Google Scholar]

RESOURCES