The effects of cognitive remediation in patients with affective psychosis: a systematic review.

Abstract

Background:

Schizophrenia, schizoaffective disorder, and related illnesses are associated with significant impairment in cognitive functioning which is among the strongest predictors of disability and poor quality of life. Cognitive remediation (CR) was developed as a set of behavioral interventions directly targeting cognitive symptoms. Studies have shown that CR produces cognitive improvements in patients with schizophrenia and bipolar disorder that may be associated with improvements in functioning. However, the relative efficacy of CR across diagnoses has not been established. Indirect evidence suggests that CR is effective in patients with affective illness as well as patients with schizophrenia (SZ); however, the one study to evaluate the effects of diagnosis on outcomes directly in patients with SZ versus schizoaffective disorder (SZA) found no differences by diagnosis.

Methods:

In this systematic review, we evaluated cognitive and functional outcomes after CR in studies including patients with SZA, and examined specificity of training content to outcomes.

Results:

Sixteen studies met inclusion criteria: 10 studies that compared CR to a control condition (n=779) and 6 comparative effectiveness studies. None of the studies explicitly compared patients by diagnosis. Studies included a mixture of patients with SZA or SZ. Of the CR versus control studies, effect sizes for cognitive outcomes were moderate-large (d=.36-.94). Studies comparing CR paradigms targeting different cognitive domains showed specificity of training focus to outcomes. Five of studies reported significant functional improvement after CR as secondary outcomes.

Conclusions:

In this review, we found support for the use of CR paradigms in patients with affective psychosis, with evidence that reported treatment effects in mixed affective and non-affective samples are at or above the levels previously reported in SZ. However, lack of availability of data directly comparing patients by diagnosis or examining moderator or mediator effects of diagnosis or diagnosis-related patient characteristics limits our understanding of the relative efficacy of CR across patient group.

Introduction

Schizophrenia (SZ) and related disorders are associated with marked impairments in neurocognitive functioning that are strong predictors of functional outcomes (Bora et al., 2009; Green et al., 2004). While some studies report that patients with affective psychosis such as schizoaffective disorder (SZA) or bipolar disorder with psychosis (BDP) perform better on neurocognitive tasks than patients with SZ (Burdick et al., 2014; Sperry et al., 2015), other reports find neurocognitive impairment that is at or near the levels seen in SZ (Lewandowski et al., 2013). Thus, attempts to ameliorate this common and disabling symptom dimension across the psychoses is warranted (Green et al., 2004; Miskowiak et al., 2017).

Cognitive remediation (CR) is a set of behavioral interventions directly targeting cognitive symptoms with the aim of improving cognitive functioning. On average, CR paradigms produce moderate, durable effects on cognitive performance in patients with SZ (Wykes et al., 2011). The effects of CR in related illnesses such as SZA, however, are less well-established. A review in patients with SZA or affective disorders (Anaya et al., 2012) reported moderate effects (0.32) on cognition across studies; however, most study samples included a mixture of patients with SZ, SZA and/or bipolar or unipolar mood disorders. The pooled effect size weighted for proportion of the sample comprised of SZA cases, which was calculated to examine the effects of CR solely in patients with SZA, was .41. Across all included studies, the authors found a significant positive association between cognitive effects and percentage of cases with SZA and affective disorders. These findings provide indirect evidence that the effects of CR on cognition are as strong or stronger in patients with SZA or affective disorders. However, only one study to date has explicitly compared the effects of CR in patients with SZA and SZ, and found that diagnosis did not induce differential effects on primary outcomes or mediate CR effects on cognitive or functional outcomes (Lewandowski et al., 2011). As previous work suggests that patients with SZA benefit from CR, we aimed to update the review of the literature examining cognitive outcomes after CR in patients with SZA, and examine specificity of training to outcomes.

Methods

1. Search Strategy

This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Moher et al., 2009). Peer-reviewed, English-language research articles were selected for the review; relevant review and meta-analytic reports were also selected. We identified studies for inclusion through searching the electronic databases PubMed, PsycINFO, and EMBASE. A 2012 systematic review (Anaya et al., 2012) covered the topic of CR in SZA (as well as in bipolar disorder and unipolar depression) between January 1990 and December 2010; thus, we restricted our search to the timeframe of January 2011 to July 2018, and refer to the findings of Anaya and colleagues for earlier works.

Two sets of keyword search algorithms were used, linked with the Boolean operator AND. The first was related to diagnosis and included the term “schizoaffective”. The second set of search terms was related to cognitive remediation: “cognitive training” OR “cognitive remediation” OR “cognitive rehabilitation.” Using these criteria, all three authors screened title and abstract of search results. During this screening phase, we excluded study protocols that did not include primary data, as well as studies that clearly failed to meet inclusion criteria (below). Whenever at least one author raised concerns about study inclusion, the full text was inspected and all three authors and discussed until a consensus was reached. For all search results that passed the first screening, we retrieved and reviewed the full texts. Additionally, at this stage we cross-referenced lists of included studies to gather any papers that the search terms had not identified.

2. Eligibility Criteria and Study Selection

We aimed to evaluate the effects of CR on cognition compared to another intervention or treatment as usual in patients with SZA. Thus, studies were included if they: 1) presented findings from randomized controlled trials (RCTs) of a psychosocial cognitive remediation; 2) included performance on at least one validated neuropsychological measure distinct from the trained tasks as a primary outcome; 3) included patients with a diagnosis of SZA; 4) were peer-reviewed English language original articles published within the specified date range above. Studies were excluded if: 1) they only provided data on feasibility, acceptability or engagement, single case studies, or no data (e.g. published study protocols); 2) they did not include participants with diagnoses of SZA confirmed by a clinician or through an initial assessment; 3) cognitive remediation was only one element in the context of a multi-component intervention tested in the treatment arm or included as an adjunctive element of another intervention; 4) data presented were secondary analyses or extended follow-up studies of original trials previously reported; 5) data presented were from a single-arm study. If there were several relevant articles based on a single sample, studies with a larger sample were selected. Note that CR is a broad-based term that can be used to refer to range of approaches for the amelioration of cognitive symptoms. We did not specify any parameters regarding the type of CR paradigm employed by the studies, other than the eligibility criteria above. For articles were not rated as eligible by all three authors, we held a discussion meeting where we analyzed any disagreements until a consensus about study inclusion was reached.

All articles matching our eligibility criteria were reviewed in full by the three authors.

From each study, we extracted means and SDs of both groups on measured cognitive and functional outcomes. We evaluated efficacy by examining differential changes in cognitive outcomes. In addition to cognitive data, two authors independently recorded information concerning demographics, clinical characteristics (diagnosis, age of onset, and duration of illness), as well as potential moderator variables. Whenever risk of bias was deemed not low, and/or for every mismatch in extracted data, all three authors discussed until a consensus was reached. Given the heterogeneity of study designs and samples, we did not code variables related to medication and positive and negative symptoms.

Results

We conducted full database searches in July 2018, with the inclusion and exclusion criteria identified prior to the collection period. Figure 1 shows a PRISMA flowchart of each stage of the search process. The search strategy returned 184 unique results after duplicates were removed. Of these, 32 full-text articles were examined further. Of the 32 papers reviewed in full, 16 were excluded as not meeting criteria, including 5 CR augmentation studies that compared a non-CR intervention (e.g. vocational training) to that intervention plus CR. Thus, sixteen studies were included in the review: ten studies that compared CR to a control condition were included in the primary results (see Table 1); six comparative effectiveness RCTs were also reviewed. The included CR vs. control studies contained 681 total participants with sample sizes ranging from 31–156. Five studies compared their active CR intervention to Treatment as Usual (Kariofillis et al., 2014; Lahera et al., 2013; Mueller et al., 2015; Rus-Calafell et al., 2013; Twamley et al., 2012), three to a Computer Games (CG) control condition (Bryce et al., 2018; Fernandez-Gonzalo et al., 2015; Fisher et al., 2015), one to a CG control plus group therapy (Ahmed et al., 2015), and one used leisure activities (Farreny et al., 2012). Three of the 10 included studies did not report a breakdown of patient numbers by diagnosis; of the studies that did, the proportion of the sample with a diagnosis of SZA ranged from 11% to 43%. Interventions ranged from 10 to 50 hours. (see Table 1). All studies found significant group-by-time interactions for at least one primary outcome measure. Six studies focused on neurocognitive outcomes, three studies on social cognitive outcomes, and one examined both neuro and social cognitive measures. Three studies reported MCCB Global Cognition as a primary outcome; the median effect was moderate (Cohen’s d=0.58). An additional study used a composite measure of global neurocognition that was not based on the MCCB, and found a small effect (d=0.43). Mean effects of significant outcomes across studies are reported in Table 2. Median effect sizes in these studies ranged from small to large with an average in the medium range (Cohen’s d =.65).

Figure 1.

PRISMA flow chart of selection of publications for inclusion in review.

Table 1.

Randomized controlled trials of Cognitive Remediation vs. Control

Study	First Author Year	Study Design	Total Randomized (SZA/SZ)	Completers per group (CR/Control)	Intervention Duration and Frequency	Description of CR program	Cognitive Outcome Mesaures	Significant Group × Time Findings (Effect size)
1	Farreny 2012	CR vs Leisure Activities Group	62 (not reported)	29/24	32 sessions, twice a week	Problem Solving and Cognitive Flexibility Training	Behavioral Assessment of the Dysexecutive Syndrome (BADS); Trail Making Test (TMT); Wechsler Memory Scale-III verbal and visual memory	BADS total (d= .43) and BADS key search (d=0.57)
2	Twamley 2012	CR vs TAU	69 (30/39)	23/28	12 weeks, once a week	Compensatory Cognitive Training	Memory for Intentions Screening Test; Wechsler Adult Intelligence Scale, third edition Digit Span; Hopkins Verbal Learning Test – Revised; Wisconsin Card Sorting Test; WAIS-III Digit Symbol; WAIS-III Letter-Number Sequencing; Controlled Oral Word Association Test	Post-treatment: verbal memory (d=0.53) 3-month follow up: Verbal memory (d= 0.38) Attention (d= .24)
3	Lahera 2013	SCIT vs TAU	37(4/33*)	21/16	18–24 weeks	Social Cognition and Interaction Training	Face Emotion Identification Task; Face Emotion Discrimination Task; Emotion Recognition-40; Hinting Task; Ambiguous Intentions Hostility Questionnaire	Emotion Recognition-40 (n2p =.16) Face Emotion Discrimination Task (n2p =.31) Face Emotion Identification Task (n2p =.13) Hinting Task n2p=.14) Hostile attribution bias (n2p =.18)
4	Rus-Calafel 2013	SST vs TAU	31 (6/25)	13/18	14 one-hour sessions	Group-based Social Skills Training	The screen for cognitive impairment in psychiatry (SCIP); Continuous performance test; Assertion inventory; Social interaction self-statement test; Simulated social interaction test	None
5	Kariofillis 2014	Auditory CR vs Visual CR vs. TAU	46 (9/37)	16/15/15	10 sessions	Auditory or Visuospatial Cognitive Training	Multiple Word Recognition Test; Word Fluency Test; Digit Symbol Coding; Digit Span; Wechsler Logical Memory Test	Digit Span Backward (η2 = .18)
6	Ahmed 2015	CR vs CG + Healthy Living Groups	88 (24/54)	41/36	50 hours	Auditory Cognitive Training	Wechsler Abberviated Scale of Intelligence; MCCB	MCCB Attention (η2p= 0.069) Working Memory (η2p=0.059) Verbal (η2p = 0.067) Composite (η2p =0.058)
7	Fernandez-Gonzalo 2015	CR vs CG	49 (10/39)	21/19	2h/week for 4–5 months (approx 20 hours)	Neuro Personal Trainer	WAIS-III Digit span; WMS-III Spatial span WMS-III); Conners Continuous Performance Test-II; Rey Auditory Verbal Learning Test; Logical Memory and Visual Reproduction subtests WMS-III; Trail Making Test; EEFF (flexibility, inhibition, verbal fluency and planning and problem solving); Stroop Word and Color Test; Verbal Fluency Test; Tower of London; Pictures of Facial Affect; Internal, Personal and Situational Attributions Questionnaire	Spatial Span (np2=.103) Immediate Logical Memory (np2=0.114) Facial Affect Recognition (np2= 0.167)
8	Mueller 2015	CR vs TAU	156 (not reported)	81/75	30 sessions, 90 minutes each, biweekly	Integrated Neurocognitive Therapy	Trail Making Test; Controlled Oral Word Association Test; Continuous Performance Test; Auditory Verbal Learning Test; Wechsler Memory Scale-Revised; Letter-Number Span; Wisconsin Card Sorting Test; Picture of Facial Affect Test; Emotion Recognition Questionnaire; Schema Component Sequencing Task-Revised; The Ambiguous Intentions Hostility Questionnaire	Neurocognition Composite (d=.43) Processing Speed (d=.41); Reasoning and problem solving (d=.32) Social cognition composite (d=.32) Emotion perception (d=.31) Social schema (d=.33)
9	Fisher 2015	CR vs CG	87 (not reported)	40/34	40 hours, 5 hours a week	Auditory Training	MCCB	MCCB Composite (d=.63), Attention (d=.30), Speed of Processing (d=.42), Verbal Learning (d=.28), Visual Learning (d=.50), and Problem Solving (d=.40)
10	Bryce 2018	CR vs CG	56 (16/40)	22/21	20 sessions	Cogpack	MCCB	Composite (d = 0.68)

CG Computer Games

CR cognitive Remediation

MCCB Matrics Consensus Cognitive Battery

SCIT Social Cognition and Interaction Training

SST Social Skills Training

TAU Treatment as Usual

^*33 BDP patients (not SZ)

Table 2.

Significantly changed outcomes after cognitive remediation protocols, and their effect size.

Cognitive Domain	Reported by Included Studies	Effect Si ze (Cohen’s d)
Attention	6,9	0.42*
Verbal Learning	6,9	0.41*
Speed of Processing	8,9	0.42*
Problem Solving	8,9	0.36*
Working Memory	6	0.50
Verbal Memory	2	0.53
Immediate Logical Memory	7	0.72
Visual Learning	9	0.50
Visual Memory	5	0.94
Spatial Span	7	0.68
Behavioral Assessment of the Dysexecutive Syndrome	1	0.43
Emotion Perception	3,8	0.66*
Facial Affect Recognition	7	0.90
Theory of Mind	3,8	0.57*
Hostile Attribution Bias	3	0.94
Global Cognition	6,8,9,10	0.55*

^*whenever more than on study reported significant groupxtime interactions for this outcome, we calculated the mean effect size across such studies.

Of the 6 comparative effectiveness studies, two directly compared neurocognitive training (NCT) with social cognition intervention (SCI). Bucci found that only NCT produced significant improvements, specifically in attention, verbal memory and executive functioning (Bucci et al., 2013). In contrast Wolwer found greater improvements in facial affect recognition, prosodic affect, and theory of mind with large effect size in SCI vs NCT (Wolwer and Frommann, 2011). Two other comparative studies examined the benefits of supplementing NCT with SCI. In NCT+SCI vs NCT-only, Lindenmeyer found greater improvements in emotion recognition and discrimination with large effect size, and in neurocognition with small effect size (Lindenmayer et al., 2013), whereas Fisher found greater improvements in prosody identification and reward processing with moderate effect size (Fisher et al., 2017). In a NCT+SCI vs SCI-only study, Kurtz found that the combined group showed greater improvements in attention and working memory, with moderate effect size (Kurtz et al., 2015). Finally, Bowie examined the benefits of supplementing functional training with CR, and found similar improvements in cognition, with moderate effect size (Bowie et al., 2012). All but one of the studies (Lahera et al., 2013) reported data on antipsychotic medications. Of the 9 studies that reported this information, only Fernandez-Gonzalo and colleagues reported a significant difference between groups at baseline. This difference was in percent of participants using in first generation antipsychotics and the study controlled for antipsychotic drug dosages in all analyses.

Finally, we examined functional change after CR as a secondary outcome where available. Five of the 10 studies reported group by time functional improvement post treatment (Kariofilis et al 2014; Ahmed et al 2015; Farreny et al 2012; Rus Calafell et al 2012; Mueller et al 2015). One additional study showed significant functional improvement at the 3-month follow-up though there was no significant functional improvement at the post-treatment measure (Twamley et al 2012). Four of the studies measured functioning and found no significant group by time interaction post treatment (Bryce et al 2018; Lahera et al 2013; Fisher et al 2018; Fernandez-Gonzalo et al 2015).

In the five studies that showed a group by time interaction in functioning post-treatment, various functional outcome measures were used: two studies showed improvements in overall global functioning (Kariofilis et al 2014; Mueller et al 2015) as measured by the Global Assessment of Functioning Scale (APA, 2000) and two showed improvements in social withdrawal (Farreny et al 2012; Rus Calafell et al 2012) as per Spanish versions of the Social Functioning Scale (Birchwood et al., 1990).

Discussion

Overall, findings from this review show cognitive improvements after CR versus active control in samples that include patients with SZA, and support the use of CR in affective psychosis. While findings from individual studies ranged from small to large, the average median effect size was d=.58, slightly higher than has been reported in meta-analyses of CR in SZ (Wykes et al., 2011), suggesting that patients with affective psychosis respond to treatment. Additionally, a majority of studies assessing functioning as a secondary outcome found improvements in functional measures after CR. These findings are consistent with a recent study of CR in patients with BDP, which found medium to large effects of CR versus active control in several cognitive domains and a cognitive composite (Lewandowski et al., 2017). Interpretation of relative effects of CR by diagnosis must be tempered, however, as only one study has explicitly tested the effects of diagnosis on CR outcomes, in this case in patients with SZA or SZ (Lewandowski et al., 2011).

The paucity of literature directly examining CR efficacy in SZA may reflect a theoretical framework treating cognition as a treatment target that cuts across diagnostic boundaries, and/or the practical challenges inherent in the conduct of CR treatment trials, which are long and intensive in nature, making recruitment and retention of patients from a restrictive eligibility pool (e.g. a single diagnostic category) particularly difficult. While the theoretical conceptualization of cognition as a cross-diagnostic symptom dimension is supported at least in part by a considerable literature, some aspects of illness known to distinguish SZA from non-affective illness have been identified as potential mediators or moderators of response to CR, including gray matter volume (Keshavan et al., 2011; Yüksel et al., 2012), baseline cognition (Biagianti et al., 2016; Bora et al., 2009; Scheu et al., 2013), negative and disorganized symptoms (Lindenmayer et al., 2017), and insight (Benoit et al., 2016; Pini et al., 2001). Thus, the role of these variables and diagnosis itself in CR treatment response needs clarification.

If we are to develop and implement CR interventions that are robust and efficient, an endophenotype strategy that assesses potential moderator variables, especially when applied in conjunction with machine learning techniques, may be useful in refining the delivery of personalized CR protocols tailored to individual strengths and weaknesses. This approach is supported by findings from our analysis of comparative effectiveness studies, which indicate some degree of domain specificity between the training target and improvements in primary outcomes.

Limitations

The present review has some limitations that should be noted. First, the search terms “schizoaffective” was a key criterion in the literature search; however, a number of studies that include a mixed diagnostic sample of patients with schizophrenia and schizoaffective disorder label the sample diagnosis as “schizophrenia;” therefore, we may have missed studies that also included mixed populations of SZ and SZA patients but were not identified as such. Additionally, systematic assessment of study quality was outside the scope of this brief review; however, it is possible that variability in study quality may have influenced findings. Although the use of antipsychotic medications was reported inconsistently across studies, the percentage of patients taking atypical antipsychotics was on average around 80%. Given the known association between serum anticholinergic activity in schizophrenia patients and lowered response to computerized cognitive training (Vinogradov et al., 2009), future studies should rigorously report medications and control the effects of CR for the presence of interfering antipsychotic therapies. Further, as noted above, none of the studies reviewed directly compared patient outcomes by diagnosis, or included diagnosis as a moderator or mediator in the analyses; thus, explicit evaluation of relative effects of CR by diagnosis is not possible at this time. Lastly, several factors create challenges comparing outcomes across studies, including heterogeneity in cognitive, social cognitive, and functional assessments used, and the fact that many studies do not report significance levels and effect sizes for non-significant outcomes making it difficult to compute an overall effect across studies that includes both significant and non-significant outcomes.

In sum, future research should identify significant predictors of treatment response, including potential diagnostic differences and characterization of heterogeneity within and between groups to advance knowledge on precision psychiatry practices that have the potential to enhance response to CR beyond the moderate effect size we and others have consistently found (Anaya et al., 2012).

HIGHLIGHTS:

• Schizoaffective disorder is associated with significant impairment in cognitive functioning

• Cognitive remediation produces cognitive improvements in schizoaffective disorder

• The average median effect size of cognitive remediation on cognitive outcomes was d=.65

• Patients with schizoaffective disorder respond to treatment at or above the level reported in schizophrenia

• Protocol targeting different cognitive domains showed specificity of training focus to outcomes