Feasibility of cognitive remediation therapy for adults with autism spectrum disorders: a single-group pilot study

Abstract

Background

Set-shifting (SS) difficulties and weak central coherence (CC) are commonly associated with autism spectrum disorders (ASD). Cognitive remediation therapy (CRT) aims to improve such cognitive processing; however, there are no reports on CRT for patients with ASD. This pilot study aimed to provide preliminary evidence to support the use of CRT for individuals with ASD and provide data to inform future studies.

Participants and methods

Nineteen individuals with ASD were recruited and administered a series of neuropsychological and questionnaire measures to examine cognitive function and clinical outcomes such as anxiety and depression. Participants received CRT, and cognitive function and clinical variables were re-evaluated at postintervention and after 3 months.

Results

The participants demonstrated significant improvement in CC and anxiety at postintervention, which was maintained at 3-month follow-up. Although SS scores had improved with a large effect size, this was not statistically significant.

Conclusion

CRT improved CC and anxiety scores for individuals with ASD, suggesting that CRT is an effective treatment for individuals with ASD.

Introduction

Individuals with autism spectrum disorders (ASD) characteristically display problems in social communication and a repetitive behavior pattern.¹ Poor cognitive processing, such as weak central coherence (CC) and difficulties with set-shifting (SS), is also commonly associated with ASD.^2–5 Weak CC can be defined as a bias toward the details of the information rather than the whole. Individuals with ASD tend to pay attention to details rather than to the whole and display weaknesses with the global integration of this partial information,⁶ demonstrating a weakness CC cognitive style.⁷ Anxiety is likely to influence cognitive style, and so it is important to consider this when evaluating CC.⁸

Deficits in SS in individuals with ASD are likely to manifest as cognitive inflexibility. It is also likely to be related to repetitive behaviors⁹ and the tendency to perseverate with the initial strategy and the inability to adapt to a new strategy when needed.⁹ Although these characteristics are reported to vary greatly among individuals, there are also studies suggesting similar cognitive characteristics in patients with anorexia nervosa (AN).^10–12 A recent systematic review of studies employing the most widely used neuropsychological measures of SS, the Wisconsin Card Sorting Test (WCST), demonstrated comparable performance between AN and ASD populations.⁴ The similarities in cognitive profile and the wider phenotypical similarities between ASD and AN¹³ support the idea of examining the benefits of cognitive remediation therapy (CRT), a treatment successfully implemented in eating disorders,¹⁴ within ASD.

CRT aims to improve neuropsychological processing, and it has been tailored for AN to target CC and SS specifically.¹⁴ The aims of CRT for AN are to ameliorate the rigid and poor global processing styles through the use of cognitive exercises, meta-cognitive strategies, and awareness and application to real-life situations.^15,16 In general, CRT uses short cognitive exercises to explore cognitive processing and facilitate the exploration of alternative thinking styles. CRT is delivered in a motivational style with the final aim of changing behavior in daily life through learning new and effective strategies.¹⁷

In previous studies of participants with AN, the effects of CRT were reported to improve not only cognitive task performance^18,19 but also quality of life.¹⁵ CRT has also demonstrated low dropout rates.^20,21 There have been several studies using CRT for patients with AN,^14,15,22 but the use of CRT for patients with ASD has not yet been reported. With this in mind, the present study aimed, 1) to investigate the feasibility and effectiveness of a CRT intervention for individuals with ASD, and 2) to provide preliminary data for use in future randomized controlled trials.

Participants and methods

This study was a prospective pilot study of single-group open-label trial. The study was approved by the ethics committee of the Graduate School of Medicine and School of Medicine, Chiba University (No 1715), and was registered in the national UMIN Clinical Trials Registry (ID: UMIN000017860). It was conducted in compliance with the Declaration of Helsinki. All participants gave their written informed consent. Participants under the age of 20 years needed a written consent from their parents.

Participants

A total of 21 Japanese patients with ASD were assessed for study eligibility. As two of participants met the exclusion criteria, 19 patients were eligible for participation in the intervention (Figure 1). The patients were recruited from outpatients at Chiba University Hospital and Chibaken Saiseikai Narashino Hospital and via the home page of Chiba University between January 2014 and September 2015.

Figure 1.

Flow diagram of CRT intervention.

Abbreviations: CRT, cognitive remediation therapy; IQ, intelligence quotient.

Nineteen patients were enrolled into the study. In sample size calculation, we assumed large effect size (d=0.80), α=0.05, and a power of 80%. In addition, allowing 20% dropout rate, 19 participants were required in reference to previous studies of psychological treatment for ASD or CRT for eating disorders.^23,24

Patients with ASD were diagnosed by psychiatrists according to Diagnostic and statistical of Mental Disorders (DSM-5) criteria. In addition, the Autism Diagnostic Interview-Revised (ADI-R),²⁵ Social Communication Questionnaire (SCQ),²⁶ and the Autism-Spectrum Quotient Japanese version (AQ-J)²⁷ were used to assess behavior and symptoms. The inclusion criteria for the participants were as follows: 1) a DSM-5 diagnosis of ASD, 2) aged between 18 and 50 years, and 3) an intelligence quotient (IQ) of >80 on Wechsler Adult Intelligence Scale (WAIS) – III. Exclusion criteria were as follows: psychosis, pervasive developmental disorders/mental retardation (IQ <80), current high risk of suicide, severe psychiatric disease, and brain injury.

Eight of the 19 participants were taking prescribed medications such as anxiolytics. Thirteen of the 19 participants were diagnosed with ASD after graduation from high school, three participants were diagnosed in their 30s, and four participants were diagnosed in their 40s. With respect to their work and educational status, two of the 19 participants were students, seven participants were office workers, and 10 participants were unemployed.

Measures

After enrollment, we carried out the following measures in preintervention, postintervention, and 3-month follow-up.

Cognitive function

SS

Several neuropsychological measures were employed to measure SS. The Brixton test²⁸ assesses SS, using a total of 56 trials. Participants are asked to predict the movements of a blue circle. In order to correctly predict where the blue circle will move next, the participants must work out the movement pattern. The pattern occasionally changes, and the participant must abandon the old pattern and shift their behavior to infer the new pattern and correctly predict the next position. A previous study of AN found a large effect size improvement²⁸ following CRT.

SS was also measured using the trail-making test (TMT),^29,30 Stroop test,³¹ and WCST.³² TMT is a pen and paper neuropsychological test to measure working memory and SS. It is composed of trial A and trial B. In trial A, participants are required to link numbers from 1 to 25 in numerical order (eg, 1–2–3). In trial B, the participant must shift between linking numbers in numerical order and letters in alphabetical order (eg, 1–A–2–B).

The Stroop test is performed under three conditions. In the first condition, the participant must, 1) name the colors of dots. The last two conditions involve, 2) reading the word but ignoring the color ink in which they are written, and 3) saying the color of the ink which the words are written in and ignoring the word. Stroop effect was calculated by subtracting the time of condition 1) from the time of condition 3).

WCST is a test for assessing SS. We used the Keio version of WCST,³³ which was computerized in Japan. This version has 48 trials less than the original version. The primary outcome of the WCST is the number of perseverative errors in Nelson (PEN).

CC

The Rey–Osterrieth Complex Figure (ROCF) test³⁴ was used to assess CC. Participants were asked to make a direct copy of the shape, and the test was scored according to Booth’s scoring system,³⁵ which assesses the following three elements: order index, style index, and CC index. The ROCF was video recorded and scored independently by two researchers. Reliability was examined by Kappa coefficient, and there was substantial agreement between the two researchers (κ=0.706, P<0.006).

Self-reported questionnaires

Self-report measures of anxiety, depression, cognitive flexibility, self-esteem, and quality of life were also administered. Anxiety symptoms were measured using the general anxiety disorder (GAD)-7.^36,37 It is composed of the following two scales: the first scale consisting of seven items (eg, feeling nervous, anxious, and on edge), which can be scored between 0 (not at all sure) to 3 (nearly every day), and the second scale assessing the severity of the anxiety symptoms on daily life.

The Patient Health Questionnaire (PHQ)-9^36,38 is a simple assessment tool developed to measure symptoms of depression. It is composed of the following two scales: the first scale consisting of nine items (eg, little interest and pleasure in doing things) with a 0 (not at all) to 3 (nearly every day) scale, and the second scale assessing the severity of depression symptoms on daily life.

The Hospital Anxiety and Depression Scale (HADS)^39,40 was also employed to assess self-reported depression and anxiety. This questionnaire has a total of 14 items, seven anxiety items, and seven depression items, on a 0-to-3 scale.

Self-esteem was measured using the Rosenberg self-esteem scale (RSES).^41,42 It consists of 10 items measured on a 1-to-6 scale.

Cognitive flexibility was measured using the Cognitive Flexibility Scale (CFS).⁴³ The questionnaire is composed of 12 items and uses a 1-to-5 scale. The Japanese edition has not yet been published, and it is now being prepared for standardization.

Quality of life was measured by EuroQol 5 dimension (EQ-5D).^44,45 EQ-5D is composed of five items and a feelings scale. This data was missing for 10 participants due to administrator error.

The alpha value of the self-reported questionnaires in this study was as follows: GAD-7: α=0.87; PHQ-9: α=0.85; CFS: α=0.902; RSES: α=0.84; HADS (anxiety score): α=0.83; HADS (depression score): α=0.89; and EQ-5D: α=0.55.

Interventions

The CRT manual Cognitive Remediation Therapy for Anorexia Nervosa¹⁷ was translated into the Japanese version before the start of the intervention. The main objectives of this program were to improve “bigger picture thinking”, cognitive flexibility, and multitasking.

The 19 participants individually received 10 CRT sessions, which were composed of 10 weekly or biweekly sessions, lasting 40 minutes each. All sessions were delivered by a clinical psychologist, a nurse, and a psychiatric social worker. The contents of the sessions were tailored for each participant according to their specific difficulties. Each session was composed of “a homework report”, “today’s exercise”, “reflection”, and “setting of homework”. After completing each CRT task, the participant and therapist discussed and reflected on their strategy in the feedback session (meta-cognition). In order to extrapolate the content of each session to daily life, the content of homework was decided in relation to what had been covered each session. Neuropsychological and self-report measures were readministered following the completion of the CRT intervention.

The CRT intervention was provided by three therapists who had trained at a CRT workshop program at Chiba University and Kings College London (provided by Dr Tchanturia on December 12–13, 2013). The therapists attended supervision sessions by a peer supervision/support group and received supervision by use of the fidelity rating scale.

The occurrence of adverse events was checked in each session by the therapist. In addition, the psychiatrist examined the mental state of participants. In the case of dropouts, the reasons for dropout were investigated by interview.

Statistical analyses

Data analyses were carried out with the 22nd version of the Statistical Package for Social Sciences (SPSS). Analysis of variance (ANOVA) was used to compare the following three points: preintervention, postintervention, and 3-month follow-up. Post hoc comparisons were used to compare between pre- and postintervention and between preintervention and follow-up.

Effect size was reported by Cohen’s d. Statistical significance was designated as P<0.01 to correct for multiple comparisons.

Results

Dropout rate and demographic data

Nineteen patients with ASD were eligible for participation, and three of these participants did not complete the intervention. The dropout rate was 15.8%. The reasons for dropout included high anxiety and assumptions regarding the use and difficulty of CRT. There were no adverse events for any of the patients.

Table 1 displays the demographic data of the 16 participants with ASD. The participants included four women (25%), and the mean age of the participants was 29.56 years.

Table 1.

Baseline demographic data (N=16)

Variables	n (%)	Mean (SD)	Range
Participants (%, female)	16 (25)
Age (years)		29.56 (11)	18–49
Age of diagnosis (years)
<18	4 (25)
18–30	7 (43.8)
31–50	5 (31.2)
Employment status
Employed full time	4 (25)
Full-time student	2 (12.5)
Part time/homemaker	3 (18.8)
Unemployed	7 (43.8)
Marital status
Single	12 (75)
Married	2 (12.5)
Divorced	2 (12.5)
Educational background
High school	6 (37.5)
<3 years of college/university	3 (18.8)
≥3 years of college/university	7 (43.8)
Length of education (years)		14.28 (1.93)
IQ		98.9 (10.76)	82–118
AQ-J		33.06 (4.19)	22–40

Abbreviations: AQ-J, Autism-Spectrum Quotient Japanese version; IQ, intelligence quotient; SD, standard deviation.

CRT outcomes

Neuropsychological measures

Table 2 shows the outcomes comparing the scores pre-, post-, and at follow-up of CRT interventions. An ANOVA showed a significant difference between the time points for the style and CC indices of ROCF. Post hoc comparisons revealed that these indices of the ROCF significantly increased between preintervention and postintervention and between preintervention and 3-month follow-up with large effect sizes. Although not significant, there was a trend toward improved scores in perseverative errors of the WCST and the number of errors on the Brixton test with medium to large effect sizes. There were no significant differences over the time points on any of the other neuropsychological measures.

Table 2.

Comparison among preintervention, postintervention, and follow-up

Measurements	ASD (n=16), mean (SD)			ANOVA		Direction of difference	Pre-post	95% CI	Pre-follow	95% CI
	Pre-CRT	Post-CRT	Follow-CRT	F-value	P-value		ES		ES
Brixton test	15.94 (13.65)	9.63 (7.34)	8.94 (5.63)	2.63	0.08	–	0.58	−0.16, 1.31	0.67	−0.07, 1.41
ROCF CC index	1.07 (0.48)	1.44 (0.37)	1.51 (0.34)	5.51	0.007**	Pre < post, pre < follow	−0.86	−1.62, 0.11	−1.06	−1.83, −0.29
ROCF order index	1.68 (0.93)	2.18 (0.6)	2.25 (0.57)	3.02	0.06	–	−0.64	−1.38, 0.1	−0.74	−1.49, 0.01
ROCF style index	1.08 (0.48)	1.52 (0.41)	1.60 (0.36)	7.28	0.002**	Pre < post, pre < follow	−0.99	−1.75, −0.22	−1.23	−2.01, −0.44
Stroop test	7.05 (5.37)	5.9 (3.70)	5.58 (4.15)	0.48	0.62	–	0.25	−0.48, 0.97	0.31	−0.42, 1.03
TMT trial B–trial A	30.68 (17.67)	29.98 (17.35)	28.19 (16.44)	0.09	0.91	–	0.04	−0.68, 0.76	0.15	−0.58, 0.87
WCST PEN	1.38 (2.03)	0.25 (0.58)	0.50 (0.89)	3.19	0.05	–	0.76	0.01, 1.5	0.56	−0.17, 1.3
CFS	36.75 (11.69)	39.81 (10.08)	38.19 (9.54)	0.34	0.71	–	−0.28	−1.01, 0.45	0.13	−0.86, 0.59
RSES	20.06 (4.68)	21.00 (5.72)	22.69 (6.78)	0.85	0.44	–	−0.18	−0.9, 0.54	−0.45	−1.18, 0.28
HADS (A)	10.88 (4.70)	6.69 (3.94)	7.88 (3.95)	4.19	0.02*	Pre > post	0.97	0.2, 1.73	0.69	−0.05, 1.43
HADS (D)	7.75 (5.09)	6.38 (5.21)	6.56 (5.15)	0.34	0.72	–	0.27	−0.46, 0.99	0.23	−0.49, 0.96
GAD-7	9.38 (5.27)	4.88 (5.45)	5.45 (4.12)	3.88	0.03*	Pre > post	0.84	0.09, 1.59	0.83	0.08, 1.58
PHQ-9	10.75 (6.50)	6.44 (5.83)	6.69 (4.59)	2.89	0.07	–	0.70	−0.05, 1.44	0.72	−0.02, 1.47
EQ-5D	0.73 (0.13)	0.77 (0.12)a	0.79 (0.12)	0.761	0.47	–	−0.32	−1.06, 0.42	−0.48	−1.21, 0.25

Notes:

^an=15.

^*P<0.05.

^**P<0.01.

Abbreviations: ASD, autism spectrum disorders; ANOVA, analysis of variance; CC, central coherence; CFS, Cognitive Flexibility Scale; CRT, cognitive remediation therapy; EQ-5D, EuroQol 5 dimension; ES, effect size; GAD, general anxiety disorder; HADS, Hospital Anxiety and Depression Scale; CA; PEN, perseverative errors in Nelson; PHQ, Patient Health Questionnaire; ROCF, Rey–Osterrieth Complex Figure; RSES, Rosenberg self-esteem scale; SD, standard deviation; TMT, trail-making test; WCST, Wisconsin Card Sorting Test.

Self-report measures

An ANOVA demonstrated a significant effect of time for anxiety scores on the HADS (anxiety) and the GAD-7. Post hoc comparisons demonstrated that there was significant difference between preintervention and postintervention, with large effect sizes. However, there were no significant differences between preintervention and follow-up.

Discussion

The aim of this study was to investigate the feasibility and effectiveness of a CRT intervention for individuals with ASD. There were significant differences between preintervention and postintervention and between preintervention and follow-up on the style and CC indices of the ROCF, with large effect sizes. These findings indicate an improvement in global processing in the individuals with ASD after CRT intervention which was maintained for 3 months. Although not significant, there was also a decrease in the error scores on the Brixton test with large effect sizes and decreases in the perseverative error score of the WCST with medium to large effect sizes. Such findings may suggest that the present study may have lacked power to detect statistical differences on these tests, so future studies with larger samples are warranted.

There were significant reductions in anxiety following CRT with medium to large effect sizes, suggesting that CRT might be effective in reducing anxiety symptoms of patients with ASD. However, this finding seemed circumscribed to anxiety symptoms and was not mirrored in the other self-report measures such as depression and quality of life.

Overall, these findings produced preliminary evidence to suggest that CRT is a feasible and effective treatment for improving cognitive processing in adults with ASD.

Three (15.8%) of the 19 participants dropped out from this study. This dropout rate was higher than previous studies of CRT for adults with AN, such as Tchanturia et al (10%),¹⁹ Abbate Daga et al (0%),²⁰ Dahlgren et al (5%),¹⁵ and Lock et al (13%).²¹ However, the dropout rate of the present study is in line with previous studies of adults with ASD,^23,46 from intervention methods other than CRT; dropout rates (0%–18%). While the dropout is comparable to the existing literature, it suggests that some modifications to the typical CRT intervention may be beneficial and reduce dropout rates in individuals with ASD. For example, many individuals with ASD may have difficulty in generalizing what was learned in sessions to daily life.⁴⁷ Therefore, using more concrete examples and modeling may aid learning and maximize effectiveness. Combining social skills training with CRT might be useful to solve daily life problems.

In addition, we found that participants who improved their outcomes showed good compliance with homework from the reviewing interview in the final session. Noticing their concrete problems in daily life about CC and SS might induce patient’s motivation, and they did their homework well and tried to use new strategies. From this clinical finding, we can hypothesize that CRT for ASD is more effective when patients understand the relations between their daily problems and CC or SS. This hypothesis should be considered in modifying treatment manual (eg, providing explanation of CRT) and also be verified in future studies.

This study has several limitations. First, the sample size of this study was small and may have lacked the power required to detect statistical differences on some of the tests. The lack of a control group in this study is the second limitation. The third limitation is that the result might be affected by learning effects as shown in the previous study.¹⁵ In future study, it will be necessary to allow for consideration of learning effects.

Although the results of this study are promising and suggest that CRT is likely to be beneficial for individuals with ASD, its results are preliminary and larger studies are needed to explore this further.

Conclusion

The findings of this study suggest that CRT is a feasible and acceptable treatment for adults with ASD. Future studies should aim to include larger sample sizes and control groups, as well as more in-depth analyses of the specific content and dosage required to maximize the efficacy of CRT for this population.