Abstract
Objective
Cognitive remediation is a promising approach to treating core cognitive deficits in adults with autism, but rigorously controlled trials of comprehensive interventions that target both social and non-social cognition over a sufficient period of time to impact functioning are lacking. This study examined the efficacy of Cognitive Enhancement Therapy (CET) for improving core cognitive and employment outcomes in adult autism.
Method
Verbal adult outpatients with autism spectrum disorder (N = 54) were randomized to an 18-month, single-blind trial of CET, a cognitive remediation approach that integrates computer-based neurocognitive training with group-based training in social cognition, or an active Enriched Supportive Therapy (EST) comparison focused on psychoeducation and condition management. Primary outcomes were composite indexes of neurocognitive and social-cognitive change. Competitive employment was a secondary outcome.
Results
Intent-to-treat analyses indicated that CET produced significant differential increases in neurocognitive function relative to EST (d = .46, P = .013). Both CET and EST were associated with large social-cognitive improvements, with CET demonstrating an advantage at 9 (d = .58, p = .020), but not 18 months (d = .27, p = .298). Effects on employment indicated that participants treated with CET were significantly more likely to gain competitive employment than those in EST, OR = 6.21, p = .023, which was mediated by cognitive improvement.
Conclusions
Cognitive Enhancement Therapy is a feasible and potentially effective treatment for core cognitive deficits in adult autism spectrum disorder. The treatment of cognitive impairments in this population can contribute to meaningful improvements in adult outcomes.
Lay Summary
Cognitive Enhancement Therapy (CET), an 18-month cognitive remediation intervention designed to improve thinking and social understanding, was found to be more effective than supportive therapy at improving mental quickness, attention, and employment in adults living with autism. Social understanding was equally improved in CET and supportive therapy. Cognitive remediation interventions are feasible and may confer significant functional benefits to adults with autism.
Trial Registration
clinicaltrials.gov Identifier: NCT00902798
Keywords: adult autism, cognitive remediation, cognitive enhancement, social cognition, neurocognition
Autism spectrum disorder (ASD) is a life-long neurodevelopmental condition characterized by core, neurobiologically-based deficits in social and non-social information processing (Minshew & Williams, 2007; Philip et al., 2012), which significantly limit adaptive function (Pugliese et al., 2015; Plitt, Barnes, Wallace, Kenworthy, & Martin, 2015; Hudepohl, Robins, King, & Henrich, 2015; Gilotty, Kenworthy, Sirian, Black, & Wagner, 2002) and quality of life (Sikora, Vora, Coury, & Rosenberg, 2012; de Vries & Geurts, 2015). The majority of intervention research for ASD has focused on early detection and intervention programs for children, which have significantly improved intellectual and behavioral outcomes (Warren et al., 2011). However, many individuals remain markedly disabled throughout adulthood (Howlin, Moss, Savage, & Rutter, 2013; Magiati, Tay, & Howlin, 2013), and there are few rigorously controlled studies to guide policy and practice for the treatment of adults living with ASD (Fitzpatrick, Minshew, & Eack, 2013).
Cognitive remediation has emerged as an effective approach for treating core cognitive deficits in other neuropsychiatric conditions, including schizophrenia, attention deficit hyperactivity disorder, and traumatic brain injury (Wykes, Huddy, Cellard, McGurk, & Czobor, 2011; Keshavan, Vinogradov, Rumsey, Sherrill, & Wagner, 2014), which may confer significant benefits to adults with ASD (Eack et al., 2013b). Although most evidence-based psychosocial interventions for this population are behaviorally-focused, autism is a neurodevelopmental disorder of complex information processing that is characterized by broad social-cognitive deficits in such domains as theory of mind (Chung, Barch, & Strube, 2014), perspective-taking (Hamilton, Brindley, & Frith, 2009), social context appraisal (Chawarska, Macari, & Shic, 2012), and emotion perception (Harms, Martin, & Wallace, 2010) and management (Mazefsky, 2015). Further, neurocognitive deficits in processing speed (Eack et al., 2013a), attention (Murphy et al., 2014), complex memory (Minshew & Goldstein, 1993), and executive functions (Rosenthal et al., 2013) have also been observed in adults with this condition. Such impairments are predictive of adaptive function (Hudepohl, Robins, King, & Henrich, 2015; Gilotty, Kenworthy, Sirian, Black, & Wagner, 2002), exist even in verbal individuals without intellectual disability (Eack et al., 2013c), and may represent important targets for intervention (Keshavan, Vinogradov, Rumsey, Sherrill, & Wagner, 2014).
Initial studies of cognitive remediation in adults with ASD have been small and short-term, frequently used non-randomized or uncontrolled designs, and primarily targeted isolated aspects of social cognition (Fitzpatrick, Minshew, & Eack, 2013). Long-term trials of more comprehensive interventions are non-existent. Recently, we adapted a cognitive remediation intervention originally developed for patients with schizophrenia, Cognitive Enhancement Therapy (CET; Hogarty & Greenwald, 2006), to verbal adults with ASD (Eack et al., 2013b). This 18-month, developmental approach was selected for adaptation to ASD because of its comprehensive integration of neurocognitive and social-cognitive training (Hogarty & Greenwald, 2006), prior evidence of efficacy in schizophrenia (Hogarty et al., 2004; Eack et al., 2009; Eack et al., 2015), and focus on domains that are similarly impaired across the two disorders (Eack et al., 2013a; Couture et al., 2010). The present study conducted the first randomized-controlled trial of CET in verbal adults with ASD, in order to examine its impact on cognitive and employment outcomes, relative to an active, Enriched Supportive Therapy (EST) comparison. It was hypothesized that participants treated with CET would evidence improved neurocognitive, social-cognitive, and employment outcomes relative to those receiving EST.
Method
Participants
Participants were 54 individuals with ASD recruited for an 18-month parallel arm randomized-controlled trial of CET. Eligibility criteria included: (1) a diagnosis of ASD confirmed by the Autism Diagnostic Observation Schedule-2 (Lord et al., 2000) or the Autism Diagnostic Interview-Revised (Lord, Rutter, & Couteur, 1994), (2) IQ ≥ 80, (3) age 16 to 45 years, and (4) significant social and cognitive disability on the Cognitive Styles and Social Cognition Eligibility Interview (Hogarty et al., 2004), which is a brief assessment of social and cognitive impairment indicative of the need for treatment. This videotaped, interview-based measure assesses three distinct styles of cognitive dysfunction (i.e., unmotivated, disorganized, and inflexible), as well as social-cognitive disability (e.g., interpersonal ineffectiveness, lack of foresight, poor understanding of the social gist). Each cognitive style was assessed with three items (i.e., basic impairment, functional disability, and social handicap associated with that style), and social-cognitive impairment was assessed with five items, all rated on a 5-point severity scale (1 = Rare, 5 = Very Severe). To be eligible for the study, participants were required to score greater than 7 for any single cognitive style and greater than 12 for social-cognitive disability, indicating significant impairment warranting treatment. No participant was excluded due to not meeting these criteria for cognitive and social disability. A broad age range for eligibility was used in this feasibility trial of CET for autism, and individuals 16 to 17 years of age (n = 7) were included to gain experience with those transitioning to adulthood. Exclusion criteria consisted of: (1) significant substance abuse within the 3 months prior to study enrollment, (2) disruptive behavior that would contraindicate participation in a group treatment, (3) untreated attention deficit hyperactivity disorder, (4) persistent homicidal or suicidal behavior, and (5) the presence of a comorbid personality disorder.
Characteristics of enrolled participants are presented in Table 1. Participants were in their early 20s on average (range = 16 to 44 years), the majority were male, and few were employed or living independently at baseline assessment. Although IQ scores were in the normal to above-normal range, cognitive functioning on the MATRICS Consensus Cognitive Battery (Green et al., 2004) was at the 29th (SD = 28.08) percentile, indicating significant cognitive impairment. Approximately half of the sample was receiving psychotropic medications, predominantly antidepressants. Randomized treatment groups were well-matched on most demographic characteristics, as well as IQ and receipt of psychotropic medications. Further, there were no significant differences in changes in receipt of psychotropic medications between those treated with EST and CET, χ2(2, N = 52) = 4.07, p = .131, over the course of the study. However, participants randomly assigned to EST were slightly more likely to have a minority ethnic background and to have attended college. These variables are included as confounding covariates in analyses.
Table 1.
Variable | CET (N = 29) | EST (N = 25) | pb |
---|---|---|---|
| |||
N (%) | N (%) | ||
Age, mean (SD) | 22.55 (6.38) | 23.60 (5.61) | .527 |
Male | 24 (83%) | 23 (92%) | .547 |
White | 27 (93%) | 17 (68%) | .044 |
Attended College | 15 (52%) | 20 (80%) | .060 |
Employed | 7 (24%) | 10 (40%) | .338 |
Living Independentlya | 4 (14%) | 4 (17%) | 1.00 |
IQ, mean (SD) | 108.69 (13.83) | 105.40 (15.71) | .417 |
ADOS Score | |||
Communication | 3.21 (1.08) | 3.52 (1.48) | .374 |
Reciprocal Social Interaction | 6.55 (2.25) | 7.60 (2.89) | .140 |
Stereotyped Behavior | 2.45 (1.40) | 3.08 (1.71) | .142 |
Receiving psychiatric medicationsa | 13 (46%) | 15 (62%) | .379 |
Antidepressants | 13 (46%) | 14 (58%) | .563 |
Mood stabilizers | 1 (4%) | 0 (%) | - |
Anxiolytics | 1 (4%) | 4 (17%) | .261 |
Antipsychotics | 2 (7%) | 1 (4%) | 1.00 |
Note. ADOS = Autism Diagnostic Observation Schedule; CET = Cognitive Enhancement Therapy; EST = Enriched Supportive Therapy
N = 52, as 2 participants (1 in CET, 1 in EST) withdrew prior to completing pre-treatment data collection.
χ2 test or analysis of variance, two-tailed, for significant differences between treatment groups.
P-values are adjusted for multiplicity using Benjamini and Hochberg’s (1995) approach.
Measures
Neurocognition
Assessments of neurocognitive ability consisted primarily of those from the MATRICS Consensus Cognitive Battery (MCCB; Green et al., 2004) developed and validated for clinical trials of cognitive enhancers for patients with schizophrenia (Nuechterlein et al., 2008; Kern et al., 2008). The MCCB is a brief, 60-minute compendium of field standard neuropsychological tests that assesses the domains of processing speed (Trails A [War Department, Adjutant General’s Office, 1944], Brief Assessment of Cognition Symbol Coding and Category Fluency [Keefe, 1999]), attention (Continuous Performance Test-Identical Pairs [Cornblatt et al.,, 1988; Nuechterlein et al., 1986]), verbal and non-verbal working memory (Wechsler Memory Scale-III Spatial and Letter-Number Span [Wechsler, 1987]), verbal learning (Hopkins Verbal Learning Test-Revised [Brandt & Benedict, 2001]), visual learning (Brief Visuospatial Memory Test-Revised [Benedict, 1997]), problem-solving (Neuropsychological Assessment Battery Mazes Task [White & Stern, 2003]), and social cognition (Mayer-Salovey-Caruso Emotional Intelligence Test-Managing Emotions Branch [Mayer et al., 2003]). The Wisconsin Card Sorting Test (Heaton, Chelune, Talley, Kay, & Curtiss, 1993) was also included to assess cognitive flexibility. An overall neurocognitive composite index using these measures was constructed by scaling unadjusted cognitive domain scores from the MCCB (excluding the social cognition domain) and items from the Wisconsin Card Sorting Test to a common (z) metric, reverse-coding relevant tests, and averaging them, such that higher scores reflect greater neurocognitive function. The internal consistency (α= .80) and 9-month re-test reliability of this composite were acceptable (r = .87).
Social cognition
A diverse battery of performance-based, behavioral, and family assessments were used to examine the impact of treatment of social cognition. The Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT; Mayer, Salovey, Caruso, & Sitarenios, 2003) provided performance-based measures of emotion processing and regulation abilities that assesses the domains of emotion perception, facilitation (using emotions to facilitate thought), understanding, and management, which has been extensively validated in the general population (Mayer, Salovey, Caruso, & Sitarenios, 2003; Mayer, Salovey, & Caruso, 2004). The Penn Emotion Recognition Test (Kohler et al., 2003), Penn Emotion Discrimination Task (Erwin et al., 1992), and Penn Emotional Acuity Test (Kohler, Bilker, Hagendoorn, Gur, & Gur, 2000) were also included to expand coverage of facial emotion recognition and discrimination. These three computerized, performance-based tests generally ask participants to identify and match emotional labels to grayscale pictures of human faces, and they have been previously validated in the psychiatric literature (e.g., Kohler et al., 2003; Erwin et al., 1992; Kohler, Bilker, Hagendoorn, Gur, & Gur, 2000). In addition, the Social Cognition Profile (Hogarty et al., 2004) was used to provide a broader behavioral assessment beyond the emotional focus of performance-based measures. This is a 50-item interviewer-rated measure of supportive (e.g., empathic, reciprocal, friendly), tolerant (e.g., accepting, respectful, cooperative), perceptive (e.g., foresightful, self-aware, gistful), and confident (e.g., comfortable, assertive, involved) social-cognitive behaviors gleaned from the social cognition literature (Baldwin, 1992; Brothers, 1990; Selman & Schultz, 1990; Wyer & Srull, 1994) that has been previously validated in studies of CET in schizophrenia (Hogarty et al., 2004; Eack et al., 2009). Because the Social Cognition Profile was originally developed as a clinician-rated measure, blind-raters interviewed both the participant and a collateral (e.g., family member, clinician) to enhance rating accuracy. Finally, a family member completed the Social Responsiveness Scale (Constantino et al., 2003) and the social cognition domain score was used to assess family perceptions of changes in social-cognitive behaviors. A composite index of social-cognitive function was constructed by scaling these measures to a common (z) metric, reverse-coding relevant items, and averaging them, such that higher scores indicate better social cognition. Internal consistency analyses revealed low reliability for the supportive and confident factors on the Social Cognition Profile, and thus these were excluded from the composite. Final internal consistency (α = .76) and 9-month retest reliability estimates for the composite were acceptable (r = .77).
Employment
Competitive employment was assessed using the Major Role Adjustment Inventory, an interview-based assessment of role functioning (Hogarty et al., 1974) that was completed with each participant with ASD by trained interviewers who were blind to treatment assignment. Competitive employment on this measure is defined as full or part-time employment for pay in a non-sheltered, unassisted environment.
Interventions
Cognitive Enhancement Therapy
CET is a comprehensive, developmental approach to the remediation of social and non-social cognitive impairments that was originally developed for patients with schizophrenia (Hogarty et al., 2004). Over the course of 18 months, CET integrates 60 hours of neurocognitive training in attention, memory, and problem-solving with 45 social-cognitive group sessions that aim to facilitate the development of adult social-cognitive milestones (e.g., perspective-taking, cognitive flexibility, social context appraisal). Neurocognitive training is conducted in participant pairs to support socialization and engagement, and utilizes the attention training software of Ben-Yishay, Piasetsky, and Rattok (1985) with memory and problem-solving routines developed by Bracy (1994). CET begins with approximately 3 months of weekly neurocognitive training in attention, after which 6–8 participants come together to form a social-cognitive group. Because of the diverse age of participants in the trial, effort was made to pair participants from similar age groups and to construct social-cognitive groups that were as developmentally homogeneous as possible. A purposely broad array of theoretically-driven social-cognitive abilities are the focus of the group, including perspective-taking, emotion perception and management, social context appraisal, cognitive flexibility, and foresightfulness. Although restricted and repetitive behaviors are not a direct focus of CET, the enhancement of perspective-taking and cognitive flexibility supports the development of broader interests and behavior, and promotes the discussion of shared topics of interest in conversations and other interpersonal encounters. Each group session is structured to contain a welcome back, homework presentation (chaired by one of the group members), in vivo social-cognitive exercise, psychoeducational lecture on a new topic on social cognition, and a homework assignment to facilitate real world application. Social-cognitive group sessions are held weekly for 1.5 hours and conducted concurrently with neurocognitive training throughout the remainder of treatment. Adaptations of CET for adults with ASD have been outlined previously (Eack et al., 2013b) and consisted of replacing psychoeducational content regarding schizophrenia with the latest information on ASD and its impact on cognition and functional outcome in adulthood; muting sounds on select neurocognitive exercises to accommodate those with heightened sensory susceptibility; providing greater clinical outreach and training on the use of the clinician/coach; and using a more guided, repetitive, and elaborated approach to group-based social cognition training that afforded more opportunity for in-group practice. Beyond these revisions to the treatment protocol, surprisingly few adaptations were needed to translate CET from schizophrenia to adults with ASD, as much of the existing neurocognitive and social-cognitive content was perceived to be helpful in our pilot studies (Eack et al., 2013b). Given the overlap in social and non-social cognitive impairment between adults with autism and schizophrenia (Eack et al., 2013a; Couture et al., 2010), as well as the comprehensiveness of domains targeted in CET, the core cognitive components of the intervention appeared to be applicable to both conditions. A complete description of CET for schizophrenia is available in the training manual (Hogarty & Greenwald, 2006), and a supplement describing the application to adults with autism is forthcoming.
Enriched Supportive Therapy
EST is an 18-month, individual condition management and psychoeducation approach based on Personal Therapy (Hogarty, 2002), which implements the established principles of supportive therapy (e.g., active listening, correct empathy, basic psychoeducation) and basic skills training on managing stress. In Phase I, participants meet weekly with their therapist to receive psychoeducation about ASD, learn about the role of stress in their condition, and generate ways to avoid or minimize stress. In Phase II, participants meet every two weeks with their clinician to identify personal cues of distress, enhance awareness of situations most likely to trigger such cues, and to learn how to implement basic coping skills.
EST was selected as an active control condition for this trial to provide a more stringent comparison to CET than usual care, which is frequently non-existent for adults with ASD, and to account for the non-specific effects of CET (e.g., psychoeducation, provision of a skilled, empathic therapist). CET consisted of 3 hours of treatment per week (1 hour of neurocognitive training, 1.5 hours of social-cognitive group, and 30 minutes of individual therapy), usually completed in two days. EST consisted of 1 hour of individual therapy per week. No attempt was made to artificially match hours of treatment, given the different content and focus of the two interventions. All therapists for both treatments were master’s-level trained clinicians with experience with autism, and fidelity was monitored continuously through random review of video and audio recorded sessions. There were no defined roles for family members in either CET or EST; however, family often provided valuable support in facilitating interest in the study, assisting with transportation, reinforcing motivation for attendance, and collaborating with the treatment team to address challenges as they occurred.
Procedures
Participants were recruited for an 18-month trial of CET from the University of Pittsburgh Center for Excellence in Autism Research and the local community. Upon recruitment, individuals were screened for eligibility, with final determinations based on consensus conferences. Eligible participants were randomized by an independent data manager using computer-generated random assignments, treated for 18 months, and assessed every 9 months by trained interviewers and testers who were blind to treatment assignment. All participants provided written, informed consent prior to participation and this study was reviewed and approved annually by the University of Pittsburgh Institutional Review Board. Participants received payment for the completion of study assessments, but were not paid to attend treatment sessions. The study was conducted from August, 2010 to April, 2016. Among randomized and treated participants, retention was high for both CET (78%) and EST (87%), with no significant differences in attrition, χ2(1, N = 54) = .11, p = .741 (see Figure 1).
Data Analysis
The effects of CET and EST on cognitive and employment outcomes were assessed using a series of intent-to-treat analyses with all 54 randomized participants. A sample size of 54 participants, with 40 completing treatment, was determined a priori to be sufficient to detect medium-to-large effect sizes for this first trial. Primary outcomes were neurocognitive and social-cognitive composite indexes. Employment was a secondary outcome. Analyses of cognitive outcomes used linear mixed-effects models with random intercept and slope parameters, where appropriate, examining the effect of treatment assignment (CET or EST), time (0, 9, or 18 months), and their interaction on outcome. The treatment x time interaction was the primary effect of interest. Given group imbalances in ethnic status and education, these were entered as covariates. Missing data were handled using the expectation-maximization approach Dempster, Laird, & Rubin, 1977), with treatment effect sizes calculated based on recommendations for mixed-effects models (Hedges, 2007).
Effects on employment were examined using the same analytic strategy as primary outcomes, but fit using generalized estimating equations due to the binary nature of the outcome. Inference testing was only conducted at the composite level for primary outcomes to avoid inflating Type I error. Effect sizes were calculated for within-composite subdomains to explore efficacy on individual measures. Finally, the longitudinal association between cognitive and employment outcomes was examined with linear mixed-effects models using penalized quasi-likelihood estimation predicting employment status from time-varying scores on each cognitive composite, separately (Singer & Willet, 2003). The mediating effect of changes in cognitive outcomes on CET-related changes in employment was estimated using a mediator-analytic framework for clinical trials (Kraemer, Wilson, Fairburn, & Agras, 2002), with asymptotic z′ tests of statistical significance (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002). All analyses were conducted in R version 3.1.2 (R Development Core Team, 2014).
Results
Main Effects on Neurocognitive and Social-Cognitive Outcomes
We began our analysis by first examining the impact of CET and EST on composite measures of neurocognitive and social-cognitive function in adults with ASD. As can be seen in Figure 2, neurocognitive improvement in CET was significant and medium-to-large in magnitude (d = .76). Participants treated with EST also evidenced some neurocognitive gains, although effect sizes were smaller (d = .31). Direct comparison of the differential effects of CET versus EST on neurocognition indicated a significant advantage for CET, t(78) = 2.55, d = .46, p = .013. With regard to social cognition, participants in CET (d = .89) and EST (d = .62) showed significant improvements over the course of the study (see Figure 2). Examination of differential patterns of social-cognitive change indicated an overall advantage for CET versus EST, F(2, 78) = 2.85, p = .064. However, social cognition effects favoring CET were only significant at 9 months (mid-treatment), t(78) = 2.38, d = .58, p = .020, and by treatment completion both groups displayed improvements in social cognition that were not significantly different in magnitude, t(78) = 1.05, d = .27, p = .298 (see Table 2 for full means and standard errors).
Table 2.
Composite | CET (N = 29) | EST (N = 25) | Analysis | ||
---|---|---|---|---|---|
| |||||
M (SE) | M (SE) | t | p | d | |
Neurocognition | |||||
Baseline | 49.82 (2.29) | 49.68 (2.10) | - | - | - |
Month 9 | 51.94 (2.33) | 52.42 (2.15) | −.40 | .691 | −.07 |
Month 18 | 56.63 (2.35) | 52.39 (2.16) | 2.55 | .013 | .46 |
Social Cognition | |||||
Baseline | 52.31 (2.43) | 49.14 (2.23) | - | - | - |
Month 9 | 59.04 (2.51) | 50.37 (2.33) | 2.38 | .020 | .58 |
Month 18 | 60.70 (2.60) | 54.99 (2.39) | 1.05 | .298 | .27 |
Note. Results are from linear mixed-effects models adjusting for education and racial status. CET = Cognitive Enhancement Therapy; EST = Enriched Supportive Therapy
Analysis of effect sizes on individual cognitive domains indicated a broad neurocognitive advantage for CET, particularly for processing speed and attention (see Figure 3a, Supplemental Table 1). Effect sizes on social-cognitive domains at 18 months revealed that the greatest advantages for CET were in managing emotions, emotional acuity, and tolerant and perceptive social-cognitive behaviors (see Figure 3b, Supplemental Table 2). Conversely, EST was more effective at improving emotion understanding. Family member ratings of social-cognitive improvement were equivalent for both groups.
Main Effects on Competitive Employment
Having found that CET was associated with significant cognitive improvements in adults with ASD, we proceed to examine the degree to which these changes generalized to improvements in competitive employment. As can be seen in Figure 4, while individuals treated with EST were slightly, but not significantly (p = .214) more likely to be competitively employed at baseline, no improvement in their employment was observed over treatment. In contrast, participants treated with CET displayed a rapid and significant differential increase in competitive employment at 9 months (p = .001), which remained significant at 18 months (p = .023). The overall advantage of CET compared to EST on increased likelihood of competitive employment was significant, χ2(2, N = 54) = 10.47, OR = 6.21 [95% CI = 1.29 – 29.99], p = .005, and similar among participants 21 years or older, χ2(2, N = 32) = 9.30, OR = 8.53 [95% CI = 1.34 – 54.40], p = .010.
Examination of the degree to which cognitive improvement contributed to increased competitive employment revealed that neurocognitive change was significantly and positively associated with a greater likelihood of becoming employed (B = .32, p < .001). Similarly, improved social cognition was also significantly associated with increased likelihood of employment (B = .16, p = .003). Mediator analysis indicated that neurocognitive improvement partially, but significantly (z′ = 2.11, p = .009) accounted for the differential advantage of CET in competitive employment. Although 18-month social-cognitive improvement did not mediate CET effects on employment (z′ = .99, p = .270), early 9-month change in social cognition was a significant mediator of increased employment (z′ = 1.88, p = .015).
Discussion
Evidence-based psychosocial interventions for adults living with ASD are remarkably limited (Fitzpatrick, Minshew, & Eack, 2013), with the majority of intervention research in autism focusing on children. Cognitive remediation represents a promising approach to treating cognitive impairments in adult ASD (Keshavan, Vinogradov, Rumsey, Sherrill, & Wagner, 2014; Eack et al., 2013c), which may have considerable functional benefits (Eack et al., 2013b), but comprehensive interventions have yet to be developed and tested in rigorously controlled trials. We conducted an 18-month, randomized-controlled trial of CET (Hogarty & Greenwald, 2006), a neurocognitive and social-cognitive remediation intervention originally developed for patients with schizophrenia (Hogarty et al., 2004), and compared its effects to an active EST condition in verbal adults with ASD.
Overall, findings indicate that CET was effective for enhancing neurocognitive function, primarily in the domains of attention and processing speed, as previously found in schizophrenia (Hogarty et al., 2004; Eack et al., 2015). These improvements were observed by independent testers who were blind to treatment assignment and identified using assessments that were dissimilar from the cognitive exercises on which participants were trained. Furthermore, the increases in processing speed and attention associated with CET generalized to broader functional improvement, as they significantly mediated effects on employment. Many studies have documented neurocognitive impairment in adults with ASD (Murphy et al., 2014; Minshew & Goldstein, 1993; Rosenthal et al., 2013), yet few have attempted to intervene on these core deficits. Such neurocognitive challenges appear to be related to meaningful functional outcomes and should not be overlooked in treatment, even among verbal adults without intellectual disability.
Social-cognitive change was also large in participants treated with CET, but only significantly greater than EST at 9 months. Interestingly, the reduction in differential treatment effect sizes at 18 months (treatment completion) was not due to a lack of improvement in CET, but rather due to an unexpectedly large social-cognitive response among EST-treated participants. While CET showed an advantage in some domains, particularly emotion management and social-cognitive behaviors, such findings underscore the benefits of disorder-relevant supportive therapy focused on stress reduction and psychoeducation for autism. However, the significant advantage of CET for social-cognitive improvement at 9 months suggests that cognitive remediation was effective for producing faster gains in social cognition, and it was these early improvements that mediated later effects on employment.
The increases in employment among CET participants are encouraging, and indicate the potential functional benefits of integrated neurocognitive and social-cognitive remediation for adults with ASD. Cognitive impairment in autism extends to many social and non-social domains, including processing speed, executive function, emotion perception, and theory of mind (Chung, Barch, & Strube, 2014; Minshew & Goldstein, 1993), which likely converge to limit adaptive function. Previous cognitive remediation interventions in ASD have been short-term and commonly targeted isolated aspects of cognition (Fitzpatrick, Minshew, & Eack, 2013), with limited functional impact (Turner-Brown, Perry, Dichter, Bodfish, & Penn, 2008; Vries, Prins, Schmand, & Geurts, 2015). Integrated treatment approaches that target both social and non-social cognitive impairments over a longer period of time may be necessary to achieve meaningful functional results. The high (> 75%) retention rates over 18 months among participants who were not paid for treatment attendance is consistent with our previous CET trials in schizophrenia (Hogarty et al., 2004; Eack et al., 2009), and points to the feasibility of providing longer-term support to adults with autism. Nonetheless, there remains room for improvement in employment gains, and the combination of cognitive remediation with supported employment interventions may represent a promising direction for future research.
Although the results of this study hold important implications for the treatment of adult ASD, they need to be understood in the context of several limitations. First, the study sample size was modest, which led to some imbalances between treatment groups and limited power to detect smaller effect sizes. The sample also consisted, on average, of young adults by design, as we aimed to focus on transitional-age individuals struggling in their early adjustment to adulthood. It is likely that both CET and EST are applicable to and may be effective for older adults with ASD, but the younger age of our sample precludes generalization to older individuals and future studies will need to specifically investigate applicability in older adult samples. Second, performance-based measures of social cognition were limited to those assessing aspects of emotion processing, which may have precluded detecting other social-cognitive effects associated with CET. A confirmatory efficacy trial with expanded social cognition measurement is currently underway to address these limitations and to replicate the current findings. Third, the mechanisms behind CET effects on employment warrant further investigation. Although cognitive improvement mediated CET effects on employment, such effects likely have multiple mechanisms, such as improving work readiness, motivation to seek employment, and job interviewing skills. These work-related factors were not measured in this trial, and it will be important for future studies to examine more precisely how cognitive gains translated into improved employment outcomes in this population. Finally, treatment conditions were not matched with regard to hours of treatment or group modality, which could have influenced outcome. However, EST provided a more stringent comparison than usual care and was selected to account for the most influential non-specific effects of CET (e.g., provision of a skilled empathic therapist, psychoeducation).
Despite the above limitations, findings indicate that CET is a feasible and potentially effective treatment for core cognitive deficits in adults with ASD. The remediation of social and non-social cognitive impairments may provide a new therapeutic opportunity for reducing disability in adults living with autism.
Supplementary Material
Acknowledgments
This work was supported by NIH grants MH-85851 (NJM and SME), MH-95783 (SME), RR-24154 (SME), and HD-55748 (NJM), as well as grants from Autism Speaks (NJM and SME), the Department of Defense (NJM and SME), and the Pennsylvania Department of Health (NJM). We would like to thank Summer A. F. McKnight, M.S.W., Patricia J. McCarroll, M.S.W., Michelle Perrin, M.B.A., and John J. Markiewicz III for their assistance in completing various aspects of this clinical trial, as well as the participants and families who generously volunteered their time to improving the treatment of adults living with autism.
References
- Baldwin MW. Relational schemas and the processing of social information. Psychological Bulletin. 1992;112(3):461–484. [Google Scholar]
- Ben-Yishay Y, Piasetsky EB, Rattok J. A systematic method for ameliorating disorders in basic attention. In: Meir MJ, Benton AL, Diller L, editors. Neuropsychological rehabilitation. New York: Guilford Press; 1985. pp. 165–181. [Google Scholar]
- Benedict RHB. Brief Visuospatial Memory Test—Revised: Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc; 1997. [Google Scholar]
- Bracy OL. PSSCogRehab [computer software] Indianapolis, IN: Psychological Software Services Inc; 1994. [Google Scholar]
- Brandt J, Benedict RHB. The Hopkins Verbal Learning Test—Revised: Professional Manual. Odessa, Fla: Psychological Assessment Resources, Inc; 2001. [Google Scholar]
- Brothers L. The social brain: A project for integrating primate behavior and neurophysiology in a new domain. Concepts in Neuroscience. 1990;1:27–51. [Google Scholar]
- Chawarska K, Macari S, Shic F. Context modulates attention to social scenes in toddlers with autism. Journal of Child Psychology and Psychiatry. 2012;53(8):903–913. doi: 10.1111/j.1469-7610.2012.02538.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chung YS, Barch D, Strube M. A meta-analysis of mentalizing impairments in adults with schizophrenia and autism spectrum disorder. Schizophrenia Bulletin. 2014;40(3):602–616. doi: 10.1093/schbul/sbt048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Constantino JN, Davis SA, Todd RD, Schindler MK, Gross MM, Brophy SL, Metzger LM, Shoushtari CS, Splinter R, Reich W. Validation of a Brief Quantitative Measure of Autistic Traits: Comparison of the Social Responsiveness Scale with the Autism Diagnostic Interview-Revised. Journal of Autism and Developmental Disorders. 2003;33(4):427–433. doi: 10.1023/a:1025014929212. [DOI] [PubMed] [Google Scholar]
- Cornblatt BA, Risch NJ, Faris G, Friedman D, Erlenmeyer-Kimling L. The Continuous Performance Test, identical pairs version (CPT-IP): I. New findings about sustained attention in normal families. Psychiatry Research. 1988;26(2):223–238. doi: 10.1016/0165-1781(88)90076-5. [DOI] [PubMed] [Google Scholar]
- Couture SM, Penn DL, Losh M, Adolphs R, Hurley R, Piven J. Comparison of social cognitive functioning in schizophrenia and high functioning autism: More convergence than divergence. Psychological Medicine. 2010;40(4):569–579. doi: 10.1017/S003329170999078X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- de Vries M, Geurts H. Influence of autism traits and executive functioning on quality of life in children with an autism spectrum disorder. Journal of Autism and Developmental Disorders. 2015;45(9):2734–2743. doi: 10.1007/s10803-015-2438-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data using the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 1977;39(1):1–38. [Google Scholar]
- Eack SM, Bahorik AL, McKnight SAF, Hogarty SS, Greenwald DP, Newhill CE, Phillips ML, Keshavan MS, Minshew NJ. Commonalities in social and non-social cognitive impairments in adults with autism spectrum disorder and schizophrenia. Schizophrenia Research. 2013a;148(1–3):24–28. doi: 10.1016/j.schres.2013.05.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eack SM, Greenwald DP, Hogarty SS, Bahorik AL, Litschge MY, Mazefsky CA, Minshew NJ. Cognitive Enhancement Therapy for adults with autism spectrum disorder: Results of an 18-month feasibility study. Journal of Autism and Developmental Disorders. 2013b;43(12):2866–2877. doi: 10.1007/s10803-013-1834-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eack SM, Bahorik AL, Hogarty SS, Greenwald DP, Litschge MY, Mazefsky CA, Minshew NJ. Is cognitive rehabilitation needed in verbal adults with autism? Insights from initial enrollment in a trial of Cognitive Enhancement Therapy. Journal of Autism and Developmental Disorders. 2013c;43(9):2233–2237. doi: 10.1007/s10803-013-1774-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eack SM, Greenwald DP, Hogarty SS, Cooley SJ, DiBarry AL, Montrose DM, Keshavan MS. Cognitive Enhancement Therapy for early-course schizophrenia: Effects of a two-year randomized controlled trial. Psychiatric Services. 2009;60(11):1468–1476. doi: 10.1176/appi.ps.60.11.1468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eack SM, Hogarty SS, Greenwald DP, Litschge MY, McKnight SAF, Bangalore SS, Pogue-Geile MF, Keshavan MS, Cornelius JR. Cognitive Enhancement Therapy in substance misusing schizophrenia: Results of an 18-month feasibility trial. Schizophrenia Research. 2015;161(2–3):478–483. doi: 10.1016/j.schres.2014.11.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Erwin RJ, Gur RC, Gur RE, Skolnick B, Mawhinney-Hee M, Smailis J. Facial emotion discrimination: I. Task construction and behavioral findings in normal subjects. Psychiatry Research. 1992;42(3):231–240. doi: 10.1016/0165-1781(92)90115-j. [DOI] [PubMed] [Google Scholar]
- Fitzpatrick LB, Minshew NJ, Eack SM. A systematic review of psychosocial interventions for adults with autism spectrum disorders. Journal of Autism and Developmental Disorders. 2013;43(3):687–694. doi: 10.1007/s10803-012-1615-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gilotty L, Kenworthy L, Sirian L, Black DO, Wagner AE. Adaptive skills and executive function in autism spectrum disorders. Child Neuropsychology. 2002;8(4):241–248. doi: 10.1076/chin.8.4.241.13504. [DOI] [PubMed] [Google Scholar]
- Green MF, Nuechterlein KH, Gold JM, Barch DM, Cohen J, Essock S, et al. Approaching a consensus cognitive battery for clinical trials in schizophrenia: The NIMH-MATRICS conference to select cognitive domains and test criteria. Biological Psychiatry. 2004;56(5):301–307. doi: 10.1016/j.biopsych.2004.06.023. [DOI] [PubMed] [Google Scholar]
- Hamilton AFC, Brindley R, Frith U. Visual perspective taking impairment in children with autistic spectrum disorder. Cognition. 2009;113(1):37–44. doi: 10.1016/j.cognition.2009.07.007. [DOI] [PubMed] [Google Scholar]
- Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: A review of behavioral and neuroimaging studies. Neuropsychology Review. 2010;20(3):290–322. doi: 10.1007/s11065-010-9138-6. [DOI] [PubMed] [Google Scholar]
- Heaton RK, Chelune GJ, Talley JL, Kay GG, Curtiss G. Wisconsin Card Sorting Test Manual: Revised and Expanded. Odessa, FL: Psychological Assessment Resources Inc; 1993. [Google Scholar]
- Hedges LV. Effect Sizes in Cluster-Randomized Designs. Journal of Educational and Behavioral Statistics. 2007;32(4):341–370. [Google Scholar]
- Hogarty GE. Personal Therapy for schizophrenia and related disorders: A guide to individualized treatment. New York: Guilford; 2002. [Google Scholar]
- Hogarty GE, Greenwald DP. Cognitive Enhancement Therapy: The Training Manual. University of Pittsburgh Medical Center: Authors; 2006. Available through www.CognitiveEnhancementTherapy.com. [Google Scholar]
- Hogarty GE, Flesher S, Ulrich R, Carter M, Greenwald D, Pogue-Geile M, Keshavan M, Cooley S, DiBarry AL, Garrett A, Parepally H, Zoretich R. Cognitive enhancement therapy for schizophrenia. Effects of a 2-year randomized trial on cognition and behavior. Archives of General Psychiatry. 2004;61(9):866–876. doi: 10.1001/archpsyc.61.9.866. [DOI] [PubMed] [Google Scholar]
- Hogarty GE, Goldberg SC, Schooler NR the Collaborative Study Group. Drug and sociotherapy in the aftercare of schizophrenic patients: III. Adjustment of nonrelapsed patients. Archives of General Psychiatry. 1974b;31(5):609–618. doi: 10.1001/archpsyc.1974.01760170011002. [DOI] [PubMed] [Google Scholar]
- Howlin P, Moss P, Savage S, Rutter M. Social outcomes in mid-to later adulthood among individuals diagnosed with autism and average nonverbal IQ as children. Journal of the American Academy of Child & Adolescent Psychiatry. 2013;52(6):572–581. doi: 10.1016/j.jaac.2013.02.017. [DOI] [PubMed] [Google Scholar]
- Hudepohl MB, Robins DL, King TZ, Henrich CC. The role of emotion perception in adaptive functioning of people with autism spectrum disorders. Autism. 2015;19(1):107–112. doi: 10.1177/1362361313512725. [DOI] [PubMed] [Google Scholar]
- Keefe RSE. Brief Assessment of Cognition in Schizophrenia (BACS) Manual—A, version 2.1. Durham, NC: Duke University Medical Center; 1999. [Google Scholar]
- Kern RS, Nuechterlein KH, Green MF, Baade LE, Fenton WS, Gold JM, Keefe RSE, Mesholam-Gately R, Mintz J, Seidman LJ, Stover E, Marder SR. The MATRICS Consensus Cognitive Battery, Part 2: Co-norming and standardization. American Journal of Psychiatry. 2008;165(2):214–220. doi: 10.1176/appi.ajp.2007.07010043. [DOI] [PubMed] [Google Scholar]
- Keshavan MS, Vinogradov S, Rumsey J, Sherrill J, Wagner A. Cognitive training in mental disorders: Update and future directions. American Journal of Psychiatry. 2014;171(5):510–522. doi: 10.1176/appi.ajp.2013.13081075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kohler CG, Bilker W, Hagendoorn M, Gur RE, Gur RC. Emotion recognition deficit in schizophrenia: association with symptomatology and cognition. Biological Psychiatry. 2000;48(2):127–136. doi: 10.1016/s0006-3223(00)00847-7. [DOI] [PubMed] [Google Scholar]
- Kohler CG, Turner TH, Bilker WB, Brensinger CM, Siegel SJ, Kanes SJ, Gur RE, Gur RC. Facial Emotion Recognition in Schizophrenia: Intensity Effects and Error Pattern. American Journal of Psychiatry. 2003;160(10):1768–1774. doi: 10.1176/appi.ajp.160.10.1768. [DOI] [PubMed] [Google Scholar]
- Kraemer HC, Wilson G, Fairburn CG, Agras W. Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry. 2002;59(10):877–884. doi: 10.1001/archpsyc.59.10.877. [DOI] [PubMed] [Google Scholar]
- Lord C, Risi S, Lambrecht L, Cook EH, Leventhal BL, DiLavore PC, Pickles A, Rutter M. The Autism Diagnostic Observation Schedule—Generic: A Standard Measure of Social and Communication Deficits Associated with the Spectrum of Autism. Journal of Autism and Developmental Disorders. 2000;30(3):205–223. [PubMed] [Google Scholar]
- Lord C, Rutter M, Couteur A. Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders. 1994;24(5):659–685. doi: 10.1007/BF02172145. [DOI] [PubMed] [Google Scholar]
- MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychological Methods. 2002;7(1):83–104. doi: 10.1037/1082-989x.7.1.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Magiati I, Tay XW, Howlin P. Cognitive, language, social and behavioural outcomes in adults with autism spectrum disorders: A systematic review of longitudinal follow-up studies in adulthood. Clinical Psychology Review. 2013;34(1):73–86. doi: 10.1016/j.cpr.2013.11.002. [DOI] [PubMed] [Google Scholar]
- Mayer JD, Salovey P, Caruso DR. Emotional intelligence: Theory, findings, and implications. Psychological Inquiry. 2004;15(3):197–215. [Google Scholar]
- Mayer JD, Salovey P, Caruso DR, Sitarenios G. Measuring emotional intelligence with the MSCEIT V2.0. Emotion. 2003;3(1):97–105. doi: 10.1037/1528-3542.3.1.97. [DOI] [PubMed] [Google Scholar]
- Mayer JD, Salovey P, Caruso DR, Sitarenios G. Measuring emotional intelligence with the MSCEIT V2.0. Emotion. 2003;3(1):97–105. doi: 10.1037/1528-3542.3.1.97. [DOI] [PubMed] [Google Scholar]
- Mazefsky CA. Emotion regulation and emotional distress in autism spectrum disorder: Foundations and considerations for future research. Journal of Autism and Developmental Disorders. 2015;45(11):3405–3408. doi: 10.1007/s10803-015-2602-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Minshew NJ, Goldstein G. Is autism an amnesic disorder? Evidence from the California Verbal Learning Test. Neuropsychology. 1993;7(2):209–216. [Google Scholar]
- Minshew NJ, Williams DL. The new neurobiology of autism. Archives of Neurology. 2007;64(7):945–950. doi: 10.1001/archneur.64.7.945. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murphy CM, Christakou A, Daly EM, Ecker C, Giampietro V, Brammer M, Smith AB, Johnston P, Roberston DM, Murphy DG, Rubia K MRC AIMS Consortium. Abnormal functional activation and maturation of fronto-striato-temporal and cerebellar regions during sustained attention in autism spectrum disorder. American Journal of Psychiatry. 2014;171(10):1107–1116. doi: 10.1176/appi.ajp.2014.12030352. [DOI] [PubMed] [Google Scholar]
- Nuechterlein KH, Edell WS, Norris M, Dawson ME. Attentional vulnerability indicators, thought disorder, and negative symptoms. Schizophrenia Bulletin. 1986;12(3):408–426. doi: 10.1093/schbul/12.3.408. [DOI] [PubMed] [Google Scholar]
- Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, Essock S, Fenton WS, Frese FJ, III, Gold JM, et al. The MATRICS Consensus Cognitive Battery, Part 1: Test Selection, Reliability, and Validity. American Journal of Psychiatry. 2008;165(2):203–213. doi: 10.1176/appi.ajp.2007.07010042. [DOI] [PubMed] [Google Scholar]
- Philip RC, Dauvermann MR, Whalley HC, Baynham K, Lawrie SM, Stanfield AC. A systematic review and meta-analysis of the fMRI investigation of autism spectrum disorders. Neuroscience & Biobehavioral Reviews. 2012;36(2):901–942. doi: 10.1016/j.neubiorev.2011.10.008. [DOI] [PubMed] [Google Scholar]
- Plitt M, Barnes KA, Wallace GL, Kenworthy L, Martin A. Resting-state functional connectivity predicts longitudinal change in autistic traits and adaptive functioning in autism. Proceedings of the National Academy of Sciences. 2015;112(48):E6699. doi: 10.1073/pnas.1510098112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pugliese CE, Anthony L, Strang JF, Dudley K, Wallace GL, Kenworthy L. Increasing adaptive behavior skill deficits from childhood to adolescence in autism spectrum disorder: Role of executive function. Journal of Autism and Developmental Disorders. 2015;45(6):1579–1587. doi: 10.1007/s10803-014-2309-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- R Development Core Team. R: A language and environment for statistical computing (Version 3.2.1) [Computer software] Vienna, Austria: R Foundation for Statistical Computing; 2014. [Google Scholar]
- Rosenthal M, Wallace GL, Lawson R, Wills MC, Dixon E, Yerys BE, Kenworthy L. Impairments in real-world executive function increase from childhood to adolescence in autism spectrum disorders. Neuropsychology. 2013;27(1):13–18. doi: 10.1037/a0031299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Selman RL, Schultz LH. Making a Friend in Youth. Chicago, IL: University of Chicago Press; 1990. [Google Scholar]
- Sikora DM, Vora P, Coury DL, Rosenberg D. Attention-deficit/hyperactivity disorder symptoms, adaptive functioning, and quality of life in children with autism spectrum disorder. Pediatrics. 2012;130(Supplement 2):S91. doi: 10.1542/peds.2012-0900G. [DOI] [PubMed] [Google Scholar]
- Singer JD, Willet JB. Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press; 2003. [Google Scholar]
- Turner-Brown LM, Perry TD, Dichter GS, Bodfish JW, Penn DL. Brief report: feasibility of social cognition and interaction training for adults with high functioning autism. Journal of autism and developmental disorders. 2008;38(9):1777–1784. doi: 10.1007/s10803-008-0545-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vries M, Prins PJ, Schmand BA, Geurts HM. Working memory and cognitive flexibility-training for children with an autism spectrum disorder: A randomized controlled trial. Journal of Child Psychology and Psychiatry. 2015;56(5):566–576. doi: 10.1111/jcpp.12324. [DOI] [PubMed] [Google Scholar]
- War Department, Adjutant General’s Office. Army Individual Test Battery: Manual of Directions and Scoring. Washington, DC: Author; 1944. [Google Scholar]
- Warren Z, McPheeters ML, Sathe N, Foss-Feig JH, Glasser A, Veenstra-VanderWeele J. A systematic review of early intensive intervention for autism spectrum disorders. Pediatrics. 2011;127(5):e1303. doi: 10.1542/peds.2011-0426. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Manual for the Wechsler Memory Scale-Revised. San Antonio, TX: Psychological Corp; 1987. [Google Scholar]
- White T, Stern RA. Neuropsychological Assessment Battery: Psychometric and Technical Manual. Lutz, Fla: Psychological Assessment Resources, Inc; 2003. [Google Scholar]
- Wyer RS, Srull TK, editors. Handbook of social cognition. Vol. 1: Basic Processes. Hillside, NJ: Lawrence Earlbaum Association; 1994. [Google Scholar]
- Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. American Journal of Psychiatry. 2011;168(5):472–485. doi: 10.1176/appi.ajp.2010.10060855. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.