Development of the Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills program for adults on the autism spectrum: Results of initial study

Abstract

The population of adults on the autism spectrum continues to increase, and vocational outcomes are particularly poor. Longitudinal studies of adults with autism spectrum and without intellectual disability have shown consistent and persistent deficits across cognitive, social, and vocational domains, indicating a need for effective treatments of functional disabilities as each impact employment. This initial pilot study is an open trial investigation of the feasibility, acceptability, and initial estimates of outcomes for the newly developed Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills intervention, a manualized “soft skills” curriculum, to enhance both cognitive and social development in adults with autism spectrum. A total of eight adults with autism spectrum, without intellectual disability (78% males), participated in the study. Results support the original hypothesis that adults with autism spectrum can improve both cognitive (i.e. executive functioning) and social cognitive (i.e. social thinking and social communication) abilities. Further Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills was found to be feasible, acceptable, and highly satisfactory for participants and parents. Employment rates more than doubled post-intervention, with an increase from 22% to 56% of participants employed. Conclusion is that Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills has promise as an intervention that can be easily embedded into exiting supported employment vocational training programs to improve cognitive, social, and vocational outcomes.

The increasing population of adults on the autism spectrum (AS) is considered a pressing, challenging public health issue (Bailey, 2012). Longitudinal studies of adults on the AS without intellectual disability have shown consistent and persistent deficits across cognitive, social, and vocational domains, indicating a significant need for effective treatments for these functional disabilities (Howlin, 2000). The cognitive and social skill deficits that are core features of autism have been identified as major challenges to employment success for these adults (Hillier et al., 2007). Such disabilities impact these adults in numerous ways, including the need for functional support as they enter the workforce, giving rise to a critical need for evidence-based methods. Interestingly, recent research suggests that individuals with AS without intellectual disability have even worse employment outcomes than those with intellectual disability (Taylor and Seltzer, 2011), and there is an increasingly identifiable number of younger individuals entering this demographic. Even for those individuals who have post-secondary educational experience, employment difficulties are common (Howlin, 2000). Currently, there are large estimated costs associated with adults with AS, including unemployment and underemployment, but a limited evidence base for understanding what interventions can optimize employment and other life outcomes for these individuals (Taylor et al., 2012). One study found that only 20% of the AS men in their sample held jobs, with 10% working in competitive employment and another 10% working in sheltered employment settings (Cederlund et al., 2008). Similarly, the vast majority of adults with AS continue to be unemployed even with the advent of evidence-based supported employment (Kessler Foundation and National Organization on Disability, 2010; Wagner et al., 2005). Multiple studies have found that supported employment increases work outcomes, skills, and quality of life when used for individuals with AS (García-Villamisar and Hughes, 2007; García-Villamisar et al., 2002; Howlin et al., 2005). Yet, another body of research in this area suggests that supported employment services provided through vocational rehabilitation programs are less than optimal for individuals with AS (Lawer et al., 2009). There are calls for the development of treatment manuals to encourage replication of promising vocational support programs and recommendations to apply interventions that demonstrate effectiveness with other populations to inform the advancement of approaches for individuals with AS (Taylor et al., 2012).

The cognitive executive functioning and social skill deficits that are core features of AS diagnosis have been identified as major challenges to employment success for adults with AS and these skills are referred to as “soft skills” in vocational settings. Soft skills are often described as a cluster of executive functioning and social abilities that make someone a good employee and compatible with coworkers and have been found to predict vocational outcomes. Qualitative studies examining vocational outcomes report that employment success is highly contingent upon social abilities as opposed to completing job duties (Hurlbutt and Chalmers, 2004; Müller et al., 2003). Furthermore, the uneven executive functioning and social abilities that adults with AS display have been found to create problems with both finding and keeping jobs, and result in isolated work opportunities (Hillier et al., 2007; Mawhood and Howlin, 1999). Support programs must address soft skills because both research and experience show that they can be an important indicator of job performance, possibly even more so than job-specific abilities (Hurlbutt and Chalmers, 2004; Müller et al., 2003).

Remediating executive functioning and social deficits in autism

As the neurobiological basis of autism is further investigated (Abrahams and Geschwind, 2008; Minshew and Williams, 2007), new interventions are focusing on remediating the core deficits specific to social and executive dysfunction in AS (Eack et al., 2013). Similar to schizophrenia and bipolar disorder, impairments in information processing and executive functioning are considerable in individuals with AS. It is likely that these impairments are a leading cause of disability in AS, as they are in severe mental illness (SMI; Green et al., 2000). In fact, in a landmark meta-analysis study examining SMI populations, executive dysfunction was shown to explain 20% to 60% of the variance in community and social functioning (Green et al., 2000). There is far less information available to understand the gross impacts these impairments have on adults with AS; however, it is known that the executive functioning deficits in AS, as in SMI, include impaired attention/vigilance, working memory, learning, processing speed, cognitive flexibility, problem-solving, and goal-oriented thinking/planning (Corbett et al., 2009; Grenada et al., 2014; Happé et al., 2006; Hill, 2004; Jolliffe and Baron-Cohen, 2001; Kenworthy et al., 2008; Minshew et al., 1997; Ozonoff, 1995). Specifically, one research review (Hill, 2004) highlights impairments in two known executive functioning deficits, planning and flexibility, while another review examining executive functioning skills in autism (O’Hearn et al., 2008) denoted impairments in response inhibition, working memory, planning, and attention in individuals with AS. Both reviews incorporated studies utilizing neurocognitive batteries that assess skills independently in laboratory settings. More recent studies examining executive functioning as a package of higher order cognitive abilities (i.e. flexibility, problem-solving, planning) have shown much deficit in comparison to typical adults (Zimmerman et al., 2016) and similar deficits to those with attention deficit hyperactivity disorder (ADHD) conditions (Craig et al., 2016). Studies with adolescents and young adults also demonstrate that executive functioning impairments predict poor daily living skills (Pugliese et al., 2016). Yet, executive functioning is an often ignored target for treatment, and there are no known medications that correct most executive functioning deficits of AS. On a positive note, the human brain has extraordinary plasticity (James, 1890), making cognitive executive functioning skills a promising treatment target using cognitive-behavioral methods (Twamley et al., 2012).

Social cognition and social skills are proposed as additional treatment targets because these abilities reflect separable, distinct domains from standard cognitive executive functioning skills (Pinkham and Penn, 2006) and are strongly associated with functional outcomes (Couture et al., 2006). Social cognition involves “Theory of Mind,” which refers to the capacities to understand or infer the thoughts or feelings of others (Baron-Cohen and Belmonte, 2005). It also encompasses the use of social communication such as the ability to understand the meaning or intent of others beyond the literal, concrete meaning of stated words (e.g. irony, metaphors, sarcasm; Stone et al., 2012). Social perception, or the ability to read social cues (Fiske, 1992), and social knowledge which is the knowledge of how to respond in particular social situations and understand what is expected and unexpected (Stone et al., 2012) are additional components of social cognition. It has been well documented that individuals with AS have social cognitive challenges and are in need of direct intervention to develop social skills (Baron-Cohen et al., 2000; Gallese, 2006). These various components of social skills are all included in the definition of vocational soft skills.

Due to the well-known similarities in social and cognitive executive functioning deficits of autism and schizophrenia, interventions for schizophrenia have begun to be adapted and tested for intellectually able adults with autism. To date, Cognitive Enhancement Therapy (CET; Hogarty and Greenwald, 2006) has been piloted with adults with autism in an 18-month feasibility study and has demonstrated positive results (Eack et al., 2013). Additionally, Social Cognition and Interaction Training (SCIT; Roberts et al., 2004) was adapted and pilot tested with adults with AS without intellectual disability, showing improvements in social cognitive and communication skills (Turner-Brown et al., 2008). Although both of these adapted interventions showed promising results for adults with AS, neither were tested nor implemented within existing vocational service settings for adults with AS.

To be disseminated, a program must be feasible, cost-effective, and acceptable to the end users, such as vocational agencies and training centers (Curran et al., 2012). To this end, it is critical that an intervention be developed in a way that attends to the needs of the population and service system from the onset and be tested within community settings. An example of a cognitive enhancement intervention developed and tested within supported employment services for SMI is Compensatory Cognitive Training (CCT; Mendella et al., 2015; Twamley et al., 2008, 2012). CCT teaches compensatory cognitive strategies as a way of improving executive functions by developing new cognitive habits that are needed in work and daily living through group instruction and real-life practice activities. CCT specifically targets prospective memory, attention, learning and memory, and problem-solving. To date, there is no empirical social cognitive intervention developed and tested within supported employment services for SMI or AS populations; however, a recent review of social skills group interventions conducted in the community for adults with AS revealed four different interventions in which two discussed social skills within a vocational setting (Spain and Blainey, 2015). Two interventions, SCIT (discussed above) and Howlin and Yates (1999), taught social cognitive skills as emotional awareness and social problem-solving. However, Howlin and Yates intervention did not follow a manualized intervention. The other two manualized interventions described in the review focused on friendship skills and conversational skills (Program for the Education and Enrichment of Relational Skills (PEERS), Gantman et al., 2012; Aspirations; Hillier et al., 2007, 2011). One additional social cognitive intervention which lacks empirical support but does offer practice-based support for adults is Social Thinking© at Work (Winner and Crooke, 2011). This program targets perspective-taking, social perception, social knowledge, and social communication.

This initial pilot study is the culmination of a 6-month open trial to investigate the feasibility, acceptability, and initial estimates of outcomes for the newly developed Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills (SUCCESS) intervention, a manualized curriculum to enhance both cognitive and social skill development in young adults with autism in a community vocational training setting.

Methods

Development of the integrated SUCCESS intervention

The SUCCESS intervention was developed through an established research-community partnership group—ACHIEVE (Active Collaborative Hub for Individuals with AS to Enhance Vocation and Education). A participatory research process approach was utilized which involved multiple stakeholders from inception through refinement and testing. The ACHIEVE group in Southern California included (at the time of this study) 22 investigators and community stakeholders working together to develop and test sustainable interventions to improve vocational and educational outcomes for transition-age youth and adults with AS. ACHIEVE group members are researchers and representatives from key community stakeholder groups/systems (i.e. educators, developmental disability services, mental health, vocational rehabilitation, community-based service providers, caregivers, and individuals with AS) with the shared goal of improving the careers and quality of life for individuals with AS. The ACHIEVE group, built as a bottom-up grass-roots effort, decided that targeting vocational outcomes for adults with AS was a priority.

Two developmental efforts took place simultaneously: (1) research members of ACHIEVE conducted a comprehensive review of the literature on vocational interventions, and (2) community agency members conducted a needs assessment with local vocational agencies. It became evident from both the literature review and the community discussions that intervening within vocational services was the appropriate setting to target the identified outcomes. Discussions among the group focused on the literature and implementation of an evidence-based supported employment approach to be used in vocational rehabilitation agencies. Both providers and consumers noted that no existing intervention model adequately supported vocational success for adults with AS due to the lack of attention to soft skills necessary in job placements. There was a clear need identified to specifically increase cognitive and social skills.

The consensus to focus on both executive functioning and social cognitive skills led to choosing two models, one cognitive enhancement and one social cognitive, from an extensive literature review. The two models were (1) Cognitive Compensatory Training (CCT) for serious emotional illness (Twamley et al., 2012), and (2) Social Thinking at Work (Winner and Crooke, 2011), a semi-structured “self-help” style curriculum teaching social awareness and social knowledge, and Social Behavior Mapping (Winner, 2007), a template for teaching social awareness and perspective-taking (Crooke et al., 2016). Both CCT and Social Thinking were selected as each had either empirical support (CCT) or practice-based support (Social Thinking) within vocational settings (Taylor, 2011) and appeared to be cost efficient, but neither was sufficient for use with adults with AS within existing community-supported employment settings. Thus, SUCCESS was developed as an integrated curriculum, pulling concepts and vocabulary from CCT and Social Thinking, but adapted significantly to be a sustainable, cost-effective intervention specific to the needs of adults with AS for delivery within existing community-based vocational rehabilitation programs.

SUCCESS is implemented within supported employment by (1) delivering it through small groups within the vocational agency, (2) training employment staff to support the use of strategies within vocational training and/or employment settings, and (3) linking executive functioning and social skills content to the individual’s job search and/or work activities. Small workgroups were formed from the larger ACHIEVE group; one group focused on cognitive enhancement sessions and one group focused on social cognition sessions to create the SUCCESS manualized intervention. Each workgroup included researchers, interventionists, vocational agency staff, parents of adults with AS, and individuals with AS. Each group met twice a month for 6 to 7 months (12 meetings for cognitive enhancement, 15 meetings for social cognition) with members providing input on the curriculum topics, concepts, language, activities, materials, and dosage of the intervention. After two intervention manuals were written, one participant workbook (Baker-Ericzén et al., 2015b) and one facilitator manual (Baker-Ericzén et al., 2015a), the manuals were sent out for external review to seven vocational program leaders: two other California counties (Los Angeles and San Francisco), two other states (Kansas and Michigan), and three other countries (Spain, Canada, and India). Feedback from key stakeholders, both oral and written, was incorporated into the SUCCESS manuals prior to the initial pilot test.

Intervention procedures

The study was approved by the University of California, San Diego, and Rady Children’s Hospital San Diego joint Social/Behavioral Science Institutional Review Board and all participants provided written informed consent. The SUCCESS intervention was designed to target neurocognitive skills specific to executive functioning first and then progress to social cognitions and social skills. SUCCESS was conducted over 25 sessions across a 6-month period, with 1.5 h sessions per week by two facilitators (each facilitating one group). Facilitators received weekly supervision from the program developer (first author) but had no prior training in adult AS. Each SUCCESS group session included a review of the agenda, check-in, assignment review psychoeducation on cognitive or social cognitive topic, teaching of strategies, experiential learning activities, discussion, application activities, and practice assignment (called “Try it and Apply it”). The SUCCESS intervention is comprised of two main sections: cognitive enhancement (executive functioning and memory skills; 13 meetings) and social skills (social cognitive and communication skills; 12 meetings) with each meeting’s concepts threading into subsequent meetings for a comprehensive learning experience. This sequenced approach allowed for the development of the requisite executive functioning skills necessary for social abilities. The cognitive and executive functions targeted include sustained attention, prospective memory (and use of memory aids), organization, “gistful” thinking, cognitive flexibility, maintaining and shifting cognitive sets, goal-oriented thinking and planning, working memory, memory (i.e. encoding and retrieval), problem-solving, and reasoning and logic. The social cognitions targeted include social context appraisal, perspective-taking, social knowledge, social reasoning, reading of social cues (verbal and non-verbal), emotion recognition and management, and social skill competence (i.e. building social relationships, using social networking, initiating meaningful activities). For a summary of SUCCESS curriculum, see Table 1.

Table 1.

SUCCESS curriculum.

Session(s)	Constructs	Content	Strategies
Cognitive enhancement
1	Executive functioning, social cognitive skills	Overview of constructs, explanation of terminology, initial goal-setting	Psychoeducation on the relevant neuroscience Explanation of skills to be developed during course Establish expectations for course Set initial work-related goals
2–3	Prospective memory	Memory for future tasks,a organizational skills	Use of external systemsa as calendars, alarms, reminders, to do lists, set places for item Use of internal systemsa as memory aids, planning ahead, prioritizing, linking tasks together
4–7	Learning and memory	Sustained attention, encoding information,a memory retrievala	Sustained attention strategies as eliminating distractions, sensory input awareness, use of small movements, cognitively tuning in, self-talk,a “mini” breaks Encoding strategies,a including taking notes, categorizing, mnemonics, visuals, focus on gist, associations, over-learning Retrieval strategies involve relaxation (breathing, progressive muscle relaxation), piecing memory together, retracing steps,a checking common places,a associations, recreating the contexta
8–9	Cognitive flexibility and problem-solving	Cognitive shifting within perceptions, communications, thoughts and actions, novel thinking, problem- solving	“Seeing” things differently, “saying/hearing” things differently, “thinking” differently, “doing” things differently Generating novel thoughts, ABC problem-solving, evaluating solutions
10	Goal-orientated thinking	Make a SMART (specific, measurable, attainable, realistic, time-bound) goal, plan prepare, persevere, evaluate	Follow five steps: Set a SMART goal, planning ahead, preparing in advance, try plan and persevere, evaluate and change plans as needed
11–12	Context awareness and appraisal	Context clues,b formulating the big picture, smart guesses,b predicting	Observing and identifying “external” context (setting, situation, non-verbals, verbals), think and remember “internal” context (experience, memory, personality of others), determine contextual relevance, piece together to formulate the full context Use perspective-taking and make smart guesses,b predict occurrences
13	Self-insight and social adjustment	Notice, accept, acknowledge, correct, compensate, prevent mistakes; social impacts	“Repair and recover” strategies to fix mistakes and/or social errors, take action to mend social relationships, and prevent themselves from making that mistake in the future
Social skills
14–15	Social awareness and perspective- taking	Social awareness, make a smart guess,b be self-aware, expected and unexpected actions,b social outcomesb	“Detect and Duplicate”b by observing and replicating social behaviors, analyze self and others’ behaviors, link intent and actions to social outcomes,b evaluate actions as expected or unexpected,b understanding social impactsb
16	Emotional awareness and regulation	Emotional identification, linkage of thoughts–emotions–actions/reactions; compression and coping skillsb	Notice, identify, and rate emotions,b link thoughts–emotions–actions Compress or cope by deep breathing, mental breaks, positive thinking, calming movements, A-B-C problem-solving, cognitive flexibility, communication
17–19	Social communication	Verbal and non-verbal communication,b social interaction types, conversation skills,b filtering thoughtsb	Cognitions and behaviors involved in social communication “Think and Do” model,b formality continuum of social interaction types, using appropriate and verbal and non-verbal skillsb-spacing, body movements and posturing, eye gaze Conversation skillsb as joining, initiating, maintaining, and departing interactions, pauses, switching topics, commenting, question-asking, filtering thoughtsb
20–21	Social knowledge and competencies	Give, receive, and seek: Help, compliments, constructive feedback	Applying verbal and non-verbal skills to receive help, seek help, and give help; offer, accept, and seek compliments; give, receive, and seek constructive feedback and make adjustments
24	Social networking and social media	Social knowledge exchange, social relationships,b social experiences	Social networking strategies include make a mental social file,b collaborate and participate in teamwork, share positives about self, expand conversations, request assistance, maintain a positive image, workplace etiquette, telecommunication
25	Pride, concept review, and integration	Self-achievement, content review, intertwining concepts, future goals	Taking pride in accomplishments, summarizing concepts, integrating strategies, application, formulating and planning for future employment goals

^aVocabulary or concepts from the Compensatory Cognitive Training intervention.

^bVocabulary or concepts borrowed, adapted, and renamed from the Social Thinking^® methodology.

The participants were split into two groups (n = 4 per group). Each participant received a workbook with the 25 sessions given to them one at a time, one per week. The participants were guided to read along and complete written activities in their workbooks. The facilitator played an important role in delivering the curriculum in an engaging and individualized manner, monitoring comprehension, adjusting pacing to the group, and leading rich discussions targeting generalization of skills to immediate and future work settings.

Data were collected at baseline and post-intervention (6 months post-baseline) by adult participants and one parent/caregiver that reported interacting regularly with the participant.

Setting and participants

SUCCESS implementation was conducted within a specialized supported employment program focused on teaching adults with AS technical skills specific to software testing called the National Foundation for Autism Research Technical Skills (NFAR Tech) program. The program enrolls adults with AS without intellectual disability interested in careers in the technological field of software testing. Eligible adults (18+ years of age) had a clinical/medical diagnosis of AS made by a community professional (trained psychiatrist, psychologist, or school psychologist) and reported on by both the participant and parent. Inclusion criteria for the study were consistent with the enrollment criteria of the program which included age 18+, diagnosis of AS, verbal in English language (fluent conversational speech), and without intellectual disability (no intellectual disability on record). The participants participated in the supported vocational training program for the entire 6 months of the SUCCESS program. The vocational program involved teaching the technical skills involved in software testing within a simulated work environment. Nine adults originally enrolled in the program and eight of the nine completed all SUCCESS sessions. One subject voluntarily exited the program prior to completion. A total of eight parents completed baseline and post-assessment measures about their adult child participant.

Study participants were young adults with AS with an average age of 22.44 (standard deviation (SD) = 3.55), ranging from 18 to 29 years. The majority were males (78%) and Caucasian (75%). All were high school graduates (100%), with some college attendance (78%) and a few college graduates (22%). A third had drivers’ license (33%) and about half traveled independently on public transportation (44%). Some participants were concurrently involved in disability services (33%) and/or Department of Rehabilitation (22%) but none of the participants received Social Security Income (0%). A total of 75% of the participants met criteria for current clinical impairment on a common autism diagnostic assessment measure (Social Responsiveness Scale–2 (SRS-2) t-score ≥60; Constantino et al., 2003). A few participants were employed part-time at the start of study (22%) or previously (33%). No participant had current or previous full-time employment. Parents were mostly mothers (75%) with some fathers (25%) completed the assessment forms.

Measures

Executive functioning skills

Delis–Kaplan Executive Functioning System

The Delis–Kaplan Executive Functioning System (D-KEFS) consists of executive function tests that assess a broad range of higher level cognitive skills (Delis et al., 2001). There are a total of nine subtests. Each subtest yields multiple scores; however, one executive functioning domain score was pulled from each test for this study. The following were used: (1) cognitive flexibility (switching condition of Trail Making test), (2) verbal fluency (category switching condition), (3) design fluency (switching condition), (4) inhibition (inhibition/switching condition of Color-Word Interference test), (5) problem-solving (sort recognition description and confirmed correct sorts of Sorting test), (6) categorical processing (weighted achievement score of Twenty Questions test), (7) deductive reasoning (total consecutively correct of Word Context test), (8) planning (total achievement score of Tower test), and (9) verbal abstraction (total achievement score of Proverb test). Test–retest reliability across subtest scores is between 0.06 and 0.90. The internal consistency across subtests is between 0.33 and 0.90 (Delis et al., 2001). Higher scores indicate more executive functioning ability.

Behavior Rating Inventory of Executive Function–Adult

The Behavior Rating Inventory of Executive Function–Adult (BRIEF-A) is an assessment of executive function behaviors at home and work for adults (ages 18–65; Gioia et al., 2000; Roth et al., 2005). It is an 86-item measure with eight clinical scales (Inhibit, Shift, Emotional Control, Initiate, Working Memory, Plan/Organize, Organization of Materials, Monitor) and two validity scales (Inconsistency and Negativity). There are two broader Indexes (Behavioral Regulation and Metacognition) and an overall total score, the Global Executive Composite. High internal consistency (0.80–0.98) and test–retest reliability (0.82 for parents) were found along with established validity with other executive functioning measures (0.73–0.84; Roth et al., 2005). Normed t-scores were used for analyses. Higher scores indicate more executive dysfunction. T-scores of 65 or higher are categorized as clinically significant.

Social cognitive and communication skills

SRS-2

The SRS-2 is a 65-item self- and parent- or other-report Likert scale that objectively measures social impairment (Constantino and Gruber, 2012). It reports t-scores in five subdomains: social awareness, social cognition, social communication, social motivation and restricted repetitive behaviors, and a total t-score. It has established psychometric properties. The psychometric properties are stable and strong (Constantino and Gruber, 2012). Normed t-scores were used for analyses. Higher scores indicate greater autism-related social impairment. A clinical cutoff score of impairment is ≥60 (mild to moderate) and >76 (severe).

Social Skills Performance Assessment

The Social Skills Performance Assessment (SSPA; Patterson et al., 2001) is a performance-based measure of social skills requiring role-plays of neutral and conflictual social situations. This version, adapted for the current study, included two additional scenes related to a vocational setting and additional coding schemes for the AS population (Baker-Ericzén et al., 2015c). The administration involved role-playing social interaction scenes with the examiner. After a 1-min practice period, the participant was requested to initiate and maintain social conversations for 3 min per scene. The two new scenes used in this study were scene 3—a “water cooler” conversation with a coworker and scene 4—speaking with a supervisor to request time off for a personal reason. The sessions were video-taped and scored by trained raters. Ratings are scored on a 5-point Likert scale in dimensions of social skills, including fluency, clarity, focus, social interest, negotiation ability, persistence, social appropriateness, overall conversation/argument (original) and eye gaze, body language (posture, spacing, gesturing), intonation/pattern of speech, facial expression, reading social cues, and perspective-taking and connectedness (additional codes). Two coders were trained to reliability and each coded all role-plays at both time points: baseline (intraclass correlation coefficient (ICC) = 0.82) and post-treatment (ICC = 0.87). Higher scores on items indicate better social skills.

Daily living and vocational skills

Functional Daily Living Questionnaire

Functional Daily Living Questionnaire (Baker-Ericzén et al., 2015f) was developed by the ACHIEVE group for this study to track daily living skills appropriate for this sample of adults with AS. The questionnaire was completed by participants and parents and included responses indicating frequency of occurrence of a number of daily living tasks. There are 14 items about the following types of tasks: hygiene (showering and teeth brushing), cooking, finances (money management), household, and vocational. Each question asked to rate on the frequency of occurrence without reminders. Items used a 5-point Likert scale (ranging from 1 = never to 5 = always). Mean scores were reported for each item. Higher scores indicate better functioning.

Employment Interview

Employment Interview (Baker-Ericzén et al., 2015d) was developed for this study by the ACHIEVE group and included questions regarding previous and current job attainment, work hours, days worked, wages, and job duties. Part-time work was calculated as ≤34 h a week and full-time was calculated as ≥35 h/week.

Program adherence and satisfaction

Program adherence measures included attendance logs, SUCCESS in-session work completion, homework assignment completion (weekly assigned Try it and Apply it), and kept scheduled work-specific appointments. These scores were calculated as percent of total.

Satisfaction questionnaires (Baker-Ericzén et al., 2015g) were developed for this study. The Participant Satisfaction Questionnaire included seven items using a 10-point Likert scale (10 = excellent). Participants rated on their satisfaction with the SUCCESS program overall and the cognitive and social cognitive curriculum separately. Parents rated on their overall satisfaction with the SUCCESS program and the Technical Skills program separately on a 5-point scale (5 = very satisfied, 3 = neutral, 1 = very dissatisfied). Only overall ratings are reported. The questionnaire also included open-ended questions prompting for both positive and negative remarks. Representative quotes are reported.

Results

Treatment feasibility, adherence, and acceptability

The primary goal of this study was to assess the feasibility, adherence, and acceptability of the newly developed SUCCESS intervention within an existing community vocational program for adults with AS. All nine participants were recruited into the SUCCESS program and each of the 25 SUCCESS sessions were delivered to fidelity in the time allotted. The curriculum melded into the existing technical skills program with ease. Of the nine participants enrolled, eight completed the entire 6-month program. The one participant who dropped from the program after the sixth group was not interested in learning the technical skills of the vocational program, and therefore dropped from the entire program. Group attendance levels were excellent (63% perfect attendance of all 25 group sessions (n = 5), two participants missed one group due to illness and one participant missed three groups due to vacation). Treatment adherence, defined by completing both in-session and homework assignments and work appointments, was high overall. In-session assignment completion was 100% for all participants. Homework assignment completion was also high (mean = 88% completion; range = 71%–100%) and was high for both the neurocognitive (mean = 88%; range = 71%–100%) and social cognitive (mean = 86%; range = 60%–100%) sessions. The mean percent of completed work appointments was lower at 78% (range = 38%–100%). Three participants really struggled with this skill through the first half of the sessions missing the appointment each week for the first 10 weeks, but then began to keep the appointments when additional incentives were added (i.e. group reward, group accountability, and positive peer support). In addition, SUCCESS group satisfaction was high for both participant and parent raters. Overall satisfaction scores were 8.13 (SD = 1.89) out of 10 for participants. The satisfaction for the cognitive enhancement curriculum was 7.25 (SD = 2.49) and 8.13 (SD = 1.89) for the social cognitive curriculum. Parents reported overall satisfaction with the SUCCESS program 4.00 (SD = 0.76) out of 5 and 3.63 (SD = 1.19) out of 5 for the technical skills training. The open-ended questions asking about positive changes noted increases in soft skills, life skills, and social life, specifically, independence, confidence, positive mood, interest, and effort toward employment, for both participants and parents. Refer to Table 2 for a sample of participant and parent quotes.

Table 2.

Qualitative feedback from participants and parents about the SUCCESS program.

Participant

Parent

“It gives [me] more motivation to learn new things and helps me improve my skills.” “I loved it and want to refer back to it when I need to.” “The course was amazing. It makes me very proud.” “Learning to stay focused and not let negativity get to me” “Very helpful.” “It was fairly good. Ok, quite good.” “The best part is managing your emotions and remaining focused on the topic.”

“Happier and more confident. He takes responsibility for himself and the home.” “Calmer, happier, more responsive and communicating more.” “He is more determined to perform independent tasks in daily life and school. He’s planning ahead and using organizational systems to help himself with assignments and deadlines.” “Brighter outlook. More sense of connection to others.” “More mature and has a more positive attitude.” “Greater ease in social situations. Great level of commitment towards obtaining a job. More communicative.” “She is making a conscious effort to be more organized and independent.” “He’s working towards being more responsible and accountable.” “Now he wants to get a job!”

SUCCESS: Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills.

Preliminary outcomes

Paired-samples t-tests were conducted on the baseline and post-intervention scores for each skill domain and sub-scales to determine if differences were statistically significant. Chi-square analysis was completed on the percentage of the sample working at baseline and post. However, due to the small sample size of this pilot study, significance was not expected so within-group effect sizes (Cohen’s d) were calculated for all scores. Because this was a pilot study to assess the potential value of the intervention with a small community sample, the Type I error rate was not adjusted for the number of comparisons and was kept at p< 0.05 level for each comparison.

Cognitive executive functioning skills

As can be seen in Table 3, participants reported increased executive functioning skills on the BRIEF-A global composite score (p = 0.018, d = 1.18), behavioral regulation index (p = 0.069, d = 0.90), and a number of subscales: self-monitor, initiate, working memory, plan/organize, and task monitor, all with moderate to large effects (d = 0.46–1.52). Parents did not report improvements on this measure.

Table 3.

Univariate effects of SUCCESS program on participant- and parent-reported cognitive and social functioning of adults with AS.

n = 8	Participant							Parent
	Baseline		Post		Analysis			Baseline		Post		Analysis
	M	SD	M	SD	t	p	d	M	SD	M	SD	t	p	d
Executive functioning
BRIEF-A
Inhibit	56.75	15.30	52.00	11.44	1.38	0.21	0.53	47.63	7.43	48.38	8.56	−0.30	0.78	0.11
Shift	61.75	13.84	59.00	15.33	1.41	0.20	0.52	56.50	13.23	57.13	13.76	−0.20	0.85	0.07
Emotional control	53.50	10.99	53.38	10.57	0.06	0.95	0.02	48.75	5.04	47.13	7.20	0.50	0.64	0.18
Self-monitor	58.75	14.67	51.63	12.06	3.01	0.02*	1.15	49.88	7.92	48.38	10.14	0.53	0.61	0.19
Initiate	59.88	15.77	56.68	14.44	1.28	0.24	0.46	56.25	16.44	60.63	17.54	−1.64	0.15	0.59
Working memory	64.38	15.96	61.63	16.78	1.41	0.20	0.50	55.75	13.91	59.00	13.89	−1.55	0.17	0.55
Plan/organize	64.38	16.11	58.50	15.58	4.21	0.00**	1.52	56.38	15.62	62.38	12.82	−2.04	0.08	0.76
Task monitor	60.13	17.91	57.63	16.84	3.42	0.01*	1.44	55.63	12.83	57.00	14.31	−0.52	0.61	0.19
Organization of materials	53.50	13.85	52.63	15.10	0.58	0.58	0.21	49.88	12.10	53.38	8.98	−2.30	0.06	1.16
Behavioral regulation index	58.25	15.15	54.63	12.49	2.15	0.07	0.90	49.63	7.17	49.00	8.96	0.20	0.84	0.07
Metacognition index	62.38	17.62	60.75	17.68	0.76	0.47	0.27	55.38	16.73	59.50	13.97	−1.88	0.10	0.73
Global executive composite	61.38	16.71	57.50	15.40	3.08	0.02*	1.18	53.13	12.65	55.13	11.78	−0.79	0.46	0.28
Social cognition and social functioning
SRS-2
Social awareness	54.50	12.13	48.75	10.50	2.56	0.04*	0.93	59.38	7.44	57.00	8.09	0.69	0.51	0.25
Social cognition	58.63	9.92	57.63	11.60	0.55	0.60	0.21	61.63	9.43	59.75	12.13	1.32	0.23	0.62
Social communication	55.63	10.35	55.13	12.06	0.20	0.85	0.07	63.25	9.00	60.63	10.85	1.86	0.11	0.74
Social motivation	58.63	11.04	54.88	12.12	2.43	0.05*	0.89	63.25	12.23	60.13	13.67	1.45	0.19	0.53
Restricted and repetitive behavior domain total	57.13	13.13	59.13	10.45	−1.01	0.34	−0.36	65.50	10.17	64.00	11.80	1.47	0.18	0.62
Social communication and interaction domain total	57.50	10.45	55.25	12.06	1.18	0.28	0.44	63.50	8.83	60.63	11.76	1.90	0.10	0.91
SRS-2 total	57.63	11.21	55.88	12.62	0.99	0.35	0.37	64.13	8.85	62.13	11.64	1.39	0.21	0.67

SUCCESS: Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills; AS: autism spectrum; SD: standard deviation; BRIEF-A: Behavior Rating Inventory of Executive Functioning–Adult Version; SRS-2: Social Responsiveness Scale–2.

Lower scores on both the BRIEF-A and the SRS-2 indicate better performance.

^*p < 0.05;

^**p < 0.01.

According to the performance-based measure, D-KEFS, four out of nine subtests showed improvements: inhibition, problem-solving, deductive reasoning, and planning with large effect sizes (d = 0.72–1.30), and significance ranging from p = 0.09 to 0.01 (see Table 4 for data).

Table 4.

Univariate effects of SUCCESS program on performance-based executive functioning of adults with AS.

n = 8	Baseline		Post		Analysis
	M	SD	M	SD	T	p	d
D-KEFS
Cognitive flexibility	8.63	3.34	9.25	3.69	−0.76	0.47	0.27
Verbal fluency	9.13	3.83	9.13	3.72	0.00	1.0	0
Design fluency	11.00	4.44	10.00	4.41	1.60	0.15	0.56
Inhibition	8.25	4.20	9.75	3.45	−1.93	0.09	0.72
Problem-solving
Sorting (confirmed correct sorts)	10.75	2.19	12.50	3.20	−2.97	0.02*	1.30
Sorting (description)	11.00	2.20	12.75	3.54	−2.50	0.04*	1.16
Categorical processing	10.25	2.55	10.88	2.70	−0.41	0.69	0.14
Deductive reasoning	8.13	3.40	10.88	3.83	−2.75	0.01*	1.09
Planning	8.50	4.07	10.75	3.58	−2.30	0.05*	0.82
Verbal abstraction	7.50	4.38	8.25	5.18	−0.75	0.47	0.27

SUCCESS: Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills; AS: autism spectrum; SD: standard deviation; D-KEFS: Delis–Kaplan Executive Functioning System.

Higher scores indicate better performance.

^*p < 0.05.

Social cognitive and communication skills

Both participants and parents report increases in social cognitive and social communication skills on the SRS-2 in the areas of social awareness, social motivation, and the social communication and interaction domain total with moderate to large effect sizes (d = 0.44–0.93), and statistical significance p < 0.05 for social awareness and social motivation (participant report). Parents also reported improvements in total score (d = 0.67) and sub-scales of social cognition, social motivation, social communication, and the social communication and interaction domain total (d = 0.53–0.91), but without statistical significance (refer to Table 3).

On the social skills performance-based measure, SSPA 2.0, participants demonstrated meaningful improvements indicated by moderate to large effect sizes on 12 out of the 15 ratings across both vocational scenes. Scores improved for clarity, overall conversation, social appropriateness, body language, facial expression, reading social cues, perspective-taking, connectedness, negotiation ability, and submissive/persistence (d = 0.53–1.83), and statistical significance ranging from p < 0.01 to p = 0.08. The fluency rating also significantly improved in the “chat with a coworker” scene (p = 0.021, d = −1.04; refer to Table 5).

Table 5.

Univariate effects of SUCCESS program on performance-based social skills of adults with AS.

n = 8	Chat with a coworker							Request time off from work
	Baseline		Post		Analysis			Baseline		Post		Analysis
	M	SD	M	SD	t	p	d	M	SD	M	SD	t	p	d
Interest level	4.88	0.35	4.77	0.71	1.00	0.35	0.00	4.88	0.35	4.75	0.71	1.00	0.35	0.00
Fluency	2.63	0.74	3.50	0.76	−2.97	0.02*	1.04	3.38	1.06	3.5	0.93	−0.26	0.80	0.09
Clarity	3.75	0.71	4.38	0.74	−3.42	0.01*	1.02	4.0	0.93	4.5	0.53	−2.65	0.03*	1.32
Focus	3.88	0.99	4.50	0.93	−1.11	0.31	0.39	4.0	0.93	4.75	0.46	−2.05	0.08	1.08
Overall conversation/argument	2.75	0.89	3.75	0.46	−2.65	0.03*	0.97	3.25	1.04	4.25	0.71	−2.65	0.03*	0.96
Social appropriateness	3.63	1.06	5.00	0.00	−3.67	0.01**	1.29	3.13	1.13	4.13	0.83	−1.60	0.16	1.60
Eye contact	3.50	1.41	4.13	0.64	−1.49	0.18	0.47	3.5	1.20	4.13	0.64	−1.49	0.18	1.62
Intonation and pattern of speech	3.25	1.16	3.88	0.99	−1.67	0.14	0.46	3.13	1.13	3.75	0.89	−1.26	0.25	0.45
Body language	3.00	1.07	4.25	0.71	−2.76	0.03*	1.40	3.13	0.99	4.0	0.76	−2.50	0.04*	0.89
Facial expression	3.63	1.06	4.38	0.74	−2.39	0.05*	0.89	3.38	0.92	4.5	0.53	−2.55	0.04*	1.29
Reading social cues	3.25	1.04	4.25	0.71	−2.37	0.05	0.85	3.5	1.07	4.5	0.53	−2.65	0.03*	1.02
Perspective-taking	3.63	0.92	4.25	1.16	−1.50	0.18	0.53	3.0	0.93	4.25	0.71	−3.42	0.01*	1.22
Connectedness	2.88	0.83	4.13	0.64	−5.00	0.00**	1.83	3.13	0.83	4.13	0.64	−2.65	0.03*	0.95
Negotiation abilitya	-		-		-			3.13	1.25	4.00	0.93	−2.50	0.04*	0.79
Submissiveness/persistencea	-		-		-			3.63	1.30	4.25	1.04	−1.93	0.09	0.53

SUCCESS: Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills; AS: autism spectrum; SD: standard deviation. Scores range from 1 to 5, with higher scores indicating better performance.

^aNot applicable to the chat with a coworker scene.

^*p < 0.05;

^**p < 0.01.

Daily living and vocational skills

Parents reported increased hygiene specific to showering, teeth brushing, and dressing in professional attire with moderate to large effect sizes (d = 0.60–0.79), but no statistical significance. Participants reported small to no effects in daily living skills with the exception of those related to work (work attendance, requesting time off, attending appointments; refer to Table 6).

Table 6.

Univariate effects of SUCCESS program on adaptive functioning of adults with AS.

n = 8	Participant							Parent
	Baseline		Post		Analysis			Baseline		Post		Analysis
	M	SD	M	SD	t	p	d	M	SD	M	SD	t	p	d
Hygiene
Showers consistently	4.63	0.74	3.88	1.13	2.05	0.08	0.76	4.14	1.64	4.43	1.13	−1.55	0.17	0.79
Brushes teeth regularly	4.75	0.46	4.63	0.74	0.55	0.60	0.18	4.00	1.41	4.29	1.50	−1.55	0.17	0.60
Cooking
Prepares meals for self	3.63	0.92	4.00	1.07	−1.00	0.35	0.37	3.50	1.07	3.38	1.06	0.42	0.69	0.11
Prepares meals for others	2.50	1.31	3.13	0.99	−0.92	0.39	0.54	2.38	1.41	2.75	1.58	−1.00	0.35	0.35
Finances
Manages own finances	2.75	0.89	3.25	1.28	−0.94	0.38	0.45	2.63	1.69	2.75	1.75	−0.24	0.82	0.08
Household
Wakes up with an alarm	3.25	1.49	3.25	1.58	0.00	1.00	0.00	3.57	1.51	3.71	1.60	−0.55	0.60	0.09
Gets ready for the day	4.88	0.35	4.63	0.52	1.52	0.17	0.56	4.43	0.79	4.29	1.11	0.35	0.74	0.15
Completes chores	3.50	0.93	3.50	0.54	0.000	1.00	0.00	3.50	0.97	3.38	0.92	0.36	0.73	0.13
Runs errands	3.13	1.64	3.25	1.17	−0.28	0.79	0.10	3.13	1.25	3.13	1.64	0.00	1.00	0.00
Schedules appointments	3.50	1.07	2.75	1.17	3.00	0.02*	0.67	3.00	1.07	2.88	1.13	0.42	0.69	0.11
Attends scheduled appointmentsa	3.75	1.04	3.63	0.92	0.42	0.69	0.12	-	-	-	-
Vocational
Follows workplace dress expectations	4.50	0.54	4.63	0.52	−0.36	0.73	0.13	4.43	0.79	4.71	0.49	−1.55	0.172	0.70
Clocks in and outa	4.13	1.36	4.13	1.25	0.00	1.00	0.00	-	-	-	-
Requests time offa	3.00	1.41	3.75	1.39	−1.16	0.29	0.54	-	-	-	-

SUCCESS: Supported Employment, Comprehensive Cognitive Enhancement, and Social Skills; AS: autism spectrum; SD: standard deviation.

Scores range from 1 to 5. Higher values indicate higher frequency of behavior.

^aItem was administered only to participants.

^*p < 0.05.

Two of the nine participants (22%) reported paid part-time work at baseline with a mean of 6 h of work per week (range = 2–10). A total of five participants (56%) reported paid work post-intervention, with a mean of 20.2 h/week (range = 10–40). This is 34% difference (χ²(1) = 1.833, p = 0.18). The salaries of the working participants at post-treatment ranged from US$10 to US$18 an hour. One participant kept the same job from baseline to post-intervention, however increased both the number of hours worked and salary received at post-intervention.

Discussion

Results of this pilot study support the original hypothesis that adults with AS can improve both cognitive executive functioning and social cognitive (i.e. social thinking and social skills) abilities within a supported vocational employment setting using a manualized intervention program, SUCCESS. Furthermore, the structured implementation of the newly developed SUCCESS intervention was found to be feasible, acceptable, and highly satisfactory for adults with AS. Voluntary attendance and participation rates during SUCCESS intervention were high, and homework completion rates were acceptable. Both participants and parents reported high rates of satisfaction and qualitatively described improvements in confidence, mood, cognitive skills, social skills, daily living skills, and vocational skills. Many also reported increased interests in forming social relationships and using social networking toward career building. Employment rates more than doubled post-intervention, with an increase from 22% to 56% of participants employed. There was also a large increase in the mean number of hours worked per week (from 6 to 20 h a week) with individuals receiving competitive wages (US$10–US$18 an hour).

Results further revealed improvements on self-report, parent-report, and performance-based measures of neurocognitive executive functioning and social cognitive skills. Improvements were found across measures, with some variability by reporter and subscale, but with gains in various areas across both executive functioning and social abilities. These findings are particularly of interest as the sample size was small, with eight participants completing baseline and post-intervention assessments. Many subscales showed moderate to large effect sizes across the four measures. Findings of increasing cognitive and social cognitive functioning with newly developed interventions for adults are being demonstrated across a few other recent studies, for example, CET for Adults with AS (Eack et al., 2013) and SCIT for adults with AS (Turner-Brown et al., 2008). However, this is the first study to investigate a skill-based intervention targeting both neurocognitive and social cognitive abilities within an existing community-based vocational program. The initial outcomes of the SUCCESS intervention are encouraging, suggesting that it may be an efficacious and cost-effective intervention that can be implemented within existing vocational rehabilitation settings.

These findings are important in further understanding the needs of young adults with AS and without intellectual disability. At the onset of the study, most participants were unemployed and/or lacked exposure to work environments (through volunteer or internship placements). Yet, all were motivated and devoted to learning new skills and developing a vocational career. They committed to spending substantial time in this training program, attending 9 h/week over 6 months on site, often with an additional 2 to 3 h of independent study (including technical skills and SUCCESS assignments) each week. They reported enjoyment of the simulated work environment and opportunities for social networking. They appreciated the structure of the program and reported disappointment when the program ended. Anecdotal reports from participants revealed many formed friendships that continued after the program and many showed interest in “stopping by” the program to check in and socialize with others. To this end, the NFAR Tech program is continuing to expand its services and is moving toward offering drop-in social and technical support groups based on this feedback and the requests of both parents and participants for continuation of the collegial environment offered by the program. Although the program aimed to offer a cost-effective service within 6 months, it may not be a long enough duration for sustainable skill development and support. Eack et al. (2013) found significant increases in skills from 9 to 18 months in Cognitive Enhancement Therapy for adults with AS.

Another set of interesting anecdotal reports came from program staff and outside observers. Multiple individuals commented on the comprehensive nature and visual appeal of the SUCCESS curriculum. Vocational program staff were immediately interested in implementing it upon viewing the participant workbook. Public agency administrators (i.e. rehabilitation agencies) commented that the curriculum was a good fit for their services and showed interest in funding participants to receive it. The vocational program involved in piloting SUCCESS requested it be embedded into their program and it currently is.

The positive findings of this study and further implications must be taken in light of the limitations. The results should be interpreted with caution due to both the small sample size and open trial design in which there was no control group. Additionally, many statistical tests were conducted without adjusting for Type 1 errors. The small sample is appropriate for a first feasibility trial but does not allow for inferences to be made on the intervention effectiveness or generalizability. Another study is currently underway testing the SUCCESS intervention in a pilot randomized controlled trial to further investigate the effectiveness of the intervention with adults with AS in community settings. Second, the repeated nature of the performance-based tests could have introduced practice effects between baseline and post-intervention which occurred about 6 months apart; however, the magnitude of change was large enough to extend beyond what is usually accounted for by repeated assessment. To maximize objective coding on the performance-based assessments, the coders of the SSPA and D-KEFS had no relationship with the participant at the time of assessment and were the same rater at baseline and post-intervention. Third, the limited resources available for this study did not allow diagnostic confirmation of AS. Thus, it used an ecologically valid eligibility assessment utilizing community-based assessment for diagnosis of AS.

An overall strength of the SUCCESS program is that it is designed to be used within community-supported employment programs; however, in doing so, it is difficult to ascertain whether the signals of positive outcomes revealed in this pilot study are due to the SUCCESS curriculum. Future study should involve multi-arm research designs and sample sizes large enough to investigate the components of the intervention as well as the combined model (Parmar et al., 2014). Curran et al. (2012) encourages researchers to use “hybrid effectiveness-implementation” methods to speed up translational gains by allowing for more effective implementation strategies and providing valuable information that service systems can use. This pilot study displayed many similarities to a hybrid design by attending to the needs of the service setting at the outset.

Another limitation was the overall lack of awareness of participants and parents of executive functioning skills. Both parents and participants often commented that they had difficulty reporting on such items as on the BRIEF-A due to not understanding the question or not being aware of the behaviors being asked about. This population had rarely (if ever) been asked about executive functioning skills and did not appear familiar with the terminology. This may be one reason why the mean scores at both baseline and post-intervention were lower than reported in other AS samples for youth and adults (Christ et al., 2010; Stichter et al., 2010). Additionally, the results of a worsening effect from parent-report may be a reflection of the increase in awareness and observations of their adult child’s lack of skills as a result of the line of questioning from the baseline assessment. The overall reports of executive functioning skills by parents and participants were in stark contrast to observations from group facilitators and study staff and to the poor performances found on the D-KEFS. The baseline assessment appeared to prime parents and participants for behavioral observation of executive functioning skills. It is recommended to assess executive functioning skills with additional repeated time points and also provide additional psychoeducation on the constructs for those who are unfamiliar.

This study’s goals of investigating the feasibility and acceptability of the SUCCESS intervention were achieved, and we conclude that SUCCESS has promise as an intervention that can be easily embedded into existing supported employment or vocational training programs. It was found to increase vocational soft skills as intended and received high satisfaction ratings. A larger study is underway to more fully test its efficacy in improving neurocognitive and social cognitive skills as well as vocational outcomes.