Randomized Controlled Effectiveness Trial of Executive Function Invention for Children on the Autism Spectrum

Abstract

Background

Unstuck and On Target (UOT) is an executive function (EF) intervention for children with autism spectrum disorders (ASD) targeting insistence on sameness, flexibility, goal-setting and planning through a cognitive-behavioral program of self-regulatory scripts, guided/faded practice, and visual/verbal cueing. UOT is contextually-based because it is implemented in school and at home, the contexts in which a child uses EF skills.

Methods

To evaluate the effectiveness of UOT as compared to a social skills intervention (SS), 3rd-5th graders with ASD (mean IQ=108; UOT n=47; SS n=20) received interventions delivered by school staff in small group sessions. Students were matched for sex, age, race, intelligence, ASD symptomotolgy, medication status, and parent education. Interventions were matched for “dose” of intervention and training. Measures of pre-post change included classroom observations, parent/teacher report and direct child measures of problem-solving, EF, and social skills. Schools were randomized and evaluators, but not parents or teachers, were blind to intervention type.

Results

Interventions were administered with high fidelity. Children in both groups improved with intervention, but mean change scores from pre- to post-intervention indicated significantly greater improvements for UOT than SS groups in: problem-solving, flexibility, and planning/organizing. Also, classroom observations revealed that participants in UOT made greater improvements than SS participants in their ability to follow rules, make transitions, and be flexible. Children in both groups made equivalent improvements in social skills.

Conclusions

These data support the effectiveness of the first contextually-based EF intervention for children with ASD. UOT improved classroom behavior, flexibility and problem-solving in children with ASD. Individuals with variable background/training in ASD successfully implemented UOT in mainstream educational settings.

Introduction

This study investigates whether Unstuck and On Target (UOT; Cannon, Kenworthy, Alexander, Werner, & Anthony, 2011) reduces insistence on sameness, improves flexibility, and/or increases organized, goal-directed behavior in children with an autism spectrum disorder (ASD) and age-appropriate basic language skills. UOT is a cognitive-behavioral, school-based intervention that targets specific executive functions (EF) related to flexibility, big picture thinking, and planning that have previously been found to be deficient in ASD (see reviews: Hill, 2004; Kenworthy, Yerys, Anthony, & Wallace, 2008; Pennington & Ozonoff, 1996; Sergeant, Geurts, & Oosterlaan, 2002). Executive dysfunction and insistence on sameness (which is an autism repetitive/restricted behavior and interest (RRBI) symptom), are associated with impairments in adaptive behavior (Gabriels, Cuccaro, Hill, Ivers, & Goldson, 2005; Gilotty, Kenworthy, Sirian, Black, & Wagner, 2002; Lopata et al., 2012), learning (Akshoomoff, 2005; Blair & Razza, 2007; Koegel & Covert, 1972; Pierce & Corchesne, 2001), and social adaptation (e.g. Kenworthy, Black, Harrison, Della Rosa, & Wallace, 2009; Klin, Danovitch, Merz, & Volkmar, 2007; Loftin, Odom, & Lantz, 2008) in ASD. RRBI are also linked to increased family stress (e.g., Bishop, Richler, Cain, & Lord, 2007; Lounds, Seltzer, Greenberg, & Shattuck, 2007) and executive deficits, including inflexibility, and have been associated with decreased independence and poor outcomes in adulthood (see review Hume, Loftin, & Lantz, 2009), a major area of difficulty for people with ASD without intellectual disability (Howlin & Moss, 2012; Smith, Maenner, & Seltzer, 2012).

Research on interventions targeting EF and/or RRBI in ASD without intellectual disability is limited (Stichter, O'Connor, Herzog, Lierheimer, & McGhee, 2012). In 2005, Fisher and Happé contrasted brief clinic-based EF and Theory of Mind skills training. They found that both contributed to improvements in Theory of Mind and concluded that longer-term school- or home-based interventions that assess real life behaviors were warranted. In their recent review of behavioral intervention research studies in ASD, however, Kasari and Lawton (2010) report that over 80% targeted social and communication impairments. Of the 10% that targeted RRBIs, all but one focused on anxiety. The expanding evidence for cognitive-behavioral therapy (CBT) targeting anxiety in ASD (see review Dawson & Burner, 2011) raises the possibility that this treatment modality can be used for other targets as well. There are several social skills CBT interventions that also briefly address problem-solving skills related to EF (Bauminger, 2002; Solomon, Goodlin-Jones, & Anders, 2004; Stichter et al., 2010, 2012) and find improvements in parent report of EF (Stichter et al., 2010, 2012) and social problem-solving in the laboratory (Solomon et al., 2004; Stichter et al., 2010). These investigations either had no control (Bauminger, 2002; Stichter et al., 2010, 2012) or a waitlist control group (Solomon et al., 2004), and pre-/post-intervention assessment was not conducted by treatment-blind examiners. They indicate, however, promise for CBT approaches to enhance social awareness and problem-solving in ASD.

The Unstuck and On Target curriculum (UOT) builds on this work with a novel, phenotype-specific intervention targeting EF impairments associated with, and linked to negative outcomes in, ASD: inflexibility, impaired goal-setting and planning, and difficulty using internalized language to support problem-solving (Wallace, Silvers, Martin, & Kenworthy, 2009). It is designed to teach new skills and accommodate impairments (Feeney & Ylvisaker, 2008; Ylvisaker & Feeney, 1998) through a CBT program that teaches what flexibility, goal setting and planning are, why they are important, and how to use self-regulatory scripts that guide flexible, goal-directed and planful behavior. UOT uses “supported cognition” (Ylvisaker & Feeney, 1998) techniques to address the EF weaknesses that typically impair learning in ASD (e.g. new skills are taught with extensive scaffolding and guided practice with fading supports; parents and school personnel are trained to model use of desired scripts; visual cues are provided throughout; vocabulary and scripts are the same across home/school settings). Some of the scripts taught in UOT were first developed for children with EF deficits stemming from traumatic brain injuries (Feeney & Ylvisaker, 2003; Feeney, 2010; Ylvisaker & Feeney, 2008).

To investigate the effectiveness of UOT, we randomly assigned elementary schools to the UOT or comparison social skills (SS) intervention, which provided an equal dose of small group intervention, and interventionist, parent and classroom teacher training, but taught social skills based on an established SS curriculum (Baker, 2003). Both interventions used published curricula, fidelity was measured, and outcome data collection was treatment-blind. Measures of generalization included blinded classroom observations and un-blinded parent/classroom teacher report on standardized measures of EF and SS. Children in the UOT intervention were predicted to improve from pre- to post-intervention in flexibility, organization, and planning, while children receiving the SS curriculum were expected to improve in social skills.

Methods

Participants

Recruitment occurred in several stages. See Figure 1. IRB approval was obtained from Children's National Medical Center and from participating school districts. A total of 107 children were identified by school districts as possible participants, based on expressed interest from specific schools that could identify at least three potentially eligible students. Individual schools entered into the study if they had three or more students whose families provided informed consent and who met the eligibility requirements. Children were then scheduled for cognitive/diagnostic evaluation (see below). Recruitment continued until the target enrollment (N=45) was reached, and all remaining interested students from enrolled schools were included (N=67).

Figure 1. Study Participant Recruitment and Enrollment.

Next, potential participants were evaluated individually and included in the study if they achieved a Full Scale IQ score>70, a verbal mental age≥8 years old, and met criteria for ASD. IQ and verbal mental age were measured with the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) Verbal, Performance and Full Scale IQ scores (represented as Standard Scores: mean=100; SD=15). Higher scores indicate better performance. To evaluate for ASD, research-valid clinical psychologists administered the Autism Diagnostic Observation Schedule, Module 3 (ADOS; Lord et al., 2000), a structured play/conversational interview that elicits symptoms of ASD. It produces non-standardized raw scores; higher scores indicate more symptoms. To be included in the study, each child obtained an ADOS diagnostic algorithm ≥‘ASD’ threshold (Lord, Rutter, DiLavore, & Risi, 1999) and met DSM-IV-TR (American Psychiatric Association, 1994) criteria for a Pervasive Developmental Disorder.

Three participants were excluded because they did not meet the ADOS threshold, and six children were excluded because they had verbal mental ages<8 years old. After eligibility ascertainment, schools were randomly assigned to treatment group. Randomization occurred at the school, not the participant, level to avoid cross-contamination of the two treatments. Of the 14 participating schools, four were randomized to the SS intervention, and the remaining 10 to the UOT intervention.

Pre- and Post-Intervention Assessments

Direct Child Measures

The WASI Block Design (BD) subtest is a timed visual construction task that requires efficient nonverbal problem-solving. Performance is represented as age-corrected T scores (mean=50;SD=10), with higher scores indicating better performance.

The Challenge Task (CT) is an unpublished, un-normed measure designed by the authors. It is an ADOS-like 30-minute interview that challenges children to be flexible and planful in the context of five activities with an examiner. Specific challenges are posed (e.g. the child is asked to sculpt a food item using clay and then trade it with the examiner before completing the sculpture), and the child's flexibility and planning are scored on a 3-point scale for each task. The scale (0-good, 1-intermediate, 2-poor performance) has task-specific behavioral markers to guide scoring (e.g. for the sculpture task described above, a flexibility score of 2 is assigned if: “The participant is unwilling to switch sculptures…”). The CT yields average Flexibility and Planning scores (higher scores indicate greater impairment), and an overall rating of Social Appropriateness ranging from 1 (severe impairment) to 10 (strength). Examiners achieved inter-rater agreement ≥90%. For further description of the CT, see online Appendix S1. The full CT instructions/scoring criteria are available from the first authors.

Parent and Teacher Report Measures

Parent and teacher report was collected on:

Behavior Rating Inventory of Executive Function (Gioia, Isquith, Guy, & Kenworthy, 2000), an informant report of EF in everyday situations comprised of 8 subscales, only two of which were administered in this study: Shift and Plan/Organize. Higher T scores (mean=50;SD=10) indicate greater impairment.

Social Responsiveness Scale (SRS; Constantino & Gruber, 2005), is an informant report on ASD-related social, communication and repetitive behaviors. Total T scores were analyzed. Higher scores indicate greater impairment.

Classroom and Intervention Observation Measures

To further assess generalization, at least two 10-15 minute classroom observations were conducted by a treatment-blind research assistant. They occurred randomly during the academic school day, but not during intervention groups. Classroom observers achieved 90% inter-rater reliability on the Classroom Observations Coding Form developed for this purpose. The following behaviors were coded as present, present with support, or absent: Handles the Unexpected, Compromise, Reciprocity, Follows Rules, Transitions, Gets Stuck, Negativity/Overwhelm, and Participates.

Fidelity was measured through 2-3 observations of intervention groups by trained research staff. Adherence to intervention content, participant engagement, and the interventionists' confidence were coded as dichotomous variables. Overall curriculum fidelity was rated on a 10-point scale (1=low fidelity, 10=high fidelity), and each leader was given individualized fidelity feedback. Intervention content coverage was also monitored at monthly interventionist trainings.

Treatment Conditions

Both interventions were delivered during school, by school staff in groups of 3-6 students; classroom teachers and parents were trained to reinforce the UOT/SS lessons. The two interventions were matched for dose of intervention with the child and the amount of parent/teacher/interventionist training provided. Both interventions were provided during one school-year in 28, 30-40 minute lessons with games, visual supports, role-plays and positive reinforcement. Intervention-specific trainings for parents (two sessions) and classroom teachers (one session) addressed targeted skills and autism phenomenology. Interventionists received an initial two-hour intensive training followed by six monthly trainings.

The UOT intervention (Cannon, et al., 2011) is designed to teach what physical/mental flexibility, goal setting and planning are, and why they are useful skills. Lessons are taught through concrete experiments (e.g. can you get through an obstacle course faster with a rigid or flexible body?), videos, visuals, and discussion of scenarios. UOT also teaches children how to be more flexible, set goals and make a plan through the use of self-regulatory scripts (e.g. Plan A/Plan B, Big Deal/Little Deal), which are explained and practiced with: activities (e.g. developing a pretend video game with flexibility challenges and solutions), games, and role plays. Motivation to be more flexible and goal oriented is addressed through discussion/games regarding the advantages of flexibility and goal setting and the teaching of coping skills. Parents were also provided with a manual describing how to model and support the use of flexibility and planning scripts at home.

The SS intervention provided social-communication skills lessons from Baker's curriculum (Baker, 2003): appropriate physical distance, turn taking, starting/joining/maintaining/ending conversations, how/when to interrupt, joining/inviting others to play, and making/maintaining friendships (lessons 1-25, 28, 33, 41). Each skill was introduced in a didactic lesson with visual supports, followed by role-plays, games, and extension activities. Most groups also included reward systems and less structured game time during which the skills could be practiced and supported. Interventionists, parents, and teachers received training in social cognition weaknesses in ASD and suggestions for supporting these skills at home and in school. The content of each intervention is further described in their published manuals.

Data Analytic Plan

Data were analyzed using SPSSv20 (IBM, 2011) and R (R Development Core Team, 2008). Group comparisons on demographic variables used chi-square analyses for the dichotomous variables and t-tests for the continuous variables. Change scores were compared through a series of analyses of covariance (ANCOVAs), with pre-test score, age, gender, Full Scale IQ, and parents' education levels included as covariates. Cohen's d, an index of effect size, was calculated using the t-statistic for each of the change scores. For the non-standardized measures (classroom observations and CT), scores were dichotomized for “improved” (any improvement in score) or “not improved” (no change or worsening). Compromising and Handling the Unexpected in the classroom observations occurred infrequently (<10 occurrences) and were dropped from analyses. Percent improved was calculated for each intervention group with the Pearson-Clopper “exact” method (Clopper & Pearson, 1934) to obtain the binomial confidence intervals. For standardized measures (WASI; BRIEF; SRS), the reliable change index (RCI; Chelune, 2003) was used to define a threshold for meaningful improvement (one-tailed at the 80% confidence level) using the standard deviation (SD) of scores at test-retest and the test-retest reliability coefficient from the measure's standardization sample, when available (WASI; BRIEF). When SD at retest was not available, the time one SD was used (SRS-Parent). When the reliability coefficient was not available, the closest, most conservative estimate was used (e.g. the lowest SRS-Parent reliability was used for SRS-Teacher). The data in Figure 2 represent the proportion of participants in each intervention for which RCI is exceeded (indicating how many participants in each condition made improvements outside error estimates on a given measure).

Figure 2. 95% Confidence Intervals of the Proportion of Individuals in Unstuck and on Target (UOT) and Social Skills (SS) Interventions Who Improved on Post-intervention, as Compared to Pre-intervention, Assessment on Classroom (CO) and Challenge Task (CT) Observations^a, WASI Block Design, and parent and teacher BRIEF and SRS scores^b.

^aClassroom Observations and Challenge Task confidence intervals indicate the percent improved.

^bWASI, parent/teacher BRIEF, and SRS confidence intervals indicate the percent exceeding an 80% reliable change index.

Results

The two groups were well matched on all characterization variables, except that a greater proportion of SS participants came from private parochial schools (see Table 1). Comparable proportions of participants completed the intervention (UOT-91%; SS-95%). Study completers did not differ from non-completers in: age, gender, race, IQ, mother's education, or pre-intervention scores from the BRIEF, SRS, or ADOS. Fathers of non-completers had fewer years of education than fathers of study completers (t=2.23,p=0.03).

Table 1.

Baseline Demographic/Clinical Features of Unstuck and On Target (UOT) and Social Skills (SS) Study Participants.

Means and (standard deviations) reported

	UOT (n=47)	SS (n=20)	Chi-Sq
Male	87%	90%	0.102
White	70%	55%	3.406
Public School	96%	75%	6.45*
On Psychotropic Medication	54.5%	60%	2.95
			t
Age	9.49(1.00)[7.83-11.08]	9.58(1.10)[7.92-11.08]	0.326
Mother's education	1.91(0.88)	1.95(0.76)	0.155
Father's education	2.04(1.12)	1.95(0.91)	-0.328
WASI FSIQ[range]	108.80(18.52)[75-151]	107.63(17.20)[82-150]	-0.023
ADOS Social+Communication	11.77(3.64)[7-21]	12.40(4.17)[7-20]	0.343
ADOS Stereotyped Behavior	1.98(1.71)[0-6]	1.90(1.33)[0-5]	-0.183

^*p<.05

Treatment Integrity

Treatment integrity procedures included staff training, intervention observations, and observer ratings of curriculum fidelity. SS and UOT interventionists attended an average of 4.38 and 4.33 of the six follow-up sessions, respectively. Treatment fidelity was assessed in 10% of randomly selected sessions (13 SS sessions, 33 UOT sessions). Eighty-five and 91% of UOT and SS sessions respectively adhered to curriculum content. Both treatment groups received equivalently high fidelity ratings (UOTM=7.57,SD=1.67; SSM=8.08,SD=1.44; t(38)=-.92,p=.36).

Group differences on Laboratory Tasks and Parent and Teacher Report Measures

See Table 2 for results of ANCOVAs assessing the relative improvement in performance following UOT and SS interventions. The main effect of intervention group (more improvement for UOT than SS participants) was found to be significant (p<0.05), both with and without co-varying the effects of pre-test score, age, gender, Full Scale IQ, and parents' education on the following measures: WASI Block Design, CT Flexibility score, Parent and Teacher BRIEF Shift rating scores, and Parent and Teacher BRIEF Plan/Organize rating scores. Effect sizes are medium-large. Figure 3 provides graphical representation of these data. In contrast, the type of intervention group did not significantly predict improvement on the CT Planning and Social Appropriateness score or the Parent or Teacher SRS Total scores. Improvement occurred on these measures in both intervention groups., Individual factors, such as performance at baseline and baseline IQ predicted outcome scores on some measures. Details on the effects of covariates are provided in online supplementary Table S1. Similar results were obtained when data were analyzed with random effects regression, taking into account the nested nature of the design by including student and school as random effects, and testing a group by time interaction. RCI analyses are presented in Figure 2, and numerical representation of the percent of participants in each group who made reliable change following intervention is provided in Table S1.

Table 2. Group Differences in Unstuck and On Target (UOT) and Social Skills (SS) Change Scores (Post-Intervention minus Pre-Intervention), and Estimated Coefficients for Treatment Effects from Group by Treatment ANCOVAs, Controlling for Age, IQ, Gender, and Parental Education.

Measure	N (UOT, SS)	UOT Mean Change (SEM)	SS Mean Change (SEM)	ANCOVA treatment effecta (SEM)	Cohen's d Effect Size (CI)b
Direct Child Measures
WASI Block Design	41,17	3.00(1.03)	-0.94(1.11)	4.36*(1.76)	0.65(0.18-1.17)
Challenge Task Flexibility	43,19	-0.53(0.07)	-0.15(0.14)	-0.29*(0.12)	-0.72(-1.38-0.14)
Challenge Task Plan	43,19	-0.33(0.07)	-0.22(0.06)	-0.10(0.10)	-0.27(-0.77-0.18)
Challenge Task Social	43,19	0.47(0.16)	0.26(0.30)	0.24(0.26)	0.17(-0.42-0.77)
Teacher-Rated Measures
BRIEF Shift T Score	27,18	-24.44(3.30)	-9.78(3.59)	-11.04**(3.64)	-0.89(-1.62--0.33)
BRIEF Plan/Org T Score	28,18	-19.14(2.39)	-11.72(3.16)	-6.88*(3.05)	-0.57(-1.26-0.01)
SRS Total T Score	25,19	-5.40(1.34)	-4.79(2.05)	-1.63(1.88)	-0.08(-0.78-0.51)
Parent-Rated Measures
BRIEF Shift T Score	41,19	-9.56(2.31)	-0.16(2.99)	-9.92**(3.41)	-0.66(-1.240.15)
BRIEF Plan/Org T Score	42,18	-5.17(2.00)	0.61(2.90)	-6.75*(3.26)	-0.45(-0.97—0.07)
SRS Total Score	42,18	-7.31(1.65)	-4.11(2.97)	-4.34(2.78)	-0.28(-0.84-0.33)

^*p<.05

^**p<.01

^aEstimated coefficient (βˆ) for treatment effect (UOT vs. SS) followed by standard error (SE). The dependent variable=post-pre of measure with pre-test score, age, gender, Full Scale IQ, mother/father education included as covariates. The coefficient is the estimated effect of a 1-point increase in the covariate on pre-post change score of the outcome measure.

^bConfidence Interval

Figure 3.

Mean Scores, with Standard Error Bars, for Problem Solving Skills, Parent and Teacher Report of Flexibility, and Flexibility Ratings on the Challenge Task for the Unstuck and On Target (UOT) and Social Skills (SS) Groups at Pre- and Post-Intervention.

Group differences on classroom observations

Classroom observations also indicated improvements in most participants, with stronger improvements in the UOT participants. More improvement was observed in UOT participants' ability to follow directions (χ²=11.41,p<.001; UOT=65.2% improved), transition smoothly (χ²=15.75,p<.001; UOT=63% improved) and avoid getting stuck (χ²=6.38,p<.05; 47.8% improved) when compared to SS participants. There was also a trend of reduced negativity in the classroom for UOT participants (χ²=3.74,p=.053; UOT=39.1% improved). Participants in both groups showed similar improvements in social reciprocity (χ²=1.76,ns; SS=45% and UOT=62.8% improved) and classroom participation (χ²=2.47,ns; SS=20% and UOT=40% improved). See Figure 2.

Discussion

Comparison of the effectiveness of two interventions for children with ASD revealed that participants in both interventions improved, but that significantly greater improvements resulted from UOT than the SS intervention. The interventions were contextually-based, or occurred in real world school and home settings, in order to maximize the potential for generalization of skills (Ylvisaker et al., 2003; Dingfelder et al., 2011). The results are notable, as this study compares a new EF intervention to an established SS intervention, with both interventions being delivered with equal intensity and high fidelity. Elementary school participants were randomly assigned to intervention type and matched at the start of the intervention for autism symptomatology, age, parent education, IQ, minority and medication status. Improvement was statistically significant both before and after performance at baseline, IQ, age, gender and parents' education were co-varied in the analyses. Furthermore, we report change on treatment-blind, lab-based measures and “real world” measures of outcome. Thus, study methodology met high standards (e.g. Dawson & Burner, 2011; Kasari & Lawton, 2010) and improvements in performance at post-test are attributed to the effect of the interventions themselves. As predicted, children in UOT showed significantly greater improvements than children in SS on measures of problem-solving, flexibility, and planning/organizing. Participants in UOT also generalized greater improvements in classroom behavior than SS participants. The hypothesized advantage for SS participants in social skills improvements was not supported, however, as both groups showed equivalent social gains.

Problem-solving ability was assessed with Block Design, a timed visual-construction task that is an excellent predictor of nonverbal intelligence (Wechsler, 1999), and taps EF abilities because it requires, flexible, efficient, goal-directed, and well-organized responses. It has been reported to improve following EF intervention in other populations (Greenberg, Kusche, & Riggs, 2004). The study finding of improvement in Block Design scores in the UOT group as compared to the SS group provides evidence that UOT improved participants' abilities to problem-solve efficiently on tasks that are distal to the curriculum material and could have wide-ranging impact on a child's ability to effectively solve a variety of problems.

Improving cognitive and behavioral flexibility and reducing problems related to insistence on sameness symptoms are major goals of UOT, and these data support its effectiveness in this regard with multiple measures and across multiple modalities. Participants in UOT improved significantly more (medium effect size) than those in SS in their ability to be flexible during the Challenge Task. Over 83% of UOT participants made meaningful gains on this measure. UOT participants improved in flexibility-related mainstream classroom behaviors, such as making transitions, following rules/instructions, and getting unstuck, all with significantly higher rates of positive response to intervention than was seen in SS participants. Corroborating the classroom observations, parents and teachers reported a clinically significant drop of a full standard deviation or more on the BRIEF Shift scale for UOT participants. Multiple previous studies (e.g. Gioia, Isquith, Kenworthy, & Barton, 2002; Mackinlay, Charman, & Karmiloff-Smith, 2006; Rosenthal et al., 2013; Winsler, Abar, Feder, Schunn, & Rubio, 2007) have documented clinically elevated BRIEF Shift scores in children with ASD similar to those reported during the pre-intervention assessment in this study; therefore, the drop following UOT to scores within the normal range indicates atypically strong flexibility for children with ASD. Stichter and colleagues (2010, 2012) report decreases in parent report of global EF problems on the BRIEF following social skills treatment, but we are unaware of other CBT studies reporting specific gains in flexibility in children with ASD without ID.

Study data also reveal significantly greater gains in planning and organization for UOT than for SS participants, although the effects are not as robust as those seen for flexibility. Parents and teachers, who were not blinded to treatment condition, report greater improvement on the BRIEF Plan/Organization score for UOT than for SS participants, with medium effect sizes, but the treatment-blind CT data did not reveal an advantage for UOT, which could reflect a problem with the measure's sensitivity. Classroom observations did reveal that a significantly higher percentage of UOT than SS participants made meaningful improvements in following directions, but it is unclear how much of this reflects greater flexibility in complying with teacher instruction versus more organized, goal-directed behavior. The relatively weaker impact of UOT on planning and organization may reflect a dosage effect. Although UOT contains more direct instruction in goal-setting and planning multiple-step tasks than other interventions for ASD, flexibility is addressed in UOT in more than twice as many lessons.

Contrary to prediction, the SS intervention was not superior to UOT in advancing participants' social skills. SS specifically targeted social communication skills, like inviting another child to play, and taking turns and making conversation, while UOT did not. Both interventions led to gains in social skills as measured by social appropriateness during the CT and parent and teacher report on the SRS. Although not expected, this finding is consistent with previous work showing that intervention to improve EF enhances social reasoning (Fisher & Happé, 2005) and that EF abilities may be precursors to Theory of Mind abilities (Flynn, 2007), or in any case are correlated with them (Pellicano, 2007; Perner & Lang, 1999). It is also the case that UOT explicitly addresses how flexibility helps people be good friends. Finally, enhancing a child's ability to regulate his/her behavior when challenged to be flexible and organized should reduce outbursts and other negative behavior that may alienate peers and make a child less available for social interaction. Classroom observations indicated, for example, that 39% of participants who completed UOT were less negative in the classroom following intervention while only 15% of the SS group improved on this measure.

Change following treatment with UOT and SS was investigated through a series of analyses of covariance in which a variety of possible contributing factors were considered in addition to treatment condition, the main variable of interest. Performance at baseline, age, gender, intelligence and parent's education were all considered. Although treatment condition remained significant in most analyses as described above, baseline performance on each measure (except Block Design) prior to the intervention also predicted the amount of improvement achieved, indicating that better skills initially increased participants' ability to benefit from intervention. Baseline intelligence also predicted improvement following intervention, but only for the challenge task flexibility and planning variables. For all other measures intelligence did not predict response to treatment, indicating that in this study of individuals with ASD without ID (Full Scale IQ scores ranged from 75-151), intelligence was not a determining factor in response to treatment. In addition, parent, but not teacher, report of changes in both flexibility and planning/organization was related to the age of the participant.

This investigation has several limitations, including relatively small samples, which prevents meaningful investigation of moderator variables. Future work with larger samples could address this and also follow participants longitudinally to investigate what changes are preserved over time (Dawson & Burner, 2011). Isolation of the active ingredients in the intervention through comparison of outcomes following treatment with specific modules of UOT will allow streamlining this relatively lengthy intervention (Kasari & Lawton, 2010). Future investigations should also address how specific characteristics of the interventionists affect treatment outcomes and should sample global EF skills as well as flexibility and planning/organization. Finally, the lack of test-re-test reliability, validity and normative data on the Challenge Task limits our understanding of how findings of reliable change on this measure relate to real world social and EF functioning.

This investigation corroborates consistently positive anecdotal reports on UOT (e.g. teacher report: “UOT helped tremendously. … For example, one student had 3 meltdowns per day, and UOT reduced her outbursts to 2 occurrences in one quarter. Not only that, the intensity was reduced as well”; parent report: “This has been a life-changing experience for my daughter!”) It provides evidence that UOT helps children with ASD to be more flexible, planful, and organized on well-controlled problem-solving tasks, and in socially-mediated real-world classroom and home settings. It holds the promise of targeting a core symptom of ASD that can prevent otherwise verbal, bright individuals from participating in mainstream settings, and it targets school age, verbal individuals for whom few evidence based interventions are available. Its successful implementation in mainstream educational settings by individuals with variable background/training in ASD means that UOT may offer a powerful contextually-based complement to other proven social skills and behavioral treatments for ASD.

Supplementary Material

Supp Appendix S1 & Table S1

Appendix S1 Challenge Task Description

Table S1 Percent of Children with Improved Change Scores

Key points.

• This study evaluated the effectiveness of a school based executive function (EF), as compared to a social skills, intervention for children with high functioning autism.

• The EF intervention, Unstuck and On Target, uses a novel cognitive-behavioral small group teaching model that emphasizes self-regulatory scripts, modeling behavior, visual supports and consistent language between home and school.

• Children in both interventions improved, but mean change scores from pre- to post-intervention indicated significantly greater improvements from the EF intervention in: problem-solving, flexibility, planning/organizing, and ability to follow rules, make transitions, and resist getting stuck in the mainstream classroom.

• Children in both groups made equivalent improvements in social skills.

• These data support the effectiveness of the first contextually-based EF intervention for children with autism.