Modular Cognitive Behavioral Therapy for Autism-Related Symptoms in Children: A Randomized, Controlled Trial

Abstract

Objective:

To date, no one-on-one psychotherapy protocol for elementary and middle school-aged children with autism spectrum disorder (ASD) has been found to be efficacious for treating autism-related symptoms such as failure to initiate social interactions. This study compared modular cognitive behavioral therapy (CBT) with enhanced standard community treatment (ESCT) in terms of impact on the severity of autism-related symptoms.

Method:

Children with ASD (N=107; aged 6-13 years) were randomly assigned to a treatment condition (CBT or ESCT). Both treatments provided 32 therapy sessions. The CBT condition utilized a modular design, matching specific evidence-based treatment elements to each child’s clinical needs (e.g., social communication symptoms). The ESCT condition provided social skills training and cognitive behavioral training in a structured and linear group-therapy format. The primary outcome measure was independent evaluator ratings of peer engagement during school recess using a structured and validated observation system. Parents also made session-by-session ratings on personalized autism-related symptom profiles throughout treatment.

Results:

CBT outperformed ESCT on the primary outcome measure (p<.001; d=.50; 95% CI [.06, .93]) and the secondary outcome measure (p=.003; d=.87; 95% CI [.45, 1.27]).

Conclusions:

The modular one-on-one CBT program evaluated in this study may be beneficial for reducing the severity of autism-related symptoms in some children with ASD. Further research is needed to clarify the extent of the treatment effect and the feasibility of implementation for therapists in the community.

Autism spectrum disorder (ASD) affects approximately 1 out of every 59 children (Baio et al., 2018). Among school-aged children (e.g., youth aged 6-13 years) with ASD, psychotherapy¹ is one of several psychosocial interventions sometimes allocated to address autism-related symptoms² (Stuart et al., 2017; Zablotsky et al., 2015). However, no one-on-one psychotherapy protocol meets contemporary evidentiary standards (cf. Southam-Gerow & Prinstein, 2014) as efficacious/well-established for addressing autism-related symptoms, such as social-communication difficulties, in school-aged children (Wood, Klebanoff, Renno, Fujii, & Danial, 2017). To ensure that high-quality treatment is provided to youth with ASD, it is critical that efficacious protocols are identified and implemented.

ASD is characterized by specific social-communication challenges and restricted/repetitive behaviors (RRB). In childhood and adolescence, ASD has pervasive impacts on adaptive functioning across domains such as friendship, daily living skills, and academic performance, including for youth with intellectual skills in the typical range (e.g., Ashburner, Ziviani, & Rodger, 2010; Kanne et al., 2011; Pugliese et al., 2015). The severity of a child’s autism-related symptoms is associated with her/his degree of functional impairment (e.g., Duncan & Bishop, 2015; Golya & McIntyre, 2018). A psychotherapy protocol that reduces the severity of autism-related symptoms could have positive downstream effects on adaptive functioning.

Historically, psychotherapy was presumed to be unsuitable for the treatment of ASD (Kanner & Eisenberg, 1954; Rutter, 1968). However, research on psychotherapy models for ASD has emerged in recent years (see Weston, Hodgekins, & Langdon, 2016). While overall treatment and education costs for children with ASD are often high (e.g., Lavelle et al., 2014), time-limited, one-session per week outpatient treatment can be relatively economical (Roundfield & Lang, 2017). Given the prevalence of ASD, the utilization of psychotherapy in this population, and the potential cost effectiveness of psychotherapy, there is a clear need for an efficacious psychotherapy protocol to address autism-related symptoms.

Cognitive Behavioral Therapy Programs for Children and Adolescents with ASD

Psychotherapy employing cognitive and behavioral practices offers an approach that may be well-suited to the clinical needs of school-aged children with ASD (Attwood & Scarpa, 2013). For example, CBT-based psychotherapy has been adapted to treat concurrent emotional dysregulation in school-aged children with ASD, with evidence of probable efficacy for anxiety symptoms (e.g., Wood et al., 2020).

However, few one-on-one psychotherapy protocols for children have focused on autism-related symptoms such as social-communication difficulties. In two recent reviews of the literature, one of which was also a meta-analysis (Weston et al., 2016; Wood et al., 2017), only one one-on-one psychotherapy protocol was identified that targeted autism-related symptoms (as well as clinical anxiety), and this protocol was a precursor of the CBT intervention used in the present randomized, controlled trial (RCT), which showed promise in a small pilot study (N=13; Wood, Fujii, Renno, & van Dyke, 2014). One additional protocol utilized mixed group-based and one-on-one CBT and also yielded promising results, as compared to a waitlist condition, based on parent ratings of autism-related symptoms (White et al., 2013). Other RCTs that have chiefly targeted clinical anxiety have also found a significant effect of one-on-one CBT on parent-reported autism-related symptoms (e.g., Storch et al., 2013; Wood et al., 2020). In contrast with these previous protocols, in the present trial, anxiety was neither an inclusion criterion nor a primary focus of the CBT treatment protocol; instead, the present treatment focused primarily on autism-related symptoms and, hence, is the first one-on-one CBT-based psychotherapy protocol with this clinical focus to be evaluated in an RCT.

Numerous group therapy programs with CBT elements have also been developed that address one or more autism-related symptoms (cf. Weston et al., 2016). Many of these programs entail group-based social skills training that include certain cognitive therapy elements such as reframing and perspective taking training (cf. Weston et al., 2016). Some of these programs have yielded promising effects on parent-reported autism-related symptoms in RCTs generally utilizing waitlist comparison groups (e.g., Frankel et al., 2010). However, historically, many other social skill training programs in group therapy settings have not yielded measureable effects on generalizable outcomes apart from behavior observed with other group members (cf. Ferraioli & Harris, 2011).

Although there are numerous advantages to CBT-informed interventions delivered in a group setting (e.g., cost-efficiency; opportunity to practice skills with peers), one-on-one CBT-based psychotherapy has several unique advantages of its own, including the opportunity for individualization of the intervention to each child. In developing a CBT program capable of addressing autism-related symptoms in children, it has been noted that although CBT may be a promising general treatment modality for school-aged children with ASD, there is substantial heterogeneity in children’s clinical profiles, which poses a challenge for designing successful protocols (Wood, McLeod, Klebanoff, & Brookman-Frazee, 2015). This clinical heterogeneity appears to be a good match to modular psychotherapy protocols that can flexibly address individualized clinical needs for each child. Research on children without ASD suggests that modular CBT-based psychotherapy may yield mental health outcomes superior to structured, linear CBT protocols that emphasize a single domain of mental health (e.g., Weisz et al., 2012). Potentially this is because highly structured, invariant intervention programs may provide insufficient intervention dosage for some severe symptoms that particular children have, while in some cases concurrently providing unneeded intervention for symptom domains that other children are not struggling with. The CBT protocol evaluated in the present trial used a modular format to allow treatment to focus on the most pressing of the commonly experienced clinical needs of children with ASD within the commonly impacted domains of social-communication skills, peer relationships, RRBs, emotion dysregulation, and self-care skills (e.g., Charman et al., 2017).

Another family of interventions sometimes used to address certain autism-related symptoms in school-aged children is applied behavior analysis (ABA), which is often delivered by paraprofessionals in school or home settings for several hours or more per week (Croen, Shankute, Davignon, Massolo, & Yoshida, 2017). For example, the ABA technique of self-management has been used to increase social initiations in children with ASD (see Koegel, Koegel, & McNerney, 2001). Evaluations of ABA practices addressing autism-related symptoms for school-aged children have generally been conducted as single-case experimental designs, with promising but not conclusive evidence of efficacy (cf. Southam-Gerow & Prinstein, 2014).³

Theoretical Framework for One-on-One CBT Applied to Autism-Related Symptoms

Although CBT is a broad intervention strategy with numerous associated practices that, depending on emphasis and implementation, might yield widely varying effects in specific clinical groups, a number of intervention developers have contributed to a growing rationale for the use of specific applications of CBT for children with ASD over the past 20 years (e.g., Attwood & Scarpa, 2013). Our CBT framework for this population (Wood, Fujii, & Renno, 2011) draws upon the memory retrieval competition model of CBT developed by Brewin (2006, 2015) and several of the principles of pivotal response treatment (PRT) for ASD developed by Koegel and colleagues (2001). In Brewin’s (2006, 2015) model, the development of adaptive appraisals and behaviors during treatment can be thwarted by competing memories of maladaptive responses evoked under real world conditions outside the therapy office. Brewin’s model identifies three key strategies for enhancing the retention of adaptive appraisals and behavioral responses and the concurrent suppression of maladaptive appraisals and responses in daily life: (1) Individuals need to engage in activities that promote deep semantic processing when learning adaptive thoughts and behaviors in CBT (as opposed to passive listening). (2) Adaptive appraisals and responses should be learned and practiced in settings, and under conditions, that cue maladaptive appraisals and responses (the encoding specificity principle). (3) There should be positive and reinforcing features associated with adaptive memories; a memory with a positive emotional valence is likely to suppress competing memories of appraisals and behavioral responses at recall.

The CBT framework adopted herein (Wood et al., 2011) assumed that these principles of memory retrieval competition have important implications for the implementation of CBT-based psychotherapy for autism-related symptoms. Some applications of these principles to specific autism-related symptoms are as follows:

To promote the conceptual development (as opposed to mere behavioral change) called for in the retrieval competition account of CBT, skill-building behavioral tasks (e.g., initiating play with an unknown peer at the park) should immediately precede or follow guided Socratic questioning focused on the rationale for the target skills (e.g., how the social behavior may positively impact the mental state and attitude of another child, and hence, what social goal it could achieve). In this form of Socratic questioning, the clinician’s questions provide hints for the correct answer within the wording of the question so that children are led towards accurate concepts while still engaging in deep processing by reflecting and putting answers in their own words. This, in turn, may augment conceptual understanding of others’ mental states and the impact of such states on others’ behavior in relation to the child with ASD (i.e., this type of deep processing may promote mentalizing skills).

Second, the retrieval competition model suggests that some skill rehearsal should occur in the actual settings where social deficits are exhibited (i.e., an application of the encoding specificity principle), rather than exclusively in simulated social situations such as a therapy room, as is done in traditional social skills training. Relatedly, caregiver-mediated social coaching in school and community settings can multiply the dosage of intervention, building from the CBT practice of caregiver-mediated in vivo exposure (e.g., Sze & Wood, 2008; Wood & Wood, 2011). On a similar note, peer intervention at school and friendship skills training emphasizing one-to-one playdates are important treatment elements that follow from this model (e.g., Frankel et al., 2010).

Finally, the intervention concepts and activities should be interesting and fun to enhance the distinctiveness of the memory for the adaptive concepts and behaviors being taught. Whether through the use of humorous role-plays to convey the rationale for self-monitoring for RRBs, fun sorting games like “treasure box/trash can” to help children practice selecting between appropriate and inappropriate social responses, or the therapist’s generally sunny disposition and tendency to use shaping and behavioral momentum instead of demands and requirements, the emphasis on generating positive affective experiences within the sessions coincides with the emphasis on motivation and choice in PRT (Koegel et al., 2001).

Hence, a key feature of the present application of CBT is its balanced integration of cognitive (e.g., perspective-taking) and behavioral (e.g., in vivo social coaching) practices throughout each specific intervention technique. A consequence of this integration is the resulting demands on logical reasoning skills and, to some extent, receptive and expressive language. Although methods for enhancing access to verbal interventions are common to many CBT (and other) programs for children with ASD (e.g., use of illustrations, pretend play toys, and modeling as various forms of visual support to lessen demands on verbally mediated learning; e.g., Attwood & Scarpa, 2013), these supports may only partially mitigate the challenges to accessing the cognitive portions of the intervention for some children with ASD. For this reason, this study included children who were no more than one standard deviation below the mean in general cognitive abilities; further, we explored the impact of level of development (age, IQ) on treatment response to examine whether children with greater developmental maturity might benefit the most from this form of CBT.

In this study, a modular CBT-based psychotherapy protocol for children with ASD (Schema, Emotion, and Behavior-Focused Therapy for Children; SEBASTIEN) was evaluated. SEBASTIEN is novel in being the first one-on-one CBT protocol for children that we are aware of to focus primarily on addressing autism-related symptoms. The trial included a comparison of two active and plausible treatments, the use of independent observations for the primary outcome measure, and an adequately powered sample. It was hypothesized that children randomized to SEBASTIEN would exhibit greater improvement in autism-related symptoms relative to children randomized to a structured and linear psychological treatment emphasizing social skills training.

Methods

Participants

Participants included 107 children (6-13 years old) with ASD living in a major metropolitan area of the western United States. Eligibility was determined through an initial screening appointment in which diagnostic measures were administered and inclusion/exclusion criteria were reviewed (see below). Participants were drawn from two interconnected samples funded by parallel grants that had been sought to increase sample size and, hence, statistical power for this study (clinical trial registration on Clinicaltrials.gov: NCT02010086 and NCT01784263). The two parallel studies shared the same objectives and recruitment methods and differed only in target age ranges (Sample 1: ages 6–10 years; Sample 2: ages 9–13 years]. After the initial Screening appointment, 118 children and their families were found eligible for the study (Autism Speaks Grant Sample [Sample 1]: N=48; NIMH Grant Sample [Sample 2]: N=70). Of these eligible children and families, 107 completed baseline measures and were randomized (see eFigure 1 in the Online Supplement for patient flow through the study). One family discontinued the study following randomization but before beginning treatment. Table 1 presents descriptive and clinical information for participating families.

Table 1.

Demographic Characteristics for the CBT and ESCT Conditions

	CBT	ESCT
Characteristic	n %	n %	Test Statistic; p
Child’s gender (female)1	8/52 (15.4%)	6/51 (11.8%)	χ2=1.30; .52
Child’s ethnicity/race2
Latino/a	7/52 (13.5%)	6/51 (11.8%)	χ2=0.07; .82
African American/African	0/52 (0%)	1/51 (2.0%)	--
Asian/Pacific Islander	5/52 (9.6%)	6/51 (11.8%)	χ2=0.12; .72
Caucasian	27/52 (51.9%)	26/51 (51.0%)	χ2=0.01; .92
Middle Eastern	4/52 (7.7%)	1/51 (2.0%)	χ2=1.83; .18
Native American	0/52 (0%)	1/51 (2.0%)	--
Multi-ethnic/racial	9/52 (17.3%)	10/51 (19.6%)	χ2=0.09; .76
African American and Asian	1/52 (1.9%)	0/51 (0%)	--
Asian and Caucasian	3/52 (5.8%)	3/51 (5.9%)	χ2=0.00; .98
Asian and Latino	1/52 (1.9%)	2/51 (3.9%)	χ2=0.36; .55
Latino and Caucasian	4/52 (7.7%)	3/51 (5.9%)	χ2=0.13; .72
Latino and Native American	0/52 (0%)	1/51 (2.0%)	--
Unspecified	0/52 (0%)	1/51 (2.0%)	--
Total household income < $40,000	8/43 (18.6%)	3/44 (6.8%)	χ2=2.74; .10
Primary parent’s education
High school degree or less	2/52 (3.9%)	0/50 (0%)	--
4-year college degree or more	40/52 (76.9%)	45/50 (90.0%)	χ2=3.14; .08
Parents currently married	39/52 (75.0%)	42/51 (82.4%)	χ2=0.83; .36
Child’s IQ	M=103.4, SD=19.3	M=105.1, SD=13.2	t=.55; .58
Child’s age in months	M=116.5, SD=24.8	M=113.0, SD=20.2	t=.80; .43

Note. CBT=cognitive behavioral therapy. ESCT=enhanced standard community treatment. Test statistics andp values reflect group comparisons between CBT and ESCT. Chi-square tests were not conducted when cell sizes were zero. The sample sizes vary within group because some demographic data was not provided by some families.

¹Unfortunately, options for reporting a child’s gender were limited to male and female at the time of study onset. Other gender identities were not intentionally excluded from the survey and future research will correct this omission.

²Race and ethnicity were queried in the same section of the survey, leading families to report on race or ethnicity but not both. Future research will more clearly separate race and ethnicity in the survey.

Children were referred by medical centers, regional centers, parent support groups, and schools; study flyers were used in recruitment. The study was approved by a university-based institutional review board. Contact was initiated by parents to the study coordinator, who conducted an initial phone screening. Parents gave written informed consent and children gave assent to participate in the study after receiving a complete description of the study. Families received $50 for participating in the assessments. Assessments and intervention occurred at the university with which the first author is affiliated.

Eligibility criteria included a clinical diagnosis of ASD confirmed at screening (see below), IQ>85 (see below), age 6 to 13 years, no current psychotic episode or suicidality, and an agreement to adhere to restrictions on concurrent treatments during the study: (1) a stable dosage of psychiatric medication, if used, was to be maintained for at least one month prior to intake and there were to be no foreseeable plans for medication changes (confirmed telephonically by the child’s prescribing physician); (2) children were not to receive concurrent CBT but were permitted to maintain the use of school counseling or other school services such as speech therapy. The principal investigator reviewed eligibility criteria for each participant in order to determine eligibility status at Screening. Participants were notified of their eligibility status by the study coordinator.

Randomization and Masking

Participants were randomized to (a) modular cognitive behavioral therapy (CBT) or (b) enhanced standard community treatment (ESCT; social skills training enhanced with cognitive and behavioral practices) in a parallel study design with a 1:1 allocation ratio. At the time of randomization, pairs of participants were matched on age, IQ, and ADOS-2 algorithm score by the study coordinator. Using a computerized random number generator that concealed the randomization sequence from the investigators, CBT was then randomly assigned to a member of each matched pair and ESCT was automatically assigned to the other member. Children were then randomly assigned to an available therapist. Randomization was conducted by the study statistician, who had no contact with participants; the study coordinator was informed of the random assignment for each participant by the statistician and subsequently notified participants about their assignment. With regard to the two parallel samples noted above, the study statistician randomized children from both samples into the two treatment conditions irrespective of sample. For example, if a therapy group in ESCT was comprised of four children, two of them could have been recruited for Sample 1 while the other two could have been recruited for Sample 2. Hence, the two parallel samples became one at the point of randomization. Participants were not informed about study hypotheses in an effort to avoid setting positive or negative expectations about the two treatment conditions. Participants were also not informed of the condition to which they had been assigned. The two conditions were structured to be similar enough in clinical focus and practices utilized (see Interventions, below) to render differences between the conditions ambiguous and, thus, to equalize participant attitudes towards the two conditions. Parents’ informed consent form stated: “One program is experimental and addresses perspective taking abilities, initiating and responding to others, and becoming flexible and tolerant of changes in routine. The other program is based on commonly used community treatment approaches”.

Interventions

Families in both conditions received 32 weekly 90-minute sessions in a university setting. Therapists included 14 psychology graduate students and 3 postdoctoral psychology fellows. Therapists had previous experience with child psychotherapy and received eight hours of training in the treatment protocols, read the treatment manuals, and attended weekly supervision with a licensed psychologist with five years of experience with the treatments.

CBT condition.

The CBT condition utilized the SEBASTIEN psychotherapy manual (Wood & Wood, 2011), the conceptual rationale for which is described in detail above. SEBASTIEN combines multiple evidence-based practices (e.g., reappraisal, self-management, friendship skills) that target six clinical problem areas often experienced by children with ASD (e.g., repetitive behavior, peer engagement difficulties, emotional dysregulation). There is a therapy module for each primary clinical area (see eTable 1 for details). SEBASTIEN is implemented in a flexible modular format; each youth receives only the modules specific to the problems he/she experiences, and these are prioritized in the order listed in eTable 1. Clinicians and clinical supervisors determined which modules to utilize on a session-by-session basis for each child based on an algorithm.

Therapeutic concepts are taught using multimodal stimuli (e.g., discussion scaffolded by drawing, writing, or demonstration with toys) and guided Socratic questioning, using children’s interests as metaphors to maintain enthusiasm and motivation. The cognitive elements of SEBASTIEN are interwoven with its behavioral practices and involve, in part, recognizing the types of thoughts that accompany maladaptive feelings and challenging those thoughts. More unique to SEBASTIEN, perspective-taking skills (i.e., deducing others’ perceptions of and attitudes towards the child) are used as a cognitive support for the behavioral skills (e.g., self-management; conversation skills) and are practiced to understand why others respond in unwanted versus supportive ways to the child, and what actions the child might take to elicit more positive and desired responses (as well as related perceptions and attitudes) from others. This procedure is utilized to increase motivation to learn new behavioral skills and to help children develop a flexible capacity to make adjustments in a range of naturalistic social interactions guided by imagining others’ mental states. Because friendship and social engagement are often challenging for children with ASD, in most cases, children and parents are taught friendship skills (elements of the Frankel et al. [2010] friendship skills training approach were adopted, namely, hosting regular 1:1 play dates and acting as a “good host” by offering guests choices of what to do together), and parents and school personnel support children in entering peer situations and maintaining conversations or play. If needed for RRBs, habit reversal procedures are implemented, including self-monitoring and incompatible replacement behaviors, as well as exposure with response prevention.

For all children in SEBASTIEN, a comprehensive hierarchy is developed early in treatment identifying target behaviors for the child (e.g., conversational skills and repetitive behaviors). Ultimate goals are set forth as measureable outcomes (e.g., initiate and respond to a range of topics other than just special interests when together with family throughout the day), which permits the delineation of specific proximal goals that gradually increase in difficulty (e.g., at dinner with one’s family, mentioning a special interest no more than twice). Proximal goals, including the use of specific cognitive or behavioral skills at school, in the community (e.g., playgrounds), and at home (e.g., during get-together with peers), are clearly defined and reinforced with a comprehensive reward system. Small steps towards ultimate goals are intended to promote steady progress and success, leading to regular positive reinforcement in working towards each goal. A free online version of the SEBASTIEN manual, including automated clinician guidance support based on the treatment algorithm, is available at meya.ucla.edu.

ESCT condition.

The ESCT condition entailed group social skills training enhanced with cognitive and behavioral practices to address mood and behavioral challenges. The study protocol called for a social skills training intervention as the comparator for CBT because such interventions for ASD are often implemented in community treatment settings (Zablotsky et al., 2015) and focus on social skills development, relevant to autism-related symptoms (Ferraioli & Harris, 2011). The ESCT condition was based, in part, on a child-focused social skills training intervention (Kasari, Rotheram-Fuller, Locke, & Gulsrud, 2012), and was enhanced with additional cognitive and behavioral practices for this study. The therapist targets specific social skills via direct instruction, modeling, coaching, and role-playing. Skills include initiating conversations, responding appropriately, focusing on age-appropriate topics, managing negative comments and behaviors, joining in games, making eye contact, using smiling to make a positive impression, staying on-topic, being respectful to adults, defusing conflict—including walking away or getting help—asking questions, and giving compliments. Intervention topics from the Kasari and colleagues (2012) session content were enhanced with age-appropriate examples when 11-13 year olds were in the group.

Reframing, in vivo exposure for anxiety, and exposure with response prevention for rigidity and anger/frustration cues were added to the ESCT manual to supplement the social skills training practices. The inclusion of these practices was intended to render the differences between the two conditions ambiguous from the participants’ perspective, to improve the consumer acceptability of the ESCT condition, and to reduce differential positive or negative bias towards either condition. However, to maintain the emphasis on social skills comparable to social skills training implemented in many community settings, the majority (59.4%) of ESCT sessions focused primarily or partially on social-communication goals. The content of CBT and ESCT is compared in eTable 1. Groups ranged in size from four to eight families. For each ESCT group, there was a child meeting (90 minutes per session) and a concurrent parent meeting (60 minutes per session), each led by one therapist. Child meetings also had two assistant leaders (either graduate students or advanced undergraduates). ESCT employed a standardized format delineated in a manual in which all families received all session content. In the child meetings, topics were introduced through brief didactic discussions supplemented by visual aids as well as role-plays. The majority of each child meeting entailed practicing specific behavioral or cognitive skills (e.g., acting as a host during a get-together). The parent meetings used a didactic approach along with group discussion and role-playing to teach parents how to support children in the use of specific skills at home and in the community.

Treatment attendance.

The average number of sessions attended was 28.41 (SD=9.20) and 28.75 (SD=9.51; t=−.19, ns) for CBT and ESCT, respectively.

Therapist fidelity.

A random selection of 240 audio-recorded treatment sessions was coded for fidelity to the treatment manuals by trained raters. Raters listened to each recording and noted the presence or absence of required topics for each module. For CBT, the fidelity rating items were specific to each module such that if the “exposure” module was used in a given session, the fidelity coder would use the fidelity checklist items for the exposure module to assess fidelity in that session. Sample items from the checklists were: “Planned and/or practiced making phone calls (yes/no)” and “Conducted a social exposure immediately after social coaching discussion (yes/no)”. Study therapists adhered to 96% of the required topics in CBT, and 90% in ESCT, on average. A second rater randomly coded 114 of the sessions, and interrater reliability was high (two-way random, single score ICC [2,1]=.91).

Measures

Screening measures.

All assessments were conducted by independent evaluators (IEs; doctoral candidates and post-doctoral fellows) unaware of treatment condition assignments. Three measures were administered at the first (screening) appointment to determine eligibility: the Autism Diagnostic Observation Schedule-Second Edition (ADOS-2; Lord et al., 2012) Module 3, the Autism Diagnostic Interview-Revised (ADI-R; Le Couteur, Lord, & Rutter, 2003), and the Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV; Wechsler, 2003). IEs were research-certified in the administration of the ADOS-2 and the ADI-R, and were trained by the study PI to administer the WISC-IV.

The ADOS-2 (Lord et al., 2012) is a semi-structured observational assessment administered directly to participants to elicit social interaction, language samples, and potential restricted or repetitive behaviors. The ADOS-2, Module 3 diagnostic algorithm has acceptable sensitivity and specificity for ASD diagnostic discrimination (e.g., Gotham, Risi, Pickles, & Lord, 2007).

The ADI-R (Le Couteur et al., 2003) is a 93-item standardized interview providing a comprehensive autism diagnostic assessment. The ADI-R provides reliable and valid diagnoses of ASD for individuals with a mental age above 2 years. Domains assessed by the ADI-R include social, communication, and repetitive behaviors. The ADI-R includes an algorithm with cut-scores for the classification of autism (Le Couteur et al., 2003). A research diagnosis of ASD was given for study eligibility purposes if children had ADI-R and ADOS-2 Module 3 algorithm scores above the cut-score for ASD and a study psychologist confirmed the diagnosis.

Interrater reliability for the ADOS-2 and ADI-R was examined: a second IE unaware of the original scores coded video and audiotapes of 14 (13.1%) ADOS-2 and 10 (9.4%) ADI-R interviews selected randomly. Reliability for meeting the algorithm cut-scores, based on percent agreement, was adequate for both measures: ADOS-2 Algorithm Scores (92.9%) and ADI-R Algorithm Scores (100%).

The WISC-IV (Wechsler, 2003) is a test of general intelligence for youth 6 to 16 years old with established reliability and validity (e.g., Ryan, Glass, & Bartels, 2009). In this study, an abbreviated WISC-IV was administered to provide an estimated full scale score based on the Vocabulary and Matrix Reasoning subscales (see Table 1), parallel to the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999), which uses these two subscales to estimate full scale IQ. The sum of WISC-IV Vocabulary and Matrix Reasoning scaled scores generally correlate highly with full scale IQ scores (e.g., r=.78; Aubry & Bourdin, 2018).

Eligibility status was determined after the screening visit; eligible families were invited to a second (baseline) appointment in which additional descriptive and experimental measures (e.g., personality questionnaires, symptom checklists) were administered. Manuscripts reporting analyses with these measures will be submitted following publication of this report of primary and secondary outcomes.

Playground Observation of Peer Engagement (POPE; Kasari et al., 2012).

School observations were conducted by a trained IE for approximately 30 one-minute intervals during school recess at baseline and again at post-treatment. Two days of data per assessment (15 one-minute intervals each) were collected (i.e., 30 minutes total) unless the child was unexpectedly absent or the school declined for one of the days, in which case scores from the single day of observations were prorated. IEs used an ASD-specific observational protocol (Kasari et al., 2012; Locke, Kasari, & Wood, 2014). The primary outcome variable in this study is the POPE Joint Engagement/Games with Rules (POPE-JEGA) scale: the number (ranging from 0 to 30) of one-minute intervals during each observation that the child spent initiating social behaviors, responding to social initiations, engaging in reciprocal communication with peers, and/or participating in cooperative play (or games with rules like tag) in an outdoor free play setting at school (e.g., the recess yard, track, field, play structure). Scores for observation days with fewer than 15 intervals of observations but with at least 5 intervals of valid observations were prorated for the day. If the child was not in an outdoor free play setting for a given 1-minute interval (e.g., was indoors, was eating, was lined up to return to class), the interval was not scored.⁴ Over the course of this study, a team of six IEs was assigned to make school observations based on availability. Regular coding meetings were scheduled in order to minimize coder drift and maintain a common set of coding practices. Coders were initially trained for eight hours through protocol review, discussion, role play, and live practice coding with guided feedback. Coders were certified for data collection if they reached 90% agreement with the trainer over two successive recess observations. On 32 observation days, a second IE randomly selected from the general pool of available IEs for the day accompanied the assigned IE (each IE was involved in at least 5 of these overlaps) and interrater reliability for POPE-JEGA scores was high (ICC=.89). In previous research, POPE scores correlated with other measures of autism-related symptoms and were treatment sensitive (Kasari et al., 2012; Locke et al., 2014).

Youth Top Problems (YTP) scale (Weisz et al., 2011).

The YTP scale is a valid and reliable personalized symptom assessment method that is sensitive to psychotherapy treatment response in children (Weisz et al., 2011). The YTP was initially administered after the ADI-R at screening; at the end of the ADI-R interview, parents were asked to state in their own words what symptoms/problems were the most concerning to them. Diagnosticians then obtained severity ratings for each symptom/problem on a scale ranging from 0 (not at all) to 10 (very, very much). The three highest-rated problems were transcribed in the parents’ own words and were rated by the parent at each treatment session on the same 0-10 scale and summed (the possible range of summed YTP scores for each treatment session was therefore 0-30). Internal consistency for the total score across the first five sessions of treatment was high (alpha=.93). To further characterize the content of the parent-described top problems in this sample, all parent-described YTP symptoms (N=318) were categorized into one of 43 YTP descriptive symptom categories (see https://meyaucla.wixsite.com/woodlab/measures). which consists of 20 social-communication (SC) related symptoms (e.g., one sided conversation / perseverative speech with others), 7 restricted/repetitive behavior (RRB) related symptoms (e.g., restricted interests), 12 externalizing symptoms (e.g., noncompliance [often with anger]), and 4 internalizing symptoms (e.g.. non-social, general anxiety [e.g., germs, math, worry, movies, new places]). Raters were trained to k>.7 criteria against consensus gold standard coding, and a second rater made ratings of 50 (15.7%) randomly selected parent-described YTPs. For categorizing individual YTP symptoms into the 43-category YTP descriptive symptom categories, rater concurrence was in the good range (k=.66). and it was higher when the 43 symptom categories were further collapsed to the construct level of SC symptoms, RRB symptoms, externalizing symptoms, and internalizing symptoms (k=.80).

Parent Expectancies for Therapy Scale (PETS; Kazdin & Holland, 1991).

The PETS scale includes 25 items rated on a 1-5 Likert scale. It was administered to parents after written correspondence about the assigned condition was reviewed and discussed with the study coordinator following randomization. The PETS has established reliability and predictive validity in studies of child psychotherapy (e.g., Nock & Kazdin, 2001). Higher scores reflect more positive expectancies. Internal consistency in this study was acceptable (alpha=.81).

Consumer Satisfaction Questionnaire (CSQ; MTA Cooperative Group, 1999).

The 11-item CSQ, which has a 7-point response scale with higher values indicating greater satisfaction, was completed by parents at post-treatment. One item was modified for the current study to query whether the treatment would be recommended to other families of children with ASD (as opposed to ADHD). Internal consistency in this sample was acceptable (alpha=.71).

Adverse events.

Clinicians completed a structured progress note following each session to monitor adverse events. Possible serious adverse events (SAEs) were coded following the criteria of the ICH Guideline for Good Clinical Practice (2018), and study investigators discussed and reached consensus on attributions of whether SAEs were possibly related to the study treatments.

Power

Prior to study onset, a simulation study was used to estimate statistical power. Assumptions included no more than 10% attrition per 6 months and a moderate within-subjects correlation of r=.5 between repeated measures. Using a two-sided significance level of α=.05, this design had 80% power to detect a linear change in treatment effect from no difference at baseline to d=.62 at the end of treatment (a medium ES) with an initial sample of 80 children allocated in a 1:1 ratio to CBT and ESCT.

Data Analysis

A hierarchical generalized linear model (HGLM) using HLM 8.0 software (Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2019) was used for the primary outcome analysis. HGLM/HLM is a full-information analytic procedure that provides unbiased parameter estimates for the intent-to-treat sample when data are MAR. An HGLM model was fit for the POPE-JEGA scale using a Poisson distribution and a log link function, with fixed effects for sample (a dummy variable for Sample 1 versus Sample 2, included to control for any differences between the two study samples), treatment condition (CBT coded 0; ESCT coded 1; Bauer, Sterba, & Hallfors, 2008), and pretreatment score. Due to the partially nested nature of the data (i.e., ESCT participants were treated in groups and CBT participants were treated individually), the HGLM model accounted for this using the multilevel data structure recommended by Bauer and colleagues (2008) (i.e., what Bauer et al. refer to as “Approach 3” for modeling partially nested data). Using this approach, a group identification variable was created in which ESCT participants in the same treatment group shared a group identification number (i.e., since there were 8 ESCT treatment groups total, the 53 children in ESCT each had one of eight group identification numbers), and in which CBT participants each received their own “group” identification number since they were treated individually (i.e., there were 54 additional group identification numbers—one for each child in CBT). In the Bauer and colleagues model, the intercept is fixed and the slope is random. Applying this model to POPE-JEGA data, the mixed HGLM equation was:

$${\eta }_{ij}={\gamma }_{00}+{\gamma }_{10}\ast PretreatmentPOPE-JEGAScor{e}_{ij}+{\gamma }_{20}\ast Stud{y}_{ij}+{\gamma }_{30}\ast TreatmentConditio{n}_{ij}++u_{2j}\ast TreatmentConditio{n}_{ij}$$

Due to a moderate level of missing pre- and post-treatment POPE-JEGA data (see Missing Data below), missing post-treatment POPE-JEGA scores were multiply imputed by fully conditional specification, assuming an arbitrary pattern of missing data, using all variables in the HGLM model to produce 20 imputed datasets. Pooled model parameters utilizing the imputed datasets were computed via HLM 8.0 software and are presented in the Primary Analyses. In addition, several sensitivity analyses were conducted as a check on the imputed model parameters.

The HLM model for YTP scores differed in several ways from the primary HGLM model used for POPE-JEGA scores. Unlike the POPE-JEGA dataset in which only pre- and post-treatment data were collected, the YTP dataset included scores from each treatment session (i.e., up to 32 datapoints per child). In order to incorporate the Bauer and colleagues (2008) approach for partially nested treatment designs into a multilevel model that included up to 32 repeated measures per participant, a 3-level HLM model using a normal distribution was employed (repeated measures nested within participants nested within the group identification number used to account for nesting in ESCT treatment groups). The mixed HLM equation for YTP scores was:

$$\begin{gathered} YTPTotalScor{e}_{tij}={\gamma }_{00}+{\gamma }_{010}\ast Stud{y}_{ij}+{\gamma }_{020}\ast TreatmentConditio{n}_{ij}+{\gamma }_{100}\ast SessionNumbe{r}_{tij} \\ +{\gamma }_{110}\ast SessionNumbe{r}_{tij}\ast Stud{y}_{ij}+{\gamma }_{120}\ast SessionNumbe{r}_{tij}\ast TreatmentConditio{n}_{ij} \\ +r_{1ij}\ast SessionNumbe{r}_{tij}+u_{12j}\ast SessionNumbe{r}_{tij}\ast TreatmentConditio{n}_{ij}+e_{tij} \end{gathered}$$

In this model, the parameter of greatest interest is γ₁₂₀. which represents the fixed effect for the interaction between treatment condition and session number (which reflects the amount of exposure to treatment as well as the passage of time). The familywise error rate stemming from multiple comparisons was addressed with the Holm-Bonferroni method applied to the POPE-JEGA and YTP models (α_famiiywise=.05). Because no statistically significant findings were negated by these corrections, unadjustedp values are reported.

Results

Recruitment for Sample 1 began on December 1, 2011; final post-treatment data were collected on November 30, 2015. Recruitment for Sample 2 commenced on March 1, 2012 with final post-treatment data collected on February 1, 2017. These study periods correspond with the funding cycles of the two grants supporting this research. As illustrated in eFigure 1, of the 106 children who began treatment, 83 completed treatment and 23 discontinued treatment prematurely. All but one family who discontinued treatment did so for personal reasons such as scheduling problems. One family discontinued treatment due to an SAE (psychotic symptoms resulting in hospitalization). Psychotic symptoms are relatively common in ASD and this child had a history of psychotic symptoms prior to study entry. One other SAE was reported to a study clinician: a participant reported thoughts about wanting to harm a classmate, which was reported to appropriate agencies. This child had a history of aggressive thoughts and behaviors. These two SAEs occurred in the CBT condition. Neither child had reacted negatively to CBT procedures or to their therapists. These SAEs were deemed unlikely to be study-related.

At pre-treatment, 25 of 103 parents (24.2%) indicated that their child was currently taking psychiatric medication (stimulants, 12.6%; SSRIs, 10.7%; alpha agonists, 8.7%; atypical antipsychotics, 5.8%; anticonvulsants, 1.9%; and SNRIs, 1.0%). Some children used multiple medications. There were no statistically significant group differences in pre-treatment medication utilization.

Missing Data

Table 2 presents baseline descriptive data for the outcome measures and eFigure 1 provides a summary of the sources of missing data in the study. POPE-JEGA observations were attempted for all participants at pre-treatment, though 23 of the participants were not present in an outdoor free play setting for at least 5 observation intervals on at least one of the observation days (as described above in Measures; e.g., the child was indoors, was eating, was lined up to return to class, etc., for 11 or more intervals on both observation days). As a result, 84 of 107 children (n_CBT=40, n_ESCT=44) had valid POPE-JEGA observational data at pre-treatment. Of these 84 children, 46 (21/40 [52.5%] for CBT and 25/44 [56.8%] for ESCT; χ²=0.16,p=.69) had pretreatment POPE-JEGA scores of 20 out of 30 or lower (i.e., below a 66.6% rate of observed intervals with joint engagement). If a child discontinued treatment prior to session 32, an attempt was made to conduct school observations. At post-treatment/post-discontinuation (hereafter referred to as “post-treatment”), 84 children (n_CBT=39, n_ESCT=45) were successfully observed at school, but only 66 of them (n_CBT=31, n_ESCT=35) were actually present in an outdoor free play setting for at least 5 observation intervals for at least one of the observation days (others were, e.g., eating lunch and not coded). Overall, 58 children (n_CBT=26, n_ESCT=32) had valid POPE-JEGA data at both pre- and post-treatment, 26 children (n_CBT=14, n_ESCT=12) had valid data at pre-treatment but not at post-treatment, 8 children (n_CBT=5, n_ESCT=3) had valid data at post-treatment but not at pre-treatment, and 15 children had no valid data at either time point.

Table 2.

Baseline Observed Autism Symptom Scores (POPE-JEGA) and Parent Ratings of Symptom Severity (YTP) in the CBT and ESCT Conditions

Scale	CBT	ESCT
POPE-JEGA
n	40	44
M	16.33	15.27
SD	10.18	11.51
YTP
n	50	47
M	22.33	21.09
SD	5.51	5.47

Note. POPE-JEGA=Playground Observation of Peer Engagement Joint Engagement/Games with Rules scale. YTP=Youth Top Problems scale. CBT=cognitive behavioral therapy. ESCT=enhanced standard community treatment. POPE-JEGA scores reflect the number of intervals out of 30 possible intervals in which a participant was rated by an independent evaluator as jointly engaged during school recess. Pre-treatment YTP scores presented here are the mean of YTP Total scores from sessions 1-3.

Prior to data analysis, a missing data analysis was conducted on the outcome variables, using the variables planned for the statistical models simultaneously as probes of patterns of missing data. Little’s missing completely at random (MCAR) test was not statistically significant (χ²=64.34, df= 60, p=.33), offering no evidence of a systematic pattern of missingness. Univariate tests showed that the missingness in YTP scores at post-treatment was associated with higher YTP scores at pre-treatment (ps<.05), suggesting that, at best, YTP scores could be missing at random (MAR) but not MCAR. No systematic pattern of missingness was found for POPE-JEGA scores.

Primary Outcome Analysis

All analyses were by the original assigned groups. In the HGLM model for POPE-JEGA scores using the 20 datasets, 26 (n_CBT=14, n_ESCT=12) missing post-treatment scores were imputed (see Data Analysis, above), for a total of 84 (n_CBT=40, n_ESCT=44) children in the primary outcome analysis. The intraclass correlation coefficient (ICC) for the ESCT condition was .25 (25% of POPE-JEGA variation at post-treatment occurred across the mean scores for the eight ESCT groups, while the remaining 75% occurred among children nested within these groups). There is no ICC for the CBT condition in this model (cf. Bauer et al., 2008). There was a statistically significant treatment effect (p<.001) showing that children in CBT had higher POPE-JEGA scores at post-treatment than did children in ESCT (see Table 3 and Figure 1), with a medium Cohen’s d effect size (calculated using formulae from Feingold [2013]). Estimated marginal means (EMMs) and effect size (d with 95% CI) at post-treatment are provided in Table 4. The Figure 1 graph, d, and EMMs are based on a linear variant of the model (with a comparable pattern of statistical significance) to preserve the original scaling for descriptive purposes.

Table 3.

HGLM/HLM Models for the Effect of CBT Versus ESCT on POPE-JEGA Scores and YTP Scores

POPE-JEGA Scores
Fixed Effect	Coefficient	Standard error	t-ratio	p-value
Intercept, γ00	2.5563	0.0739	34.604	<.001
Pre-Treatment POPE-JEGA Score, γ10	0.0152	0.0030	5.140	<.001
Study, γ20	0.2263	0.0936	2.418	.021
Treatment Condition, γ30	−0.3211	0.0932	−3.443	<.001
Random Effect	SD	Variance Component	df	χ2	p-value
Treatment Condition, u2	0.1182	0.0140	7	17.0318	.017

YTP Scores
Fixed Effect	Coefficient	Standard error	t-ratio	p-value
For Intercept-1, π0
Intercept-3, γ000	21.6858	0.2330	93.043	<.001
Study, γ010	−3.4546	0.3402	−10.154	<.001
Treatment Condition, γ030	−1.1277	0.3365	−3.351	<.001
For Session Number slope, π1
Intercept-3, γ100	−0.3550	0.0421	−8.414	<.001
Study, γ110	0.0406	0.0621	0.653	.517
Treatment Condition, γ120	0.2007	0.0643	3.121	.003
Random Effect	SD	Variance Component	df	χ2	p-value
Session Number, r1	0.2726	0.0743	41	946.7305	<0.001
level-1, e	3.9564	15.6528
Session Number/Treatment Condition, u12	0.0649	0.0042	7	11.1192	.133

Note. HGLM/HLM=hierarchical (generalized) linear model. POPE-JEGA=Playground Observation of Peer Engagement Joint Engagement/Games with Rules scale. YTP=Youth Top Problems scale. CBT=cognitive behavioral therapy. ESCT=enhanced standard community treatment. HGLM for POPE-JEGA scores uses a Poisson distribution and a log-link function. HLM for YTP scores uses a normal distribution. The POPE-JEGA model is based on 20 multiply imputed datasets of the 84 participants with valid pre-treatment POPE-JEGA scores. The YTP model is based on all available weekly YTP data, including participants who have less than 32 weeks of data.

Figure 1.

Estimated marginal means (EMMs) for POPE-JEGA scores at post-treatment for CBT and ESCT based on a linear variant of the primary HGLM model (to preserve the original scaling for descriptive purposes) (figure above; p_Condition<.001); and EMMs for YTP scores at Session 1 (beginning of treatment) and Session 32 (end of treatment) for CBT and ESCT based on HLM growth model parameters (figure below; p_{Condition x Time}=.003). POPE-JEGA=Playground Observation of Peer Engagement Joint Engagement/Games with Rules scale. YTP=Youth Top Problems scale. CBT=cognitive behavioral therapy. ESCT=enhanced standard community treatment.

Table 4.

Estimated Marginal Means of Post-Treatment Scores from HGLM/HLM Models and Effect Size (Cohen’s d) Estimates for CBT Versus ESCT on POPE-JEGA Scores and YTP Scores

	Post-treatment Estimated Marginal Means (SEs)
Scale	CBT	ESCT	Cohen’s d (95% CI)
POPE-JEGA Scores	18.18 (1.53)	13.15 (1.54)	.50 (.06, .93)
YTP Scores	10.33 (.72)	15.62 (.71)	.87 (.45, 1.27)

Note. HGLM/HLM=hierarchical (generalized) linear model. POPE-JEGA=Playground Observation of Peer Engagement Joint Engagement/Games with Rules scale. YTP=Youth Top Problems scale. CBT=cognitive behavioral therapy. ESCT=enhanced standard community treatment. Effect size estimates were calculated using formulae provided by Feingold (2013).

Sensitivity analysis.

Two types of sensitivity analyses were conducted. The primary model’s inclusion of baseline scores as a covariate led to a loss from the model of 8 valid post-treatment POPE-JEGA scores (see Missing Data, above). As a result, a second E1GLM model was conducted with the 20 imputed datasets in which no pre-treatment POPE-JEGA score covariate was included in the model; hence, the entire sample (N=107) was included in this model. Although the proportion of imputed data increased in this model (41 post-treatment scores were imputed), the treatment effect was still statistically significant (γ_treatment=−265, SE=.121; p=.032). Secondly, an HGLM model using only raw, non-imputed data (i.e., those with valid pre- and post-treatment data; n=58) was evaluated to test the limits of the primary model; this model also yielded a significant treatment effect (γ_treatment=−.33, SE=.093; p=.001). In summary, the sensitivity analyses resembled the findings of the primary analysis.

Secondary Outcome Analyses

For personalized symptom severity rated by parents at each treatment session (total YTP scores), all children who began treatment were included in the analysis (n_CBT=54, n_ESCT=52) and the number of session-by-session data points per child ranged from 1 to 32 (M_CBT=22.15, SD=8.36; M_ESCT=25.22, SD=8.22; ns). ICCs for ESCT were computed for each level of the unconditional model: 37% of YTP variation occurred across repeated measurements within children nested within ESCT groups (level 1), 49% of YTP variation occurred across the mean YTP scores for children nested within ESCT groups (level 2), and 14% of YTP variation occurred across the mean YTP scores for the 8 ESCT groups (level 3). There was a statistically significant effect (p=.003) for the treatment condition by time interaction showing that CBT improved more on YTP scores over the course of 32 sessions than did ESCT, with a large effect size (see Tables 3 and 4, and Figure 1).

Exploratory Analyses

To further explore the YTP results, we examined the content of the YTP ratings at the construct level. Of the 318 individual YTP symptoms generated by parents, 181 (56.4%) were social-communication (SC) symptoms, 58 (18.1%) were restricted/repetitive behavior (RRB) symptoms, 67 (20.9%) were externalizing symptoms, and 12 (3.7%) were internalizing symptoms based upon rater coding into one of the 43 YTP descriptive symptom categories (see Methods). HLM models were fit examining treatment effects for each domain separately. These analyses are not independent of the primary YTP analysis above, but rather a decomposition of that analysis by symptom type. The treatment condition by time interaction effects suggested that CBT improved more on YTP scores in each domain (SC, RRB, externalizing, and internalizing; allps <.01) over the course of 32 sessions than did ESCT, with estimated reductions in symptom scores from pre- to post-treatment in CBT of 46.0% for SC symptoms, 53.2% for RRB symptoms, 39.5% for externalizing symptyoms, and 66.9% for internalizing symptoms (see eTable 2).

To explore the credibility of the two treatment conditions, pre-treatment parental expectancies (assessed with the PETS) and post-treatment parent satisfaction with treatment (assessed with the CSQ) were examined. Mean differences between the two groups on the PETS (Ms=4.05 and 3.96, SDs=.35 and .37 for CBT and ESCT, respectively) were small and nonsignificant (t=1.22, df=102, p=.23), providing no evidence of differential parental expectations of the two treatment conditions. In contrast, at post-treatment, parental satisfaction as rated on the CSQ was higher in CBT than in ESCT (Ms=6.43 and 6.00, SDs=.52 and .62, respectively; t=3.22, df=76, p=.002). However, given that the CSQ items are rated on a 7-point scale where 7 is the highest possible score, consumer satisfaction was high, in an absolute sense, in both groups.

To explore the possible effect of age and level of development on the treatment outcomes, two additional models were examined for each outcome: one adding age (in months) as a predictor, as well as its interaction with treatment condition; and one adding estimated IQ as a predictor, as well as its interaction with treatment condition. For POPE-JEGA scores, the child’s age was inversely associated with post-treatment gains (B=−.0050, SE=.0023, p=.033) such that older children exhibited less improvement in either treatment condition than younger children (there was no age by treatment condition interaction). For YTP scores, the child’s age was also inversely associated with session-by-session improvements (B=.0012, SE=.0005, p=.008) such that older children exhibited a slightly less rapid decline in symptom scores over time than did younger children (again, there was no age by treatment condition interaction). A similar pattern was observed for estimated IQ, which was inversely associated with session-by-session improvements in YTP scores (B=.0026, SE=.0007, p=.001) such that children with higher estimated IQ exhibited a slightly slower decline in scores over time; there was no IQ by treatment condition interaction. Estimated marginal means from the model including estimated IQ illustrated that this effect is partially a function of higher symptom severity at session 1 for children with lower estimated IQ, suggesting regression to the mean as a possible determinant of the significant IQ by session number interaction effect for YTP scores (see eFigure 2).

Clinically Significant Change

The clinical significance of the treatment effect was explored by examining the reliable change index (RCI; Jacobson & Truax, 1991) on the POPE-JEGA scale, using the following equations:

$$\begin{gathered} S{E}_{Measure}=SD\ast \sqrt{1-r_{11}} \\ RCI=\left(rawposttest-rawpretestscores\right)/S{E}_{measure} \end{gathered}$$

The standard error of the measure (SE_Measure) was derived based on pre-treatment POPE-JEGA scores and interrater reliability estimates (r₁₁) reported in the Method section. Reliable change was assessed for all children who were jointly engaged less than 67% of the time during free play at pretreatment (those with higher baseline levels of engagement had essentially no room for improvement using this method) and had both pre- and post-treatment data. In CBT, 11 of 13 (84.6%) of these children exhibited positive reliable change, in comparison with 8 of 17 (47.1%) of children in ESCT (χ²=4.47, df=1, p=.03).

RCI analyses were also conducted for YTP scores. For the YTP, scores aggregated across the first five sessions with valid YTP data and the last five sessions with valid YTP data (e.g., for children who ended treatment early, this would include the YTP scores from the last 5 sessions they received) served as the input parameters for the model (note: for this analysis, we only included children with at least 10 sessions of YTP data; due to random missing data and early discontinuation, this resulted in the loss of 11 cases for this analysis). Cronbach’s alpha for the YTP total scores from sessions 1–5 was used to compute the SE_Measure. In CBT, 39 of 50 (78.0%) participants exhibited positive reliable change, in comparison with 25 of 46 (54.3%) of children in ESCT (χ²=6.03, df=1, p=.01). Overall, the RCI analyses mirrored the HGLM/HLM analyses, illustrating that children in CBT had a higher probability of positive treatment response on the primary outcome variable and on parent-defined treatment goals than did children in ESCT.

Discussion

Results suggest that SEBASTIEN, a modular one-on-one CBT-based psychotherapy program, may facilitate greater improvement in social communication with peers and greater reduction of high priority symptoms relative to an analogue community care intervention (i.e., the format used in social skill training groups, supplemented with additional cognitive and behavioral practices). SEBASTIEN matched evidence-based therapeutic practices to each child’s clinical profile based on a treatment algorithm, seeking to maximize the dosage of relevant therapeutic techniques (e.g., exposure and response prevention, friendship skills) for each child. Both treatment conditions employed evidence-based practices to support social communication and reduce RRBs and emotion dysregulation. However, in a multifaceted disorder like ASD, a structured, linear treatment approach such as that used in ESCT may ultimately allocate both unneeded interventions for clinical problems that some group members do not have as well as an insufficient intervention dosage for persistent symptoms that they do have. Comparatively, the modular approach utilized in SEBASTIEN may facilitate the allocation of specific practices relevant to each child’s symptom profile that may promote greater clinical gains. This capacity for individualization of treatment—potentiating focal symptom reduction—as well as the flexibility of the one-on-one psychotherapy format (e.g., it does not require a group of similarly-aged children with similar clinical needs to commence treatment) and the fit of the one-on-one modality with existing community-based treatment allocation patterns (e.g., Stuart et al., 2017; Zablotsky et al., 2015) represent some of the possible advantages of modular one-on-one CBT-based psychotherapy for autism-related symptoms. Nonetheless, a circumspect appraisal of these findings is appropriate due to the moderate amount of missing data in the primary outcome measure and a need for replication by other research teams.

A recent meta-analysis of psychological treatments with cognitive-behavioral elements for individuals with ASD (Weston et al., 2016) offers a useful reference point for situating these findings. This meta-analysis, along with other recent narrative reviews (e.g., Wood et al., 2017) illustrates that, to date, one-on-one CBT protocols to treat autism-related symptoms have been essentially untested; only 1 of 24 studies reviewed in the Weston and colleagues meta-analysis used a one-on-one CBT treatment protocol focusing on autism-related symptoms, and that protocol was a precursor of the presently-evaluated CBT protocol (i.e., Wood et al., 2014). Wood and colleagues (2014) was a small pilot study (N=13) and it included clinical anxiety as an inclusion criterion and a major treatment focus. The remaining 23 studies reviewed by Weston and colleagues entailed group CBT interventions, mainly social skills training groups with some cognitive therapy components.⁵ In general, these protocols were compared to waitlist or treatment-as-usual conditions. As a group, these studies yielded an average effect size for autism-related symptoms ranging from .10 (self-report) to .52 (parent/informant report) after potential outliers and/or studies with risk of bias were removed, reflecting small to medium-sized treatment effects. To put these meta-analytic findings in context, large-scale meta-analyses of general child psychotherapy treatment effects have found that effect sizes are higher in studies using waitlist control groups or when relying on informants who were aware of or involved with the child’s treatment (e.g., parents, youth) (Weisz et al., 2017). In comparison with the set of treatment protocols and their corresponding trials in Weston and colleague’s meta-analysis, the present study (a) evaluated a one-on-one CBT-based psychotherapy protocol focused primarily on treating autism-related symptoms, (b) used a modular approach to individualize CBT for autism and concurrent clinical features for each child, and (c) implemented stringent methodological features such as an active comparison group, IE-rated observations in a naturalistic setting, and an adequately powered sample, each of which tends to constrain effect sizes (Weisz et al., 2017). Nonetheless, the effect size point estimates for SEBASTIEN in the present trial compared favorably to the average effect sizes reported by Weston and colleagues in studies without these methodological features.

Additional CBT-based interventions have been studied that address clinical anxiety in school-aged children with ASD (e.g., Reaven et al., 2018; Storch et al., 2013; White et al., 2013; Wood et al., 2009; Wood et al., 2009, 2020). Although autism-related symptoms, such as impaired social reciprocity, have been at most a secondary emphasis in these CBT protocols, it is notable that at least parent-report measures of autism-related symptoms have sometimes had positive response to these CBT interventions as well (e.g., Storch et al., 2013; White et al., 2013; Wood et al., 2009; Wood et al., 2020). Collectively, these results suggest that modular one-on-one CBT with an intensive focus on a child’s most pressing clinical needs may facilitate improvement in autism-related symptoms for some school-aged children.

The relatively efficient and inexpensive nature of time-limited CBT-based psychotherapy in comparison with other therapies (Roundfield & Lang, 2017) may make the utilization of a one-on-one modular CBT protocol like SEBASTIEN with a primary focus on autism-related symptoms sensible in some cases for treatment-seeking youth. Of course, practicing clinicians will likely need to adhere sufficiently to the various practices within the protocol, including the in vivo exposures and other behavioral supports that may be less familiar to some therapists, to achieve similar positive outcomes. Further, even 32 sessions of CBT may not be sufficient to achieve optimal outcomes for many children with ASD, and CBT is not an alternative to intensive or developmental treatments for language and cognitive skill development. Still, the capacity of a modular CBT-based psychotherapy program like SEBASTIEN to individualize treatment across a wide variety of clinical target areas may make it a good fit for some school-aged youth and their treating clinicians. To this end, a free online clinician training and guidance app for the CBT practices evaluated in this study is available at meya.ucla.edu.

Exploratory analyses revealed that about three quarters of the highest priority symptoms addressed in this trial from the standpoint of parents were autism-related symptoms (i.e., social communication and RRB-related challenges) with the remainder reflecting manifestations of emotion dysregulation. CBT had a relatively similar advantage over ESCT across these symptom domains, illustrating its potential applicability to both autism-related symptoms as well as concurrent mental health needs in children. In addition, for both treatment conditions, children who were developmentally less mature (as indexed by both chronological age and estimated IQ) responded somewhat better to treatment. In analyzing the pattern of data, one possible explanation could be regression to the mean, since there were (nonsignificant) trends towards higher baseline scores for children who were younger and with lower IQs. Conversely, perhaps the interventions were simply better suited to less mature children, suggesting that adaptations to engage more mature children could be helpful.

The strengths and weaknesses of the ESCT condition as a comparator merit consideration. One strength is that ESCT uses a familiar format (group training) seen in many ASD interventions. Furthermore, the study was designed to mask hypotheses about which of the two treatments was expected to be superior; as noted above, the consent form and treatment elements in ESCT were designed to render the distinction between the two treatments ambiguous from a parent’s perspective, and parents were not told which treatment their child had been assigned to or what the alternative treatment entailed (i.e., 1:1 CBT). Parent expectations of treatment were similar at baseline in the two groups (i.e., small effect sizes and nonsignificant effects). However, ESCT differed from CBT through its limited capacity for individualization and its group format, which may have impacted parental and child expectancies and engagement in unmeasured ways. While treatment satisfaction at post-treatment was high in both groups, it was statistically significantly higher in CBT, suggesting parent perceptions of the two intervention conditions separated at some point during the treatment.

Several limitations are noted. First, a recruited sample treated in a university setting may not be representative of typical treatment-seeking families served in community clinics and schools; effectiveness research will be needed to evaluate CBT-based psychotherapy for children with ASD under real-world conditions. Second, this study only included children no more than one standard deviation below the mean in general cognitive abilities; the applicability of CBT-based psychotherapy to other children and adolescents with ASD, including minimally verbal children, is important to investigate. Third, the study entailed a relatively long treatment period and attrition occurred at each phase of the trial. Even in the CBT condition, the premature discontinuation rate was relatively high at 18.5%, raising questions about the ecological validity of engaging children ranging in age from 6-13 years in a 32-session intervention. A more flexible modular approach to implementation could make both the modules used and the length of treatment responsive to clinical need, remission of high priority symptoms, and family preference. Furthermore, given our comparison of two multicomponent intervention packages, the study is unable to shed light on which therapeutic elements accounted for the differential outcomes among the two conditions.

Another important limitation is that the collection of school observational data was at times hampered by a lack of alignment among the multiple stakeholders involved. Ultimately, the school observation assessment selected a priori as the primary outcome measure for the study (POPE-JEGA) had two drawbacks: only 58 of the 107 participants had both pre- and post-treatment POPE-JEGA data that was usable (i.e., the child was physically present in an outdoor free play setting at the time of the observations), and only 46 (54.8%) of 84 children with usable pretreatment data exhibited joint engagement difficulties for more than 33% of the observed intervals at pre-treatment, constraining the room for improvement for nearly half of the sample. A missing data analysis and sensitivity analysis with multiple imputation helped allay concerns about the nature and impact of the missing data. Nonetheless, it is preferable to have less attrition, more complete cases, and a primary outcome that is more sensitive to potential improvement. Although observational measures are desirable to document treatment response in a more objective way than interviews with informants who are aware of treatment condition assignment, alternative approaches for objective assessment of autism-related symptoms in children with ASD have been developed and could be considered in future RCTs (e.g., Ness et al., 2019; Usher, Burrows, Schwartz, & Henderson, 2015; Zhang et al. 2020).

Additionally, missingness of YTP scores at post-treatment was associated with higher YTP scores at pre-treatment, suggesting that premature dropout was more likely for youth whose parents rated them as having more severe top problems over the first five sessions of treatment. Perhaps this finding identified children who did not begin to make clinical gains from the parents’ perspective early in treatment, and whose parents were less likely to persevere through 32 sessions of treatment; how these children may have fared with the whole course of treatment is unknowable and their absence could have had an effect of unknown magnitude on the YTP analysis. However, the YTP analysis was based on all available session by session YTP ratings (i.e., as many as 32 scores), rather than pre-post data, allowing the trajectories of even those who went on to prematurely dropout to be modelled with relative precision, reducing concerns about the effect of some missing data in this analysis. Further, although the protocol is modular, the modules/treatment foci were selected based on an algorithm dependent on the clinician’s and parents’ appraisals of clinical need and progress as a guide, adding some complexity and ambiguity to treatment implementation; the treatment might be improved with a personalized medicine approach in which a priori information about the child is used to select modules. Relatedly, the study design cannot clarify whether the greater improvements seen in the CBT condition were actually associated with the modular nature of the intervention, or were simply due to the additional individual attention that was given by the therapists.

The treatment of autism-related symptoms with one-on-one psychotherapy has rarely been evaluated in controlled research. While challenges with achieving generalization have been evident in some psychological treatments for autism-related symptoms, the current findings suggest that with a modular design focused on a child’s most pressing clinical needs, some children with ASD may be able to achieve clinical benefit from a CBT-based psychotherapy protocol like SEBASTIEN. The limitations noted above related to missing data suggest that further controlled evaluation of SEBASTIEN (and similar modular one-on-one CBT-based psychotherapy protocols for autism-related symptoms) will be needed to probe the scope of its treatment effects and to identify which children are most likely to benefit from this type of intervention. Additional research on interventions like SEBASTIEN that aim to address multiple autism-related symptoms should consider using a primary outcome measure with a greater breadth of symptom and setting coverage as well as less susceptibility to ceiling effects than the structured school recess observations used in this study. Lastly, in subsequent studies it will be worthwhile to examine whether, with the use of different dosages of this type of treatment—alone or in combination with other treatments—some children may be able to achieve positive downstream effects on future adaptive functioning.

Public Health Significance:

There is currently no well-established psychotherapy treatment for autism-related symptoms. The present study identified cognitive behavioral therapy as a potential treatment for reducing symptom severity among children with autism spectrum disorder.

Appendix A

Please note that there are no other published, in press, or under review studies that utilize the submitted manuscript’s present dataset.

Data collected for the study, including individual participant data and a data dictionary defining each field in the set is available through the National Database for Autism Research (NDAR) via the following URL: https://nda.nih.gov/edit_collection.html?id=2007. The NDAR repository sets standard criteria for accessing data across all studies, which are published on its website. The study protocol is available upon email request to the first author.