Adolescents with autism spectrum disorder and social skills groups at school: A randomized trial comparing intervention environment and peer composition.

Abstract

This study used a randomized controlled trial to compare two distinct models of group social skills interventions with adolescents with autism spectrum disorder (ASD). Participants had a confirmed diagnosis of ASD, an IQ greater than or equal to 70, and were educated in the general education setting. Data from 62 adolescent participants who were randomized to one of two treatment conditions (SKILLS vs. ENGAGE) were analyzed. SKILLS participants had a diagnosis of ASD, or social difficulties. ENGAGE groups included adolescents with ASD and typically developing (TD) peer mentors. SKILLS and ENGAGE participants both improved joint engagement and reduced solitary engagement, however, SKILLS participants reported higher social stress and lower quality interpersonal relationships at exit, and increased emotional symptoms and problem behaviors at follow-up compared to the ENGAGE group. The findings suggest that within inclusive secondary school settings, it may be beneficial to include TD peers in social intervention groups.

Introduction

Adolescents with autism spectrum disorders (ASD) without intellectual disabilities are increasingly being placed into general education classes (U.S. Department of Education, National Center for Education Statistics, 2012), which provide opportunities for them to interact with typically developing (TD) peers (Ochs et al., 2001). With little peer relationship support or access to social interventions within inclusive settings (Wainscot, Naylor, Sutcliffe et al., 2008), it is difficult for adolescents with ASD to take advantage of the social opportunities afforded to them. Adolescents with ASD spend less time socializing with peers, have smaller peer groups, and are less likely to have reciprocal friendships compared to TD adolescents (Locke, Ishijima, Kasari, & London, 2010; Petrina, Carter, & Stephenson, 2014; Symes & Humphrey, 2010). Similarly, they are less likely than TD adolescents to socialize outside of school (Kuo, Orsmond, Cohn, & Coster, 2013; Orsmond & Kuo, 2011). Within existing relationships, they often report increased feelings of loneliness and insecurity (Humphrey & Lewis, 2008; Locke, Ishijima, Kasari, & London, 2010; Pisula & Łukowska, 2012), and due to difficulties interpreting social rules, they are vulnerable to bullying and victimization (Fisher & Taylor, 2016; Humphrey & Lewis, 2008; Humphrey & Symes, 2011). These challenges reflect the impact of core deficits in social communication and social understanding (Humphrey & Symes, 2011) on social engagement and success.

Although inclusion provides propinquity to TD peers, the successfulness of inclusion hinges on the extent to which TD peers are accepting of individuals with ASD within the inclusive setting (Ochs et al., 2001; Symes & Humphrey, 2010). Individuals with ASD benefit from having peer relationships with disabled and non-disabled peers (Bauminger, 2008; Petrina, Carter, Stephenson, & Sweller, 2016). Bauminger et al. (2008) examined the social relationships of children with ASD in mixed-dyad friendships (with TD and ASD peers) and non-mixed dyad friendships (ASD only). Children with ASD in mixed-dyad friendships were more socially responsive, exhibited more sophisticated play skills, and had more stable friendships compared to children with ASD who had non-mixed friendships. Yet, forming mixed relationships is difficult for many children with ASD, who tend to be more socially connected to TD peers in early grades, but move to the periphery of social networks or are isolated in later grades (Rotheram-Fuller, Kaari, Chamberlain, & Locke, 2010). Locke, Ishijima, Kasari, and London, (2010) examined the social networks of adolescents with ASD enrolled in an inclusive high school and found that these adolescents tended to hang out in small groups with other adolescents with ASD, or they were isolated. They described their relationships as lonelier and less intimate, compared to the TD control group. Thus, propinquity to TD adolescents does not appear to be enough to improve social network status or co-occurring social-emotional symptoms. More research is needed to examine the social experiences of adolescents with ASD within inclusive secondary settings, and to study the extent to which these individuals benefit from interventions that are designed to provide opportunities for mutual social engagement with others.

Being on the periphery of social groups and isolated from TD groups means that adolescents with ASD have fewer opportunities to learn social norms within the dominant peer culture at school. The transactional relationship between adolescents with ASD and the social environment at school highlights bidirectional social misunderstandings. Individuals with ASD tend to have difficulty interpreting social norms within the general education school culture, due to social communication challenges related to their diagnosis. Concomitantly, TD adolescents may avoid social interactions with individuals with ASD, because they feel that they lack skills to do so effectively (Bottema-Beutel & Li, 2015). Given the salience of peer relationships and reputational concerns (Rivet & Matson, 2011), TD adolescents tend to exclude individuals with comparatively lower social status than their own status (Horn, 2006).

Social Skills Interventions

Group social interventions are empirically supported and have shown to be efficacious in improving social outcomes for adolescents with ASD (Laugeson, Ellingsen, Sanderson, Tucci, & Bates, 2014; Laugeson, Frankel, Mogil, & Dillon, 2009, Laugeson, Frankel, Gantman, Dillon, & Mogil, 2012). Many group-based social interventions use common evidence-based active ingredients, including didactic instruction targeting specific skills, modeling, and role play (Gutman et al., 2010; Herbrecht et al., 2009; Laugeson et al., 2009; Laugeson et al., 2012; Olsson, Rautio, Asztalos, Stoetzer, & Bölte, 2016). Some interventions also use parent-mediated training and homework to promote generalization by providing opportunities to practice targeted skills between sessions (Herbrecht et al., 2013; Laugeson et al. 2009; Laugeson et al., 2012; Minihan, Kinsella, & Honan, 2011). The evidence concerning social interventions for adolescents, however, is primarily from clinic-based social skills programs often composed of children from different schools who share a diagnosis of ASD (Gutman et al., 2019; Herbrecht et al., 2009; Laugeson et al., 2009; Laugeson et al., 2012; Olsson et al., 2016). Although adolescents with ASD have shown improvements in clinic-based interventions, the extent to which newly acquired social skills generalize to the school setting is unclear.

Two different types of group social skills interventions have been shown to be effective when implemented in the secondary school setting: clinic-based and peer-mediated interventions. Laugeson, Ellingsen, Sanderson, Tucci, & Bates (2014) implemented PEERS, a clinic-based intervention, into a secondary special education school setting. The school-based PEERS intervention utilized didactic instruction, modeling, and role-play. A special education teacher facilitated the intervention every day for 14 weeks during homeroom, and in line with the clinic-based approach, all participants had a diagnosis of ASD. Results indicated that teachers reported a significant improvement in the areas of social motivation, social awareness, and social communication. Adolescents also reported increased get-togethers with classmates outside of school. Parent reports suggested elevated social stress in the control group, however the results did not reach significance. The findings support the application of clinic-based interventions in special education settings, but the dose and setting of the PEERS intervention may be difficult on inclusive campuses, where individuals with ASD attend homeroom with TD students.

Peer-mediated interventions, in which TD peers are trained as intervention agents to facilitate social skills interventions with individuals with ASD, may serve as an alternative to clinic-based interventions (Chan, 2009). By providing proximity to peers, peer-mediated interventions have been shown to be effective at increasing successful social opportunities (Symes & Humphrey, 2010; Ochs, Kramer-Sadik, & Solomon, 2001), and promoting the generalization and maintenance of skills (Watkins et al., 2015). Schmidt & Stichter (2012) used multiple-treatment single-subject design to compare two different types of peer-mediated interventions for adolescents with ASD at school. In one condition, TD peers were trained to move close to the individual with ASD, to respond to conversational initiations, and to initiate conversations. Generalization was promoted by situating the intervention in real-life social periods during the day, thereby improving the social validity of the intervention. Results indicated that peer-mediated interventions were effective in improving the generalization of skills to outside of the intervention setting, however there was a greater improvement in social engagement during the peer-initiation condition. Thus, adding a peer-mediated component to group-based interventions may improve the social engagement of adolescents with ASD within inclusive secondary settings and the generalization of skills to non-intervention settings.

To our knowledge, no studies have used a randomized controlled trial to examine the effects of clinic-based and peer-mediated social interventions for adolescents with ASD who are educated in inclusive secondary school settings. In primary school populations, however, Kasari et al. (2015) examined the efficacy of two different models of school-based group social intervention (SKILLS and ENGAGE). The SKILLS condition emulated a clinic-based intervention, in which all participants in the group had a diagnosis of ASD or social challenges. The ENGAGE condition was peer-mediated, and groups included two to three TD classmates for every one participant with ASD. Children in both conditions improved joint engagement, but SKILLS participants made significantly more gains compared to ENGAGE participants, suggesting that proximity to peers with a similar diagnosis was beneficial to participants with ASD. The findings also demonstrated the effects of the student-teacher relationship and child emotional and behavioral profiles on the intervention condition. Children with ASD whose teachers reported high conflict and low closeness benefited the most from the SKILLS condition. More research is needed to explore the effects of clinic-based versus peer-mediated interventions on the engagement of adolescents with ASD in secondary inclusive settings.

Research Purpose

This study sought to compare two distinct models of group-based social skills interventions (SKILLS and ENGAGE) with adolescents with ASD. The SKILLS and ENAGE interventions (Kasari et al., 2015) were adapted to make clinical approaches and the weekly focus topics developmentally appropriate for adolescents (Oti, Gerdts, Orlich, & Faia, 2009a; Oti, Gerdts, Orlich, & Faia, 2009b). Both interventions utilized common social intervention strategies, including didactic instruction targeting specific social skills each session, modeling, role-play, and homework. The SKILLS intervention was modeled after a clinic-based intervention approach and provided homophily by utilizing homogenous grouping, in which all participants had a diagnosis of ASD or pronounced social deficit. The ENGAGE condition was a peer-mediated intervention, and the heterogeneous grouping provided propinquity through targeted opportunities to socialize with typically developing peers.

We sought to examine the effects of the SKILLS and ENGAGE interventions on the social engagement of adolescents with ASD during natural social periods at school. Because peer-mediated interventions have been shown to promote generalization of social skills outside of the intervention setting and to provide access into the general education social culture, we hypothesized that ENGAGE participants would have significantly greater increased joint engagement and decreased solitary engagement (Schmidt and Stichter, 2012) at exit and follow-up. ENGAGE participants had more opportunities to practice skills outside of group, therefore we hypothesized that they would feel more socially competent compared to SKILLS participants. Secondary aims were exploratory and sought to examine the effects of clinic-based or peer-mediated interventions on adolescent reports of their social-emotional feelings and behaviors and also to compare the overall effect of receiving either treatment on the observed joint engagement.

Methods

After receiving university and school district IRB approval, secondary school students who received special education services under the eligibility of ASD and who were educated in the general education setting for a minimum of 80% of the school day, were recruited to participate in the study. Study personnel worked with district administrators to identify secondary schools with inclusive ASD programs, and study coordinators met with school administrators to provide information about the study. If they were interested in participating in the study, school administrators provided a signed letter of agreement to the research coordinator. School personnel disseminated recruitment flyers and consent forms to students meeting study criteria. A total of 13 schools participated in the study (Northwest = 6 [3 middle schools and 3 high schools]; Southwest = 7 [4 middle school and 3 high school]).

Eighty-one students with signed assent and written parent consent were screened. The Autism Diagnosis Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 2001) was used to confirm ASD diagnosis. Eligible participants had an IQ greater than or equal to 70 (abbreviated Stanford-Binet 5^th ed (SB5); Roid, 2003). Of the 81 screened, eleven participants did not meet eligibility requirements or declined to participate, leaving 70 adolescents with ASD that met inclusionary criteria and were randomized to treatment groups. After randomization a total of eight students were unable to be reached or decided to withdraw from the study (ENGAGE = 6; SKILLS = 2). All sixty-two participants (ENGAGE=32; SKILLS = 30) with data collected were included in the final analysis [Figure 1]. Participants were randomized to either the ENGAGE or SKILLS intervention conditions, resulting in 13 different ENGAGE intervention groups (Northwest = 7; Southwest = 6) and 11 different SKILLS groups (Northwest = 5; Southwest = 6). Two sites recruited study participants over two different academic years, with fewer participants recruited in the second year. Second year participants at both sites were randomized into the ENGAGE condition. This resulted in two sites (one Northwest and one Southwest) running two different ENGAGE groups (one group each year). SKILLS and ENGAGE groups were held on different days, and SKILLS interventionists were different from the ENGAGE interventionists.

Figure 1:

Consort Chart

Below Table 1 shows the baseline demographic characteristics of the study sample.

Table 1.

Demographic Characteristics of Adolescent Participants

			Intervention
Variable	Value	Mean (SD) or N (%)	ENGAGE (n = 32)	SKILLS (n = 30)	p-value
Age (yrs.)		14.72 (1.83)	14.77 (1.89)	14.68 (1.79)	0.86
IQ		93.48 (15.31)	94.97 (17.05)	91.79 (13.16)	0.42
Free/Reduced Lunch %		22.55 (22.28)	23.40 (22.30)	21.62 (22.63)	0.76
ASD Participants		2.88 (1.08)	2.68 (0.75)	3.11 (1.34)	0.14
Gender	Male	54 (87.1%)	28 (87.5%)	26 (86.7%)	0.99
	Female	8 (12.9%)	4 (12.5%)	4 (13.3%)
Diagnosis	Non-Spectrum	14 (22.6%)	6 (18.8%)	8 (26.7%)	0.60
	ASD	18 (29.0%)	11 (34.4%)	7 (23.3%)
	Autism	30 (48.4%)	15 (46.9%)	15 (50.0%)
School Type	Middle School	26 (43.3%)	15 (46.9%)	11 (36.7%)	0.58
	High School	36 (58.1%)	17 (53.1%)	19 (63.3%)
Ethnicity	African American	8 (12.9%)	7 (21.9%)	1 (3.3%)	0.04
	White Caucasian	25 (40.3%)	14 (43.8%)	11 (36.7%)
	Hispanic/Latino	5 (8.1%)	0 (0%)	5 (16.7%)
	Asian	12 (19.4%)	5 (15.6%)	7 (23.3%)
	Other	8 (12.9%)	4 (12.5%)	4 (13.3%)
	Missing/Not reported	4 (6.5%)	2 (6.2%)	2 (6.7%)

The ADOS modules 3 and 4 were used to determine eligibility. Following the completion of follow-up data collection, ADOS scores were re-calculated using the ADOS-2 algorithm to create standardized comparison scores (Lord et al., 2012), which generated a standardized severity score that was used to measure ASD across the sample. Fourteen participants who originally met the cut of for ASD no longer met the cut-off for autism or ASD on the ADOS-2. Participants were categorized as Non-Spectrum, ASD or Autism. There was no significant difference in diagnosis status by treatment group (p-value = 0.57). Participants who no longer met the cut-off for ASD or Autism remained in the analysis because they were randomized and received treatment. Since all participants received special education services in school with an eligibility of ASD, the participants reflect authentic ASD populations in school.

Participants attended economically and ethnically diverse public middle (n=26; 41.9%) and high schools (n=36; 58.1%) in a large southwestern city and a large northwestern city. Schools were significantly different with respect to socioeconomic status by site. On average, participating schools in the southwest served a significantly (p<0.01) higher proportion of students who were receiving a free and reduced lunch (35.79%), compared to participating schools in northwest (12.12%). Also, ethnic composition of the schools differed significantly between sites (p<0.01) with participating schools in the southwest having larger African American and Hispanic/Latino representation and schools in the northwest had larger Asian representation. Site was controlled for during the analysis, to account for these differences.

Treatment

The SKILLS and ENGAGE interventions were adapted from interventions used in Kasari et al. (2015) to meet the developmental needs of adolescents with ASD in secondary inclusive placements. Each intervention occurred one time per week for eight weeks in a classroom on school campuses (40–60 minutes per session) during lunch or after school. Using an intervention manual, adult clinicians facilitated each session, covering one topic peer week. Intervention topics and homework assignments are noted in Table 2.

Table 2.

Weekly intervention focus topics

Week	Intervention	Topic	HOMEWORK
			SKILLS	ENGAGE
0	ENGAGE	Pre Intervention Peer Training
1	SKILLS and ENGAGE	Meeting New People/Asking Questions	Asking questions to get to know someone better.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch.
2	SKILLS and ENGAGE	Using Body Talk	Observe peoples’ body language/how they present themselves.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch, or maybe eventually going to a football game, or hanging out at someone’s house.
3	SKILLS and ENGAGE	Using Signals to Express and Understand Emotions	Practice identifying how someone else is feeling.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch, or maybe eventually going to a football game, or hanging out at someone’s house.
3 1/2	ENGAGE	2nd Peer Training – Checking in and trouble shooting		Think of activities to bring to do with the group.
4	SKILLS and ENGAGE	Being Positive!!	Be positive! Give someone a compliment, support and encourage someone, focus on the positive when sad or frustrated.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch, or maybe eventually going to a football game, or hanging out at someone’s house.
5	SKILLS and ENGAGE	Keeping the Conversations Going/Active Listening	This about topics and ideas to talk about with friends.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch, or maybe eventually going to a football game, or hanging out at someone’s house.
6	SKILLS and ENGAGE	Teen Obstacles	Pay attention when you get hooked by a problem. Write about how you felt, and what strategy helped the most.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch, or maybe eventually going to a football game, or hanging out at someone’s house.
7	SKILLS and ENGAGE	Sharing Opinions	Think about the group, and write about what has been most helpful, least helpful, and most fun.	Keep the connection going outside of group. Start by saying “hi” or eating together at lunch, or maybe eventually going to a football game, or hanging out at someone’s house.
8	SKILLS and ENGAGE	Recap of Groups/	Group Reward

All SKILLS and ENGAGE intervention sessions followed the same format: (a) welcome, (b) snacks, (c) mindfulness exercise, (d) homework check in, (e) lesson/focus skill of the week, (f) activity with group, and (g) conclusion: expectations for homework were clearly explained. Participants had the opportunity to earn individual points and group points in each session. Individuals earned points by practicing newly learned skills outside of the group, which they reported to the group in the following session. Groups earned points by saying hi to each other in the hallways, walking together to the bus, and eventually earning bonus points for socializing with fellow group members outside of group sessions. The goal was to earn a party at the end of the group.

Clinicians were doctoral students in a clinical psychology program, psychological studies in education, or undergraduate student research assistants. Clinicians resided in either in the Northwest or Southwest, and all had prior experience working with individuals with ASD in their region. Clinicians were trained to facilitate either the SKILLS or the ENGAGE intervention, and each intervention group was led by 2 adults (one lead and one supporting). Trainings included didactic instruction, in which site coordinators reviewed the SKILLS or ENGAGE manual and modeled clinical strategies. Trainees shadowed an experienced clinician. All sessions were video recorded, and both clinicians received feedback from the principal investigator. The principal investigator utilized a fidelity checklist to ensure that intervention steps were implemented with fidelity. Intervention steps included a mindfulness activity, activity, wrap up, homework review, homework, and the interventionist reinforced skills and facilitated conversations among group members. Clinicians were determined ready to independently facilitate groups when they maintained intervention fidelity greater than .85 on two consecutive sessions. Twenty percent of the remaining interventions were scored for fidelity by the principal investigator, with all clinicians maintaining greater than .87 fidelity scores.

The SKILLS intervention was used to emulate a clinic-based social intervention on a general education school campus. Similar to clinic-based intervention models, SKILLS groups were homogeneous, including only students with ASD and students with significant social challenges based on teacher or administrator report. Clinicians facilitated the entire group session using lesson plans provided in the manual (Oti, Gerdts, Orlich, & Faia, 2009a; Oti, Gerdts, Orlich, & Faia, 2009b). The first half of each meeting was didactic. The second half of group involved specified activities intended to practice the skills learned in that day’s lesson. For example, participants played a modified game of charades to practice interpreting non-verbal cues. SKLLS participants were assigned homework at the end of each session to reinforce and generalize skills learned in the group. One assignment focused on social questions and initiation, and participants were instructed to ask someone from school two social questions, and to record the questions and the answers. Homework was reviewed at the beginning of each lesson. The SKILLS manual was intended to serve as a comprehensive guide to leading the group. Each lesson included lesson plans, handouts, and assigned homework.

The ENGAGE condition was peer-mediated, and group sessions included adolescents with ASD and peer mentors who had been selected and trained to deliver a portion of the intervention. To recruit peer mentors, study personnel worked with teachers and administrators, passed out recruitment materials to individuals they thought had pro-social skills, and would be good peer mentors. Potential mentors were not told that they would be working with individuals with ASD, instead, they were told that they would receive training to learn strategies to socialize with adolescents who had social challenges. Adolescents interested in being a peer mentor contacted study personnel either by phone, or in person on campus. Peer mentors went to the same school as participants with ASD, were described by their teachers as having pro-social skills, and were interested in building relationships with adolescents who have social challenges (n=91; male=35; female=56). No additional demographic data about the peer mentors were collected. In addition to using the same weekly lesson plans as the SKILLS group, the ENGAGE intervention included trainings for peer mentors prior to the start of the intervention and in between sessions 3 and 4. The primary goal of mentor training was to support and train the mentors to promote social interactions among members by encouraging conversations, involving nonparticipating group members in the activities, and organizing shared experiences outside of the group. The first peer training session focused on understanding and learning strategies to support individuals with social challenges. The second sessions focused on trouble shooting challenges, and preparing mentors to lead future sessions. Autism was not introduced during either training.

Consistent with the SKILLS groups, the first half of each session was didactic and led by clinicians following the lesson plan described in the manual. The second half of the group involved student activities to practice the skill learned in that day’s lesson. Although mentors initially lead activities, the goal was for all group members to contribute to leading the activity to de-emphasize a split between mentors and adolescents with ASD. Mentors helped to encourage students with ASD to participate and later to take a leadership role in the group activities. For example, peer mentors would suggest and facilitate games and activities related to the weekly topic area (e.g., playing Apples to Apples^© to practice perspective taking). Clinicians were available to step in if needed to help support the peer mentors.

In the earlier weeks of the program, the manual prescribed specified activities in the activity portion of the group. As the group progressed, group members (participants with ASD and mentors) began to select from an assortment of activities listed in the activities section of each week’s lesson plan. Homework in the ENGAGE group included ideas for a social activities outside of group that both mentors and participants with ASD could attend together. Activities were selected to maximize social interaction with the overarching goal of practicing the skills taught in session and improving engagement. Examples of social activities included going to a high school football game, coffee shop, or shopping mall. Passive activities such as watching a movie or playing video games were options, but participants were encouraged to include some unstructured time afterward to socialize. Mentors were invited to discuss their ideas of social activities for homework with group leaders. Participants with ASD and mentors reported back to the group about their shared social activities outside of the group setting. Group leaders used a frequency chart to keep track of shared social activities that were reported to the group. These data were used as positive reinforcement within the group, as groups earned a pizza party at the end of the eight weeks.

Measures

Two assessments were used to determine eligibility. Participants meeting the eligibility criteria had a diagnosis of ASD without intellectual disability. Intervention measures included observations of participant social engagement, teacher report, and self-report measures.

Eligibility Assessments

ADOS (Lord et al., 2001).

The ADOS was used to determine ASD eligibility. The ADOS is a standardized assessment of communication, social interaction and play, and imaginative use of materials. The ADOS-2 Calibrated Severity Metric (CSS), a standardized score that controls for age and language level when describing ASD symptom severity (Lord et al., 2012) was used to classify participants.

SB-5 (abbreviated; Roid, 2003).

The SB-5 has high reliability, and was used to determine eligibility (IQ). Two subtests yield nonverbal and verbal IQ scores. The abbreviated IQ scores are highly correlated with full-scale IQ scores.

Intervention Measures

Data were collected prior to the start of the intervention (baseline), immediately following the last session (exit), and four to six weeks following the completion of the intervention (follow-up). Social skills and social experiences were examined using blinded observations of adolescents at school, adolescent self-report of social experiences, and teacher reports of adolescent social skills.

Teen Observation of Peer Interaction (TOPI; Oti, Orlich, Eickhoff, Poole, & Mahjouri, 2010).

The TOPI is a modification of a structured behavioral observation measure that has been used to measure the social engagement of children during unstructured social periods at school (Kasari et al., 2011, 2015; Kasari and Rotheram-Fuller, 2005; Shih et al., 2019). Participants were observed, during lunch or morning snack period on their school campus (e.g., in cafeteria or school yard), twice at each time point. The TOPI consists of five two-minute observation intervals, in which a trained observer rated the participant’s primary engagement state during each interval. Three engagement states were included in the analysis: (a) Solitary, the adolescent is alone or with an adult, (b) Parallel, the adolescent is in proximity to a peer or peers, but is not mutually engaged, and (c) Joint Engage, the adolescent is mutually engaged with a peer or peers. Observers were research assistants or graduate student researchers who were blinded to the intervention condition and trained to a reliability criterion of greater than .85. Two coders randomly and independently coded 20% of the observations, maintaining a coding consistency (>.90).

Behavior Assessment System for Children 2nd Edition (BASC-2; Reynolds & Kamphaus, 2004).

The BASC-2 is a norm-referenced, multidimensional self-report of personality consisting of 150 items, in which participants were asked to rate each item according to frequency of a behavior (never, sometimes, often, almost always). Symptoms of internalizing problems, inattention/hyperactivity, and personal adjustment are calculated as composites. Social stress and interpersonal relationships are calculated as individual subscales. The composite and subscales self-report have reliability coefficients of .90 to .97 (Reynolds, 2004)

Social Skills Improvement System (SSIS; Gresham & Elliot, 2008).

The SSIS is a revision of the Social Skills Rating System (SSRS; Gresham & Elliott, 1990), which has been widely used to measure treatment effects of social interventions in adolescents with ASD (Laugeson et al., 2009; Laugeson et al., 2012). This multi-rater system evaluates social skills, problem behaviors, and academic competence; and teachers and adolescents report the frequency of behaviors (not true, little true, a lot true, very true). The SSIS has adequate test-retest reliability and validity and is correlated with other measures of social skills (r = 0.50–70). Variables used in this study include teacher and adolescent ratings of social skills and problem behaviors composite scores.

Analysis

T-tests, χ² tests, and Fisher exact tests were used to compare pre-treatment covariates between treatment groups. Treatment effects from longitudinal measurements were modeled using linear mixed models (LMM). These models included treatment (ENGAGE or SKILLS), time (baseline, exit, and follow-up), and treatment by time interaction with subject level random intercepts to model outcome trajectories while controlling for baseline child IQ, ADOS severity category (Non-spectrum, ASD, or Autism), and site (southwest or northwest) using R software with the lme4 package.

To determine the timepoint if/when the outcome was affected by the treatment assignment, two separate treatment effects were tested using separate models: immediate and delayed. Immediate treatment effect was defined as a significant treatment by time interaction from baseline to exit, and delayed treatment effect as a significant treatment by time interaction from baseline to follow-up. Both methods were fit using all available observations from the 62 participants being analyzed. Missing outcome data was assumed to be missing at random (MAR), which ensured that estimates were unbiased. In addition to the immediate and delayed treatment effects, the primary engagement measures were also tested for an overall immediate or delayed effect of receiving either intervention by excluding the treatment by time interaction and testing the main effect of time for significance. Evidence for an overall effect was determined by the corresponding coefficient for time in either the immediate or delayed model.

Significance was calculated using ANOVA type III χ² test statistics that compared a full model to a reduced model without the coefficient of interest. An unadjusted nominal p-value of 0.05 was used to identify potentially significant effects. Since outcomes were analyzed from three separate constructs (TOPI, BASC, and SSIS), an adjusted p-value of 0.0167, i.e. 0.05/3, signified strong evidence of significance. This Bonferroni correction was performed by construct rather than outcome since outcomes within the same construct were potentially correlated. Cohen’s f² effect size statistic was calculated for LMM by comparing the additional variance explained by the predictors’ effect over the random effects (Selya et al., 2012).

Results

The following paragraphs describe the results from the TOPI, teacher report and self-report analysis. Immediate and delayed intervention effects on observed engagement, and teacher report and self-report are outlined below.

TOPI

Using LMM three separate models were fit for each TOPI engagement state: Joint Engage, Parallel, and Solitary. There were no significant immediate or delayed treatment effects for any of the engagement states. However, there were significant overall effects from baseline to follow-up overall. Table 3 below shows the results from the hypothesis testing.

Table 3.

TOPI Outcomes

Time	Effect	Variable	Coefficient (SE)	χ2(1)	p-value	f2
Immediate	Treatment	Joint Engage	7.05 (9.44)	0.56	0.46	<0.01
		Parallel	−12.50 (9.67)	1.67	0.20	0.04
		Solitary	5.94 (10.33)	0.33	0.57	<0.01
	Overall	Joint Engage	−2.43 (4.64)	0.27	0.60	0.01
		Parallel	3.81 (4.92)	0.60	0.44	0.01
		Solitary	−1.31 (5.18)	0.06	0.80	<0.01
Delayed	Treatment	Joint Engage	7.81 (9.86)	0.63	0.43	0.01
		Parallel	−3.46 (9.47)	0.13	0.72	<0.01
		Solitary	−1.72 (10.03)	0.17	0.68	<0.01
	Overall	Joint Engage	10.75 (4.84)	4.94	0.03*	0.06
		Parallel	0.05 (4.73)	<0.01	0.99	<0.01
		Solitary	−11.19 (5.11)	4.78	0.03*	0.08

Results from linear mixed models for TOPI engagement state outcomes controlling for IQ, Site, and Diagnosis with random intercept for individual; For treatment effect: reference is ENGAGE and coefficient corresponds to difference between SKILLS vs. ENGAGE; For overall effect: model includes main effect of treatment and time without interaction with the coefficient corresponding to the main effect of time (baseline vs exit or baseline vs follow-up).;

^*Indicates significance at α=0.05 and

^**Indicates significance at α=0.05/3=0.0167

There was a significant overall average increase of Joint Engage from baseline to follow-up, χ²(1) = 4.94, p = 0.03, and f² = 0.06. On average students had initial Joint Engage of 24.15% at baseline and increased to 35.71% at follow-up, controlling for baseline covariates students had average increase of +10.75% from baseline to follow-up with 95% CI (+1.27%, +20.20%). This overall effect was not significant from baseline to exit, χ²(1) = 0.27, p = 0.60, and f² = 0.01.

There was no evidence of immediate or delayed overall effect for Parallel play, p = 0.44 and p = 0.99. However, on average there was a significant overall decrease in Solitary from baseline to follow-up, χ²(1) = 4.78, p = 0.03, and f² = 0.08. Combined the groups decreased solitary play from 41.02% at baseline to 26.51% at follow-up, with average decrease of −11.19% from baseline to follow-up with 95% CI (−21.2%, −1.16%) controlling for baseline differences. This overall effect was tested from baseline to exit but was found to be not significant, p = 0.80.

Self-Report

Using LMM, a variety of self-reported social scales were analyzed to test for an effect of the intervention on perceived social ability. The following self-reported behaviors from the BASC and SSIS measures were found to be non-significant for immediate and delayed treatment effect: BASC School Problems, BASC Internalizing, BASC Attention/Hyperactivity, SSIS Social Skills, and SSIS Internalizing. See Table 4 below, which includes the estimated coefficients, standard errors, chi-square values, and effect sizes for each self-reported outcome tested and Figure 3 which plots the longitudinal trajectories for these outcomes by treatment group.

Table 4.

BASC and SSIS Outcomes

			Immediate Treatment Effect				Delayed Treatment Effect
Construct	Reporter	Variable	Coefficient (SE)	χ2(1)	p-value	f2	Coefficient (SE)	χ2(1)	p-value	f2
BASC	Self-Reported	Social Stress	7.66 (3.43)	4.99	0.03*	0.04	5.19 (3.52)	2.18	0.14	0.06
		IPR	−9.99 (3.80)	6.90	0.01**	0.07	−5.91 (4.36)	1.84	0.17	<0.01
		Emotional Symptoms	2.22 (3.23)	0.47	0.49	<0.01	8.51 (3.98)	4.58	0.03*	0.11
		School Problem Composite	1.43 (2.67)	0.29	0.59	<0.01	−0.07 (3.36)	<0.01	0.98	<0.01
		Internalizing	−1.33 (2.97)	0.20	0.65	<0.01	−3.81 (3.87)	0.97	0.33	<0.01
		Attention/Hyperactivity	1.49 (3.25)	0.23	0.63	<0.01	−1.57 (3.63)	0.19	0.67	<0.01
		Social Skills	−5.42 (4.44)	1.49	0.22	<0.01	−3.30 (4.49)	0.54	0.46	<0.01
SSIS	Self-Reported	Problem Behaviors	0.28 (2.95)	0.01	0.92	<0.01	8.84 (4.08)	4.70	0.03*	0.11
		Externalizing	−0.17 (0.10)	3.14	0.08	0.08	3.77 (1.55)	5.88	0.02*	0.08
		Social Skills	−5.36 (4.56)	1.39	0.24	<0.01	−3.30 (4.49)	0.54	0.46	<0.01
		Internalizing	−0.06 (0.13)	0.24	0.62	<0.01	0.14 (0.15)	0.98	0.32	<0.01
	Teacher-Reported	Problem Behaviors	−0.07 (3.43)	<0.01	0.98	<0.01	1.11 (4.34)	0.07	0.80	<0.01
		Externalizing	0.09 (1.22)	0.01	0.94	<0.01	2.45 (1.71)	2.06	0.15	0.10
		Internalizing	0.30 (0.92)	0.11	0.74	<0.01	0.52 (1.51)	0.12	0.73	<0.01
		Social Skills	6.62 (4.42)	2.46	0.12	0.03	9.29 (4.16)	4.99	0.03*	0.09

Coefficients are obtained from linear mixed model controlling for IQ, Site, and Diagnosis with random intercept for individual. Treatment reference group is ENGAGE, so the coefficient corresponds to the treatment effect between SKILLS as compared to ENGAGE group.

^*Indicates significance at α=0.05

^**Indicates significance at α=0.05/3=0.0167

Figure 3.

Significant Social Scales Trends Over Time

BASC Self-Reported Social Stress.

The LMM model was significant for immediate treatment effect with a positive association between the SKILLS intervention and self-reported social stress post intervention, χ²(1) = 4.99, p = 0.03, and f² = 0.04. On average participants in the SKILLS group started slightly lower than the ENGAGE group at baseline (45.36 vs 47.86), but increased to 51.73 at exit while ENGAGE participants decreased to 45.96. Thus, the SKILLS group had greater increase in self-reported social stress with an average increase of +7.66 with 95% CI (+0.94, +14.40) compared to the ENGAGE group from baseline to exit. This model was also tested for a delayed treatment effect, but was not significant, χ²(1) = 2.18, p = 0.14, and f² = 0.06.

BASC Self-Reported Interpersonal Relationship.

Via LMM modeling, interpersonal relationships had a significant immediate treatment effect, with SKILLS groups rating their interpersonal relationships significantly lower from baseline to exit compared to the ENGAGE group, χ²(1) = 6.90, p = 0.01, and f² = 0.07. The SKILLS group started higher at baseline compared to the ENGAGE group (53.21 vs 46.62) and the SKILLS saw lower scores for interpersonal relationships at exit (47.50) while the ENGAGE group increased their scores to 51.25. This led to an average treatment difference of −9.99 for the SKILLS group versus the ENGAGE group from baseline to exit, with 95% confidence interval (−17.44, −2.54). There was no significant delayed treatment effect, χ²(1) = 1.84, p = 0.17, and f² < 0.01.

BASC Self-Reported Emotional Symptoms.

There was a significant delayed treatment effect for self-reported emotional symptoms. The SKILLS group started lower at baseline compared to the ENGAGE group (44.24 vs 50.58) but at follow-up the SKILLS group had increased to 48.88 while the ENGAGE group had decreased to 46.59. Controlling for baseline differences, the SKILLS group on average reporting higher emotional symptoms score changes from baseline to follow-up compared to the ENGAGE group, χ²(1) = 4.58, p = 0.03, and f² = 0.11. The SKILLS group on average increased their emotional symptoms scores by +8.51 versus the ENGAGE group, with 95% confidence interval (+0.72, +16.30). There was no evidence of a significant immediate treatment effect, χ²(1) = 0.47, p = 0.49, and f² < 0.01.

SSIS Self-Reported Problem Behaviors.

Using LMM modeling, on average the SKILLS group increased their frequency of self-reported problem behaviors compared to the ENGAGE group, χ²(1) = 4.70, p = 0.03, and f² = 0.11. On average the SKILLS group and ENGAGE group started with similar reports of problem behaviors (100.1 vs 101.9), but the SKILLS group saw an average increase to 104.5 at follow-up while the ENGAGE group decreased on average to 96.8. After controlling for baseline covariates in the model, the SKILLS group on average increased their self-reported problem behaviors by +8.84 from baseline to follow-up compared to the ENGAGE group with 95% confidence interval (+0.85, +16.80). There was no observed immediate treatment effect, χ²(1) = 0.01, p = 0.92, and f² < 0.01.

Teacher-Reported Social Skills.

There was a significant delayed treatment effect, χ²(1) = 4.99, p = 0.03, and f² = 0.09, showing that teachers in the SKILLS groups on average saw an greater improvement in social skills from baseline to follow-up after controlling for baseline covariates of +9.29 versus the ENGAGE groups with 95% confidence interval (+1.14, +17.4). At baseline the SKILLS group had average social skills scores of 80.77 as reported by teachers and increased to 87.42 at follow-up while the ENGAGE group started at 85.63 and increased to 87.42 There was no significant immediate treatment effect, χ²(1) = 2.46, p = 0.12, and f² = 0.03.

Discussion

This study used a randomized controlled trial to compare the effects of clinic-based and peer-mediated interventions for adolescents with ASD who were educated in inclusive secondary settings. Participants in both the SKILLS and ENGAGE interventions significantly increased joint engagement and decreased solitary engagement at follow-up. Despite the increased engagement, neither SKILLS nor ENGAGE participants self-reported any significant improvement in social skills. Secondary outcomes indicated that compared to the ENGAGE participants, individuals in the SKILLS condition reported a greater impairment in social stress and interpersonal relationships at exit, and significantly increased problem behaviors and emotional symptoms at follow-up. The findings suggest that due to the utilization of peer mentors, ENGAGE participants experienced a better quality of social engagement.

The data highlight the emotional factors related to increasing one’s social repertoire, and that the use of a peer mentor may help to buffer social stressors that can interfere with one’s ability to socially engage with peers. SKILLS participants increased joint engagement between baseline and follow-up; yet, they also reported a significant increase in social stress and greater impairment with interpersonal relationships between baseline and exit. These findings suggest an initial discomfort period where participants felt that their stress and interpersonal relationships were worsening relative to the broader social environment of secondary school. These findings are consistent with the body of research that shows that adolescents with ASD who perceive themselves as having social difficulties also tend to have increased social stress (Bellini, 2004; Corbett et al., 2014). Lopata et al. (2008), measured cortisol levels in adolescents with ASD without cognitive impairment and found a significant increased stress response when adolescents interacted with a familiar peer. Knowing that all members of the group have social challenges may have had a negative impact on individual evaluations of one’s own social difficulties.

Research has shown that in context of an inclusive secondary school environment, adolescents with ASD are aware of their difficulties with understanding and interpreting social rules (Humphrey & Lewis, 2008), and this awareness is associated with increased emotional symptoms and reputational concerns (Cage, Bird, & Pellicano, 2016). Increased joint engagement at follow-up suggests that SKILLS participants were able to generalize social skills to the natural social setting, yet at the same time, they also perceived themselves as having significantly higher problem behaviors and emotional symptoms. Thus, when SKILLS participants socialized outside of the intervention group, they experienced a heightened sense of social impairment in relation to the dominant social peer culture at school. The SKILLS homework model may have inadvertently exacerbated these social insecurities by placing the onus of social initiations solely on the individual with ASD. Thus, these findings highlight potential challenges of using homogenous group structures and certain types of homework protocols in secondary inclusive settings.

In contrast to SKILLS participants, however, ENGAGE participants reported a decrease in social stress at exit, suggesting that using supportive peer mentors to provide entre into the normative social culture and to facilitate social engagement may be effective in mitigating social stress as adolescents with ASD build their social repertoire. Given the complexity of adolescent social groups, challenges related to ASD may make it difficult for adolescents to keep up with the fast-paced social environment in TD peer groups. The peer-coaching model, where individuals with ASD have opportunities to practice and develop social skills with socially competent peers, appears to be more effective in protecting adolescents from additional mental health co-morbidities. Compared to the SKILLS homework model, the structure of the ENGAGE homework was more conducive to practicing social skills and increasing social engagement outside of the group. Homework in the ENGAGE condition provided explicit directions to encourage group members to hang out with each other outside of group. Thus, the responsibility to complete homework was on the group rather than the individual. These findings suggest that promoting a shared responsibility to complete social group homework may be an effective way to incorporate generalization activities outside of group meetings without increasing social stress.

Teachers reported greater improvement in social skills among participants in the SKILLS condition, but not the ENGAGE condition, suggesting that certain qualities of social improvement were noticeable in classroom settings. Yet, most teachers are not with students during lunch, passing periods, or other unstructured social periods at school, and therefore would not be aware of increased social engagement outside of the classroom. Teacher reports of classroom behavior may not provide enough information to improve the physical and social environment in which adolescents with ASD will socialize.

Limitations and Future Directions

More research is needed to examine the extent which environmental factors like intervention type, dose, and homework can affect the social engagement and social cognitive outcomes for adolescents with ASD within inclusive secondary settings. There were site differences in the length of sessions and in the time of day that sessions occurred. Southwest sessions lasted about 30 to 40 minutes and occurred during lunchtime. Northwest sessions lasted about an hour and occurred after school. Homework in the ENGAGE condition provided explicit directions to encourage group members to hang out with each other outside of group. Future studies should utilize the same homework model with both the SKILLS and ENGAGE conditions. Teacher demographic data were not collected, and therefore it is not clear how well teachers knew the students for whom they were reporting. We did not collect demographic information from TD peers or group participants with significant social challenges, nor did we collect data measuring the extent to which peers socialized outside of group. Studies are needed to examine qualities of peer mentors, and whether certain peer characteristics facilitate positive social outcomes for participants with ASD. This is the first study to use the TOPI in an adolescent population, which has not been normed on a typically developing adolescent sample. This is an area for future research. Also, future studies could explore the extent to which TD adolescents were or were not told that the study participants had ASD. Finally, future social skills intervention studies should ask adolescents with and without ASD to rate the social validity of the intervention.

General Implications

The SKILLS and the ENGAGE participants increased joint engagement and decreased solitary at follow-up. They were also able to generalize social skills to the natural social environment at school. Yet, the findings highlight the potentially beneficial and harmful effects of the content of social skills curriculum and homework models selected. Including TD peers in social group sessions and homework assignments yielded additional social-emotional benefits for adolescents with ASD in the ENGAGE group. In the general education setting, individuals with ASD have a much greater exposure to TD peers who are in all of their classes, in the cafeteria, and hallways. Thus, this study provides a guidepost for school personnel who work with adolescents with ASD within inclusive settings. Identifying TD adolescents who have prosocial behaviors and who are interested in working with students who have social challenges may be useful when implementing social skills interventions at school. Schools facilitating social interventions should also include peer mentor training before and throughout the social intervention period.

Figure 2.

Joint Engagement Trends Over Time

Author Biographical Statements

Michelle Dean Ph.D. is an associate professor of Special Education in the School of Education at California State University, Channel Islands. She received her Ph.D. and completed a postdoctoral fellowship at the UCLA Center for Autism Research and Treatment. Her research interests include the social development of children with disabilities at school, school-based intervention services for children with autism in economically disadvantaged communities, and the study of gender differences in the social behaviors of boys and girls with autism. Michelle is a former special education teacher for the Los Angeles Unified School District.

Justin Williams received his M.S. at UCLA in the Department of Biostatistics, and is currently a Ph.D. candidate in the department. His research focuses on data analysis in clinical trial settings using longitudinal mixed modeling. Other research interests include analysis methods for non-randomized or pseudo-randomized studies using propensity scores, big data challenges associated with large educational databases, and bias from truncated and censored distributions.

Felice Orlich is the Director of Community Outreach at Seattle Children’s Autism Center and a Clinical Associate Professor at the University of Washington. Felice received her PHD in psychology from UC - San Diego. She completed a fellowship in child Psychology at Stanford University School of Medicine. Dr. Orlich’s clinical interests include diagnosis and interdisciplinary evidence- based interventions for children and adolescents with Autism Spectrum disorders, and co- occurring medical, educational and psychiatric needs, working with gender diverse children and adolescents, adjustment to chronic medical, in particular neurological issues, and neuropsychological assessment. Dr. Orlich’s research centers on health equity for individuals with ASD and gender diverse youth, and improving social outcomes and engagement in schools.

Connie Kasari, Ph.D. is a Professor of Human Development & Psychology in Education with a joint appointment in Psychiatry at UCLA. She is the Principal Investigator for several multi-site research programs and a founding member of the Center for Autism Research and Treatment at UCLA. She has been actively involved in autism research for the past 25 years, leading projects under the CPEA, STAART, and Autism Centers of Excellence programs from NIH. Since 1990, she has been on the faculty at UCLA where she teaches both graduate and undergraduate courses and has been the primary advisor to more than 60 PhD students. Dr. Kasari has a wide range of publications on topics related to social, emotional, and communication development and intervention in autism. She is on the science advisory board of the Autism Speaks Foundation, and regularly presents to both academic and practitioner audiences locally, nationally and internationally.