Efficacy of group social skills interventions for youth with autism spectrum disorder: A systematic review and meta-analysis

Abstract

Group-based social skills interventions (GSSIs) are widely used for treating social competence among youth with autism spectrum disorder (ASD), but their efficacy is unclear. Previous meta-analysis of the literature on well-designed trials of GSSIs is limited in size and scope, collapsing across highly heterogeneous sources (parents; youths; teachers; observers; behavioral tasks). The current meta-analysis of randomized control trials (RCTs) was conducted to ascertain overall effectiveness of GSSIs and differences by reporting sources. Nineteen RCTs met inclusion criteria. Results show that overall positive aggregate effects were medium (g = 0.51, p < 0.001). Effects were large for self-report (g = 0.92, p < 0.001), medium for task-based measures (g = 0.58, p < 0.001), small for parent- and observer-report (g = 0.47 and 0.40, respectively, p < 0.001), and nonsignificant for teacher-report (p = 0.11). Moderation analyses of self-report revealed the effect was wholly attributable to youth reporting that they learned about skilled social behaviors (social knowledge; g = 1.15, p < 0.01), but not that they enacted them (social performance; g = 0.28, p = 0.31). Social skills interventions presently appear modestly effective for youth with ASD, but may not generalize to school settings or self-reported social behavior.

1. Introduction

Impairment in social functioning is the defining feature of autism spectrum disorder (ASD). Youth with ASD are at increased risk for social isolation and peer victimization (Hobson, 2014; Mendelson, Gates, & Lerner, 2016). Moreover, social impairments in youth with ASD do not tend to improve merely with development, but rather may become more pronounced during adolescence when the social demands exceed the social skills (Picci & Scherf, 2015), underscoring the need for appropriate interventions to promote social competence in this population.

Group-based social skills interventions (henceforth referred to as GSSIs) are the most widely used approach to address social impairment and foster social skills development in school-age and teenage youth with ASD (McMahon, Lerner & Britton, 2013). As such, it is vital to know the degree to which, and according to what metrics, GSSIs yield benefits. However, despite their widespread use, very little rigorous, well-designed research has been conducted to examine their efficacy (Kasari, Shire, Factor, & McCracken, 2014). Indeed, while there have been more than a dozen systematic reviews of GSSIs conducted in recent years (e.g., Barry et al., 2003; Blacher, Kraemer, & Schalow, 2003; Elder, Caterino, Chao, Shacknai, & De Simone, 2006; Flynn & Healy, 2012; Kaat & Lecavalier, 2014; Kasari & Patterson, 2012; Koenig, De Los Reyes, Cicchetti, Scahill, & Klin, 2009; McMahon, Lerner et al., 2013; Miller, Vernon, Wu, & Russo, 2014; Otero, Schatz, Merrill, & Bellini, 2015; Rao, Beidel, & Murray, 2008; Reichow & Volkmar, 2010; Schreiber, 2011; Spence, 2003; White, Keonig, & Scahill, 2007), there has only been one meta-analysis to evaluate efficacy of GSSIs (Reichow, Steiner, & Volkmar, 2012). This sole meta-analysis – which focused on well-designed randomized controlled trials (RCTs) with a wait-list control, of which only five were evident at the time – provided initial support for the efficacy of GSSIs for improving social competence for ASD youth. Due to the very small number of included trials, though, little beyond a headline effect size (g = 0.47) could be determined. However, in the years since, the number of published RCTs of GSSIs has tripled, making evaluation of overall efficacy of GSSIs timely and vital, and facilitating the possibility of a more robust, comprehensive, and precise estimation of their efficacy. Additionally, as evidence of efficacy accumulates, and sufficient studies are available, it is important to identify moderating factors that may influence the strength of intervention effects. Further, extensive literature demonstrates that complex skills expressed in youth are rarely unitary in nature, and so meaningful information can be obtained by disaggregating these constructs into constituent assessments obtained via multiple informants and other sources (De Los Reyes et al., 2015). This information may reveal informative circumstantial variations in expression of skills, especially for highly contextually-dependent domains such as social skills in youth with ASD (De Los Reyes, 2011; Lerner, Calhoun, Mikami & De Los Reyes, 2012; Murray, Ruble, Willis, & Molloy, 2009). Hence, the current meta-analysis sought to ascertain overall efficacy of GSSIs and whether the intervention effects differ by reporting sources.

1.1. Current evidence for GSSIs in ASD

The sole small previous meta-analysis of GSSIs for youth with ASD found a medium effect, primarily according to parent report (Reichow et al., 2012). This effect was comparable to the mean ES found for behavioral interventions impacting social skills for individuals with ADHD (g = 0.47; Daley et al., 2014) and social skills treatment for individuals with schizophrenia (g = 0.52; Kurtz & Mueser, 2008), but somewhat larger than that found in social skills treatment for youth with learning disabilities (g = 0.21; Forness & Kavale, 1996). Despite these promising results, the study contained only four studies, reflecting the limited state of the literature at the time. As a result, it was unable to consider any potential predictors of variation in effects or test for publication bias, and failed to consider variation in the reports of social competence change across different informants.

1.2. Variations in assessment of intervention efficacy

Although individual studies have yielded some support for the efficacy of GSSIs (e.g., Koning, Magill-Evans, Volden, & Dick, 2013; Lopata et al., 2010), results have not been consistent. One evident source of this variation is the outcome of interest – that is, the source of information on social functioning (e.g., Koning et al., 2013; Laugeson, Frankel, Mogil, & Dillon, 2009). Such a finding is not uncommon. Indeed, extensive literature has consistently shown that ratings of child symptomatology vary between various reporting sources (De Los Reyes et al., 2015). These differences reflect the complex nature of “social skills” and may reveal important and meaningful information about contextual variations in the expression of skills and in the perceived impact of the intervention (Koenig et al., 2009; Lerner, Calhoun et al., 2012). Understanding the shape and nature of changes in GSSIs according to different informants not only offers practical information about what outcomes can be affected, but also provides deeper insight into the nature of the intervention itself and its mechanisms of action (Lerner, White & McPartland, 2012).

Hence, it is important to consider differences in assessment measures used to evaluate the efficacy of a GSSI, as the efficacy is dependent on the quality of assessments (McMahon, Lerner et al., 2013). In fact, GSSIs commonly utilize various sources to assess efficacy. Parent-report questionnaires, in which parents rate broad metrics of frequency or quality of social behavior across windows of time ranging from several days to several months (e.g., Constantino & Gruber, 2007; Gresham & Elliott, 1990; Reynolds & Kamphaus, 2004), are one of the most regularly utilized assessment methods, owing in part to easy and quick administration. Teacher-report questionnaires, in which teachers rate broad metrics of frequency or quality of social behavior as they appear in the classroom (e.g., Gresham & Elliott, 1990; Pekarik, Prinz, Liebert, Weintraub, & Neale, 1976), are frequently used to gather information about child’s functioning in school settings. Participants themselves also frequently rate their social competence, via questionnaires, where participants rate broad metrics of frequency or quality of their own social behaviors as they perceive them to be occurring (e.g., Gresham & Elliott, 1990; Reynolds & Kamphaus, 2004) and indicate their own social knowledge, whereby participants indicate what they believe is the right thing to do (regardless of whether they believe they themselves do it) in various social situations (e.g., Laugeson & Frankel, 2006; Lopata, Thomeer, Volker, Nida, & Lee, 2008). Participants also complete task-based assessments that are often aimed at measuring specific skills related to social competence, such as social cognition, emotion recognition, and social response planning (e.g., Nowicki, 2004; Sofronoff, Eloff, Sheffield, & Attwood, 2011). Observer-report measures in which (typically blinded) independent, trained observers evaluate and rate the frequency or quality of social behaviors of the participants in either naturalistic (such as playgrounds) or structured (such as lab-based interactions) settings (e.g., Koning et al., 2013; Lerner & Mikami, 2012), are also used, though not as commonly as parent or teacher ratings (McMahon, Vismara & Solomon, 2013). Some observer-report measures are uniquely designed by the intervention staff to assess participant’s behavior (Kamps et al., 2015; Rodgers et al., 2015), whereas others are standardized across settings (e.g., Bauminger, 2002). Therefore, examining change in ratings of social competence as a result of these interventions by different sources is useful to gain a clear understanding of GSSI outcomes.

1.3. Plausible moderators of treatment effects

The mixed results of individual studies may also be due to differences in participant or intervention-related characteristics – that is, plausible moderators (Kazdin, 2007; Lerner & White, 2015; Lerner, White et al., 2012). Indeed, GSSIs vary widely according to a broad range of characteristics, from participant age to length of treatment to the cognitive profile of the participants (McMahon, Lerner et al., 2013). This heterogeneity invites the likelihood that such factors may contribute to differences in intervention efficacy. Therefore, we consider factors that may moderate the effects of GSSIs, which can help to better identify efficacious methods of GSSI as well as for whom, and under what conditions, these interventions are best suited (Kazdin & Nock, 2003).

1.3.1. Participant characteristics

The target age range of GSSIs can vary widely (e.g., Ichikawa et al., 2013; Laugeson, Gantman, Kapp, Orenski, & Ellingsen, 2015). Several studies have shown different effects of GSSI by age, though these findings are inconsistent. For example, some have indicated relatively greater improvements in older participants (e.g., Herbrecht et al., 2009; Mathur, Kavale, Quinn, Forness, & Rutherford, 1998), while others suggest greater benefits for younger children in some approaches (McMahon, Vismara et al., 2013; Wang, Cui, & Parrila, 2011). Considering the inconsistent findings of the effects of GSSIs for each age group, it is currently unclear whether effects should be larger or smaller among older youth, but is vital for guiding service recommendations.

Participants in GSSIs are generally thought to exhibit average to above-average cognitive ability (McMahon, Lerner et al., 2013), however, participants above this level still display a wide range of overall cognitive and verbal ability (Lerner & White, 2015). Research has shown that more cognitively-able participants with a higher IQ and better verbal ability have demonstrated greater improvements in emotion recognition (e.g., Solomon, Goodlin-Jones, & Anders, 2004) and benefit most from participating in a GSSI (Herbrecht et al., 2009). Therefore, participants with higher cognitive and verbal ability may show greater improvements after participating in the GSSI.

In addition, males and females with ASD may have unique social challenges that could potentially moderate treatment outcomes (Dean et al., 2014). However, while many interventions do include at least a few female participants, there has rarely been sufficient sample size in an individual study to empirically examine effects of gender on treatment outcomes. Intriguingly, the one study that examined gender effects of a GSSI showed relatively greater improvements in girls (McMahon, Vismara et al., 2013). Owing to the dearth of evidence of gender on the outcomes of GSSI, it is unclear whether effects are indeed larger in females than males, but current clinical practice urges examination of potential differences.

Individuals with ASD commonly experience other comorbid psychiatric conditions (Simonoff et al., 2008) that may affect treatment outcomes. There is strong reason to believe this may be the case for GSSIs. For example, one study found that those with ADHD comorbidity showed less improvement in social skills, while those with anxiety comorbidity showed greater improvements (Antshel et al., 2011). A more recent study found attenuated effects associated with comorbid anxiety (Pellecchia et al., 2015). Therefore, there is evidence to suggest that participants with (versus without) psychiatric comorbidities (that is, the preponderance of ASD youth seeking GSSIs) should show less improvement in social competence following GSSIs.

Relatedly, psychopharmacological medication is often prescribed to youth with ASD due to frequent psychiatric comorbidities (Malone, Maislin, Choudhury, Gifford, & Delaney, 2002). The literature investigating the effects of medication on interventions has been inconclusive, with some showing greater improvements in a medicated group (Herbrecht et al., 2009) and others showing greater improvements in an un-medicated group (Frankel, Myatt, & Feinberg, 2007). Given the current evidence, it is unclear whether effects should be larger or smaller among participants who are on psychopharmacological medication, but the high rate of medication use necessitates their investigation.

1.3.2. Intervention characteristics

GSSIs for youth with ASD show a wide range of variation in their content, type, structure, and therapeutic targets. For example, the length of interventions often varies dramatically, ranging from a few weeks to spanning two academic years (e.g., Kamps et al., 2015; Lopata et al., 2010). Moreover, while the majority of GSSIs are conducted for 1- to 2-h weekly sessions, there are more intensive interventions, often during the summer, that meet for 5–6 h/weekday for several weeks (McMahon, Lerner et al., 2013). There has been a recent effort to evaluate how duration/intensity of an intervention may be associated with intervention efficacy (i.e., “dose-response relationship”; Turner-Brown, Perry, Dichter, Bodfish, & Penn, 2008), however results of this investigation are mixed. While in one study intervention length was not correlated with social improvement (Tyminski & Moore, 2008), another study found greater increase in peer interactions among those who attended more intervention sessions (McMahon, Vismara et al., 2013). Considering the inconsistent findings, it is currently unclear whether effects should be larger in longer or more intense interventions, but guidance is needed for service providers to optimize dosage (and minimize costs) to individuals and families.

In some cases, non-ASD youths of similar age participate in GSSIs as peer tutors or peer models (Kamps et al., 2015; Wang et al., 2011). This strategy involves the use of socially competent students to model, interact, and occasionally use intervention strategies to promote social skill development in individuals with ASD (DiSalvo & Oswald, 2002; Rogers, 2000). Although a meta-analysis of single-subject research studies showed evidence that peer-mediated intervention may be an effective strategy for social skill deficits (Wang et al., 2011), the impact of such peers as an adjuvant in the context of a group intervention has not explicitly evaluated, and it is unclear whether it is an effective treatment component (Kaat & Lecavalier, 2014).

1.3.3. Intervention content

GSSI strategies often vary in terms of the specific content they focus on. That is, some focus (at least in part) on didactically presenting information about correct behaviors in social contexts (i.e., social knowledge training), while others aim more squarely on providing a context in which successful peer interactions may occur and reinforcing them when they happen without prescriptive teaching of rules (i.e., social performance; Gresham, 1997; Lerner & Mikami, 2012; Lerner & White, 2015; White et al., 2007). As such, the specific content targeted in these GSSIs may be assessed using measures that aim to collect information on either gains made in the participants’ self-reported social knowledge or social performance (i.e., enacted social behavior), which may conclude different information about the changes in skills of the participant as a result of the intervention. Though there is reason to believe that social knowledge and social performance may be independent constructs (Lerner & Mikami, 2012; Lerner & White, 2015; Lerner, White et al., 2012), many studies assess social knowledge and social performance together via self-report. Hence, it is important to consider whether there may be a difference between self-reported gains in social knowledge (i.e., knowing what to do in a social situation) and self-reported gains in social performance (i.e., applying the social skills and displaying appropriate social behaviors) when evaluating efficacy of GSSIs.

1.4. Measurement of intervention effect

A crucial question when examining complex, group-based interventions such as GSSIs is whether group effect comparisons are valid and accurate. Most meta-analytic studies of RCTs examine treatment effects by using post-test scores to obtain standardized mean difference (SMD) between experimental and control conditions, based on the assumption that randomization will produce two equivalent groups (Durlak, 2009). However, especially for a complex construct like social competence that is both treated and measured in the context of other people (Koenig et al., 2009), intervention effects may be confounded by many unmeasured constructs (Rosenbaum & Rubin, 1983), which may lead to biased intervention effects. Meta-analyses of such interventions that examine treatment effects by comparing endpoint may be susceptible to such confounding (Preece, 1983). As such, it is especially crucial to not take for granted that the two groups are equivalent and consider whether the pattern of change itself over the course of intervention is different across the two groups.

1.5. The present systematic review and meta-analysis

Examining whether, how much, and according to whom GSSIs may be efficacious is essential for guiding delivery and improvement of this core clinical service for ASD youth. Thus, the present study first sought to meta-analytically examine the efficacy of GSSIs as assessed using well-designed RCTs according to the contemporary literature. Second, we aimed to consider whether these effects differed according to all known sources of information in the literature (parent report, teacher report, self report, observer report, measured behavior on a relevant task), as well as according to intervention characteristics, content, and change measurement.

2. Methods

2.1. Identification and selection of studies

The databases of PsycINFO, PubMed and Web of Science were searched using the following Boolean String: (ASD OR autism spectrum disorder OR Asperger OR autism OR pervasive developmental disorder) AND (social skills OR peer interaction OR social competence OR social functioning OR friendship OR social interaction OR social play) AND (treatment OR intervention) NOT (early intervention OR toddler OR early intensive behavior intervention) NOT (pharmacological OR medical).

2.2. Study selection and literature search

The systematic literature search was conducted in two stages. In an initial search, the entire literature was examined, up to July 2014. A subsequent search was then conducted to ensure the current literature was covered; thus, an identical second search was conducted covering the literature from July 2014 to January 2016.

2.2.1. First stage of literature search

Fig. 1 gives a detailed layout for the identification and selection of studies process. This search yielded 2620 results. The following inclusion criteria were used to refine the results, such that eligible studies were: a) empirical, b) peer reviewed or dissertations, c) evaluated an intervention claiming to be used by providers to address core social deficits, d) included school-aged children and adolescents aged 5–21, e) included participants with a diagnosis of autism, including PDD-NOS and Asperger’s syndrome, f) written in English, g) did not include medical or pharmacological interventions, and h) did not include early intervention. In this first pass, 298 abstracts with duplicates removed were identified. The abstracts were screened again using additional criteria: i) published between January 2010 and July 2014; j) peer-reviewed; k) a randomized controlled trial; l) a social skills group intervention; m) included a treatment as usual, wait-list control, or no-treatment control group, and selected by two independent reviewers; 100% of articles were screened by both reviewers, with excellent reliability, (ICC (2,1) > 0.93; Cicchetti, 1994). All discrepancies were resolved via group consensus. The conclusion of part one yielded 14 studies eligible for full-text review.

Fig. 1.

PRISMA flowchart representing the identification and selection of studies.

2.2.2. Second stage of literature search

In an effort to include the most up-to-date articles, a second stage of searching was conducted up to January 2016. An additional 637 results were searched from PsycINFO, PubMed, and Web of Science using the same criteria listed above, with the exception of date, which was set from August 2014 to January 2016. Out of these 637 articles, 6 additional articles were eligible for full-text review. One study was excluded because it did not fit the definition of a social skills group intervention (Kretzmann, Shih, & Kasari, 2015). After removing this study, five studies remained for full data extraction. Therefore, the conclusion of part two yielded another five studies for full-text review.

Recent review articles as well as their reference sections were searched. We included two additional studies published before January 2010 (Laugeson et al., 2009; Solomon et al., 2004) that were included in the most recent meta-analysis of social skills intervention conducted by Reichow et al. (2012), which thoroughly searched the published literature up until 2010.

2.3. Data extraction

Twenty articles were double-coded by the same two blind raters on various demographic variables, as well as outcome data on social competence (See Appendix F for a full list of articles included in the meta-analysis and notable articles in the search process). The a priori definition was: The measure must assess the level of ability or skill an individual possesses when required to engage in socio-cognitive processes and display social behaviors (Beauchamp & Anderson, 2010), and/or involve the active and skillful coordination of multiple processes and resources available to the child to meet social demands and achieve social goals in a particular type of social interaction (e.g., parent-child, peer relations) and within a specific context (e.g., home, school; Iarocci, Yager, & Elfers, 2007). We also included any measures that were specifically identified in the text of the studies as a social competence measure (see Appendix A for included measures). Agreement among coders was excellent, ICC (1, 2) > 0.90.

Appendix A.

Measures used for each study by source.

Citation	Parent	Teacher	Self-report	Observer	Task
Andrews et al., 2013	–Affection for Others Questionnaire –General Affection Questionnaire –Social Competence with Peers Questionnaire	–	–	–	–Walk in the Forest
Begeer et al., 2011	–The Children’s Social Behavior Questionnaire	–	–The Index of Empathy for Children and Adolescents	–	–Levels of Emotional Awareness Scale for Children –The Theory of Mind Test
Begeer et al., 2015	–Social Responsiveness Scale –Social Skills Questionnaire –Theory of Mind Behavior Checklist	–Social Skills Questionnaire	–	–	–Advanced Theory of Mind Test –Levels of Emotional Awareness Scale for Children –The Theory of Mind Test
Corbett et al., 2015	–Adaptive Behavior Assessment System	–	–	–	–Developmental NEuroPSYchological Assessment
Frankel et al., 2010	–Social Skills Rating Scale	–Pupil Evaluation inventory	–Piers-Harris Self-Concept Scale –The Loneliness Scale	–	–
Gantman et al., 2012	–Social Responsiveness Scale –Social Skills Rating Scale	–	–Social Skills Inventory –Test of Young Adult Social Skills Knowledge	–	–
Ichikawa et al., 2013	–Strengths and Difficulties Questionnaire	–	–	–Interaction Rating Scale	–
Kamps et al., 2015	–	–Teacher Impression Scale	–	–Study-Developed Observation Form	–
Koenig et al., 2010	–Social Competence Inventory	–	–	–	–
Koning et al., 2013	–Social Responsiveness Scale –Vineland Adaptive Behavior Scale	–	–Social Knowledge	–Peer Interaction Measure	–The Child and Adolescent Social Perception measure
Laugeson et al., 2009	–Social Skills Rating Scale	–	–Test of Young Adult Social Skills Knowledge	–	–
Laugeson et al., 2015	–Social Responsiveness Scale –Social Skills Rating Scale	–	–Friendship Qualities Scale –Test of Young Adult Social Skills Knowledge	–	–
Lopata et al., 2010	–Adapted Skill streaming Checklist –Behavior Assessment System for Children –Social Responsiveness Scale	–	–Skill streaming Knowledge Assessment	–	–Diagnostic Analysis of Nonverbal Behavior
Rodgers et al., 2015	–The Facial Affect Rating Form	–	–	–	–
Schohl et al., 2014	–Social Responsiveness Scale –Social Skills Rating Scale	–Social Responsiveness Scale –Social Skills Rating Scale	–Friendship Qualities Scale –Social Interaction Scale –Test of Adolescent Social Skills Knowledge	–	–
Solomon et al., 2004	–	–	–	–	–Diagnostic Analysis of Nonverbal Behavior –Faux Pas Stories Task –Strange Stories Task
Thomeer et al., 2012	–Adapted Skillstreaming Checklist –Behavior Assessment System for Children –Social Responsiveness Scale	–	–Skillstreaming Knowledge Assessment	–	–Diagnostic Analysis of Nonverbal Behavior
White et al., 2007	–Social Responsiveness Scale	–	–	–	–
Yoo et al., 2014	–Asperger Syndrome Diagnostic Scale –Social Communication Scale –Social Responsiveness Scale –Vineland Adaptive Behavior Scale	–	–Friendship Qualities Scale –Social Skills Rating Scale	–Autism Diagnostic Observation Schedule	–Autism Diagnostic Observation Schedule

After the variable set was completed, all authors were contacted for additional data needed including demographic and outcome variables. After corresponding with authors, it was revealed that two studies (Schohl et al., 2014; Van Hecke et al., 2015) contained overlapping participants (AV. Van Hecke, personal communication, April 27, 2016). It was decided that the study with the more complete set of data would be included (Schohl et al., 2014). Similarly, one study (Rodgers et al., 2015) contained completely overlapping participants with two other studies (Lopata et al., 2010; Thomeer et al., 2012; J.D. Rodgers, personal communication, June 6, 2016). Therefore, this article was excluded from the overall analysis. However, we included this study in the observer-report analysis, as it was the sole study from this sample that included data from observers. Consequently, data from 19 articles were included in the meta-analyses.

2.4. Meta-analytic procedure

The meta-analytic procedures used in this paper adhered to all applicable Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for Meta-Analysis (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009). The current study included six separate meta-analyses. The first meta-analysis included all measures of social competence from each study regardless of informant. This analysis averaged all relevant measures of social competence in each study. In order to examine informant influence on social competence outcome ratings, the other meta-analyses were composed of measures reported exclusively from different sources: parent, teacher, self, observer, and behaviors measured on a relevant task.

2.4.1. Parameterization of terms

Parent-report was defined as social competence data collected using standardized assessments, questionnaires, and/or interviews about frequency and quality of the child’s social interactions with others from the perspective of a parent or caregiver of the child or teenager with ASD. Teacher-report was defined as social competence data collected using standardized assessments, questionnaires, and/or interviews from the participant’s teacher that assess frequency and quality of the child’s social interactions with others in school settings. Self-report was defined as social competence data collected from the perspective of the participant (child or teenager with ASD) who was receiving the intervention using standardized assessments, questionnaires, and/or interviews about the frequency and quality of their own behavior, emotions, and/ or knowledge. These informant-rated questionnaires largely overlap with the National Institute of Mental Health Research Domain Criteria (RDoC; Insel et al., 2010; Morris & Cuthbert, 2012) “Subjective/Self-Reports” unit of analysis within the social processes domain. Observer-report was defined as behavioral observation data on the frequency, duration, and/or degree of appropriateness of participant’s social behavior, usually in the context of a dyadic or group interaction, typically rated by at least two trained observers that are assessed for the reliability of their observations using a standardized coding system (Kamps et al., 2015; Koning et al., 2013; Rodgers et al., 2015). The observer-report outcome data corresponds to the “Behaviors” unit of analysis within the RDoC social processes domain. Task-based source was defined as data collected through the completion of a task or activity in which the participant must utilize specific skills and knowledge related to social competence, such as emotion recognition or Theory of Mind. The assessments could be completed independently by the participant, on a computer or electronic device, or administered by a researcher. The task-based outcome data corresponds to “Paradigms” unit of analysis, which refers to scientific tasks that are especially useful for studying the construct (Morris & Cuthbert, 2012; Sanislow, Quinn, & Sypher, 2015), within the RDoC social processes domain.

2.4.2. Statistical analyses

The effect size statistic provides information about the direction and magnitude of quantitative research findings (Lipsey & Wilson, 2001). The effect size (g), is calculated using the difference between means of the treatment group and the control group, divided by the standard deviation, and weighted for sample size to correct for small sample bias (Hedges & Olkin, 1985; Lipsey & Wilson, 2001). An effect size of 0.2 is considered to be a small effect, 0.5 is considered a medium effect, and above 0.8 is considered a large effect (Cohen, 1992).

Mean gain scores were used for the treatment and control groups to calculate Cohen’s d. This was used to yield better estimates of the treatment effects by taking into account pretest differences (Durlak, 2009). Effect sizes (d) were calculated for each measure, and then averaged together to make an overall effect size for each study. Similarly, for the informant-analyses, effect sizes for the relevant informant measures were averaged together to create a single informant effect size for each study. The Standardized Mean Difference using Hedges’ g for small sample correction was calculated using Comprehensive Meta Analysis, Version 2 (CMA-2; Borenstein, Hedges, Higgings, Rothstein, & Englewood, 2007). A random-effects model was used in all calculations. For each analysis, if variation was detected, (Q; I²) moderator analyses were conducted. The Q-test instructs whether there is heterogeneity by summing the squared deviation from each study’s effect size from the overall effect size, and weighing each study by variance (Higgins & Green, 2011; Huedo-Medina, Sánchez-Meca, Marín-Martínez, & Botella, 2006). The I² measures the degree of heterogeneity and provides a percentage of the amount of variance that is attributable to between-study variation (Huedo-Medina et al., 2006). To evaluate whether it would be appropriate to conduct moderator analyses on an obtained effect, the following criteria were established: (a) at least 10 studies included in the analysis; and (b) in the absence of significant Q statistics, evidence of at least a nontrivial amount of heterogeneity according to the I² statistic (≥20%; Huedo-Medina et al., 2006). If these criteria were met, fixed effects meta-regression moderator analyses were conducted for potential moderators. The a priori moderators of interest coded during data extraction were age, gender, overall cognitive ability, verbal ability, medication status, presence of psychiatric comorbidity, intervention length, and inclusion of a peer tutor (Lerner & White, 2015; Wang et al., 2011).

The continuous moderators (age, overall cognitive ability, verbal ability, intervention length, percent of total variance of self-report, percent of total measures of self-report) were conducted using analogue to regression and the categorical moderators (gender, medication status, comorbidity, inclusion of a peer tutor, presence of a social knowledge measure) were conducted using analogue to ANOVA.

Due to qualitative differences between child’s report of his/her knowing what to do in social situations versus of actually doing such things (Lerner, Calhoun et al., 2012), we considered inclusion of social knowledge measures (binary variable: social knowledge measure included vs. excluded) as a potential moderator of self-report effect. A social knowledge measure was defined as a test that directly measured what the participant knew about social competence and what they were supposed to do (e.g., “The most important part of having a conversation is to…”) vs. answering a question or rating a statement about whether the person actually does them (e.g., “I make friends easily,” or “I try to understand how others feel”). If the binary social knowledge measure was a significant predictor in the self-report meta-regression, we further examined the effect of including social knowledge measures in two additional ways. First, we calculated the proportion of the total number social knowledge measures relative to the total number of self-report measures in each study (e.g., if 4 self-report measures were used and 1 of them was a social knowledge measure, this value would be 25%), and re-ran the meta-regression with this variable included. Second, we calculated the percent of variance in change in self-reported social skills attributable to the social knowledge measures (e.g., if the changes in social knowledge measures accounted for 25% of the total effect of change in social skills according to child self-report, this value would be 0.25), and re-ran the meta-regression with this variable included. If social knowledge was a significant predictor in all of these moderator models, an additional self-report meta-analysis was conducted in which all social knowledge measures were excluded (i.e., only “pure” self-report of social skills was included).

In an effort to better measure true within-group change between pretest and posttreatment over the course of intervention, we calculated all effect sizes for the primary analyses as mean gain scores (the difference between the posttest mean and the pretest mean; Dimitrov & Rumrill, 2003). The mean gains score effect sizes were then compared to the traditional method of calculating an effect size (using the unadjusted posttest scores for the treatment group and the control group). The differences in g values divided by the square root of the sums of the variances of the individual g is distributed as Z, and we used this Z-distributed statistic to test the significance of the difference between two Hedges’ g values as outlined by Rosenthal (1991, p. 65).

2.4.3. Tests for publication bias

Publication bias was assessed when appropriate (k > 10) using funnel plots (plots represent a symmetrical, inverted funnel, with smaller studies spread across the bottom; Egger & Davey Smith, 1998) and a combined tandem method suggested by Ferguson and Brannick (2012). This method includes Egger’s regression test, wherein significant findings suggest publication bias (Egger & Davey Smith, 1998); the trim-and-fill method (this method analyzes an asymmetrical funnel plot, identifies the unbalanced plots, “trims” the studies responsible for asymmetry, and assigns new effect sizes to correct for suspected publication bias, if the effect is no longer significant, then publication bias is suspected; Duval & Tweedie, 2000); and Orwin’s Fail Safe N (the number of non-significant missing studies [i.e., file-drawer articles] needed for the effects to be no longer significant is lower than the number of studies in the analysis; Ferguson & Brannick, 2012). If indicated by all three criteria, publication bias was deemed “probable,” if indicated by one or two criteria met, bias was deemed “possible,” and if no evidence of bias was found, it was deemed “unlikely.”

3. Results

3.1. Descriptive characteristics

For the 18 studies included in the overall meta-analysis, data were collected from 735 participants (see Appendix B). Sample sizes ranged from 11 to 97 participants, M (SD) = 40.83 (25.56). The age of participants ranged from 5.30 to 20.42 years, M (SD) = 10.54 (4.18). 33.3% of studies had study samples with >90% male, and all other samples were between 50% and 90% male. The mean overall standardized cognitive ability of participants was 102.27, ranging from 87.55 to 112.45 across studies. The mean standardized verbal ability of participants was 100.01, ranging from 86.3 to 106.26 across studies. 44.4% of studies reported data on comorbidity of participants, and 50% of studies included participants taking medications. Intervention length ranged from 5 to 97 sessions. 22.2% of the studies included peer tutors.

Appendix B.

Descriptive information for each study.

Citation	Mage	N	% Male	Mean Overall Cognitive Ability	Mean Verbal Ability	Intervention length	Comorbidity data	Medication	Peer Tutors	Sources
Andrews et al., 2013	9.02	58	50–90%	112.45	N/A	5 Sessions	N/A	N/A	No	P, TA
Begeer et al., 2011	10.3	36	≥90%	101.39	105	16 Sessions	Yes	N/A	No	P, S, TA
Begeer et al., 2015	9.6	97	≥90%	N/A	106.26	8 Sessions	Yes	Yes	No	P, T, TA
Corbett et al., 2015	11.04	30	50–90%	101.64	101.19	10 Sessions	N/A	Yes	Yes	P, TA
Frankel et al., 2010	8.53	68	50–90%	N/A	103.8	12 Sessions	N/A	No	No	P, S, T
Gantman et al., 2012	20.4	17	50–90%	102.3	95.93	14 Sessions	Yes	Yes	No	P, S
Ichikawa et al., 2013	5.3a	11	50–90%	87.55a	98.5	20 Sessions	N/A	N/A	No	P, O
Kamps et al., 2015	5.8	95	50–90%	N/A	86.3	97 Sessions	N/A	N/A	Yes	T, O
Koenig et al., 2010	9.24	41	50–90%	96.2	N/A	16 Sessions	N/A	Yes	Yes	P
Koning et al., 2013	11.07	15	≥90%	109.3	102.31	15 Sessions	Yes	Yes	No	P, S, O, TA
Laugeson et al., 2009	14.6	33	50–90%	95.24	92.27	12 Sessions	No	Yes	No	P, S
Laugeson et al., 2015	20.42	20	50–90%	105.03	98.17	16 Sessions	Yes	N/A	No	P, S
Lopata et al., 2010	9.47	36	≥90%	103.04	104.32	30 Sessions	N/A	N/A	No	P, S, TA
Rodgers et al., 2015	9.33	60	50–90%	102.77	104.03	30 Sessions	N/A	N/A	No	O*
Schohl et al., 2014	13.65	58	50–90%	100.34	100.31	14 Sessions	Yes	Yes	No	P, S, T
Solomon et al., 2004	9.37	18	≥90%	105.22	105.94	20 Sessions	N/A	N/A	No	TA
Thomeer et al., 2012	9.31	35	50–90%	103.83	103.6	30 Sessions	N/A	N/A	No	P, S, TA
White et al., 2013	14.58	30	50–90%	N/A	97.39	20 Sessions	Yes	Yes	Yes	P
Yoo et al., 2014	13.78	47	≥90%	99.87	100.04	14 Sessions	Yes	Yes	No	P, S, O

Note. P- Parent, S-Self, O- Observer, T- Teacher, TA- Task,

^aMedian score.

^*This study was not included in the overall analyses due to sample overlap with Lopata et al. (2010) and Thomeer et al., (2012). However, it was included in the Observer-report analyses. The Mean Cognitive Ability and Mean Verbal Ability are standard scores. Medication use was categorized by “Yes,” “No,” or “N/A = not available,” if it was indicated by the article or through communication with the author that any of the participants of the intervention were taking medication. Inclusion of a peer tutor was defined as the presence of a typically developing peer that assists or participates in the group social skills intervention.

3.2. Overall analysis

The effect sizes (g) ranged in magnitude from g = 0.20 to g = 1.19, with positive effects indicating increases in ratings of social competence (see Table 1). Fig. 2 shows individual effect sizes for this analysis.

Table 1.

Aggregate effect sizes for all analyses.

Informant	k	Total N	Effect size (g)	Q	I2
Overall combined	18	745	0.51**	6.36	0.00
Parent	16	632	0.47**	4.08	0.00
Teacher	4	318	0.41	5.27	43.06
Self	10	365	0.92**	11.52	21.87
Observer	5	228	0.40**	1.16	0.00
Task	8	325	0.58**	1.64	0.00

Note.

^**p < 0.01.

Fig. 2.

Overall effect sizes for all measures of social competence, regardless of informant or source. All models are random effects. All effect sizes are Hedges’ g. Plots with a square indicate Hedges’ g for individual studies within the analysis. Diamond indicates overall effect size for the analysis.

Participants who received treatment made significantly greater improvements on measures of social competence compared to those in the control group (g = 0.51, K = 18, 95% [CI 0.30, 0.72], Z = 4.75, p < 0.001). This is a medium effect. While the number of studies included in the overall analysis was sufficient, the I² (0.00) was not large enough to justify proceeding with moderator analyses.

3.2.1. Publication bias analysis

Evidence of publication bias was found via Egger’s regression test (b = 0.93, p < 0.01), implying asymmetry of studies was detected in the funnel plot (See Appendix C). Publication bias was not evident according to the trim and fill analysis, where six studies were removed to the left of the mean, making the adjusted effect of GSSI intervention smaller (g = 0.39, 95% [CI 0.20, 0.58]), though still significantly different from zero. In contrast, no evidence of publication bias was found according to the Fail Safe N (98 studies). The combined tandem criteria suggest publication bias is possible.

3.3. Informant analysis

In order to investigate the effect of informant on ratings of social competence, five separate meta-analyses were conducted which included measures reported exclusively by parent, teacher, self, observer, or completed as a task. Moderator analyses were conducted when appropriate.

3.3.1. Parent-report

Sixteen studies included parent-report measures. The effect sizes for parent-report measures ranged from g = 0.06 to g = 1.03. Fig. 3 displays the individual effect sizes for the parent-report analysis. According to parent-report, participants who received treatment had significantly greater improvements in social competence relative to controls (g = 0.47, K = 16, 95% CI [0.24, 0.70], Z = 4.01, p < 0.01). This is a small effect. Although the sample of studies was sufficient, the I² (0.00) was not large enough to justify proceeding with moderator analyses.

Fig. 3.

Effects sizes for parent-report measuresofsocial competence. All models are random effects. All effect sizes are Hedges’ g. Plots with a square indicate Hedges’ g for individual studies within the analysis. Diamond indicates overall effect size for the analysis.

3.3.1.1. Publication bias analysis

No evidence of publication bias was detected according to Egger’s regression test (b = 0.59, p = 0.14), implying symmetry of the funnel plot (See Appendix D). Publication bias was not evident according to the trim and fill method, as three studies were removed to the left of the mean and the adjusted effect of intervention was smaller (g = 0.41, 95% [CI 0.20, 0.63]), but still significantly different from zero. Further, no evidence of publication bias was found according to the Fail Safe N (55 studies). Thus, no evidence for publication bias was found using the tandem method, suggesting such bias is unlikely.

3.3.2. Teacher-report

Four studies contained teacher-reported measures of social competence. The effect sizes (g) for teacher-report ranged from 0.11 to 0.98. Fig. 4a displays the individual effect sizes for the teacher-report analysis. According to teacher-report, there was not a statistical difference in social competence between the treatment and control groups (g = 0.41, K = 4, 95% CI [-0.10, 0.93], Z = 1.58, p = 0.11). This is a small effect size. While the I² (43.06) value was sufficient to support exploratory moderator analyses, the sample of studies was too small to do so; for the same reason, publication bias could not be analyzed.

Fig. 4.

4a. Effects sizes for teacher-report measures of social competence. 4b. Effects sizes for self-report measures of social competence. All models are random effects. All effect sizes are Hedges’ g. Plots with a square indicate Hedge’s g for individual studies within the analysis. Diamond indicates overall effect size for the analysis.

3.3.3. Self-report

There were 10 studies that included self-report measures. The effect sizes (g) for child-report measures ranged from g = 0.13 to g = 2.15. Fig. 4b shows the individual effect sizes for the self-report analysis. According to self-report informants, those who received treatment showed significantly greater improvements on measures of social competence compared to controls (g = 0.92,K =10, 95%CI [0.58, 1.26], Z= 5.26, p < 0.01). This is a large effect. The I² (21.87) value was nontrivial, and there were at least 10 studies included; thus, exploratory putative moderator analyses were conducted (see Moderator Analysis below). Evidence of publication bias was found via Egger’s regression test (b = 3.01, p = 0.03), implying asymmetry of studies in the funnel plot (See Appendix E).

3.3.3.1. Publication bias analysis

Publication bias was not indicated by the trim and fill method: while three studies were removed to the left of the mean and the adjusted effect of intervention was smaller (g = 0.72, 95% [CI 0.45, 0.99]), it was still significantly different from zero. No evidence of publication bias was found according to the Fail Safe N (92 studies). Thus, the combined tandem criteria suggest publication bias is possible.

3.3.4. Observer-report

Five studies contained observer-report measures related to social competence. The effect sizes (g) for observer-report measures ranged from 0.07 to 0.84. Fig. 5a displays the individual effect sizes for the observer-report analysis. According to observer report, participants who received treatment had significantly greater improvements in social competence than controls (g =0.40,K=5,95% CI [0.28, 0.52], Z = 6.75, p < 0.001). The sample of studies was not sufficient and the I² (0.00) did not meet the criteria to justify proceeding with moderator or publication bias analyses.

Fig. 5.

5a. Effects sizes for observer-report measures of social competence. 5b. Effects sizes for task-report measures of social competence. All models are random effects. All effect sizes are Hedges’ g. Plots with a square indicate Hedges’ g for individual studies within the analysis. Diamond indicates overall effect size for the analysis.

3.3.5. Task-based measures

Eight studies contained task-based measures of social competence. The effect sizes (g) for task-report ranged from 0.26 to 1.07. Fig. 5b displays the individual effect sizes for the task-source analysis. According to task-based measures, those children who received treatment performed significantly better on task-based measures relative to those in the control groups (g = 0.58, K = 8, 95% CI [0.24, 0.92], Z = 3.314, p < 0.01). This is a medium effect. The sample of studies was not sufficient and the I² (0.00) did not meet the criteria to justify proceeding with moderator or publication bias analyses.

3.4. Moderator analysis

Exploratory analyses of putative moderators were conducted for the self-report informant meta-analysis. All of the putative descriptive moderators (i.e., age, gender, overall cognitive ability, verbal ability, intervention length, comorbidity, medication, and peer tutors) were non-significant. Interestingly, the three self-report social knowledge moderators did demonstrate significant variability.

3.4.1. Social knowledge - presence

The categorical variable of presence of a social knowledge measure was significant (Q (1) = 7.01, p = 0.01). Effect sizes were larger when a social knowledge self-report assessment was included (g = 1.149, K = 8, 95% CI [0.80, 1.50], Z = 6.35, p < 0.01) than when they were not (g = 0.280, K = 2, 95% CI [-0.28, 0.82], Z = 1.02, p = 0.31).

3.4.2. Social knowledge - percent of total measures

The variable of percent of total measure was significant, Q (1) = 7.27, p = 0.01. Effect sizes were larger when a greater total percentage of self-report measures were social knowledge assessments, β = 1.05, SE = 0.331, 95% CI [0.29, 1.82].

3.4.3. Social knowledge - percent of variance

The variable of percent of variance was significant Q (1) = 6.74, p = 0.01. Effects were larger when the percentage of the overall effect of change in self-reported social skills that is attributable to social knowledge was greater, β = 0.889, SE = 0.346, 95% CI [0.220, 1.578].

3.5. Post-hoc analyses

3.5.1. Self-report meta-analysis without social knowledge

A post-hoc analysis of the self-report data excluding the social knowledge measures was conducted. Six studies were included in this analysis. Effect sizes ranged from g = 0.06 to g = 1.095. When social knowledge measures were excluded from the analyses, there was no significant difference from controls g = 0.20, K = 6, 95% CI [-0.137, 0.54], Z = 1.17, p = 0.14. That is, improvements in self-report were only found for measures of social knowledge. The test of heterogeneity was not significant (Q (5) = 2.99, p = 0.70; I² = 0.00).

3.5.2. Post-test SMD

Post-hoc analyses comparing the gain score SMD to post-test SMD scores for each meta-analysis were conducted (Table 2). None of the Hedges’ g scores calculated using post-test SMD were significantly different from mean gain scores (all p > 0.13). Moderator analyses were not conducted for the post-test SMD analyses.

Table 2.

Social skills intervention effect sizes according to Mean Gain Score SMD vs. Post-Test only SMD.

Informant	k	Mean gain score SMD (g)	Mean gain score 95% CI	Post-test SMD (g)	Post-test 95% CI	Z1
Overall	18	0.51**	0.20, 0.58	0.49**	0.354, 0.64	0.06
Parent	16	0.47**	0.24, 0.70	0.47**	0.31, 0.62	0.01
Teacher	4	0.41	−0.10, 0.93	0.44**	0.15, 0.73	0.88
Self	10	0.92**	0.58, 1.26	0.78**	0.47, 1.09	0.58
Observer	5	0.40**	0.28, 0.52	0.18	−0.10, 0.45	1.45
Task	8	0.58**	0.24, 0.92	0.59**	0.30, 0.89	−0.06

Note.

^**p < 0.01.

¹No Z scores reached the 0.05 significance threshold. SMD - Standardized Mean Difference. Z = differences in g between the two SMD scores, divided by the square root of the sums of the variances of the individual g; this produces a Z-distributed statistic for the difference in effect size between the two SMD scores.

4. Discussion

This study was the largest meta-analysis conducted to date evaluating the efficacy of GSSIs for youth with ASD. Moreover, this was the first study to examine effect sizes by varying reporting sources of social competence. Results indicated that GSSIs led to moderate overall improvements in social competence, reflected in data from parents, youth, observers, and tasks, but not teachers, supporting variations by reporting sources. Notably, the large self-report effect was wholly attributable to increased social knowledge, but not perceived changes in their own social behavior.

4.1. Comparison with other meta-analytic studies

The effect size found in the overall meta-analysis (g = 0.51) was comparable to the only other meta-analysis of GSSI to date (g = 0.47; Reichow et al., 2012), suggesting medium effects of GSSI interventions on social competence. This was, again, similar in magnitude to the effect found for interventions affecting social skills for youth with ADHD (Daley et al., 2014) and schizophrenia (Kurtz & Mueser, 2008), but larger than that for youth with learning disabilities (g = 0.21; Forness & Kavale, 1996). It was also larger than the effects of school-wide (g = 0.15; January, Casey, & Paulson, 2011) and after-school (g = 0.19; Durlak, Weissberg, & Pachan, 2010) interventions focusing on improving social skills in general populations of children and adolescents. This suggests that social skills intervention efficacy may be augmented somewhat for populations with primary (rather than secondary) deficits in this domain, suggesting a “deficit consonance” for this treatment modality.

Interestingly, though, the effect was broadly similar to that found across all child and adolescent group treatment (d = 0.61; Hoag & Burlingame, 1997), indicating that grouping youth with common concerns and providing a venue to directly, collaboratively, and strategically address them may yield a generalized benefit, perhaps via a common, nonspecific factor such as group cohesion (Lerner et al., 2013).

Further, a recent meta-analysis of CBT for individuals with ASD (Weston, Hodgekins, & Langdon, 2016) across all age ranges and treatment modalities (group and individual) found similarly small and nonsignificant effects for self-report (g = 0.25) and medium effects (g = 0.48) for informant (e.g., parent-)report of ASD symptoms; however, the effect of CBT for task-based measures was small (g = 0.35), in comparison to a medium effect found in the present meta-analysis. Thus, GSSIs for youth with ASD may produce similar (and, according to some tasks, larger) effects on ASD symptoms than more traditional CBT, while CBT may be more effective for treating anxiety in this population. This contrast provides foundational guidance for symptom-specific treatment recommendations for individuals treating youth with ASD.

4.2. Variations by sources in intervention effect

Results of the parent-report meta-analysis revealed a small effect of GSSIs on social competence. Past literature has demonstrated that parents tend to stably report positive effects of intervention, possibly due to expectancy effects reflecting their investment in and allegiance to the intervention and/or hope and expectations for intervention to increase skills (McMahon, Lerner et al., 2013). That said, intervention effects reported by parents in this study were small. Several elements may contribute to this finding: social competence is a complex and multidimensional construct and many parent-report measures combine assessment of both social knowledge and social performance. Previous literature suggests that while parents generally report high satisfaction with the GSSIs and gains in social knowledge, their report of changes in behaviors in naturalistic settings is infrequent (White et al., 2007). Thus, the small parent-report effect may be attributable specifically to gains in only a limited number of settings. Alternatively, due to expectancy effects, parents may already anticipate some changes in the behaviors of their children as a result of participating in the intervention at all; thus, they may report small effects even if robust gains are not being seen.

On the other hand, no differences were found between treatment and control groups in the effect of GSSIs according to teacher-report. There may be several reasons for this. Teachers often may not be aware of the skills being targeted in the GSSI, making them less susceptible to expectancy effects or allegiance effects in their reports of improvement. This supports the proposition that the aforementioned small parent-report effects may be a function of expectancy effects rather than robust, cross-contextual changes. Relatedly this finding provides further support for differential expression of psychopathology across settings (Achenbach, McConaughy, & Howell, 1987; De Los Reyes, Henry, Tolan, & Wakschlag, 2009). Generalization of acquired skills may be especially difficult in school settings where social demands are generally greater than (or at least different from) home or intervention settings. For example, classmates may continue to show negative bias, even when participants do improve in their social skills (Mikami, Lerner, & Lun, 2010). Conversely, it is possible that teachers may not have sufficient opportunity to observe certain social skills in the classroom, so they may need more observation opportunities to see any actual changes that may be evident.

Self-reported measures of social competence revealed a large effect. This finding is interesting compared to the small effects seen by parents, and a stark contrast to the lack of treatment effects seen by teachers. However, it is consistent with past literature suggesting a consistent tendency in youth with ASD to overestimate their social functioning relative to their parents (Lerner, Calhoun et al., 2012). Moreover, they also tend to show expectancy effects similar to parent-report, as children also consistently report high satisfaction with GSSIs (McMahon, Lerner et al., 2013). Taken together, these tendencies may contribute to greater estimation of their intervention-related improvements.

That said, moderator analyses indicated that the use of self-reported social knowledge measures was wholly responsible for the large effect shown in the self-report analysis. That is, while participants report gains in their social knowledge, they do not report changes in their social behaviors (Lerner, White et al., 2012). This finding suggests that the common approach in GSSIs of didactically teaching social skills may not provide opportunities to allow adequate application of social skills and rehearsal of these behaviors. As intervention characteristics (e.g., social knowledge-based vs. social performance-based methods) may affect gains in these constructs separately, this highlights the importance of GSSIs providing opportunities for participants to actively practice these skills in social situations (Lerner & Mikami, 2012; Lerner, Mikami, & Levine, 2011).

Further, this finding reveals valuable insights about the participants’ self-awareness of their own social competence. Individuals reported that they had improved in knowledge of correct social skills, but also that they were not actually performing these skills in social settings, which is more consistent with reports by parents and teachers. This indicates that the participants in the interventions themselves did not actually believe the intervention was helping them improve how they perform social skills in real-life situations. This stands in contrast to theories stating individuals with ASD have limited insight into their social competence (e.g., Frith & Hill, 2003; Williams, 2010), in that they may be able to differentiate between reflecting on their own knowledge of a behavior, versus enactment of it.

All other putative moderators (mean age, gender composition, overall cognitive ability, verbal ability, comorbidity, medication status, and inclusion of peer tutors) did not predict self-report intervention effects. Importantly, this was the only reporting source for which moderator analyses could be run. Thus, these child and group-level variables may indeed still predict outcomes according to different rater; as additional studies in this domain are published, these potential moderators should surely be explored. At present, however, this suggests that the heterogeneity in outcomes in self-report was not explained by any of these participant or intervention characteristics.

Observer-report studies revealed a small effect of GSSIs on social competence. This demonstrates that modest improvements in naturalistic social behaviors are being observed as a result of GSSI, even though participants themselves may not report this. Further, it lends support to the idea that these behaviors are malleable to intervention. That said, consistent with the findings from the parent-report analyses, the small effect suggests that large gains in social performance may indeed be difficult to achieve with current GSSIs approaches. Thus, identifying elements of GSSIs that do indeed potentiate naturalistic social behavior would be valuable.

Task-report studies yielded a medium effect of GSSIs on social competence. Many GSSIs utilize strategies of teaching specific skills and knowledge required to complete a given task (e.g., Theory of Mind) by providing explicit feedback on their performance on the task (e.g., giving them the right answer and “teaching to the test”). The medium effects are consistent with the literature that children with ASD can successfully learn these skills following didactic instructions (e.g., Wellman et al., 2002). However, the degree of improvement in tasks may not be mirrored in actual social skills use (Ozonoff & Miller, 1995). Therefore, it is important to examine whether changes in task-based measures represent generalized skill learning vs. simply effects of teaching to the test.

Across meta-analyses, no significant differences were found between the mean gain score SMD and post-test SMD scores. This finding may suggest that randomization in each study was done successfully to yield comparable pretreatment scores across treatment and control groups, and supports the use of either method in future meta-analyses.

4.3. Publication biases

Evidence for possible publication bias was found in the overall and self-report meta-analyses, suggesting that “true” effects may be smaller than what are currently reported in the literature. This finding suggests that studies with smaller effects (though, importantly, not a large number of null findings) are being omitted from the literature, perhaps due to the fact that most GSSI studies aim to evaluate “novel” treatments (i.e. studies that have a new name, even if not materially different in content from other interventions) rather than replicating and extending the current literature. Consistent with “first studies” throughout science (e.g., Tuttle et al., 2015), these “novel” GSSIs tend to show larger effects (e.g., Kamps et al., 2015; Koning et al., 2013; Lopata et al., 2010); thus, the current published literature may be selecting for these types of studies, thereby inflating overall effects. Especially considering the structural similarity of many of these interventions (see Lerner & Mikami, 2012; Lerner & White, 2015) future research should focus on follow-up and replication studies of existing GSSIs (e.g., Van Hecke et al., 2015).

Conversely, the findings here could be a result of “small study” effects (effects due to systematic methodological differences between studies with small vs. large sample size) rather than true publication bias (Ferguson & Brannick, 2012). Therefore, a greater number of studies, as well as studies that include a larger sample of participants, are needed.

Findings from the publication bias analyses bolster the importance of considering variations in intervention effects by sources. There was some evidence of publication bias in the overall meta-analysis, which included an aggregate of several sources, including those for which we could not conduct publication bias analyses (i.e., teacher, observer, and tasks sources). Results from the parent-report meta-analysis suggest that publication bias is unlikely in this domain. This finding reinforces the aforementioned prospect that parent-report may be influenced by expectancy effects; that is, these effects may provide a “floor” for parent-reported effect sizes, contributing to their stability, even in cases where objective changes may not be evident. They also highlight the differences in reporting patterns among sources and suggest that the evidence of publication bias found in the overall report may be influenced by other sources. For instance, given the above self-report findings, it may be that unpublished null findings on self-reported social skills (without social knowledge measures) may contribute to the observed bias. Further, given the nonsignificant findings in the teacher-report analyses, it is reasonable that unpublished, nonsignificant studies using teacher-report measures may contribute as well.

4.4. Implications for practice

GSSI are among the most widely used methods to improve social skills in school-age and teenage individuals with ASD (McMahon, Lerner et al., 2013). Consistent with the past meta-analysis (Reichow et al., 2012), findings from this study suggest that participants of GSSIs show some improvement in social competence after GSSIs.

Further, important variations in the reports of gains by sources reflect multidimensionality of and contextual variations in participant’s social skills and behavior (De Los Reyes et al., 2009; Murray et al., 2009). Providers of intervention may observe significant variations in participant’s behavior across contexts and may need to foster appropriate strategies that will be effective in a specific context (e.g., school vs. home environment). Developing intervention approaches that support enactment of successful social behavior across contexts may support more robust gains. Moreover, results identifying variations in self-reported gains in social competence when assessed via a social knowledge measure vs. social performance measure suggest the importance of considering and interpreting these separate constructs differently (Lerner & White, 2015; McMahon, Lerner et al., 2013). Thus, it would be important for the providers to implement strategies that aim to provide more opportunities for participants to practice the performance of these learned skills in real situations. Further, social knowledge and social performance measures divulge substantively different information on the improvement of social skills in interventions, suggesting that providers should be careful, specific, and circumspect in their use and interpretation of such measures. Moreover, the findings suggest a vital need to carefully consider variations according to source and differentiate social knowledge and social performance when assessing intervention effects.

Further, some analyses reported a nontrivial amount of heterogeneity, but this heterogeneity could not be accounted for using the demographic or intervention variables assessed in the analysis. This suggests that GSSIs may be beneficial for participants with wide range of demographic characteristics, or that additional individual differences should be considered in future RCTs of GSSIs.

4.5. Limitations and future research

The current study had several limitations that future research could address. First, we evaluated different GSSI approaches, including both social knowledge-based approaches and social performance-based approaches (Lerner & Mikami, 2012; Lerner & White, 2015). Future studies should more thoroughly identify differences between features of interventions and conduct moderation analyses in order to identify participant characteristics for which specific intervention approaches are most likely to be beneficial.

Moreover, although this was the largest meta-analysis conducted on GSSIs for youth with ASD, the total k was modest, especially considering individual source effects. As such, we were not able to conduct moderator analysis or publication bias analysis for some sources of effects.

Also, it is important to note that the reported effect sizes only provide information about improvement in social competence when compared to a participant’s own levels just before participating in a GSSI and therefore do not provide information on the level of social competence compared to individuals without ASD. Therefore, future studies may wish to consider clinically meaningful improvements (Jacobson & Truax, 1991) as well as statistical change.

Furthermore, the efficacy of GSSIs in this study is demonstrated based on RCTs with waitlist or no-treatment control groups. However, it is unclear whether the effects are attributable simply to working together in a supported group context, which has itself been shown to be beneficial for youth with ASD (e.g., Bohnert, Lieb, & Arola, 2016). Therefore, use of attention control groups that are matched for structure and contact would provide informationas to whether GSSIs outperform active conditions that control for attention and other nonspecific factors (e.g., time spent with peers). In addition, it would be important to evaluate the effectiveness (rather than just efficacy) of GSSIs to assess whether they are implemented with fidelity in community settings, and whether such fidelity contributed to obtained effects.

Lastly, this study does not include follow-up and long-term comparisons of treatment and control conditions, as most studies did not include follow-up assessments. More studies with follow-up data are needed with the same control condition throughout the follow-up period.

4.6. Summary

This meta-analysis suggests that GSSIs lead to moderate overall improvements in social competence in youth with ASD relative to non-treatment or waitlist controls. This study also indicates that parents and observers report small effects of GSSIs, and task-based measures yield medium effects. Teachers appear not to see effects of GSSIs. While youth with ASD self-report large effects, these changes appear entirely attributable to changes in social knowledge; when social knowledge measures are excluded, youth self-report no changes in GSSIs. Overall, larger and more well-controlled studies using a broad array of appropriate measures are needed to advance the study of GSSIs and identify when, how, and for whom they can be most effective.

HIGHLIGHTS.

• Meta-analysis of Group social skills interventions (GSSIs) for youth with ASD.

• GSSIs have a medium overall effect size, but there is potential publication bias.

• Parents and observers reported small effects, teachers reported null effects.

• Self-report effects were attributable to improved social knowledge, not behavior.

Appendix C

Funnel plot for examining publication bias for overall analysis. Here, Y-axis stands for the standard error of the Hedges’ g and X-axis stands for the Hedges’ g. Each dot stands for an individual study.

Appendix D

Funnel plot for examining publication bias for parent-report analysis. Here, Y-axis stands for the standard error of the Hedges’ g and X-axis stands for the Hedges’ g. Each dot stands for an individual study.

Appendix E

Funnel plot for examining publication bias for self-report analysis. Here, Y-axis stands for the standard error of the Hedges’ g and X-axis stands for the Hedges’ g. Each dot stands for an individual study.