Intervention Effects on Language in Children With Autism: A Project AIM Meta-Analysis

Abstract

Purpose

This study synthesized effects of interventions on language outcomes of young children (ages 0–8 years) with autism and evaluated the extent to which summary effects varied by intervention, participant, and outcome characteristics.

Method

A subset of effect sizes gathered for a larger meta-analysis (the Autism Intervention Meta-analysis or Project AIM) examining the effects of interventions for young children with autism, which were specific to language outcomes, was analyzed. Robust variance estimation and metaregression were used to calculate summary and moderated effects while controlling for intercorrelation among outcomes within studies.

Results

A total of 221 outcomes were gathered from 60 studies. The summary effect of intervention on language outcomes was small but significant. Summary effects were larger for expressive and composite language outcomes compared to receptive language outcomes. Interventions implemented by clinicians, or by clinicians and caregivers together, had summary effects that were significantly larger than interventions implemented by caregivers alone. Participants' pretreatment language age equivalent scores positively and significantly moderated intervention effects, such that effects were significantly larger on average when samples of children had higher pretreatment language levels. Effects were not moderated by cumulative intervention intensity, intervention type, autism symptomatology, chronological age, or the proximity or boundedness of outcomes. Study quality concerns were apparent for a majority of included outcomes.

Conclusions

We found evidence that intervention can facilitate improvements in language outcomes for young children with autism. Effects were largest for expressive and composite language outcomes, for children with initially higher language abilities, and for interventions implemented by clinicians or by caregivers and clinicians combined. However, quality concerns of included studies and borderline significance of some results temper our conclusions regarding intervention effectiveness and corresponding moderators.

Autism affects an estimated 2 million people in the United States and tens of millions of people worldwide (Baio, 2012). Though the impact of this condition varies substantially, many autistic ¹ individuals struggle to acquire adaptive, social, academic, and vocational skills (American Psychiatric Association, 2000, 2012; Billstedt et al., 2007). Long-term outcomes of persons with autism are to some extent impacted by the social communication impairments and restricted, repetitive patterns of behavior, interests, or activities that define the diagnosis (American Psychiatric Association, 2013). However, the single most replicated predictor of long-term outcomes for autistic individuals is the acquisition of language during early childhood (e.g., Billstedt et al., 2007; Eisenberg, 1956; Gillberg & Steffenburg, 1987; Kobayashi et al., 1992; Lotter, 1974; Rutter et al., 1967), making this an especially important focus of intervention efforts.

Language Development in Children With Autism

Children with autism are highly heterogeneous in their ability to understand and use language (Kjelgaard & Tager-Flusberg, 2001; Tager-Flusberg et al., 2005). Although some children with autism experience severe language impairments, others display language abilities within the average (or even above average) range relative to their typically developing peers (Bacon et al., 2019; Norrelgen et al., 2015; Rose et al., 2016; Tager-Flusberg et al., 2005). Long-term outcomes of autistic individuals have most commonly been associated with language “production” (i.e., expressive language) acquired during early childhood (Billstedt et al., 2007; Eisenberg, 1956; Gillberg & Steffenburg, 1987; Kobayashi et al., 1992; Lotter, 1974; Rutter et al., 1967). However, children with autism may also display impairments in language “understanding” (i.e., receptive language) that impact their ability to participate fully in a range of life experiences, as well as their social, academic, and vocational success (Charman et al., 2003; Ellis Weismer et al., 2010; Gillum & Camarata, 2004; Hudry et al., 2010; Luyster et al., 2008; Maljaars et al., 2012; Volden et al., 2011; Woynaroski et al., 2016).

Intervention Recommendations

When language is impaired in young children with autism, individualized interventions and family-centered supports are recommended (American Speech-Language-Hearing Association, n.d.). These interventions may include home- or clinic-based treatment delivered directly by certified professionals (e.g., certified speech-language pathologists or other trained professionals) and/or caregivers, integration of low-tech or high-tech communicative supports (e.g., Picture Exchange Communication System, speech-generating devices), and structured visual supports (e.g., activity schedules and video modeling of targeted skills). Prior work has shown that intervention (considered broadly) can facilitate improvements in language outcomes for children with autism (Hampton & Kaiser, 2016).

Putative Moderators of Intervention Effectiveness on Language Outcomes

However, it is possible that the extent to which interventions are able to influence child language outcomes varies by characteristics of the intervention, the child, and the specific language outcome being measured. We discuss each of these putative moderators of intervention effects in detail below.

Intervention Characteristics

Various intervention characteristics may moderate effects of treatment on language outcomes. The type of intervention, total amount of intervention delivered (cumulative intensity), and delivery agent (interventionist) are intervention features that may be particularly influential on whether and to what extent interventions are able to influence language outcomes of young children on the autism spectrum.

Intervention type. Multiple intervention approaches have been promoted and used to support language development in children with autism. These approaches vary in terms of their philosophical underpinnings and application. Traditional behavioral interventions, which are adult-led interventions grounded in operant learning theory (e.g., early intensive behavioral intervention [EIBI], Picture Exchange Communication System), are perhaps the most commonly recommended approach for this population. Prior meta-analyses have suggested that behavioral interventions can have positive impacts on language and communication outcomes (Reichow et al., 2012, 2018), but recent work has suggested that these effects may be inflated by the inclusion of outcomes from low-quality investigations (Sandbank et al., 2019). In contrast, developmental approaches (e.g., DIR/Floortime, Hanen models) are rooted in constructivist theories that posit that children must be active participants in their own learning experiences. Prior work has shown that developmental approaches can facilitate improvements in language (Casenhiser et al., 2013). However, recent meta-analytic work has suggested that the summary effect of developmental interventions on language outcomes is not significantly different from zero (Sandbank et al., 2019). Naturalistic Developmental Behavioral Interventions (NDBIs; Schreibman et al., 2015) employ strategies from both of the aforementioned approaches and are frequently deployed to target improvements in language in young children with autism. These interventions blend behavioral and constructivist techniques to target language milestones in the context of natural interactions between children and caregivers. Intervention targets are chosen using the developmental sequence established in research on typical child development. Prior meta-analytic work has suggested that the summary effect of NDBIs on language outcomes is positive and significantly different from zero, but this estimate may be heavily influenced by outcomes from investigations wherein assessors were aware of group assignment (Sandbank et al., 2019). Sensory-based interventions, which target improvements in sensory processing and related skills by modulating auditory, visual, tactile, and/or other sensory input, have aimed to improve language outcomes in children with autism. Recent meta-analytic and systematic reviews of sensory-based interventions, however, have suggested that sensory-based approaches may have limited effectiveness for improving language and related outcomes (e.g., Barton et al., 2015; Sandbank et al., 2019). TEACCH (Treatment and Education of Autistic and Related Communication Handicapped Children; Mesibov & Shea, 2010) is a classroom-based intervention that heavily incorporates visual cues and structured work systems. Although TEACCH is one of the most widely used interventions for children with autism, relatively few group design studies have documented the effects of TEACCH on language outcomes (Virues-Ortega et al., 2013). Finally, technology-based interventions are those that use technology (e.g., computers, video games, robots) as the primary medium of instruction and support, thereby capitalizing on autistic persons' reported special interest in technology, which provides predictable formatting and user-controlled pace of information delivery (Knight et al., 2013). Although studies have shown that the integration of technology into other intervention approaches can enhance intervention effects on language acquisition (Kasari et al., 2014), recent meta-analytic works suggest there are few studies of interventions “entirely mediated” through technology that document their effects on language outcomes (Sandbank et al., 2019). If intervention type significantly moderates language outcomes, it may provide support for the choice of specific interventions over others for targeting receptive and expressive language in children with autism.

Cumulative intensity. The cumulative amount of intervention provided may also moderate intervention effects on language of children on the autism spectrum. In general, greater amounts of intervention are thought to yield greater effects for children with autism, and recommendations for high numbers of intervention hours per week (e.g., > 25 hr per week; Odom et al., 2010) are rooted in this claim. In some cases, evidence has seemingly supported this assertion. For example, initial studies of EIBI suggested that children receiving the most hours of intervention made the greatest gains (Lovaas, 1987; Reichow & Wolery, 2009). However, in recent meta-analytic work examining intervention effects on spoken language outcomes for children with autism, Hampton and Kaiser (2016) failed to find evidence that treatment intensity moderated effects. At least one study suggested that the effect of treatment intensity on child outcomes may vary as a function of autism symptomatology. In their recent randomized comparison of different intervention intensities, Yoder and colleagues found that increased intensity did improve intervention effects, but only for participants with lower Autism Diagnostic Observation Schedule (ADOS)–calibrated severity scores (P. J. Yoder et al., 2018, 2019). If intensity does significantly moderate effects of intervention on language outcomes, such that more intense intervention yields larger effects, this will add to evidence supporting calls for increased access to high-intensity intervention for at least some children with autism. However, it is possible that a higher number of intervention hours does not yield better outcomes, or that this association is quadratic, where modest increases in intervention dosage (e.g., 5–10 hr per week) yield substantial improvements compared to lower and higher dosages.

Interventionist. Intervention effects may also vary depending on whether the intervention is delivered by a clinician, a caregiver, or a combination of both. Clinicians are trained professionals who may have special knowledge of ways to elicit communication, sustain interaction, and increase targeted opportunities for language learning. Alternatively, caregivers may be more ideal agents of change because they are familiar interaction partners, spend large amounts of time with their children, and have meaningful relationships with them. Thus, interventions that are delivered primarily through caregivers may be delivered throughout the day and across contexts and may therefore facilitate greater and more generalized changes. Moreover, when caregivers and clinicians work together to deliver interventions, they may capitalize on the expert knowledge of the clinician and the familiarity and consistent presence of the caregiver, thereby facilitating larger effects than interventions delivered by clinicians or caregivers alone. Prior meta-analytic work suggests this may be the case for interventions targeting language outcomes. Hampton and Kaiser (2016) found that interventions delivered by both parents and clinicians yielded larger treatment effects on spoken language outcomes than interventions delivered by parents or clinicians alone.

Participant Characteristics

Intervention effects on language outcomes may also vary according to mean-level characteristics of the participants enrolled in the study, such as chronological age, autism symptomatology, and entry-level language ability.

Chronological age. A common recommendation for children with autism is that intervention begins as early as possible, with some advocating for intervention to begin even prior to diagnosis (e.g., Beaudoin et al., 2014; Boyd et al., 2010; Landa, 2018; Reichow, 2012). This recommendation stems from neuroscientific theories of neuronal plasticity, which suggest brain architecture is most malleable at younger ages, potentially allowing for greater developmental improvements associated with intervention (Dawson, 2008). Additionally, early studies of EIBI have suggested treatment gains were greatest for participants that began intervention at an early age (Lovaas, 1987). However, age was not associated with treatment gains in subsequent studies of EIBI (Eldevik et al., 2012; Hayward et al., 2009; Reichow & Wolery, 2009), and a recent meta-analysis examining the effects of intervention (considered broadly) on spoken language outcomes in children with autism found that age did not significantly moderate effects of intervention on outcomes (Hampton & Kaiser, 2016). In contrast, a recent meta-analysis of intervention effects on an outcome developmentally related to language, social communication, suggested that age did moderate intervention effects (Fuller & Kaiser, 2019). If age does significantly moderate treatment effects on outcomes, such that intervention effects are larger for younger children compared to older children, it will bolster the evidence prioritizing access to earlier diagnostic and intervention services.

Autism symptomatology. The heterogeneous nature of autism may also impact the effects of intervention on language outcomes. Prior work has shown that children with fewer or less severe impairments related to the core features of autism demonstrated greater improvements in language while receiving intervention (Ben-Itzchak & Zachor, 2007; Gordon et al., 2011). However, other rigorous tests of intervention efficacy have failed to document significant associations between pretreatment autism symptomatology scores and change in language outcomes (Rogers et al., 2019). It is also possible that the association between autism symptomatology and intervention effects on language is further complicated by intervention intensity. P. J. Yoder et al. (2018, 2019) compared effects of Applied Behavior Analysis therapy and the Early Start Denver Model as delivered at relatively low and high intensities and found that, across intervention types, children with lower autism symptomatology scores benefited more from high-intensity intervention compared to those with higher autism symptomatology scores. Thus, intervention may have differential effects depending on autism severity, and a greater understanding of this association could add to evidence supporting the development of more individualized intervention protocols.

Language ability. Several studies have shown that initial language ability predicts eventual language outcomes in both developmental and intervention studies of children with autism, where children with higher initial scores on measures of language ability exhibit steeper developmental slopes and benefit more from interventions targeting language (Paul et al., 2013; Vivanti, Prior, et al., 2014; P. Yoder et al., 2015). These findings suggest that stronger foundational language skills may allow children with autism to “tune in” and learn more from linguistic interactions that occur in developmental and intervention contexts (Woynaroski et al., 2016). Given that initial language age equivalents in months are commonly reported in participant demographics and that this scale is comparable across studies even when different language measures are used, it is possible to use language age equivalency scores and meta-analytic strategies to determine if there is a significant interaction between pretreatment language ability and intervention effects on language outcomes. As several individual studies have documented a positive association between language ability and intervention gains, it is likely that average participant language age equivalency will moderate intervention effects on language, such that larger effects are observed for participant samples with greater average language age equivalence scores at study entry.

Outcome Characteristics

Effects of intervention on language in young children with autism may further vary according to a number of different characteristics of the outcome measures employed.

Language outcome subtype. Language encompasses both the comprehension and production of a spoken, written, or other communicative symbol system (e.g., American Sign Language). Language outcomes can therefore be divided into those that index receptive language, expressive language, or both receptive and expressive skills (i.e., language composite). Multiple studies have suggested that children with autism may display disproportionate delays or deficits in receptive relative to expressive language (e.g., Barbaro & Dissanayake, 2012; Hudry et al., 2014; McDaniel et al., 2018; Woynaroski et al., 2016).We hypothesized that receptive language may also be less responsive to intervention than expressive language and that effect sizes indexing intervention effects on receptive language would thus be significantly smaller than those for expressive language outcomes in this clinical population. We expected that summary effects for composite language, which index intervention effects on receptive and expressive language together, would likely be smaller than summary effect sizes for expressive language alone, but larger than summary effect sizes for receptive language.

Proximity. Intervention outcomes that are directly taught or prompted by the intervention are considered “proximal,” whereas outcomes that are broader and/or developmentally beyond what was directly taught or prompted are considered “distal.” In terms of language outcomes, children's production of particular linguistic forms that were the focus of the intervention (e.g., saying “hello” or “goodbye”) could be considered proximal, whereas performance on a standardized language assessment could be considered distal. Distal effects are more desirable because they suggest that the intervention has tapped into a developmental process, making continued child growth after the intervention has stopped more likely (e.g., Kasari et al., 2008). However, previous research syntheses have shown that interventions are more likely to show proximal effects (Yoder et al., 2013) and that proximal effects tend to be larger than distal effects in young children with autism (Sandbank et al., 2019).

Boundedness. Some intervention outcomes are measured in contexts that are very similar to the intervention itself, including the materials used, the procedures for eliciting the outcome (e.g., prompting and reinforcement), the interaction partner, and the setting. If this is the case, the outcome can be considered potentially “context bound.” In contrast, outcomes that are measured in contexts that are very different from the intervention on the dimensions listed above are considered more highly “generalized.” In most cases, generalized outcomes are more favorable, as they suggest that the outcome has become an established part of the child's repertoire and is not dependent on particular aspects of the intervention in order to occur. Similar to the aforementioned findings for proximal effects, however, research has shown that interventions are more likely to produce effects on context-bound outcomes (Yoder et al., 2013) and that effect sizes tend to be larger for context bound as compared to generalized outcomes in young children on the autism spectrum (Sandbank et al., 2019).

Current Investigation

The primary purpose of the current investigation was to estimate the summary effect of intervention (considered broadly) on language outcomes as a whole and to evaluate the extent to which intervention effects vary by language outcome subtype (e.g., receptive, expressive, composite) for children with autism during the early childhood period (i.e., between birth and 8 years of age). Although prior meta-analytic work has estimated the summary effects of specific intervention approaches (e.g., behavioral, developmental, NDBI, parent-implemented treatments) on language outcomes (Roberts & Kaiser, 2011; Sandbank et al., 2019), as well as the summary effect of intervention on spoken language outcomes (Hampton & Kaiser, 2016), no prior meta-analysis has estimated the summary effects of intervention across approaches and across broader language outcomes in young children with autism. The secondary purpose of this investigation was to examine the degree to which intervention effects on language outcomes were moderated by various characteristics of the intervention approach, participant sample, and outcomes measured. Our research questions were as follows:

1. Are the summary effects of intervention on language outcomes in children with autism, considered broadly, and by subtype of receptive, expressive, and composite language outcomes, significant and positive?

2. Are the effects of intervention on language outcomes moderated by any one of nine putative moderators (intervention type, cumulative intensity, interventionist, participant age, autism symptomatology, entry-level language ability, language subtype, proximity, and boundedness)?

Given prior evidence and theory, we hypothesized that summary effects would be significantly larger for interventions with greater cumulative intensity, as well as for those interventions that were delivered by a combination of parents and clinicians as compared to either parents or clinicians alone. Second, we hypothesized that summary effects would be larger for participant samples with lower mean age and autism symptomatology scores, as well as with higher entry-level language ability. Finally, we hypothesized that intervention effects would be smaller for receptive relative to expressive and composite language outcomes and that effects for proximal and context-bound outcomes would be significantly larger than effects for distal and generalized outcomes.

Method

The current review article is part of a larger systematic review and quantitative synthesis of all nonpharmacological intervention studies of young children with autism, referred to as the Autism Intervention Meta-analysis or Project AIM (Sandbank et al., 2019). Relevant search and coding procedures are reiterated here briefly.

Search

A wide search strategy was used for the initial project (Sandbank et al., 2019). Nine online databases (Academic Search Complete, CINAHL Plus with Full Text, Education Source, Educational Administration Abstracts, ERIC, MEDLINE, PsycINFO, Psychology and Behavioral Sciences Collection, and SocINDEX with Full Text) were searched with the following search terms: autis*, ASD, PDD, Aspergers, intervention, therapy, teach*, treat*, program, package, assign*, control group, BAU, “wait list,” RCT, random*, quasi, “treatment group,” “intervention group,” “group design,” and trial. Synonymous terms were joined with the boolean operator “OR,” and groups of terms across categories were joined with the boolean operator “AND.” The final electronic search was completed on November 16, 2017. To gather unpublished data, we also searched the National Database for Autism Research, the National Institutes of Health Matchmaker, and the Institute of Education Sciences to identify researchers who had been awarded federal grants to study autism interventions. Of 106 researchers identified through this search, 90 were contacted by e-mail with a request to share eligible outcome data. The contact information for the remaining investigators could not be found. The initial search yielded 12,933 results and captured a broad scope of the intervention literature (i.e., reporting on effects of all intervention types on all outcome types for children with autism across early childhood). Studies were eligible for inclusion in the current meta-analysis if they were experimental or quasi-experimental group design studies examining the effect of an intervention on language outcomes for young children (aged 0–8 years) with autism, regardless of whether or not the intervention was explicitly reported to directly target language. A total of 60 studies met the inclusion criteria for this subgroup analysis of the larger Project AIM data.

Coding Procedures

After the search and screening process, studies were coded for study, participant, intervention, and outcome characteristics. The coding manual used by primary and reliability coders is available upon request from the first author, and data have been deposited in the Open Science Framework: https://osf.io/ha76c/?view_only=55f42fb1e14a4af3a16d5885b301a8b5.

Participant Characteristics

Studies were coded to include pre-intervention chronological age in months, autism symptomatology, and entry-level language ability of study samples. Autism symptomatology scores were taken from eligible measures when reported. These included all versions of the ADOS, the Autism Diagnostic Interview–Revised, and the Childhood Autism Rating Scale. Scores were then categorized as either “moderate” or “high” using published guidelines for score interpretation (Gotham et al., 2009; Hus et al., 2014; Luyster et al., 2009; Schopler et al., 1988) or guidance from individuals with expertise in the quantification of autism symptomatology from the aforementioned standardized measures (K. Gotham, personal communication, November 8, 2018). When autism symptomatology scores were not reported or when reported scores could not be categorized based on published guidelines, autism symptomatology was coded as missing. When reported, initial language ages in months were also extracted from the study. Expressive language age was prioritized over receptive language age (as we suspected, expressive language age would be more commonly reported), but receptive and total language age were extracted if expressive language age was not reported.

Intervention Characteristics

Interventions were coded as belonging to one of nine categories. These included animal-assisted therapy, behavioral, developmental, NDBI, cognitive behavior therapy, sensory based, technology based, and TEACCH. Interventions that were not captured by these categories were coded as “other.” Explicit criteria that were used to categorize intervention approaches are detailed in the main project report (Sandbank et al., 2019). The cumulative intensity of intervention in hours participants received was also coded when estimable from study details. For interventions that incorporated parent training while child participants were present, training time was included in intensity calculations. Parent training hours were excluded from intensity calculations if children were not present during this time. Lastly, the person/s who delivered the intervention to the participants was coded as either caregivers, clinicians, educators, peers, or technology (e.g., when the intervention was completely mediated through a DVD or computer). If a substantial portion (> 40%) of intervention hours was delivered by individuals from two of the above categories, either working together at the same time or separately, then the type of interventionist was coded as “combination.”

Outcome Characteristics

The outcomes analyzed in this review article are those that index the postintervention language ability of the participants in treatment conditions relative to contrast/control groups. Language outcomes were further classified as indexing receptive, expressive, or composite language ability. As outlined in the study of Sandbank et al. (2019), each outcome was also coded for quality indicators, including the risk of selection bias, attrition bias, detection bias, correlated measurement error related to parent/teacher training, and reliance on caregiver report. Selection bias was coded as high for outcomes taken from studies that were quasi-experimental or that featured insufficient randomization procedures (as opposed to randomized controlled trials [RCTs]). Attrition bias was coded as high for outcomes for which data were missing for more than 20% of participants, where intent-to-treat analysis was not employed. Detection bias was coded as high for outcomes collected by assessors who were aware of group assignment. The potential threat of correlated measurement error related to parent/teacher training was coded as high when parents or teachers participated in the study as both interventionists and assessors (see Sandbank et al., 2019, for an expanded rationale regarding this quality indicator).

Outcomes were also coded in regard to their proximity to intervention targets and their boundedness to intervention contexts. Outcomes were coded as proximal if they indexed skills that were directly taught or prompted during intervention. They were coded as distal if they indexed skills (a) that were not directly targeted in the intervention and/or (b) that were measured via developmentally scaled assessments, which are designed to capture developmental progress across a given domain rather than acquisition of specific skills. Outcomes were coded as context bound if they were measured in contexts that were the same or similar to the intervention context, potentially context bound if they were measured via a report from a parent or teacher that delivered the intervention (as these assessments do not indicate the extent to which children displayed language gains with partners who were not part of the intervention), and generalized if they were measured in a context that differed from that of the intervention context on several dimensions (e.g., interaction partner, interaction style, materials, setting).

Effect Size Information

Unadjusted postintervention means, standard deviations, and ns were extracted for treatment and contrast groups. This information was used to calculate the standardized mean difference reflecting the difference between groups after intervention. Standardized mean differences were then converted to Hedge's g to correct for small sample sizes (applicable to sample sizes < 50), which was the effect size metric used for analyses. Effect sizes were transformed as necessary so that higher effects consistently reflected outcomes that favored the intervention group.

Analyses

Summary Effect Estimation

The summary effects of intervention were estimated for all language outcomes and for subgroups of receptive, expressive, and composite outcomes, using intercept-only robust variance estimation (RVE; Hedges et al., 2010) metaregression models. A primary assumption of meta-analysis is that effects are independent, but this assumption is violated when multiple effects are extracted from individual studies. To address this violation, we used RVE to account for the within-study dependence of effects. RVE estimates the covariance structure of metaregression coefficients by assuming a common correlation between all effect sizes (ρ). RVE is a useful approach because it does not require assumptions about the distribution of effect sizes or their weighting scheme (Tipton, 2015). All RVE analyses were conducted with ρ set at 0.8 and then followed by sensitivity analyses to ensure that changing the value of ρ did not substantially change summary effect estimates. We also applied the small sample adjustments proposed by Tipton (2015), which correct degrees of freedom to allow the researcher to evaluate whether the RVE approach was suitable for the given analysis. Results with fewer than four degrees of freedom should not be trusted (Tanner-Smith & Tipton, 2014).

Moderator Analyses

Moderator analyses were conducted using metaregression, which applies the logic of regression to meta-analysis, where effect sizes are treated as the dependent variable in a regression model, and study-level covariates (i.e., moderators) are entered as predictors. Prior to building metaregression models, we first examined putative associations between continuous pretreatment predictors and intervention outcomes using visual inspection of bubble plots to ensure that linear models were sufficient and that the addition of quadratic terms was not needed. For categorical predictors, metaregression analyses were used to determine the intervention effect in each subgroup in reference to nominated reference subgroups. Metaregression analyses were conducted on the following putative moderators: intervention type, intervention intensity, interventionist, chronological age, autism symptomatology, language age, and language outcome subtype. An exploratory post hoc metaregression analysis was also carried out to test a potential interaction between autism spectrum disorder symptomatology and intervention intensity that was suggested by the results of one recent RCT testing the efficacy of treatments delivered at relatively high versus low intensity in young children with autism (P. J. Yoder et al., 2018). Summary effect estimates and metaregression analyses were conducted with the R package Robumeta (Fisher et al., 2017), and plots were created with the R package Metafor (Viechtbauer, 2010).

Publication Bias

Finally, we conducted tests of publication bias, because a publication process, which gives preferential treatment to studies that document significant intervention effects over studies with null results, can yield a set of published studies that are largely unrepresentative of the complete body of evidence and lead to an overestimation of summary effects. Publication bias potentially threatens all meta-analytic results; therefore, it is essential that meta-analyses in speech, language, and hearing research include multiple tests designed to evaluate the potential influence of publication bias (Chow, 2018). Common methods for this purpose involve testing the association between effect sizes and their precision, because effect estimates from small studies should vary most (due to random error), whereas effect estimates from large studies should vary least. To this end, we visually inspected funnel plots of effect sizes plotted against their standard errors and tested the asymmetry of these plots using an Egger's regression test (Egger et al., 1997). The results of these tests provide quantitative information that allow us to evaluate the likelihood that summary effects were influenced by publication bias.

Results

Reliability

All reliability calculations were completed using the irr package (Gamer et al., 2012) in R studio (R Core Team, 2017). Coding reliability was calculated for 100% of studies included in the larger meta-analysis, using two-way random, single measures, absolute intraclass correlations for continuous variables (ICC[A, 1]; McGraw & Wong, 1996) and unweighted kappa coefficients for categorical variables (J. Cohen, 1960). Reliability was high. Average kappa values across categorical variables included in the current review article was .82 (range: .73–.91), and the average intraclass correlation across continuous variables was .88 (range: .81–.96).

Descriptives of Included Study Samples and Outcomes

A total of 221 outcomes gathered from 60 studies featuring a total of 2,908 participants were included in the current analyses. Of included outcomes, 130 were coded as expressive, 79 were coded as receptive, and 12 were coded as tapping both expressive and receptive language (composite language). The mean number of language outcomes reported per study was 3.68 (min = 1, max = 14). On average, participants were 47.5 months old (SD = 16.45). The average percentage of male participants across samples was 82.7%. Average participant language age was 20.19 months (SD = 7.52) for studies in which it was reported. See Table 1 for included studies and their corresponding participant and intervention characteristics.

Table 1.

Characteristics of participant samples and interventions in included studies.

Study authors, year	Mean CA in months	Mean LA in months	Intervention type	Interventionist type	Total intervention hours
Boyd et al., 2014	48.3	—	TEACCH	Educator	—
Boyd et al., 2014	48.2	—	Other (LEAP)	Educator	—
Brian et al., 2017	25.2	13.1	NDBI	Caregiver	—
Carter et al., 2011	21.3	7.8	Developmental	Caregiver	—
Casenhiser et al., 2013	44.5	24.6	Developmental	Caregiver	1,393.2
Chang et al., 2016	50	31.5	NDBI	Educator	—
Clionsky, 2012	61.4	—	Other	Caregiver	13.4
Cohen et al., 2006	—	—	Behavioral	Clinician	5,287.5
Corbett et al., 2008	66.7	—	Sensory	Other	90
D'Elia et al., 2014	49.2	18.6	TEACCH	Combination	1,144
Dawson et al., 2010	23.5	—	NDBI	Combination	3,276
Drew et al., 2002	22.5	—	NDBI	Caregiver	—
Eikeseth et al., 2002	65.7	—	Behavioral	Educator	1,456
Fava et al., 2011	48.2	—	Behavioral	Combination	591.68
Fletcher-Watson et al., 2016	49.6	—	Technology	Technology	8.55
Green et al., 2010	45	—	Developmental	Caregiver	214
Hardan et al., 2015	49.2	—	NDBI	Caregiver	—
Howlin & Rutter, 1989	75.4	31.4	NDBI	Caregiver	—
Hudson et al., 2017	56.1	—	Other	Educator	11.03
Hudson et al., 2017	55.7	—	Other	Educator	9.6
Ingersoll et al., 2016	43.7	—	NDBI	Caregiver	—
Jalili et al., 2014	90	—	Other	Clinician	25.8
Jocelyn et al., 1998	43.3	—	NDBI	Combination	—
Kaale et al., 2014	48.8	21.5	NDBI	Educator	25
Kasari et al., 2008	42.6	20.1	NDBI	Clinician	14.3
Kasari et al., 2008	42.3	20.5	NDBI	Clinician	17.35
Kasari et al., 2015	31.5	13.9	NDBI	Caregiver	—
Kovshoff et al., 2011	37	—	Behavioral	Combination	2,662.4
Landa et al., 2011	28.8	—	NDBI	Clinician	205.66
Lerna et al., 2012	39.9	—	Behavioral	Clinician	36
Lim, 2010	56	31	Sensory	Technology	0.9
Lim, 2010	56	31	Technology	Technology	0.6
Magiati et al., 2007	39.6	—	Behavioral	Other	3,415
Nefdt et al., 2010	38.7	—	NDBI	Caregiver	—
Paul et al., 2013	46.4	9.7	Behavioral	Clinician	27
Peters-Scheffer et al., 2013	62.5	19.4	Behavioral	Clinician	478.08
Rahman et al., 2016	65.2	—	Developmental	Caregiver	132.4
Roberts et al., 2011	42.6	—	NDBI	Combination	36
Roberts et al., 2011	43.4	—	NDBI	Clinician	72
Rogers et al., 2006	38.4	12	NDBI	Caregiver	75
Rogers et al., 2012	21	—	NDBI	Caregiver	—
Sainato et al., 2015	75.2	—	Other	Educator	1,083.6
Sallows & Graupner, 2005	33.7	—	Behavioral	Clinician	6,429.8
Salt et al., 2002	41	—	Developmental	Caregiver	473.946
Sampanthavivat et al., 2012	70.6	—	Other	Other	20
Schertz et al., 2013	26.1	—	Developmental	Caregiver	105.35
Schreibman & Stahmer, 2014	29.2	—	NDBI	Combination	247
Silva et al., 2015	—	—	Sensory	Caregiver	37.625
Simpson et al., 2015	70.8	—	Other	Technology	1.125
Simpson et al., 2015	70.6	—	Other	Technology	1.125
Smith et al., 1997	37	—	Behavioral	Clinician	3,120
Smith et al., 2000	35.9	15.7	Behavioral	Clinician	2,137.88
Sokmum et al., 2018	55	—	Developmental	Caregiver	—
Solomon et al., 2014	50.2	—	Developmental	Caregiver	627.16
Spjut Jansson et al., 2016	36.1	—	Behavioral	Combination	2,600
Spjut Jansson et al., 2016	36.6	—	Behavioral	Caregiver	1,040
Srinivasan et al., 2015	91.4	—	Sensory	Combination	17.56
Srinivasan et al., 2015	89.3	—	Technology	Technology	18.44
Strain & Bovey, 2011	50.3	29.5	Other	Educator	1,768
Strauss et al., 2012	49.4	—	Behavioral	Combination	354.75
Thompson et al., 2014	45.4	—	Sensory	Combination	8.17
Tonge et al., 2014	46.8	—	Other	Caregiver	—
Tonge et al., 2014	48.1	—	Behavioral	Caregiver	—
Vivanti, Paynter, et al., 2014	41.2	19	NDBI	Combination	1,023.4
Welterlin et al., 2012	30.5	—	TEACCH	Caregiver	—
Wetherby & Woods, 2006	18.19	—	NDBI	Caregiver	—
Whitehouse et al., 2017	39.8	26.1	Technology	Technology	26.93
Woo & Leon, 2013	79.2	—	Sensory	Caregiver	—
Woo et al., 2015	56	—	Sensory	Caregiver	—

Note. Studies are listed twice when more than one intervention group was evaluated in reference to a comparison. Em dashes indicate information was not reported. CA = chronological age; LA = language age equivalent; TEACCH = Treatment and Education of Autistic and related Communication Handicapped Children; LEAP = Learning Experiences—An Alternative Program for Preschoolers and Parents; NDBI = Naturalistic Developmental Behavioral Intervention.

Outcome Quality

Figure 1 reflects outcome-level quality indicators. Out of 221 outcomes, 135 (61%) indexed intervention effects in RCTs, 25 (11%) were coded as having high risk of attrition bias due to a large percentage of missing outcome data, and 126 (51%) were coded as having high detection bias due to reliance on assessors that were aware of group assignment. The threat of correlated measurement error related to parent/teacher training was marked as high for 71 outcomes (32%), where parents or teachers participated both as interventionists and assessors. A total of 59 outcomes (26%) were derived from caregiver report. We coded 21 outcomes (9.5%) as proximal to intervention targets. Finally, 31 outcomes (14%) were coded as bound to the context of intervention, 47 outcomes (21%) were coded as potentially context bound, and 143 (65%) were coded as generalized.

Figure 1.

Quality indicators for studies of interventions with language outcomes. RCT = randomized controlled trial.

Summary Effects of Intervention on Language Outcomes

Forest plots of included effect sizes and summary effects for receptive, expressive, and composite language outcomes are presented in Figures 2, 3, and 4. Across all language outcomes and intervention types, the RVE summary estimate with small sample corrections was 0.184 (95% CI [0.075, 0.292], p = .001). Metrics for quantifying the dispersion of effect sizes suggest there was substantial heterogeneity (I ² = 58.6%, τ² = 0.123, τ = 0.351). For expressive language outcomes, the RVE summary estimate was 0.18 (95% CI [0.077, 0.283], p = .001). For receptive language outcomes, the RVE summary estimate was 0.135 (95% CI [0.000, 0.269], p = .05). For composite language outcomes, the RVE summary estimate was 0.284 (95% CI [−0.0465, 0.614], p = .084). Following benchmarks for main effects proposed by Cohen (1988), where effect sizes of 0.2, 0.5, and 0.8 are considered small, moderate, and large, respectively, these effects are relatively small. However, it should be noted that summary effects for education interventions tend to fall within a range of 0.20–0.30 (Hill et al., 2008), so the magnitude of these effect sizes might be more appropriately interpreted as moderate.

Figure 2.

Forest plot of Hedge's g effect sizes reflecting intervention effects on receptive language outcomes. RVE = robust variance estimation.

Figure 3.

Forest plot of Hedge's g effect sizes reflecting intervention effects on expressive language outcomes. RVE = robust variance estimation.

Figure 4.

Forest plot of Hedge's g effect sizes reflecting intervention effects on composite language outcomes.

Moderator Analyses

Table 2 provides descriptive information for putative moderators. Detailed results of tested metaregression models are presented in Table 3 with both uncorrected and corrected p values (Benjamini & Hochberg, 1995). Benchmarks for small, moderate, and large moderator effects are 0.10, 0.20, and 0.40, respectively (Tipton, 2015), but these benchmarks are meant to complement Cohen's benchmarks for main effects. If we consider a main effect of 0.25 to be moderate, then the corresponding moderate metaregression coefficient estimate would be 0.13 (e.g., half of the main effect benchmark). Significant intercept estimates should not be interpreted as evidence that effects were significantly moderated by tested predictors. Rather, this suggests that the summary effect of intervention is significantly different from zero when the value of the predictor is 0 (for continuous moderators), or that the summary effect of the reference category is significantly different from 0 (for categorical moderators).

Table 2.

Features of continuous and categorical moderators.

Moderators	No. of studies	M (SD)	Min	Max
Intervention dosage	42	854.29 (1424.59)	0.6	6429.8
Chronological age	58	48.01 (16.74)	18.19	91.4
Language age	17	20.86 (7.70)	7.8	31.5
Intervention type
Behavioral	15
Computer based	4
Developmental	8
NDBI	21
TEACCH	3
Other	11
Sensory based	7
Interventionist
Educator	9
Caregiver	26
Clinician	12
Combination	12
Computer instruction	7
Other	3
Autism severity
Moderate	19
High	9
Language outcomes
Expressive	130
Receptive	79
Total	12

Note. NDBI = Naturalistic Developmental Behavioral Intervention; TEACCH = Treatment and Education of Autistic and Related Communication Handicapped Children.

Table 3.

Results of metaregression analyses.

Predictor	Studies	Outcomes	Beta	SE	t	df	p	p adj
Model 1: Intervention type	60	221
Intercept (reference: behavioral)			0.27	0.11	2.44	12.22	.028**	.116
Developmental			−0.22	0.13	−1.68	15.39	.113	.273
NDBI			−0.06	0.14	−0.46	24.83	.648	.723
Sensory based			−0.09	0.19	−0.49	9.87	.636	.723
TEACCH			−0.34	0.35	−0.98	2.85	.404	.586
Technology based			−0.08	0.21	−0.37	3.69	.732	.758
Other			−0.09	0.24	−0.38	16.39	.712	.758
Model 2: Intervention intensity	60	221
Intercept			0.20	0.06	3.28	35.10	.002***	.019**
Cumulative intensity in hours			0.09	0.08	1.16	5.10	.295	.475
Model 3: Interventionist	60	221
Intercept (reference: caregiver)			0.06	0.06	1.08	22.16	.290	.474
Clinician			0.33	0.15	2.16	16.44	.046**	.145
Combination			0.26	0.13	2.02	18.16	.058*	.153
Technology based			0.09	0.28	0.30	4.83	.774	.774
Educator			0.16	0.23	0.76	10.07	.465	.642
Other			−0.11	0.20	−0.54	2.15	.642	.723
Model 4: Chronological age	58	217
Intercept			0.19	0.06	3.33	52.40	.002***	.019**
Age in months			0.03	0.06	0.47	18.30	.641	.723
Model 5: Autism symptomatology	27	109
Intercept (reference: moderate)			0.12	0.08	1.52	16.4	.148	.330
High Symptomatology			0.09	0.20	0.47	16.1	.639	.723
Model 6: Language age	17	76
Intercept			0.23	0.07	3.07	12.89	.009***	.048*
Language age equivalency			0.25	0.07	3.45	7.18	.010**	.048*
Model 7: Language outcome type	60	221
Intercept (reference: expressive)			0.21	0.06	3.61	47.75	.001**	.019**
Receptive			−0.13	0.07	2.04	49.58	.047**	.145
Total			0.16	0.14	1.16	8.27	.277	.475
Model 8: Outcome boundedness	60	221
Intercept (reference: context bound)			0.41	1.80	2.26	9.07	.050**	.145
Potentially context bound			−0.28	0.19	−1.46	18.41	.161	.334
Generalized			−0.25	0.19	−1.30	13.11	.216	.421
Model 9: Outcome proximity	60	221
Intercept (reference: distal)			0.160	0.05	3.03	49.92	.004***	.029**
Proximal			0.29	0.29	1.01	7.68	.344	.525

Note. p _adj = p values after utilizing a Benjamini–Hochberg correction (Benjamini & Hochberg, 1995); NDBI = Naturalistic Developmental Behavioral Intervention; TEACCH = Treatment and Education of Autistic and Related Communication Handicapped Children.

^* p < .10.

^** p < .05.

^*** p < .01.

Intervention Characteristics

Results from the metaregression models that included intervention variables indicated that effect sizes were not moderated by intervention type (see Table 3) or by cumulative intervention intensity in hours. However, interventionist was a significant moderator of effect sizes, when alpha was nominally set at .05. Results from the metaregression model that included the categorical variable of interventionist, with caregiver as the reference category, indicated that intervention effects were significantly larger for interventions implemented by clinicians compared to caregivers alone (B = 0.33, p = .044) and marginally larger for those implemented by a combination of interventionists (e.g., caregivers and clinicians working together, B = 0.26, p = .058) compared to caregivers alone. However, when p values were corrected to account for multiple comparisons, they did not pass the significance threshold. Effect sizes for interventions implemented by other interventionist types (e.g., educator, computer-mediated instruction) did not significantly differ from caregiver-implemented intervention effects. Figure 5 displays summary effects and confidence intervals by interventionist type.

Figure 5.

Robust variance estimation summary effect estimates and corresponding confidence intervals for language outcomes according to interventionist type. *Effect sizes for which degrees of freedom were too few to permit confidence in the accuracy of the estimate.

Participant Characteristics

Results from metaregression models that included participant characteristics suggested that chronological age at intervention onset did not moderate intervention effect sizes (B = 0.03, p = .641). Autism symptomatology categorizations also did not moderate intervention effects: Significantly different effects were not observed for participant samples rated as “high symptomatology” compared to samples for which autism symptomatology was categorized as “moderate” (B = 0.09, p = .639). Language age in months did significantly moderate intervention effects, such that higher mean language ages were associated with larger intervention effects (B = 0.25, p = .010). This result was significant even after correcting for multiple comparisons. Figure 6 displays a bubble plot of effect sizes plotted against participant mean language age equivalents.

Figure 6.

Bubble plot of Hedge's g effect sizes against mean participant language age equivalency scores in months. The dotted line reflects a summary effect of zero. The black line reflects the linear model predicting intervention effects from mean language age, controlling for intercorrelation among outcomes within study using robust variance estimation. Bubble size is proportional to the outcome model weights.

Outcome Characteristics

Results from the metaregression model that included outcome type as a categorical variable, with expressive outcomes as the reference category, indicated that intervention effects were significantly smaller for receptive outcomes compared to expressive outcomes (B = −0.13, p = .047), though this effect was not significant after correcting for multiple comparisons. Effect sizes for composite language outcomes did not differ significantly from expressive outcomes (B = 0.16, p = .277). Figures 2, 3, and 4 display RVE summary estimates for receptive, expressive, and composite language outcomes, respectively. Effect sizes did not significantly vary as a function of outcome boundedness or distality. Compared to outcomes coded as context bound, effect sizes were smaller but not significantly different for outcomes coded as potentially context bound (B = −0.28, p = .161), as well as those coded as generalized (B = −0.25, p = .216). Effect sizes for outcomes coded as proximal were, on average, larger than those coded as distal, but this difference was not statistically significant (B = 0.29, p = .344).

Exploratory Post Hoc Analysis

Results from the exploratory metaregression model that tested a putative interaction between intervention intensity and autism symptomatology categorization indicated the interaction term was not significant (B = −0.176, p = .5).

Publication Bias Analyses

Results of the Egger's regression test of funnel plot asymmetry for the full model (i.e., across all language outcomes and intervention types) was significant, z = 2.52, p = .012, indicating evidence of publication or small sample bias. Follow-up analyses indicated that this effect was driven by the expressive language outcomes model, z = 4.54, p < .001. Neither the receptive language outcomes model, z = 1.38, p = .17, nor the composite language outcomes model, z = 0.66, p = .51, showed evidence of publication or small sample bias. Funnel plots of effect sizes plotted against their standard errors, across all outcomes, according to outcome type are presented in Figure 7.

Figure 7.

Funnel plots of Hedge's g effect sizes against standard errors.

Discussion

The purpose of this meta-analysis was to determine whether existing interventions significantly improve language of young children with autism, considered broadly and more specifically according to subtype of receptive, expressive, and composite language outcome. As well, we sought to determine whether intervention effects were moderated by various aspects of the participant samples, interventions, and outcomes. These results suggest that interventions on average yield small-to-moderate, significant improvements in language outcomes for children with autism in early childhood. Summary effect estimates of intervention are significantly smaller for receptive language outcomes relative to summary effects for expressive language outcomes for children with autism during this developmental period. In addition to type of language outcome, other significant moderators of intervention effects included the interventionist/s and the entry-level language ability of participants. However, only entry-level language ability was a significant moderator after correcting for multiple comparisons. We failed to find evidence that intervention type, cumulative intervention intensity, chronological age of participants, participant autism symptomatology, boundedness of the outcome, and distality of the outcome significantly moderated intervention effects, though it is possible that study-level factors, measurement imprecision, and a limited number of studies prevented us from uncovering a potentially true association between any of these putative moderators and intervention effects. Analysis of study quality indicates that there are several areas in which study designs will need to be improved in future research in order to draw strong conclusions about the effects of interventions on language outcomes of young children with autism. Below, we discuss findings for each moderator explored in this study and corresponding recommendations for primary intervention research.

Intervention Characteristics and Intervention Effects

Intervention Type

Though our prior work suggested that intervention effects may vary substantially by intervention type for some outcomes of interest in young children on the autism spectrum (Sandbank et al., 2019), we did not find evidence in the current review article to suggest that summary effects across language outcomes significantly vary according to intervention type. In other words, though summary effect estimates differ to some degree by intervention type, the confidence intervals surrounding each of these estimates are wide (suggesting high variability of effects within intervention type) and overlap with one another. This should not be construed as evidence that all intervention approaches targeting language outcomes in this population are equivalent. The philosophical underpinnings and techniques employed in these interventions differ extensively, and these differences likely matter. However, at least one recent randomized comparison of this nature documented nonsignificantly different improvements in groups assigned to receive distinct intervention approaches, adding to the evidence that intervention approaches motivated by opposing philosophies may be associated with similar effect sizes (P. J. Yoder et al., 2018, 2019).

It is also possible that effects of intervention type vary according to participant (sample) characteristics. Such interactions have been observed in prior clinical trials of interventions targeting language and communication outcomes in young children with autism (e.g., P. Yoder & Stone, 2006). We were unable to test this question in the present synthesis, as the limited number of studies and missing data prohibited the addition of Intervention Type × Participant Characteristic product terms as predictors in models summarizing effects of interventions on language outcomes. Future RCTs that permit the testing of differential effects of high-quality instantiations of different intervention approaches according to child characteristics are thus necessary to elucidate whether certain interventions are more optimal choices compared to others for facilitating improvements in language for specific subgroups of children on the autism spectrum and thereby advance us toward more personalized treatment planning for this clinical population.

Intensity

We did not find evidence that greater intervention intensities were significantly associated with greater gains in language. This finding is consistent with the findings of prior meta-analyses of intervention effects on language (Hampton & Kaiser, 2016; Virues-Ortega, 2010) and related outcomes (Fuller & Kaiser, 2019) in children with autism, though it departs from frequent assertions that more intervention will yield more improvement. It is possible that a true association between cumulative intensity and intervention effects is complicated by aspects of treatment intensity other than the cumulative number of hours of intervention, such as within-session dosage (i.e., number of teaching episodes effectively delivered per intervention session), dose frequency (i.e., number of intervention sessions per week), and/or the total duration (i.e., total number of weeks over which intervention is provided), as well as participant characteristics, such as autism symptomatology. Though some of the aforementioned, more fine-grained aspects of intervention intensity could not be analyzed because they were not extracted in the original coding system for the larger meta-analysis (and notably often were not reported in extant studies), we were able to conduct exploratory post hoc analyses to examine whether autism symptomatology interacted with cumulative intensity to predict intervention effects. The results of this analysis are discussed below.

If future meta-analyses are to clarify the impact of intervention intensity on outcomes, it will be important for investigators to report a range of metrics that comprehensively describe intervention dose, dose frequency, and duration (Warren et al., 2007), as well as participant characteristics that may predict a differential response to more versus less intense intervention, such as autism symptomatology. Future randomized comparisons of low- and high-intensity versions of single interventions can also increase our understanding of the overall impact of intervention intensity and provide a framework for moderator analyses that elucidate the extent to which this impact differs by participant characteristics. Finally, sequential multi-assignment randomized trial design studies may be particularly useful for determining the extent to which increases in intervention intensity alter the trajectory of change for children who may initially show no or minimal gains during intervention. Until then, it will be important for professionals recommending high-intensity intervention to weigh the potential positive impact of increased intervention with the potential negative impact of stress that high-intensity interventions can place on young children and their families.

Interventionist/s

We did find evidence that the implementer of the intervention (clinician, caregiver, educator, combination, etc.) is a key factor that impacts the magnitude of intervention effects on language outcomes. Compared to interventions implemented by caregivers alone, for which the summary effect was small and not significantly different from zero, interventions delivered primarily by clinicians and interventions delivered by a combination of interventionists (i.e., caregivers and clinicians working together) yielded significantly larger effects, though these results for moderated effects notably no longer exceeded the threshold for statistical significance after correcting for multiple comparisons. The largest summary effect estimate was observed for interventions delivered by clinicians only. This result contrasts with the findings of Hampton and Kaiser (2016), who found that the summary effect of clinician-led interventions on spoken language outcomes was very small and not significantly different from zero. The inclusion of approximately 200 more effect sizes in our analysis of this question likely explains the apparent discrepancy in findings across syntheses.

The summary effects for interventions delivered primarily by educators and for interventions mediated entirely through technology were also small and not significantly different from the summary effect for treatments delivered by caregivers. These results suggest that, though caregivers and educators are primary figures in young children's lives, their ability to directly effect change in language outcomes may be limited. Clinicians who have specialized training in language development and treatment (e.g., speech language pathologists) are likely the most equipped to facilitate language development, either by working directly with children or working with caregivers and educators to provide intervention across the natural environments that children encounter each day. However, we note that there may be language outcomes that were not measured in this analysis (e.g., those that tap more social and/or pragmatic features of language) that may be more influenced by caregivers than by clinicians.

Participant Characteristics and Intervention Effects

Chronological Age

Chronological age did not emerge as a significant moderator of intervention effects on language outcomes in this meta-analysis. This finding is consistent with prior meta-analytic work in this area (Hampton & Kaiser, 2016) but may also be due to the restricted age range defined for eligible studies (i.e., aged 0–8 years). Our results should not be construed as evidence that early intervention is not important for children with autism. Multiple studies included in this meta-analysis demonstrated that intervention can have positive effects on language outcomes for very young children with autism, and early gains in language may produce cascading effects on development in related domains as children age. Thus, early access to diagnosis and intervention continue to be necessary for children with autism. Rather, we see our results as adding to mounting evidence that advancing age does not place a limit on the amount of change that can be achieved with intervention. Rather, children can benefit from language interventions throughout early childhood (i.e., birth to 8 years).

Autism Symptomatology

We additionally did not find evidence that autism symptomatology moderated intervention effects on language outcomes for children on the autism spectrum. A number of challenges may have limited our ability to detect a true association between autism symptomatology and intervention effects, and these are detailed in the Limitations section. The relatively recent creation of the ADOS calibrated severity score algorithms (Gotham et al., 2009), which provide a continuous measure of symptomatology that is comparable across age groups and ADOS modules, could facilitate direct comparison of separate participant samples going forward. Thus, increased reporting of calibrated severity scores to quantify autism symptomatology could allow future meta-analysts to better estimate the true impact of autism symptomatology on intervention effects.

Language Ability

As we predicted, our results suggested that intervention effects on language outcomes were dependent on children's entry-level language ability, such that children with higher initial language age equivalency scores at study entry benefited more from language interventions than children with lower initial language age equivalency scores. This finding is consistent with prior evidence and theory that initial developmental achievements in language provide a foundation for further development, creating a “rich get richer” phenomenon, where children with substantial developmental delays benefit less from intervention than their more advanced peers. Further work must be done to develop and test interventions that facilitate improvement for this subgroup of children on the spectrum, especially since recent work suggests that merely intensifying available interventions (i.e., increasing from 5 to 15 hr per week) do not yield added benefit for this group on average (P. J. Yoder et al., 2018, 2019). There is at least some evidence that suggests that language development in this subgroup of children benefits from more explicit, adult-led interventions than naturalistic, child-led approaches (Paul et al., 2013, P. Yoder & Stone, 2006).

Outcome Characteristics and Intervention Effects

Language Outcome Type

As we anticipated, intervention effects varied depending on the aspect of language that was indexed, such that receptive outcomes were significantly smaller than both expressive and composite language outcomes, when alpha was nominally set at .05. Though expressive language acquisition is often a key focus of intervention for children with autism, these findings add to a large literature suggesting children with autism may have considerable difficulty with receptive language acquisition and may require interventions and Individualized Education Program goals that specifically target receptive skills (e.g., Barbaro & Dissanayake, 2012; Charman et al., 2003; Ellis Weismer et al., 2010; Gillum & Camarata, 2004; Luyster et al., 2008; Maljaars et al., 2012; Pickles et al., 2014; Volden et al., 2011; Woynaroski et al., 2016). Persistent deficits in receptive language may contribute to later academic difficulties such as poor reading comprehension, as well as social difficulties stemming from an inability to follow complex conversational exchanges, negatively impacting long-term outcomes of persons with autism. Thus, it is important that clinicians' focus on acquisition of spoken language for this population does not come at the exclusion of targeting receptive language skills.

It is notable that the expressive language outcomes showed significant evidence for publication and/or small study bias, whereas the receptive language outcomes did not. Given that our search strategy included an extensive gray literature search (see Sandbank et al., 2019), it is likely that small study bias or some other factor influenced the expressive language model (see Figure 7, which shows effect sizes to the right of the funnel have large standard errors, indicating small sample sizes). The presence of bias may have led to inflated summary effect sizes in analyses focused on effects of intervention on expressive outcomes. Therefore, more primary research (or possibly publication of previously completed but presently unpublished primary research) exploring treatment effects on both receptive and expressive language outcomes, particularly rigorous research conducted with large samples, is needed to fully understand the impact of intervention on these domains in young children with autism.

Boundedness and Distality

Though we found evidence in prior work that boundedness and distality moderated effect sizes of intervention across all outcomes of interest to young children with autism, without regard to outcome type (Sandbank et al., 2019), in the present synthesis, we did not find evidence that these outcome characteristics moderated intervention effects on language outcomes specifically. On average, language outcomes that were coded as “potentially context-bound” or generalized were smaller than those coded as context bound, but the confidence intervals for the aforementioned outcome types overlapped. Similarly, language outcomes that were coded as proximal were larger, on average, than those coded as distal, but not significantly different. Our failure to replicate prior findings regarding the effect of boundedness and distality on intervention outcomes may have been a product of reduced power, as fewer outcomes and studies were included in the current analysis compared to that of the larger project (see Sandbank et al., 2019). Even so, future investigators should consider these outcome characteristics when planning and interpreting results from intervention studies to ensure that readers do not overestimate an intervention's potential to effect change in outcomes and contexts beyond those that were directly targeted in treatment.

Outcome Quality

Our claims in regard to the efficacy of interventions on language outcomes should be tempered in light of the quality concerns apparent in the included studies. Similar to the larger meta-analysis from which this subset of study outcomes was drawn, a lack of naive coders and assessors is evident for a majority of outcomes. This quality concern indicates that the summary-level effects we report may be overestimated in comparison to “true” intervention effects, though this issue is unlikely to have affected findings from moderator analyses. Researchers should continue to work toward designing intervention studies that are randomized and increase efforts to ensure that assessors and coders are not aware of intervention assignment.

Strengths

The multiple strengths of this review article include the use of comprehensive search techniques and advanced meta-analytic tools. First, the wide initial search strategy employed in the larger project likely permitted the identification of more studies than would have been identified in a more targeted search. However, one caveat of this is that, because our search terminology did not specifically include the term “language,” we may have captured studies of interventions that were not specifically language oriented but that simply included measures of language growth and/or outcome as a secondary analysis. However, even if this was the case, presumably, investigators tracked language as a secondary outcome because they had reason to believe that this domain might be distally impacted by the intervention of interest. Thus, we see it as a strength that these effects were also included in this meta-analysis. A second strength of this review article was the use of RVE (Hedges et al., 2010), which allowed us to include multiple language outcomes from each study and then to statistically account for their intercorrelation, increasing the precision of our summary estimates.

Limitations

The primary limitations of this review article are related to power. First, we were unable to test more complex moderation models due to missing data. While the available data were sufficient to justify metaregression models with single moderator terms, we were unable to test models with multiple moderators, because listwise deletion of missing data for any single predictor would have greatly reduced the number of cases in a given model and the corresponding power of that model. Second, most significant moderator results were associated with borderline p values and were no longer significant after correcting for multiple comparisons. Multiple uncorrected comparisons are a common issue in meta-analytic literature and can give rise to Type I errors (i.e., the true effect is zero, but researchers incorrectly reject the null) and skew conclusions about the summary effects of intervention and putative moderators (Polanin & Pigott, 2015). While researchers agree that multiple comparisons can pose a threat, there is less agreement on how to address this threat in meta-analysis (Borenstein et al., 2011). In many cases, researchers choose a lower significance criterion (α) for treating results as significant. Although choosing a more stringent alpha level will reduce the risk of a Type I error, it will also reduce the power to detect an effect and consequently increase the risk of a Type II error (i.e., the true effect is not zero, but researchers fail to reject the null). In balancing the risk for Type I and II errors, researchers must consider the potential consequences of each relative to their own results. For example, if significant results are likely to be interpreted as definitive and trigger an immediate change in practice, then a stringent alpha level should be chosen to avoid a Type I error. Alternatively, if significant results are likely to be interpreted as highlighting potentially important findings and inviting replication, then a less conservative alpha level may be chosen to avoid a Type II error. In regard to the current findings, given that power for tests of moderators in meta-analysis is often already very low (Hedges & Pigott, 2004) and given that our results are unlikely to be interpreted as definitive and produce an immediate change in clinical practice, we believe it would be imprudent to rely on a more stringent alpha criterion when interpreting our results (though we have included corrected p values in Table 3 and explicitly highlighted all instances wherein significant results did not survive corrections for multiple comparisons to allow readers to draw their own conclusions). Rather, we suggest that our results highlight potentially important intervention, participant, and outcome characteristics that may influence intervention effects for children with autism, while also calling for replication.

Finally, an additional limitation of this review article was the forced dichotomization of autism symptomatology. Forced dichotomization of a continuous variable will always result in the loss of information, and in this case, that loss may have obscured a true association. These limitations further prevented us from examining with precision the potential complex interactions between intervention, participant, and outcome characteristics, which might have increased our understanding of “what works” and “for whom” and “for what” (Sneider, 2018).

Future Meta-Analytic Research

Though the results of this meta-analytic investigation are informative, they are limited in scope to language outcomes and should not be assumed to generalize to intervention effects on all outcomes of young children with autism. Moderating associations of intervention, participant, and outcome characteristics on intervention effects likely differ across outcome domains. Thus, future meta-analytic investigations should examine the extent to which these variables moderate intervention effects on other outcomes of interest, particularly those that are core challenges for autistic children, such as social communication and sensory function. It will also be important to examine moderating associations within studies of newer interventions that have recently gathered a large evidence base, such as NDBIs, across outcome types (Schreibman et al., 2015).

Conclusions

Existing interventions can likely affect small but significant improvements in language outcomes on average for children with autism between birth and 8 years of age. Effects are largest for expressive language outcomes and when interventions are delivered by clinicians or clinicians working in combination with caregivers or educators. Children with higher language ability at entry to treatment may stand to make greater gains as a result of intervention than those with more substantial language impairments or lower entry-level language ability. However, some concerns regarding study quality, in particular the high prevalence of assessors and coders who are not naive to group assignment, prevent us from making strong claims in regard to intervention effectiveness. Thus, more rigorous clinical trials are necessary to increase our confidence in the degree to which interventions can facilitate language development for this clinical population in early childhood.

Funding Statement

This work was funded in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (U54HD083211, PI: Neul) and the National Institute on Deafness and Other Communication Disorders (1R21DC016144; Woynaroski).