Early intervention for very young children with or at high likelihood for autism spectrum disorder: An overview of reviews

Abstract

Aim:

To identify which interventions are supported by evidence and the quality of that evidence in very young children with or at high likelihood for autism spectrum disorder (ASD) to improve child outcomes.

Method:

We conducted an overview of reviews to synthesize early intervention literature for very young children with or at high likelihood for ASD. Cochrane guidance on how to perform overviews of reviews was followed. Comprehensive searches of databases were conducted for systematic reviews and meta-analyses between January 2009 and December 2020. Review data were extracted and summarized and methodological quality was assessed. Primary randomized controlled trial evidence was summarized and risk of bias assessed. This overview of reviews was not registered.

Results:

From 762 records, 78 full texts were reviewed and seven systematic reviews and meta-analyses with 63 unique studies were identified. Several interventional approaches (naturalistic developmental behavioral intervention, and developmental and behavioral interventions) improved child developmental outcomes. Heterogeneity in design, intervention and control group, dose, delivery agent, and measurement approach was noted. Inconsistent methodological quality and potential biases were identified.

Interpretation:

While many early interventional approaches have an impact on child outcomes, study heterogeneity and quality had an impact on our ability to draw firm conclusions regarding which treatments are most effective. Advances in trial methodology and design, and increasing attention to mitigating measurement bias, will advance the quality of the ASD early intervention evidence base.

Autism spectrum disorder (ASD) is a heterogeneous neurodevelopmental condition marked by social communication differences and restricted, repetitive patterns of interests or behaviors.¹ Since the symptoms of ASD emerge in early childhood, the early developmental concerns of individuals who go on to receive an ASD diagnosis are increasingly documented in health records by medical providers.^2,3 While ASD prevalence data do not yet exist in many parts of the world, studies in North America, Europe, and Asia reported prevalence rates between 1% and 2.6%.^4–6 The prevalence of ASD has increased significantly over the past few decades, due in part to more widespread use of early screening and diagnostic tools, adapted to facilitate earlier diagnosis.⁷

Early intervention for young children with developmental differences, including ASD, is based on the notion that support early in life leads to better long-term outcomes.⁸ In early intervention services, active caregiver involvement in treatment is seen as critical because it helps facilitate child skill generalization across settings.⁹ Globally, there are increasing trends to provide family-centered early intervention services.¹⁰ Public policies in high-income countries, in addition to emerging policies in lower-resourced countries, promote early identification and intervention services for children with developmental challenges.^11,12 Policies matter because they mandate funding and identify a service workforce.¹³

Globally, there is growing understanding of the importance of early identification and intervention for ASD.¹⁴ Early ASD intervention can improve long-term independence and decrease medical, education, and social support costs.^8,15 In addition, expert consensus guidelines suggest that the earlier in childhood an ASD intervention is initiated, the better clinical outcomes will be.¹⁶ From a developmental neuroscience perspective, intervention early in life at a time when the brain typically expects to develop language and social skills, key areas of critical difference in ASD, is thought to result in quicker and stronger improvement than if those skills were taught later in development.^17,18 Early intensive behavioral intervention for ASD, delivered for 25 to 40 hours per week, has been reported to result in improvements across multiple child developmental domains and has been noted as the most frequently recommended approach.^19,20

With health systems around the world shifting focus from the ‘surviving’ to the ‘thriving’ focus of the Sustainable Development Goals, there is growing interest in understanding how to implement effective early intervention programs within existing systems.²¹ For example, UNICEF along with the World Health Organization and other partners, are developing comprehensive early identification and early intervention programs for children aged 36 months or younger with developmental delays and disabilities.²²

Over the past decade, multiple systematic reviews and meta-analyses have aimed to synthesize the research evidence for ASD interventions in children.^23–28 While systematic reviews and meta-analyses can be used as a basis to generate clinical guidelines for interventions in young children to inform global policy on early intervention, when many systematic reviews and meta-analyses exist on a topic it may be challenging for healthcare decision-makers to synthesize these due to variations in approach, scope, and quality. Several recent systematic reviews and meta-analyses of ASD interventions include participants across a wide age range (e.g. 0–8 years, 0–12 years, and 0–22 years), which may limit the applicability of review findings to children aged 36 months or younger.^26–28 Furthermore, very few systematic reviews and meta-analyses have critically appraised the outcome measures used to assess intervention effectiveness, a growing area of focus in autism research. In particular, few have reported whether intervention outcomes reflect generalized child change (‘outcome boundedness’) or are indicative of change beyond skills directly targeted by the intervention (‘outcome proximity’).²⁹ With global efforts underway to develop early identification and early intervention programs for children aged 36 months or younger with developmental delays and disabilities, there is a need to focus specifically on evidence since it relates to these young children to inform global policy on early intervention.³⁰

METHOD

To synthesize the literature on early intervention for very young children with or at high likelihood for ASD and identify which interventions are supported by evidence and the quality of that evidence, we performed an overview of reviews published within the past decade. When multiple systematic reviews and meta-analyses have been conducted on a specific topic, an overview of reviews may be a reasonable way to synthesize findings.³¹ Our primary Participants, Interventions, Comparators and Outcomes (PICO) question was: For children aged 36 months or younger with or at high likelihood for ASD, which therapeutic or educational interventions demonstrate evidence of greater efficacy or effectiveness for enhancing developmental outcomes compared to a control group (if present)? As best possible, we followed Cochrane guidance on how to perform overviews of reviews.³² In addition to reporting similarities or differences in conclusions across reviews, we assessed the methodological quality of the identified reviews since quality should be a key factor informing our confidence in results. Furthermore, we aimed to explore the contributions of all studies included in the systematic reviews. Notably, we examined the presence of study overlap across systematic reviews to better understand the sources of provided recommendations, individual study designs, interventions included (noting comparison groups if present and intensity, duration, and delivery agent), and measures used to assess child outcomes. We summarized the evidence from all primary randomized controlled studies included in the identified systematic reviews to synthesize which interventions were supported by evidence and assessed the risk of bias for primary studies that included a control or comparison group.

The primary objective of this overview of reviews was to synthesize the literature on early intervention for children aged 36 months or younger with or at high likelihood for ASD and identify the quality of the evidence for improved child outcomes. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart and PRISMA checklist are included in Figure S1 and Table S1.

The review objectives, inclusion criteria, and study methods were specified in advance. The manuscript inclusion criteria were: (1) children with or at high likelihood for ASD; (2) participants aged 36 months or younger at the time of intervention or data for those aged 36 months or younger analysed separately; (3) therapeutic or educational intervention; and (4) systematic review or meta-analysis. With the assistance of a National Institutes of Health librarian, a comprehensive search of the following databases was conducted: Embase, PubMed, Scopus, and Web of Science. Search terms included database-specific subject headings and keyword variants for ASD and early intervention. The individualized database search strategies are included in Appendix S1. The search was restricted to articles published in English. The publication search time span was from 1st January 2009 to 10th December 2020 to summarize the most up-to-date evidence.

Citations from the four searched databases were imported into EndNote. Duplicates were then removed. Two reviewers independently reviewed each title and abstract to determine inclusion. To determine which full-text reviews would be conducted, lists of identified articles were compared between two independent reviewers and differences resolved through discussion. Two reviewers independently completed a full-text review of the articles. Reviewers compared their final selection of articles that met the inclusion criteria and disagreements were resolved through joint review of the article and discussion. A third reviewer was available to help resolve differences. A table for data extraction was developed a priori to record information from the systematic reviews. We then extracted information from each review, including: interventions (categorized according to Sandbank et al.²⁸); measures used to assess child outcomes; and the methodological quality of reviews and strength of evidence (if reported). Two reviewers completed an initial draft of the data summary table that was reviewed and revised by another author with agreement on the final version. The extracted information is presented in Table S2 and as a narrative summary in the ‘Results’ of this overview of reviews.

The methodological quality of the included systematic reviews was assessed. A MeaSurement Tool to Assess systematic Reviews, Second Edition (AMSTAR-2), an instrument for critical appraisal of systematic review methodology, was used to assess risk of bias in the identified systematic reviews.³³ AMSTAR-2 was deemed appropriate for evaluation of the methodological quality of reviews because many systematic reviews included studies that were non-randomized. The AMSTAR-2 evaluation process identifies and highlights the bias and methodological weakness of systematic reviews across critical domains. Of the 16 items on AMSTAR-2, seven have been highlighted as critical. AMSTAR-2 developers recommend rating the methodological quality of systematic reviews based on the extent of critical flaws and non-critical weakness as follows: high, zero to one non-critical weakness; moderate, more than one non-critical weakness; low, one critical domain; critically low, more than one critical domain. Two authors independently performed an assessment of each of the included systematic reviews with full agreement on the results. The AMSTAR-2 results are presented in Table 1; a narrative summary is provided in the ‘Results’ of this overview of reviews.

TABLE 1.

Methodological quality of systematic reviews assessed with AMSTAR-2

	Baril and Humphreys23	Bradshaw et al.39	Debodinance et al.36	French and Kennedy25	Makrygianni and Reed24	Reichow et al.40	Warren et al.20
1. Research question and inclusion criteria included PICO	Yes	Yes	Yes	Yes	Yes	Yes	Yes
2. Statement that methods were decided before review; justification if deviations	No	Yes	No	No	No	Yes	Yes
3. Statement of reason for study design selection	No	No	Yes	No	No	Yes	No
4. Used a comprehensive search strategy	No	No	Yes	Partial yes	No	No	No
5. Study selection done by two reviewers	Yes	Yes	No	No	No	No	No
6. Data extraction done by two reviewers	Yes	Yes	Yes	No	Yes	Yes	Yes
7. Provided list of exclusions and reasons	Yes	No	No	Yes	No	Yes	No
8. Included studies described in adequate detail	Yes	Yes	No	No	No	Yes	No
9a. Adequate assessment of risk of bias in the included studies (RCT)	Yes	No	N/A	Yes	No	Yes	No
9b. Adequate assessment of risk of bias in the included studies (NSRI)	Yes	No	No	N/A	Yes	Yes	Yes
10. Reported sources of funding for the included studies	No	No	No	No	No	No	No
11a. If meta-analysis, appropriate statistical methods were used (RCT)	N/A	N/A	N/A	N/A	No	N/A	N/A
11b. If meta-analysis, appropriate statistical methods were used (NRSI)	N/A	N/A	No	N/A	No	N/A	N/A
12. If meta-analysis, risk of bias impact from the included studies was assessed	N/A	N/A	No	N/A	No	N/A	N/A
13. Risk of bias accounted for in the discussion/interpretation	Yes	No	No	No	Yes	Yes	Yes
14. Satisfactory explanation for/discussion of any heterogeneity in the results	No	Yes	Yes	Yes	Yes	No	Yes
15. If meta-analysis, the impact of any small study bias was investigated/discussed	N/A	N/A	No	N/A	No	N/A	N/A
16. Authors reported if they had any conflicts of interest (funding)	Yes	Yes	No	Yes	Yes	Yes	Yes

Critical AMSTAR-2 items are shown in bold.

Abbreviations: AMSTAR-2, A MeaSurement Tool to Assess systematic Reviews, Second Edition; NA, not applicable; NRSI, non-randomized study of interventions;

PICO, Participants, Interventions, Comparators and Outcomes; RCT, randomized controlled trial.

We examined the presence of study overlap across the identified reviews to better understand the sources of the provided recommendations using the corrected covered area method.³⁴ Corrected covered area values were classified as: slight (0%–5%); moderate (6%–10%); high (11%–15%); or very high (>15%) overlap (see Figure S2 for the graphical representation of primary study overlap). We compiled a comprehensive list of the child outcome measures used in the primary studies included in the identified systematic reviews. Figure S3 presents the outcome measures arranged in order from most proximal and bounded to most distal and unbounded.^29,35

Only one of the identified systematic reviews conducted a meta-analysis inclusive of the target population (children aged ≤36 months).³⁶ Therefore, we summarized the evidence from all primary randomized controlled studies included in the identified systematic reviews to synthesize which interventions were supported by evidence (Table S3). We assessed or extracted the risk of bias for primary studies that included a comparison or control group using Risk Of Bias in Non-randomized Studies of Interventions (ROBINS-I) and Cochrane risk of bias for randomized trials approaches.^37,38 This was done independently by two authors with disagreements resolved through discussion (Table 2).

TABLE 2.

Risk of bias for all primary studies with comparison or control groups

Cochrane risk of bias for randomized controlled trials37 (from the systematic review by French and Kennedy25)
	D1	D2	D3	D4	D5	D6	D7
Dawson et al.41	+	?	−	−	+	+	+
Rogers et al.42	+	+	−	−	−	+	−
Carter et al.49	+	?	−	−	−	+	+
Drew et al.50	+	?	−	−	+	+	−
Baranek et al.84	+	+	−	−	+	+	−
Green et al.85	+	+	−	+	+	+	+
Hartford86	+	?	−	−	+	−	−
Kasari et al.87	+	+	−	−	−	−	+
Kasari et al.88	+	?	−	+	−	+	+
Kasari et al.89	+	+	−	+	−	+	+
Landa et al.90	?	?	−	+	+	+	+
Schertz et al.91	?	?	−	−	?	−	−
Shire et al.92	+	+	−	+	+	+	+
Welterlin et al.93	?	?	−	−	+	+	−
Wetherby et al.94	+	?	−	−	+	+	−
Wong and Kwan95	?	?	−	−	?	+	−
Sallows and Graupner101	?	+	−	−	+	+	−
Smith et al.103	+	+	−	−	+	+	−

Judgment		Domains
Low	+	D1: Random sequence generation	D5: Incomplete outcome data
Unclear	?	D2: Allocation concealment	D6: Selective reporting
High	−	D3: Blinding of participants/personnel	D7: Other sources of bias
		D4: Blinding of outcome assessment

ROBINS-I
	D1	D2	D3	D4	D5	D6	D7
Vismara et al.46	S	NI	L	S	L	L	M
Cohen et al.97	S	S	S	NI	M	L	M
Howard et al.98	S	S	L	NI	C	L	M
Lovaas99	S	S	S	NI	NI	L	M
Remington et al.100	S	S	S	S	NI	L	M
Smith et al.102	S	S	S	NI	L	L	M
Hayward et al.105	S	S	S	NI	L	L	M
Zachor et al.106	S	S	S	NI	M	M	M

Judgment		Domains
Low	L	D1: Confounding	D5: Missing data
Moderate	M	D2: Participant selection	D6: Measurement of outcomes
Serious	S	D3: Intervention classification	D7: Selection of results to report
Critical	C	D4: Deviations from the intended interventions
No information	NI

Abbreviation: ROBINS-I, Risk Of Bias in Non-randomized Studies of Interventions.

RESULTS

The four database searches yielded 762 citations from PubMed (n = 134), Embase (n = 196), Scopus (n = 231), and Web of Science (n = 201) and 412 potentially relevant records after duplicates were removed. Seventy-eight articles met the criteria for full-text review, which was completed independently by two reviewers. Reviews were excluded if they were not ASD-specific (n = 5), participant age was not less than 3 years, and data for less than 3 years were not analysed separately (n = 56), did not describe an intervention study (n = 8), were not systematic reviews or meta-analyses (n = 8), or were duplicates (n = 3). Multiple reviews were excluded for more than one reason. Appendix S2 summarizes the reasons for full-text exclusion, including article references. A total of seven systematic reviews and meta-analyses met the inclusion criteria and were within the scope of our PICO question.^{20,23–25,36,39,40} The study selection process is outlined in the PRISMA flow diagram (Figure S1).

Table S2 includes information extracted from each of the seven reviews. Baril and Humphreys²³ aimed to evaluate the literature on the Early Start Denver Model (ESDM), a naturalistic developmental behavioral intervention (NDBI) for children with and at high likelihood for ASD.⁴ Six studies focused on children aged 36 months or younger.^41–46 All studies were conducted in the USA and study designs included single-participant, quasi-experimental, and randomized controlled trial (RCT) (control group: treatment as usual). The authors described their search strategy, which included four databases searched through to April 2015. Intervention intensity and duration ranged from 1 hour per week over 3 months to 31.5 hours per week over 24 months. The delivery agent included therapists and/or parents and the setting was home or clinic. While the authors did not provide a quantitative synthesis of the data in this review, they concluded that ESDM is a promising intervention based on rubrics from the Evaluative Method for Evaluating and Determining Evidence-Based Practice in Autism.^47,48 The authors did not determine whether ESDM was more effective than other interventions and noted that ESDM could not yet be considered an ‘established’ intervention for children with ASD.

Bradshaw et al.³⁹ reviewed studies on interventions for children aged 24 months or younger at high likelihood for ASD. Their review included nine studies involving NDBI and developmental approaches.^{41,42,49–55} Studies were conducted in the USA and UK and designs included case study, multiple baseline, quasi-experimental, and RCT (control group: treatment as usual). The authors described their search strategy, which included four databases searched through to June 2014. Intervention intensity and duration ranged from 10 1-hour sessions over 3 months to 31.5 hours per week over 24 months. Studies included both group and individual intervention delivery, with the delivery agent being therapists and/or parents and the setting being home or clinic. While the authors did not provide a quantitative synthesis of the results, they concluded that the effects of intervention on social and communication development were encouraging but more rigorous research was needed. The authors did not formally evaluate the quality of the evidence, nor directly comment on the overall strength of the evidence in their review.

Debodinance et al.³⁶ completed a meta-analysis of single-participant experimental designs aimed at determining the effects of interventions for young children with or at high likelihood for ASD. The article included 34 studies involving behavioral, developmental, NDBI, sensory, and technology-based interventions.^{43–45,52–54,56–83} All studies were conducted in the USA. The authors described their search strategy, which included five databases searched through to February 2014, in addition to reference lists and tables of contents of specific journals. Interventions, on average, occurred over 10 weeks and included 26 sessions, in the clinic or at home, and the parent was the delivery agent in 50% of studies. While the authors reported that intervention improved development and behavior by an average of 2.14 standard deviations for children with or at high likelihood for ASD, they noted significant variation in intervention effectiveness between individuals and interventions. While significant effects were found for applied behavioral analysis, pivotal response training, reciprocal imitation training, ESDM, picture exchange communication system, and video modeling, no significant differences were found in the effect sizes between interventions. The authors noted that interventions conducted at home (vs a clinical setting) and by either a parent or professional had significant effects. While the effects of the intervention tended to increase with increased intervention duration, no effect was found for the number of sessions. Moderator effects were not found for sex, age, or developmental level. The authors did not formally evaluate the quality of the evidence; while they did not directly comment on the strength of the evidence, they concluded that early interventions were effective.

French and Kennedy²⁵ aimed to identify the evidence base for early ASD intervention. Sixteen RCTs included in this review focused on children aged 36 months or younger and involved developmental, NDBI, Treatment and Education of Autistic and related Communications Handicapped Children, and ‘other’ (e.g. Autism 1-2-3) approaches.^{41,42,49,50,84–95} Studies were conducted in the USA, UK, and Hong Kong. The authors described their search strategy, which included two databases searched from 1806 to 10th May 2017, in addition to searches of reference lists and ‘researcher websites’. Intervention intensity, delivery agent, and setting were reported for two studies assessed to be at ‘low risk’ of bias across all domains (except performance bias) on the Cochrane risk of bias tool.^37,85,92 In their study, Green et al.⁸⁵ included 12 sessions (six sessions with six booster sessions) delivered by parents at home. The study by Shire et al.⁹² included 30-minute daily sessions for 10 weeks, delivered by teacher assistants in early intervention classrooms. The authors did not provide a quantitative synthesis of the data in this review and concluded that while treatment effects were significant in many studies, effect sizes were often small with wide confidence intervals; only two studies were at ‘low risk’ of bias.^37,85,92 The authors did not directly comment on the strength of the evidence and noted that all approaches warrant further research in routine clinical practice.

Makrygianni and Reed²⁴ conducted a meta-analysis of behavioral interventions to provide a synthesis of effectiveness research. Eight of the 14 studies in the meta-analysis included children aged 36 months or younger. The subanalyses examined the impact of child age at intervention intake (≤35 months vs >35 months) on intervention effectiveness.^96–103 Designs included quasi-experimental and RCT; studies were conducted in Israel, the UK, and the USA. The names of the databases and search dates were not specified; however, the authors noted that an ‘extensive search was carried out using search engines, and computerized bibliographic databases’, following the example of Smith.¹⁰⁴ The authors also noted that they reviewed citations of specific review articles in addition to articles identified in their review and that ‘recommendations from experts in the field were taken into account’. Intervention intensity, duration, and staff per child (mean and standard deviation) were reported for the high versus low methodological quality papers. Overall the high methodological quality group received 27.54 (10.47) hours per week for 27.51 (14.83) weeks, with 3.63 staff per child. Overall the low methodological quality group received 25.89 (10.27) hours per week for 37.26 (15.89) weeks, with 4.48 (1.94) staff per child. Regarding a child’s intake age, no statistically significant correlations were found with intervention effectiveness in the subanalyses; however, two trends were found. First, age at intake was negatively correlated with language abilities (r = −0.736, p = 0.059). Second, the younger they were at program initiation the greater the impact on intellectual abilities (r = 0.798). Study quality was assessed with a scale recommended by Reichow et al.⁴⁸ While the authors excluded studies categorized as of ‘weak’ quality, 4 of the 8 studies that included participants aged 36 months or younger were categorized as ‘low’ quality. The authors did not formally assess the strength of the evidence but concluded that early intervention was quite effective for children with or at high likelihood for ASD.

Reichow et al.⁴⁰ examined the effectiveness of early intervention for individuals with lower-functioning ASD conducted by non-specialist providers. Five studies involved children aged 36 months or younger and focused on behavioral, NDBI (ESDM), and other approaches (Autism 1-2-3).^{41,95,97,98,103} Designs included quasi-experimental studies and RCTs; studies were conducted in the USA and Hong Kong. The authors described their search strategy, which included nine databases searched through to 24th June 2013. Intervention intensity and duration ranged from five 30-minute sessions per week for 2 weeks to 35 to 40 hours per week over 156 weeks. Subanalyses with participants aged 36 months or younger suggested that behavior analytical techniques (the authors included behavioral and ESDM in this category) are most effective since 4 of 7 effect size estimates greater than 0.50 (in developmental and daily skills domains) were found in RCTs; 5 of 7 effect size estimates were statistically significant, including 2 of 4 estimates from RCTs. Risk of bias was assessed with the Cochrane risk of bias tool.³⁷ All studies were at risk for performance bias, most studies were at moderate risk for selection, detection, and contamination bias, and most studies were at low risk for reporting and attrition bias. The authors did not assess the overall strength of the evidence for early intervention but concluded that non-specialists can deliver effective treatment to children with lower-functioning ASD.

Warren et al.²⁰ reviewed articles on early intervention effectiveness for children with ASD. Ten studies included children aged 36 months or younger and focused on behavioral and NDBI approaches.^{41,46,50,55,96–98,103,105,106} The authors reported overall strength of evidence and study results separately for comprehensive approaches for children aged 24 months or younger. Designs included prospective case series, prospective cohort studies, non-concurrent multiple baseline studies, and RCTs, and were conducted in Israel, the UK, and the USA. The authors described their search strategy, which included three databases searched from 2000 to May 2010. The authors noted that behavioral approaches were intensive and delivered through 1:1 instruction, and that ESDM included 2 years of intensive intervention. Comparison groups (where present) varied significantly and included parent-mediated intervention, eclectic intervention, public early intervention programs, parent training, eclectic-developmental principles, treatment as usual, and a posttreatment contrast group. In the studies that included comprehensive approaches for children aged 24 months or younger, the authors reported that while improvements in adaptive behaviors, cognitive, and language abilities were seen over 2 years of ESDM, findings were not yet replicated and how core ASD symptoms respond to treatment was unclear. The authors developed a quality assessment form, with studies receiving an overall score of good, fair, or poor. One study in the comprehensive approaches for children aged 24 months or younger group received a ‘good’ rating.⁴¹ Strength of evidence was assessed based on four domains; the authors concluded that the intervention evidence base for children aged 24 months or younger was insufficient. The authors noted limited high-quality studies and studies that compared one intervention type to another. They concluded that further research was needed to determine the effectiveness of early intervention for young children with and at high likelihood for ASD.

Table 1 summarizes the extent to which each of the seven systematic reviews controlled for sources of methodological bias using the AMSTAR-2 evaluation process. In three of the systematic reviews, one to two critical weaknesses were reported, all of which included the omission of a comprehensive description of the search strategy or a statement demonstrating that the methods were determined before conducting the search.^23,25,40 Two reviews had three to four critical weaknesses, with additional weaknesses including not providing the reasoning for the exclusion of full-text articles and/or not accounting for risk of bias in the discussion.^20,36 Finally, two reviews had five or more critical weaknesses, with additional weaknesses resulting from the lack of adequate assessment of risk pertaining to RCT-type study designs that were included in the analysis.^24,39 Therefore, using the ratings recommended by the AMSTAR-2 developers, six of the seven systematic reviews had critically low methodological quality (more than one critical flaw with or without non-critical weaknesses)^{20,23–25,36,39} and one review had low methodological quality (one critical flaw with or without non-critical weaknesses).⁴⁰

Across the seven systematic reviews, 63 unique papers included participants whose ages were 36 months or younger (or whose data for ≤36 months was analysed separately). The total number of participants across these seven studies (including the comparison groups), removing numbers from duplicate studies, was 1388. We examined the presence of study overlap across the seven reviews to better understand the sources of the provided recommendations using the corrected covered area method.³⁴ Overlap ranged from slight or no overlap (e.g. Baril and Humphreys²³ and Makrygianni and Reed²⁴: 0%) to very high overlap (e.g. Reichow et al.⁴⁰ and Warren et al.²⁰: 36%). Figure S2 details the primary study overlap across reviews.

A comprehensive list of the heterogeneous child outcome measures used in the 63 unique papers is presented in Figure S3. Measures have been presented in six categories, arranged in order from most proximal and bounded to most distal and unbounded.³⁵ It is important to underline that the measures themselves are not inherently proximal or distal, or context-bound or generalized, and that these categorizations should always be made relative to the context and targets of the intervention under study.²⁹ However, the categorizations in Figure S3 represent how these heterogeneous child outcome measures are typically used in the intervention studies included in this overview of reviews. Behavioral coding measures were categorized as the most proximal and bounded because these measures were typically used to collect intervention-specific data, with study-specific scales, based on the observation of child participants in naturalistic settings, often via video recordings of intervention sessions or caregiver–child play interactions, and were often measured using frequency or duration. These were the most frequently used measures, with 23 studies using behavior coding measures exclusively. Measures in the behavior coding system group included behavioral rating and coding systems developed for use across research studies via observation of participants in naturalistic or intervention settings. Structured observational assessments, such as the Autism Diagnostic Observation Schedule and Motor Imitation Scale, are standardized procedures that probe for behaviors and skills that may be targeted in interventions. Informant report measures, such as the Autism Diagnostic Interview-Revised, ask caregivers or teachers about behaviors and skills that are the focus of the intervention but may gather data about child behavior across various settings, potentially making them less bounded than behavioral coding and structured observation assessments— depending on the context and targets of the intervention under study. Standardized assessments may be those that are most generalized (distal) and removed from the intervention context (unbounded); depending on the context and targets of the intervention under study, they may fall at the other side of the spectrum. Standardized language assessments, such as the Reynell Developmental Language Scales, were used in 18 of the primary studies, all of which examined interventions targeting language skills. The language subscales of more comprehensive standardized assessments, such as the Vineland Adaptive Behavior Scales, Bayley Scales of Infant and Toddler Development, and Mullen Scales of Early Learning, were also used frequently. Due to their distal nature, standardized assessments are the least likely to show change based on the intervention. Of the 63 primary studies, 15 used a combination of behavior coding and standardized measures. Of these, 12 studies found intervention effects using behavior coding outcomes that were not evident when analysing results from standardized measures.

Only one systematic review conducted a meta-analysis of single-participant studies on children aged 36 months or younger.³⁶ The authors of the review noted that significant heterogeneity in treatment, measurement approaches, comparison groups, participant profiles, and amount of intervention delivered to the child across primary studies had an impact on their ability to conduct meta-analyses. To synthesize the evidence on which interventions were supported by evidence, we examined all primary studies in the identified systematic reviews that included control groups. The evidence summary is presented in Table S3. Change in Cohen’s d was calculated for each RCT with available data on child outcomes (n = 16).¹⁰⁷ Effect sizes ranged from low to high across various intervention approaches. With interventions categorized using the approach by Sandbank et al.,²⁸ various NDBI (ESDM; social-pragmatic joint attention focused parent training; Joint Attention, Symbolic Play, Engagement, and Regulation; interpersonal synchrony; early social interaction), developmental (adapted responsive teaching; joint attention-mediated learning; iBASIS Video Interaction to Promote Positive Parenting), and behavioral (early intensive behavioral intervention) approaches demonstrated medium-to-large effect sizes judged by Cohen’s d (>0.50 and >0.80) on child outcomes across developmental domains (including cognitive, language, motor, social communication, and autism characteristics). The largest effect sizes were typically seen on proximal, bounded child outcome measures (i.e. intervention-specific outcomes, collected with study-specific scales, coded from caregiver–child play interactions), with smaller effect sizes typically seen on distal, unbounded child outcomes measures (i.e. standardized assessments administered by clinicians and removed from the intervention context).

Table 2 shows the extracted or assessed risk of bias information for all primary studies in the identified systematic reviews that included comparison or control groups. The Cochrane risk of bias tool for RCTs was used to rate the risk of bias in 18 primary RCTs included in the identified systematic reviews (extracted from French and Kenendy²⁵).³⁷ All RCTs demonstrated high risk of bias across at least two domains. The most common domains impacted included performance bias (lack of blinding of participants and relevant personnel) and detection bias (lack of blinding of outcome assessment). We assessed risk of bias in eight primary studies that included comparison groups in the identified systematic reviews using the ROBINS-I.³⁸ Serious concerns for bias were identified in all eight studies across at least two domains. The two most common domains impacted included confounding bias (when one or more prognostic variable[s] also predicts the intervention received) and participant selection bias (when exclusion of some eligible participants, follow-up time, or some outcome events are related to both intervention and outcome).

DISCUSSION

In this overview of reviews, we aimed to synthesize the literature on early intervention for very young children with or at high likelihood for ASD to identify which interventions are supported by evidence and examine the quality and strength of that evidence. In addition to reporting similarities or differences in conclusions across the identified reviews, we aimed to explore the contributions of primary studies. Given variable methodological reporting across reviews, we followed Cochrane guidance on how to perform overviews of reviews as best possible.³²

There was significant overlap in primary studies included in the reviews, with almost half of comparisons across reviews having high to very high primary study overlap. The identified systematic reviews included 63 primary studies with a wide range of study designs, with considerable diversity of intervention approaches and control groups (when present). Child outcomes varied widely across studies in terms of their content focus, outcome proximity, and boundedness. Furthermore, there was significant variation in the dose of intervention, delivery agent, and intervention setting across primary studies. This heterogeneity in study design, intervention, and measurement had an impact on how results were reported across the seven identified systematic reviews within the scope of our PICO question. Three reviews provided only narrative summaries of the results. Three reviews that included quantitative summaries of data were subanalyses with participants aged 36 months or younger and included few primary studies. While one review included a meta-analysis of 34 primary studies and reported improved child development and behavior across various NDBI, developmental, behavioral, sensory, and technology-based intervention approaches, primary studies included single-participant designs and study quality was not evaluated. Importantly, across systematic reviews, inconsistent methodological quality and potential biases were noted across critical AMSTAR-2 domains. Examination of all primary studies in the systematic reviews that included control groups identified various NDBI, developmental, and behavioral intervention approaches that significantly improved child outcomes across various developmental domains. The greatest intervention impact was seen on proximal, intervention-specific outcomes, with less impact being documented on child outcomes measured using distal, standardized assessments. While this is an expected finding, given that previous studies reported greater intervention effects on outcomes that measure intervention-specific skills, this finding matters for the interpretation of the autism intervention evidence in children aged 36 months or younger.^28,108 Serious concerns for bias were identified across primary studies with comparison or control groups. However, it should be noted that performance bias, which is part of the Cochrane assessment, is challenging to address since in these early autism intervention approaches, participants cannot necessarily be masked to the intervention they receive because a relevant placebo condition is not possible or available. In summary, while this overview of reviews identified many early intervention approaches that had an impact on child developmental outcomes, limitations in the quality of the evidence and heterogeneity of measurement, comparison groups, participant profiles, and intervention dose limit our ability to conclude which interventions are most effective. This conclusion is not entirely dissimilar from reviews of autism interventions that have included participants across a broader age range, suggesting that as a field, autism intervention research may face similar challenges across age ranges in approach, design, and measurement.^26–28

The recent Lancet Commission on the future of care and clinical research in autism concluded that at present, while many early intervention approaches have been shown to have an impact on child outcomes, we do not yet know which treatments ‘are most effective, when, and for whom’.¹⁰⁹ There is growing recognition of limitations in the quality and strength of available early ASD intervention literature.¹¹⁰ This matters because the current evidence base has informed clinical guidelines.¹¹¹ In high-income countries, like the USA, standard recommendations after diagnosis often include behavioral intervention, provided for 25 to 40 hours per week, even in the absence of sufficiently high quality evidence supporting such a recommendation.¹¹² With growing global recognition of the importance of early ASD identification and intervention, aligning guidelines with the evidence base is critical as stretched health and education systems in low-resource countries are tasked with supporting child developmental needs. The expense and limited availability of such intensive interventions should necessitate high-quality evidence. With ongoing advances in intervention trial methodology and design, the quality challenges facing the early ASD intervention field will certainly decrease over time.¹¹³ In addition, increasing attention toward mitigating measurement bias, in particular the challenges associated with outcome proximity and boundedness, will advance the quality of ASD early intervention evidence.³⁵ However, it is critical that calls for increasingly robust clinical trial methodologies are balanced with research strategies to bridge the community implementation gap in early ASD intervention, bearing in mind that there is tremendous disparity in who participates in and benefits from ASD intervention research globally.

This overview of reviews has limitations. First, while we did not register the review, PRISMA guidelines and Cochrane guidance on how to perform overviews of reviews were followed.³² Second, although seven systematic reviews and meta-analyses were identified, the most recent review was from 2018. Therefore, the evidence included in this overview of reviews was not as up to date as if we had conducted a systematic review with individual studies up to the end search date (10th December 2020). Third, while many systematic reviews have been published over the past decade, which informed the overview of reviews approach, the reviews identified from the search process did not meet optimal criteria for an overview. Concerns include considerable primary study overlap and the quality limitations of the included reviews. To mitigate these concerns, we calculated corrected coverage areas between each pair of reviews and evaluated the methodological quality of reviews using the AMSTAR-2 standardized tool. Finally, while we categorized measures according to their proximity and boundedness, categorization of outcome measures will always need to be made relative to the study context and intervention targets and not merely considered as an innate characteristic of the measure itself.²⁹ Limitations in the quality of the early ASD intervention literature, including limitations in the primary studies included in the reviews, highlight the urgent need to improve our evidence base.

While this overview of reviews identified many early intervention approaches that had an impact on child developmental outcomes, significant limitations in the quality of the evidence and heterogeneity of the included studies was evident. Therefore, we wish to highlight the following lessons learned from our review: first, there is limited evidence to support recommendations for very intensive interventions (25–40 hours per week) in young children with ASD. Intensity recommendations should be individualized to the child profile and family preference.¹¹⁴ To date, the only RCT of intervention intensities suggested differential benefit based on ASD symptom severity from 15 to 25 hours per week.¹¹⁵ Second, NDBI and developmental interventions have more empirical support from RCTs than behavioral interventions. A recent systematic review, which required a minimum of five RCTs of intervention effects on a given outcome to compute summary effects, found that developmental and NDBI types demonstrated positive effects.²⁸ While behavioral interventions reported positive effects, there were not enough RCTs of behavioral interventions to meet inclusion in this systematic review to compute summary effects. Therefore, recommendations on intervention type should include these approaches. Importantly, both developmental and NDBI approaches align with family-centered early intervention services emerging globally since they occur in a child’s natural environment during everyday interactions with caregivers with learning goals guided by early developmental sequences.^10,116,117 Third, when the goal of an intervention is to support child developmental gains, using behavioral coding of intervention-specific skills rated on study-specific scales as a primary outcome has significant limitations because such measures may capture only limited and transient changes in skills that should not be construed as indicative of broader developmental improvement. Recent measurement considerations provide guidance on potential approaches.³⁵ One approach would be to retain behavioral coding of caregiver–child interactions but use behavioral coding data in mediation analyses in an effort to link proximal, bounded measures with unbounded, distal child outcomes. Although outside the scope of this review, it is important to acknowledge that primary caregivers are vital in family-centered early intervention services. Therefore, understanding and bolstering support for caregiver mental health and well-being is an important aspect of early intervention.¹¹⁸ In conclusion, we recognize the heterogeneity of ASD and differences in every child and family unit; therefore, we underline that ultimately recommendations should fit a child’s unique needs, family priorities, and available resources.

Supplementary Material

Supplementary Appendix 2

Appendix S2: Reasons for full-text exclusions.

Supplementary Appendix 1

Appendix S1: Individualized database search strategies.

Supplementary Figure 3

Figure S3: Outcome proximity and boundedness of primary studies.

Supplementary Table 1

Table S1: PRISMA 2009 Checklist

Supplementary Table 2

Table S2: Summary of identified systematic reviews/meta-analyses

Supplementary Table 3

Table S3: Summary table listing results by outcomes for RCTs with data to calculate Cohen’s d

Supplementary Figure 1

Figure S1: PRISMA flow diagram.

Supplementary Figure 2

Figure S2: Corrected covered area overlap across reviews.

What this paper adds.

• Naturalistic developmental behavioral interventions, as well as developmental and behavioral interventions, improve child outcomes in autism spectrum disorder (ASD).

• If only randomized controlled trials are considered, guidelines for early intensive behavioral intervention in younger children should be revisited.

• The greatest intervention impacts were on proximal, intervention-specific outcomes.

• Inadequacies in the quality of the early ASD intervention evidence base were observed.

Abbreviations:

AMSTAR-2

A MeaSurement Tool to Assess systematic Reviews, Second Edition

ASD

Autism spectrum disorder

ESDM

Early Start Denver Model

NDBI

Naturalistic developmental behavioral intervention

PICO

Participants, Interventions, Comparators and Outcomes

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RCT

Randomized controlled trial

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.