A Multisite Randomized Controlled Trial Comparing the Effects of Intervention Intensity and Intervention Style on Outcomes for Young Children With Autism

Abstract

Objective:

This randomized, multisite, intent-to-treat study tested the effects of 2 levels of treatment intensity (number of hours) and 2 treatment styles on the progress of young children with autism spectrum disorder (ASD). We predicted that initial severity of developmental delay or autism symptoms would moderate the effects of intensity and style on progress in 4 domains: autism symptom severity, expressive communication, receptive language, and nonverbal ability.

Method:

A total of 87 children with ASD, mean age 23.4 months, were assigned to 1 of 2 intervention styles (naturalistic developmental/behavioral or discrete trial teaching), each delivered for either 15 or 25 hours per week of 1:1 intervention for 12 months by trained research staff. All caregivers received coaching twice monthly. Children were assessed at 4 timepoints. Examiners and coders were naive to treatment assignment.

Results:

Neither style nor intensity had main effects on the 4 outcome variables. In terms of moderating the effects of initial severity of developmental delay and of autism symptom severity, neither moderated the effects of treatment style on progress in any of the 4 domains. In terms of treatment intensity, initial severity moderated effect of treatment intensity on only 1 domain, namely, change in autism symptom severity; in a secondary analysis, this effect was found in only 1 site.

Conclusion:

Neither treatment style nor intensity had overall effects on child outcomes in the 4 domains examined. Initial severity did not predict better response to 1 intervention style than to another. We found very limited evidence that initial severity predicted better response to 25 vs 15 hours per week of intervention in the domains studied.

Clinical trial registration information:

Intervention Effects of Intensity and Delivery Style for Toddlers With Autism: https://clinicaltrials.gov/; NCT02272192

For young children with autism spectrum disorders (ASD), high-quality single-blind randomized controlled trials of well-defined manualized interventions delivered 1:1 at intended fidelity have demonstrated significant effects on IQ, expressive language, receptive language, and autism severity.^1-5 Although these different approaches have all demonstrated efficacy, their manuals and publications describe large differences among them in (1) intervention style (eg, adult-directed didactic vs naturalistic developmental/behavioral), and (2) intervention intensity (ie, hours of intervention per week). Although caregivers and practitioners need information about what style and intensity of early intervention is optimal for a given child, there have been no rigorously controlled comparative studies addressing these questions, which have major policy, practice, and funding implications for intervention delivery systems, professional training and practice, caregiver choice and learning, and children’s lives. Multiple previous intervention studies reported that autism symptom severity, severity of developmental delays, and treatment intensity predicted child outcomes. However, none of these studies used a randomized controlled trial (RCT) design.^6-9 Although it is common for some interventions to be delivered 1:1 at intensities as high as 35 to 40 hours per week in keeping with Lovaas’s initial report, the costs and scarcity of such intensive treatment require supportive high quality evidence, providing 1 of the rationales for this study.¹⁰

Because rigorous assessment of relative efficacy of treatment style and intensity requires a study of adequate power and the control of other factors, we designed a randomized, multisite, intent-to-treat study using naive examiners and coders to compare effects of 2 intervention styles and 2 intervention intensities on outcomes in multiple domains of toddlers with ASD.¹¹ The styles compared were 1:1 discrete trial teaching (Early Intensive Behavioral Intervention [EIBI]) and 1:1 naturalistic developmental/behavioral intervention (Early Start Denver Model [ESDM]). The intensity levels were 15 hours or 25 hours per week delivered for 12 months.^12,13

Given the similarity of outcomes from well-controlled RCTs testing very different intensities and styles, we did not predict main effects of intensity or style.^1,2,4 However, based on the slower learning rates and need for more learning repetitions of children with lower developmental quotients (DQs) than those with higher DQs, we expected baseline DQ to moderate effects of treatment intensity and style on children’s outcomes. In addition, based on the decreased social responsivity of children with more severe vs milder autism symptoms, we predicted that baseline ASD severity would moderate child response to treatment style and intensity.

The study hypotheses were as follows. Hypothesis 1 was that the initial degree of developmental delay and autism severity would moderate effects of treatment style on growth trajectories of expressive communication, receptive language, nonverbal ability, and autism symptom severity, with milder degree of delay and/or autism symptoms predicting greater progress in response to ESDM vs EIBI treatment. Hypothesis 2 was that the initial degree of developmental delay and autism severity would moderate effects of treatment intensity on growth trajectories of expressive communication, receptive language, nonverbal ability, and autism symptom severity, with more severe initial developmental delay and/or autism symptoms predicting greater progress in response to 25 vs 15 hours of treatment.

METHOD

Trial Design

We conducted an intent-to-treat, single-blind RCT at 3 universities. The study period began in 2013 and ended in 2019, as depicted in Figure 1.

FIGURE 1.

CONSORT Table. EIBI = early intensive behavioral intervention; ESDM = Early Start Denver Model

Children were recruited, screened, qualified, consented, stratified by DQ and age, and randomized to 1 of 4 cells (15 or 25 hours of ESDM; 15 or 25 hours of EIBI). Treatment was delivered for 12 months in homes (for most children) and/or in childcare settings by research staff. In addition, all families received two 1.5-hour sessions of caregiver coaching monthly in use of the assigned intervention. An independent data coordinating center (DCC) conducted the randomization of participants using a computer program allocation. All assessor-administered measures were conducted by staff researchers naive to group assignment. Observational measures were coded by observers who were naive to group assignment. Caregivers were not naive to assignment, and caregiver reports were used in some measures. Interventionists were trained across sites to acceptable levels of fidelity of implementation (FOI) before beginning to treat children, were supervised by expert professional therapists regularly, and were overseen by developers of the 2 treatments. The treatment manuals were followed rigorously; ongoing training and FOI monitoring occurred across sites throughout treatment. Supplement 1,available online, provides more information.

All children were scheduled for 4 clinic-based assessments conducted by qualified, trained staff members across a 24-month period: time of enrollment, 6 months following enrollment (mid intervention phase), 12 months following enrollment (end of intervention phase), and 24 months following enrollment (follow-up phase). Caregivers were provided verbal and written reports and referral to community-based services. Given the intent-to-treat design, all children’s data were included in the original assignment group, including those who dropped out before the study’s end, and all were assessed at all timepoints as possible. The project was approved by appropriate Institutional Review Boards. Design and data were routinely reviewed by an independent Data Safety and Monitoring Board. Main modifications to the design after commencement of the study involved reducing the minimum age to 12 months, adding inclusion/exclusion criteria described later here, some additions of measures not reported here, timing of measure administration, personnel changes, and addition of follow-up procedures.

Recruitment, Enrollment, and Randomization

Children were recruited from pediatricians, developmental disability settings, and university website postings using institutional review board—approved materials. DCC randomization was performed using a computer algorithm based on 2 pre-specified blocks: 12 to 20 months or 20+ months, and DQ <60 or DQ >60. Blocks were monitored, and a difference of 3 between ESDM and EIBI groups resulted in a compensatory assignment to maximize equal samples.

Participants

We screened 128 toddlers referred to the study using 2 different published, age-appropriate autism risk screeners, and we conducted diagnostic assessments of all those who screened positive according to screener criteria. We enrolled 87 toddlers with autism spectrum disorder (DSM-5) recruited from 3 sites: 28 from site 1, 30 from site 2, and 29 from site 3. Individuals of both sexes were recruited, and the group was ethnically/racially diverse. The Consolidated Standards of Reporting Trials (CONSORT) table in Figure 1 depicts the flow of participants through the study.

Inclusion Criteria.

Participants were required to meet the following inclusion criteria: age 12 to 30 months at the time of assessment; ambulatory and without impairments affecting hand use; criteria met for Autism Spectrum Disorder on the DSM-5 criteria and on the Autism Diagnostic Observation Schedule for Toddlers and clinical consensus of ASD diagnosis by 2 independent staff (including a licensed psychologist) based on observation as well as record review; overall developmental quotient of ≥35 on Mullen Scales of Early Learning; normal hearing and vision screen; and caregiver agreement to comply with all project requirements, including regular videotaping at home with provided equipment.^14-16

Exclusion Criteria.

Exclusion criteria were as follows: English not a primary language spoken at home; absence at 2 or more appointments without prior notice during the intake assessment; more than 10 hours per week of 1:1 ABA-based treatment; other health or genetic conditions (ie, Fragile X syndrome, seizures, prematurity).

Sample Size

Power analyses were calculated using longitudinal mixed-level models on data from measures used in Dawson et al.,¹ estimated for 108 participants. In the Dawson et al. study, longitudinal analyses of MSEL data showed a 17.6% group difference, between-subject variance of 23.7, and within-subject correlation of repeated measurements of 0.45.¹ We had 90% power to detect the main effects of the treatment intensity and the style on an average MSEL score of 2.0% to 2.25% when the between-subject variance was 30 and the within-subject correlation of repeated measurements was 0.4. We had over 90% detection power when this difference increased to 2.5%, as hypothesized for those individuals with more severe initial developmental and autism symptoms, and also when the within-subject correlation increased to 0.5.

Treatment

Children were scheduled to receive either 15 or 25 hours per week of 1:1 treatment in their homes or care/preschool settings, delivered by trained interventionists (ITs), for 12 months. Sessions were typically 1.5- or 2.5-hour blocks, 10 blocks per week, generally 1 morning and 1 afternoon block, fitted around sleep and family schedules.

ITs were employed full-time by the study and were trained and overseen by full-time supervisors with graduate degrees who monitored the treatment at all sites via Internet and live viewing and who met regularly within and across sites to ensure that all procedures were carried out consistently. Treatment procedures followed the published manuals and implementation guidelines. Primary caregiver(s) received coaching in the assigned model by their team supervisor every 2 weeks throughout their enrollment. Supplement 1,available online, provides further information.

Intervention Intensity: 15 vs 25 Hours per Week.

All delivered hours across the 48 (of 52) treatment weeks were recorded daily and reported weekly. Cancelled treatment hours were made up as possible. The actual number of hours of treatment per week for each participant was quantified using schedulers’ weekly attendance logs. Both groups received an average of 83% of the hours scheduled: means were 12.42 hour per week (SD = 1.45) and 20.82 hours per week (SD = 1.40) for the low- and high-intensity groups, respectively, which differed significantly by design (Cohen’s between-group d = 5.9).

Fidelity of Treatment Implementation.

The Leaf and McEachin manual, A Work in Progress, defined the EIBI approach and curriculum.¹² Supervisors and ITs received quarterly ongoing onsite training and ongoing consultation from Dr. McEachin. An FOI tool was developed to measure correct implementation of 9 components using a 5-point Likert scale applied to randomly selected 20-minute sections of recorded treatment sessions (Yoder P, McEachin J, Wallace E, Leaf R, 2014, unpublished. Discrete Trial Training Fidelity of Treatment Rating). During instruction, children had blocks of teaching trials interspersed with short breaks that included therapist interactions, and a 10- to 15- minute break half-way through each session.

The ESDM intervention was carried out as per the manual instructions and was measured by its published FOI tool.¹³ Instruction followed the ESDM curriculum and was embedded in typical early childhood activities, carried out within the joint activity structure first defined by Ratner and Bruner,¹⁷ using everyday play objects and activities and routines from everyday life.¹⁸ The FOI tool measured quality of instruction on a 14-item, 5-point Likert based scale from videos of 30 minutes or more of activities.¹³ ESDM credentialed therapists supervised the ITs, and their work was overseen by certified ESDM Trainers and by Dr. Rogers.

Staff were trained to fidelity on all procedures before beginning to work with children and maintained 80% or better fidelity scores measured twice a month throughout the project, assessed via 20-minute random videotaped segments of therapy gathered quarterly. FOI was quantified by using the average of the key items from each FOI rating scale. With “5” as the score that indicated highest compliance, means were 4.15 (SD = 1.3) for EIBI (83% of possible total) and 4.3 (SD = 0.15) for ESDM (86%). There were no significant differences between them (Cohen’s between-group d = −0.16; 95% CI — 0.59 to 0.27). If fidelity scores dropped below 80%, the supervisor retrained the IT until acceptable fidelity was regained. Supplement 2,available online, provides additional details.

We created a Treatment Modification instrument to measure the extent to which each child’s intervention may have deviated from the core EIBI or ESDM principles as a means of individualizing the intervention approach for a particular child’s learning needs in ways other than what was captured in the FOI instruments (Yoder P, 2014, unpublished). For the EIBI group, the instrument assessed whether 4 naturalistic modifications were made. For the ESDM group, the instrument assessed whether 4 EIBI-type modifications were made. Each quarter, supervisors rated the percent modifications made in EIBI or ESDM using a Likert-type scale (ie, 1 = 1%–25%, 2 = 26%–50%, 3 = 51%–75%, 4 = 76%–100%). The degree to which EIBI treatment incorporated naturalistic characteristics was associated with time 1 DQ (r = 0.50, p < .05) and time 1 autism severity (r = −0.35, p < .05); thus, adding ESDM principles to EIBI was related to initial milder severity. The degree to which ESDM treatment incorporated EIBI characteristics was correlated with time 1 developmental delay (r = −0.47, p < .05) and time 1 autism severity (r = 0.34, p < .05), indicating that the addition of discrete trial methods to ESDM was done for children with initial greater severity.

Measures

Screening measures.

The Early Screening of Autistic Traits Questionnaire (ESAT), an autism risk parent-report questionnaire that has been validated in several large studies, was used for children 12 to 15 months of age. The inclusion criterion required 3 or more negative responses.¹⁹

The Infant Toddler Checklist (ITC) is a standardized parent questionnaire with screening cutoffs and standard scores for children 6 to 24 months of age that is based on a normative sample of more than 2,188 children.²⁰ Criterion was a score at or below the fifth percentile.

The Modified Checklist for Autism in Toddlers (MCHAT), a 23-item autism screening questionnaire that has been validated in 2 large studies, was used for children 16 to 24 months of age.^21,22 Criterion was failure on 2 critical items or any 3 items.

Constructing the Composite Scores.

The primary outcome and moderating variables were composites representing expressive language, receptive language, nonverbal performance, and autism severity, constructed from variables of the component measures described below. Table 1 provides a list of the composites, the variables included in each composite, and relevant statistics. We computed composite scores from multiple measures of a construct to increase construct validity and to minimize the number of significance tests required to test predictions, thereby reducing familywise error. To build the composite scores, we first examined whether the proposed component variables were correlated (for 2 component variables) or factor loaded (for 3 or more component variables) at or above 0.3. Supplement 3,available online, provides Chronbach’s α data on relationships among these composites. Because a single time period’s mean and SD is required so that composites can show change over time, we used time 4 means and SD to compute the z scores.

TABLE 1.

Components of the Primary and Moderating Composite Dependent Variables

Composite DV	Component variables
Autism symptom severity
	ADOS-2a calibrated severity score
	PDDB-Ib expressive receptive social communication composite reflected raw score
Expressive communication
	15-min Communication sample; wficc
	15-min Communication sample; number of different word roots
	MSELd expressive language age equivalency
	VABS-2e expressive communication age equivalency
	MB-CDIf expressive raw score
	PDDB-I expressive social communication abilities composite raw score PDDB-I expressive language raw score
Nonverbal abilities
	MSEL fine motor age equivalency
	MSEL visual reception age equivalency
	VABS-2 daily living skills age equivalency
	VABS-2 motor skills age equivalency
	VABS-2 socialization age equivalency
Receptive language
	MSEL receptive language age equivalency
DQ	VABS-2 receptive language age equivalency
	MSEL expressive language age equivalency
	MSEL receptive language age equivalency
	MSEL fine motor age equivalency
	MSEL visual reception age equivalency

Note: ADOS-2 = Autism Diagnostic Observation Schedule, Second Edition; DQ = developmental quotients; DV = dependent variable; MB-CDI = MacArthur—Bates Communicative Development Inventories; MSEL = Mullen Scales of Early Learning; PDD = pervasive developmental disorder; PDDB-I = PDD Behavior Inventory; VABS-2 = Vineland Adaptive Behavior Scales, Second Edition; wfic = weighted frequency of intentional communication.

^aAutism Diagnostic Observational Schedule version 2

^bPervasive Developmental Disorders Behavior Inventory—parent rating form.

^cWeighted frequency of intentional communication.

^dMullen Early Learning Scales.

^eVineland Adaptive Behavior Scales version 2.

^fMacArthur—Bates Communicative Development Inventory, combined infant and toddler forms.

Measures Contributing to Composite Dependent Variables or Moderators.

The ADOS calibrated severity score was calculated from the Autism Diagnostic Observation Scale (ADOS 2) assessment. The appropriate module of the standardized ADOS based on each child’s language level was administered.²³ Assessors naive to assignment were trained to 85% reliability on the full range of scores, and all met research criteria. Interobserver reliability at individual sites was assessed on at least 15% of interviews, and any deviation from standards led to retraining.

The PDD Behavior Inventory is a rating scale filled out by caregivers and teachers that is designed to assess response to intervention in children with PDD.²⁴ Subscales measure both maladaptive and adaptive behaviors as well as a summary score that reflects overall severity. Factor analyses have confirmed the structure of the scale. Correlations with psychometrically sound instruments that assess autism severity, adaptive behavior, and maladaptive behavior reveal high correlations and document construct validity. Scores used in composite dependent variables and 1 of the moderators are listed in Table 1.

The Communication Sample Procedure is a 15-minute, videotaped, laboratory-based communication sample using a standard toy set that provides semi-structured free-play with interspersed opportunities for the child to respond to an adult’s topic change and to request clarification. The examiner’s interaction style is guided by specific principles described in the procedure manual (available from Dr. Yoder). Using a timed event sampling method, trained observers unaware of group coded number of different words and weighted frequency of intentional communication from video. Reliability of the coded variables was assessed on a random selection of 20% of the sessions, with coders kept naive to session dates. The intraclass correlation coefficient using a 2-random analysis and absolute agreement method averaged 0.85 for the weighted frequency of intentional communication and averaged 0.83 for the number of different words. Variables used in composites included weighted frequency of intentional communication and number of different word roots.

Mullen Scales of Early Learning are standardized measures of early child development across 4 domains: expressive and receptive language, fine motor skills, and visual perception.¹⁶ These were carried out by experienced assessors naive to group assignment who were employed by and trained at the research sites. Reinforcers for child attention and cooperation were used as needed to support children’s motivation to perform. Because the standard scores on the MSEL have set floors and because many of the participants had scores at the floor, we used the age-equivalent scores from each subdomain to construct the composite dependent variables and the initial DQ for a possible moderator.

The MacArthur—Bates Communicative Developmental Inventories gather information from caregivers about expressive and receptive words observed in their children in the recent past.²⁵ The expressive vocabulary lists from the Infant and Toddler inventories were combined into 1 caregiver checklist, and endorsed items were summed to form the expressive raw score variable.

The Vineland Adaptive Behavior Scales—Second Edition is a standardized caregiver questionnaire that gathers information across 4 domains: expressive and receptive language, daily living skills, and motor skills. Caregivers completed the caregiver form during assessment visits.²⁶

The overall developmental quotient (DQ) was constructed by averaging the age equivalents of the expressive and receptive language, fine motor, and visual perception subscales from the MSEL and dividing by chronological age.

Measure of Intervention Hours Received Outside of the Project

The amount of therapeutic intervention of all types was examined at each timepoint and calculated as per procedures described by Rogers et al.³ There were no significant differences in number of treatment hours delivered across the sites and across the 4 randomized experimental groups. Groups received very little outside treatment weekly in year 1, as shown in Table 2, averaging only 1 to 2 hours of additional treatment per week. In year 2, after project-delivered treatment had ended, there were no significant differences in treatment received across the 4 randomized experimental groups. There was, however, a significant difference in number of treatment hours received in year 2 across the sites.

TABLE 2.

Demographic Characteristics of the Participants

Variable	Total	Site A	Site B	Site C	Statistics	EIBI, 25 h	EIBI, 15 h	ESDM, 25 h	ESDM, 15 h	Statistic
Race, n (%)					p = .04					p = .93
African american	8.0% (7)	10.7% (3)	6.7% (2)	6.9% (2)		8.7% (2)	13.6% (3)	4.8% (1)	4.8% (1)
American indian / alaska native	1.1% (1)	3.6% (1)	0.0% (0)	0.0% (0)		4.3% (1)	0.0% (0)	0.0% (0)	0.0% (0)
Asian	10.3% (9)	7.1% (2)	23.3% (7)	0.0% (0)		4.3% (1)	13.6% (3)	9.5% (2)	14.3% (3)
White	54.0% (47)	39.3% (11)	50.0% (15)	72.4% (21)		60.9% (14)	40.9% (9)	57.1% (12)	57.1% (12)
Multiracial	23.0% (20)	32.1% (9)	20.0% (6)	17.2% (5)		17.4% (4)	27.3% (6)	23.8% (5)	23.8% (5)
Pacific Islander	1.1% (1)	3.6% (1)	0.0% (0)	0.0% (0)		0.0% (0)	0.0% (0)	4.8% (1)	0.0% (0)
Unknown	2.3% (2)	3.6% (1)	0.0% (0)	3.4% (1)		4.3% (1)	4.5% (1)	0.0% (0)	0.0% (0)
Ethnicity, n (%)					p = .17					p = .82
Hispanic/latino	19.5% (17)	28.6% (8)	16.7% (5)	13.8% (4)		17.4% (4)	18.2% (4)	19.0% (4)	23.8% (5)
Non-hispanic	73.6% (64)	60.7% (17)	83.3% (25)	75.9% (22)		69.6% (16)	72.7% (16)	76.2% (16)	76.2% (16)
Unknown	6.9% (6)	10.7% (3)	0.0% (0)	10.3% (3)		13.0% (3)	9.1% (2)	4.8% (1)	0.0% (0)
Sex, n (%)					p = .13					p = .99
Female	24.1% (21)	17.9% (5)	16.7% (5)	37.9% (11)		21.7% (5)	27.3% (6)	23.8% (5)	23.8% (5)
Male	75.9% (66)	82.1% (23)	83.3% (25)	62.1% (18)		78.3% (18)	72.7% (16)	76.2% (16)	76.2% (16)
Age, mo, mean (SD)					F2,84 = 3.6 p = .03					F3,83 = 0.9 p = .46
Time 1	23.4 (4.0)	25 (3.6)	22.7 (4.2)	22.6 (3.8)		22.6 (3.5)	23 (4.7)	24.4 (4)	23.8 (3.6)
Time 2	30.1 (4.1)	31.8 (3.6)	29.3 (4.3)	29.2 (4)		29.2 (3.8)	29.3 (4.7)	31.4 (4)	30.4 (3.8)
Time 3	36.2 (4.1)	38.1 (3.4)	35.5 (4.3)	35.2 (3.9)		35.7 (3.4)	35.2 (4.8)	37.6 (4.2)	36.5 (3.7)
Time 4	48.2 (4.3)	50.2 (3.9)	47.8 (4.4)	46.6 (3.9)		47.8 (3.8)	46.9 (5.1)	49.6 (4.4)	48.7 (3.8)
Mullen composite DQ
Time 1	64.35 (18.55)	65.89 (17.43)	73.13 (20.83)	54.36 (12.1)	F2,84 = 9.07 p < 0.001	66.96 (19.28)	64.25 (17.88)	61.36 (15.79)	65.05 (22.05)	F3,83 = 0.20 p = 0.89
Time 4	77.88 (28.18)	75.75 (23.05	88.79 (29.59)	64.05(25.41)	F2,64 = 4.98 p = 0.01	78.35(29.96)	79.44 (29.07)	73.87 (25.55)	79.5 (29.68)	F3,63 = 0.13 p = 0.94
ADOS severity score
Time 1	8.18 (1.73)	8.14 (1.75)	7.97 (1.75)	8.42 (1.7)	F2,83 = 0.57 p = .57	8.19 (1.86)	8.38 (1.28)	8.29 (1.54)	7.82 (2.22)	F3,82 = 0.76 p = 0.52
Time 4	6.78 (2.25)	6.78 (2.09)	6.42 (2.42)	7.25 (2.24)	F2,66 = 0.75 p = 0.47	6.68 (2.06)	6.94 (2.54)	6.35 (2.37)	7.19 (2.17)	F3,65 = 0.41 p = 0.75
Outside Tx (avg h/wk) year 1	1.90 (3.89)	2.18 (2.88)	1.57 (3.41)	1.97 (5.12)	F2,84 = 0.18 p = .84	0.91 (2.17)	2.43 (3.94)	2.74 (5.87)	1.58 (2.62)	F3,83 = 1.01 p = .39
Outside Tx (avg h/wk) year 2	3.18 (6.77)	5.98 (9.78)	1.45 (3.91)	2.26 (4.56)	F2,84=3.88 p = 0.02	3.53 (5.93)	4.20 (7.98)	3.13 (8.84)	1.78 (3.15)	F3,83=0.48 p = 0.70
Income, $, mean (SD)	89439.7	81018.5	99137.9	62500	F2,55 = 2.88	85000	91428.6	96833.3	84285.7	F3,54 = 0.4
	(33573.8)	(34802)	(30128.3)	(35355.3)	p = .06	(29730.9)	(36554.9)	(32956.5)	(36956.7)	p = .72
Mother’s education, y, mean (SD)	16.2 (2.7)	16.8 (2.8)	17.1 (2.4)	14.8 (2.3)	F2.81 = 7.1 p = .001	15.4 (2.8)	16.6 (3.1)	16.3 (2)	16.6 (2.8)	F3,8O = 0.9 p = .44

Note: ADOS = Autism Diagnostic Observation Schedule; avg = average: DQ = developmental quotient; EIBI = early intensive behavioral intervention; ESDM = Early Start Denver Model; TX = treatment.

Measure of Caregiver Satisfaction With Treatment

The TADPOLE Project Intervention Evaluation for Parents (Estes A, 2013, TADPOLE Project Intervention Evaluation for Parents) was a 16-item Likert-based caregiver satisfaction questionnaire constructed to capture caregiver experiences across the 2 different treatment styles and intensities and used to examine caregiver acceptability of each treatment and intensity. Total raw score was used to quantify caregiver satisfaction with the treatment that their child received.

Analysis Plan

The 4 primary outcomes (ie, autism severity, receptive language, expressive communication and nonverbal development) were analyzed using a general linear mixed model (GLMM). The GLMM is a full information method that included information from all randomized (intent-to-treat) participants, including those with only partial data because of drop-out or other reasons. We modeled the participants’ development over time by using a linear spline multilevel model to account for the specific structure of the data, involving an observation range of 24 months that included 2 distinct phases: an active treatment phase from the beginning of the study to month 12, and a follow-up phase from month 12 to the end of the study at month 24. The linear spine model simultaneously modeled an overall linear trajectory from the beginning of the study to the end of the study, and also a deviation term that modeled the degree to which the transition from active treatment to the follow-up period led to changes in the trajectories. We used a random intercept to account for individual differences in the dependent variable at baseline. Site was included as a covariate when that predictor and interactions with that predictor were significant. Time was parameterized as months since study entry was exact for each participant so that potential effects due to assessment appointments that were not exactly on schedule were included.

The 2 independent variables in this model were treatment intensity (high vs low number of hours) and treatment style (ESDM vs EIBI). A second set of analyses addressed whether effects of these treatment characteristics were moderated by baseline DQ or autism severity. Because of the complex model and the sample size, we carried out a hierarchical series of models that included site as a predictor only if site or if interactions with site were significant in the primary analyses. A final set of analyses were conducted to help interpret higher-order statistical interactions.

RESULTS

Analytic Methods and Baseline Data

Baseline Data.

Baseline data for all demographic measures are presented in Table 2. There were significant site differences on several variables: child age, race, DQ, and maternal education, as expected given geographic locations. However, there were no significant differences among the 4 randomized groups on any variable.

Change on Outcome Measures.

As seen in Table 3, all groups of children made significant gains on all composite outcome variables regardless of assignment group. Effect sizes (ie, Cohen’s d) for the within-group change ranged from |0.5| to |2.2|. All mean change scores were significantly higher than zero.

TABLE 3.

Means, SDs, and Effect Sizes of the Primary and Moderating Variables by Site for the 4 Treatment Groups, Including Effect Sizes at All Time Points

Composite variable label	Time point	EIBI, 25 h	EIBI, 15 h	ESDM, 25 h	ESDM, 15 h
Aut_sx_comp	T1	1.27 (0.71)	1.07 (0.57)	1.24 (0.46)	1.08 (0.81)
Expr_com_comp	T1	−1.54 (0.27)	−1.34 (0.41)	−1.50 (0.22)	−1.48 (0.29)
Nv_dev_comp	T1	−1.52 (0.34)	−1.43 (0.42)	−1.45 (0.33)	−1.45 (0.24)
Recp_lang_comp	T1	−1.75 (0.41)	−1.62 (0.51)	−1.63 (0.47)	−1.55 (0.43)
Aut_sx_comp	T2	0.83 (0.81)	0.43 (0.74)	0.45 (0.92)	0.37 (0.72)
Expr_com_comp	T2	−1.13 (0.65)	−0.80 (0.48)	−0.99 (0.58)	−0.92 (0.57)
Nvdev_comp	T2	−1.00 (0.46)	−0.90 (0.45)	−0.85 (0.51)	−0.94 (0.33)
Recp_lang_comp	T2	−0.97 (0.81)	−0.84 (0.63)	−0.90 (0.67)	−0.87 (0.69)
Aut_sx_comp	T3	0.27 (0.69)	0.24 (0.79)	0.24 (0.90)	0.22 (0.56)
Expr_com_comp	T3	−0.69 (0.72)	−0.41 (0.62)	−0.65 (0.68)	−0.51 (0.61)
Nvdev_comp	T3	−0.58 (0.63)	−0.51 (0.65)	−0.59 (0.63)	−0.51 (0.45)
Recp_lang_comp	T3	−0.51 (0.81)	−0.41 (0.54)	−0.50 (0.85)	−0.47 (0.61)
Aut_sx_comp	T4	0.03 (0.93)	−0.11 (0.96)	−0.07 (0.94)	0.01 (0.85)
Expr_com_comp	T4	−0.01 (1.09)	0.07 (0.88)	−0.20 (0.95)	−0.00 (0.81)
Nvdev_comp	T4	0.03 (0.86)	0.04 (1.12)	−0.14 (0.76)	0.04 (0.71)
Recp_lang_comp	T4	−0.03 (0.95)	0.04 (0.99)	−0.09 (0.93)	−0.03 (0.93)

Note: All composite scores are based on the time 4 reference point. Aut_sx_comp = Autism symptomology composite; EIBI = early intensive behavioral intervention; ESDM = Early Start Denver Model; Expr_lang_comp = expressive language composite: Nv_dev_comp = Nonverbal developmental composite: Recp_lang_comp = receptive language composite.

Effects of Treatment Style

Overall, there was no evidence that treatment style had effects on the trajectories for any of the dependent variables over time: autism severity (overall trajectory: F_1,225 = 0.9, p = .76), spline: F_1,224 = 0.2, p = .16)); expressive language (overall trajectory: F_1,224 = 0.2, p =.78, spline: F_1,223 = 0.5, p = .48)); receptive language (overall trajectory: F_1,224 = 0.3, p = .59, spline: F_1,223 = 0.0, p = .87)); and nonverbal ability (overall trajectory: F_1,222 = 1.7, p = .18, spline: F_1,221 = 0.4,p = .67)). There was no significant interaction of style × time × site.

Moderating Effects of Initial Severity on Effect of Treatment Style

Baseline autism severity did not moderate the effects of treatment style on trajectories of any dependent variable: autism severity (overall: F_1,228 = 0.1, p = .77, spline: F_1,228 = 0.0, p = .92); expressive language (overall: F_1,226 = 0.3, p =.58, spline: F_1,226 = 0.0, p = .98); receptive language (overall: F_1,223 = 2.8, p = .09, spline: F_1,223 = 1.0, p = .32); and nonverbal ability (overall: F_1,224 = 0.0, p = .83, spline: F_1,224 = 0.8, p = .37).

Similarly, baseline DQ did not moderate the effect of treatment style on trajectories of the dependent variables: autism severity (overall: F_1,231 = 0.5, p =.46, spline: F_1,229 = 0.7, p = .74); expressive language (overall: F_1,229 = 0.2, p = .66, spline: F_1,228 = 0.1, p = .90); receptive language (overall: F_1,229 = 0.7, p = .42, spline: F_1,227 = 0.9, p = .33); and nonverbal ability (overall: F_1,227 = 0.2, p = .42, spline: F_1,226 = 0.0, p = .97).

Effects of Treatment Intensity

Overall, there was no main effect of treatment intensity on trajectories of any dependent variable: expressive language (overall change: F_1,225 = 0.8, p = .36, spline: F_1,223 = 0.7, p = .36); receptive language (overall change: F_1,222 = 0.0, p = .96, spline: F_1,221 = 0.0, p = .92); nonverbal ability (overall change: F_1,223 = 0.4, p = .54, spline: F_1,211 = 0.4, p = .69); or autism severity (F_1,226 = 0.8, p = .38, spline F_2,224 = 0.1, p = .78).

For autism severity, but not for the 3 developmental variables, there was a significant higher-order interaction with time and site (F_2,226 = 4.1, p = .02). When probing this 3-way interaction effect, we found that site 1 showed no time × intensity interaction effect on autism severity (F_1,230 = 0.0, p = .87); site 2 showed a significant time × intensity interaction effect on autism severity (F_1,223 = 8.0 with greater improvement resulting from higher intensity intervention; and site 3 showed no time × intensity interaction effect on autism severity (F_1,225 = 1.2, p = .28).

Moderating Effects of Initial Severity on Effect of Treatment Intensity

Baseline autism severity did not moderate the effects of treatment intensity on the trajectories of the 3 developmental dependent variables: expressive language (overall: F_1,226 = 1.1, p = .30, spline: F_1,226 = 0.6, p = .44); receptive language (overall: F_1,223 = 0.0, p = .93, spline F_1,223 = 1.3, p = .26); and nonverbal ability (overall: F_1,224 = 0.0, p = .84, spline: F_2,223 = 1.6, p = .21).

Similarly, baseline DQ did not moderate effects of treatment intensity on the trajectories of the 3 developmental dependent variables: expressive language (overall: F_1,229 = 3.5, p =.06, spline: F_1,227 = 0.8, p = .37); receptive language (overall: F_1,228 = 1.1, p = .30, spline: F_1,227 = 1.4, p = .24); and nonverbal ability (overall: F_1,228 = 1.1, p = .30, spline: F_1,227 = 1.4, p = .24).

In examining moderation effects on autism severity trajectory, we included site in the analyses because of the significant effects of site that we described above in the primary analyses of intensity effects. When exploring baseline DQ as a moderator to the primary model, we found significant main effects of treatment intensity on the trajectory of autism severity (overall: F_1,226 = 5.1, p = .02, spline: F_1,222 = 3.9, p = .05). This effect is conditional on the significant moderation effect of DQ (time × intensity × baseline DQ) when predicting the effects of intensity on the trajectory of autism severity (overall: F_1,226 = 6.5, p = .01, spline: F_1,222 = 4.2, p = .04).

Similarly, when exploring baseline autism severity as a moderator to the primary model, we found significant main effects of treatment intensity on the trajectory of autism severity (F_1,221 = 4.1, p = .04). This effect is conditional on the significant moderation effect of baseline autism severity (time × intensity × baseline autism severity) when predicting the effects of intensity on the trajectory of autism severity (F_1,221 = 4.5, p = .03). However, increasing the complexity of these results, there is also a significant 3-way moderation among site, baseline autism severity, and time. That is, the moderation effects of baseline autism severity on the effects of treatment intensity on trajectory of autism severity differed by site (F_2,226 = 3.6, p = .03). We did not probe these site effects, because of lack of power (see Supplement 4,available online). The model is shown in Table 4.

TABLE 4.

The Moderation Model: Testing Moderation of Initial Autism Severity on Effects of Treatment Intensity on Trajectory of Change in Autism Severity Over 24 Months

Dependent variable: autism severity
Type III tests of fixed effects
Effect	DFb	DFw	F	p	f^2
Intercept	1	146	0.0	.84	0.00
Slope	1	221	6.1	.02	0.03
Spline	1	221	0.5	.50	0.00
Intensity	1	146	0.0	.98	0.00
Intensity * slope	1	221	4.1	.04	0.02
Intensity * spline	1	221	2.0	.16	0.01
Site	2	151	0.2	.84	0.00
Site * slope	2	226	2.3	.10	0.02
Site * spline	2	224	1.9	.16	0.02
Intensity * site	2	151	0.1	.91	0.00
Intensity * site * slope	2	226	3.6	.03	0.03
Intensity * site * spline	2	224	2.0	.14	0.02
Autism severity	1	146	97.3	<.01	0.67
Slope * autism severity	1	221	19.6	<.01	0.09
Spline * autism severity	1	221	15.3	<.01	0.07
Intensity * autism severity	1	146	0.0	.94	0.00
Intensity * slope * autism severity	1	221	4.5	.04	0.02
Intensity * spline * autism severity	1	221	3.1	.08	0.01

Note: Highlighted rows indicate significant interaction effects.

Caregiver Satisfaction

Caregivers were highly satisfied with the treatment their children received, regardless of the style or intensity their child was assigned to. With “5” meaning “highly satisfied” on the caregiver satisfaction measure, means for the 4 cells were 4.7 (SD = 0.31 for EIBI low intensity), 4.8 (SD = 0.34 for EIBI high intensity), 4.8 (SD = 0.30 for ESDM low intensity), and 4.8 (SD = 0.19 ESDM high). Means were not significantly different among groups (F_1,78 = 0.2, p = .73).

DISCUSSION

Significance of the Main Findings

Although the science of early intervention for autism has a long history, dating back to the 1960s, and has been dominated by 2 different treatment approaches, those derived from the principles of ABA, and those derived from the principles of developmental science, few studies have been designed to provide answers to the questions that parents receiving a first diagnosis typically ask: what kind of treatment should I seek, and for how many hours per week? Previous studies have documented that both types of treatment can result in significant child gains, but there is little comparative information from well-controlled studies. On the question of treatment intensity, we have no data from past experimental studies on which to base decisions; thus, recommendations for specific level of intensity have been based on assumptions, correlates, opinions, and common sense rather than on experimental evidence.

We conducted this trial to provide objective answers about relationships among initial child characteristics, treatment styles, treatment intensities, and child progress over time on key developmental and symptom domains. We used a rigorous randomized, controlled, multisite study design, rigorous fidelity of implementation methods, and outcome assessments conducted by raters naive to group membership. The 4 composite outcome variables—receptive and expressive language ability, nonverbal development, and autism severity—were composed to represent constructs that have been associated with long-term outcomes in ASD.²⁷

We hypothesized that differences in child outcomes related to treatment style and intensity would be associated with initial severity of children’s disability reflected in baseline DQ and autism symptom severity, as has been previously reported using quasi-experimental designs. The results did not support our primary hypothesis that the initial degree of developmental delay and autism severity would moderate effects of treatment style on growth trajectories of children’s outcomes. The severity of children’s initial DQ and autism symptoms did not differentially affect the outcomes of EIBI vs ESDM intervention on children’s progress in receptive language, expressive communication, nonverbal ability, and autism symptom change.

Although our rigorous fidelity of implementation methods ensured that the treatments were delivered as intended, 1 possible reason for the lack of treatment style effect is the tendency of both treatments to modify delivery style in specific, converging ways according to initial severity of disability. In the EIBI treatment, naturalistic teaching involving play, dyadic engagement, child initiation, and generalization occurred during child breaks from direct instruction. However, the direct instruction episodes were carried out at high levels of fidelity of implementation, as reported. In ESDM treatment, additional structure was added to by choosing and managing materials that would result in massed practice, by working at a table, and by adding external cues of temporal structure to work times and play times. ESDM fidelity of implementation principles was maintained by: allowing for children’s choice-making and initiation among the materials available, adult sensitivity and responsivity to child communications, adult use of developmentally appropriate language, management of child attention, modulation of child affect and arousal, use of turn-taking/dyadic engagement, adult positive affect, adult communications involving multiple pragmatic functions, smooth transitions between activities, working on language and/or social objectives within every activity, and theme and variation format within planned activities. Thus, the modifications were carried out within the basic fidelity principles and practices of each treatment.

Our second hypothesis predicted that initial degree of developmental delay and autism severity would moderate effects of treatment intensity on children’s outcomes. When examining this hypothesis, we included site as an additional moderator because we found that a significant higher-order interaction involving site predicted change in the outcome variables. The hypothesis was supported for only 1 of the 4 outcome variables, namely, change in autism severity, and it was conditional by site. We were underpowered to conduct within-site tests to determine the initial severity of disability values at which intensity levels mattered.

Does our finding indicate that 12 to 15 hours per week of comprehensive intervention is sufficient and that the National Academy of Sciences (2001) recommendation of 20+ hours is not supported?²⁸ It does not, because, for at least 1 important variable—namely, degree of autism symptoms—25 hours of intervention was found to be more efficacious than 15 hours for improving core autism symptoms in 1 site. Furthermore, the study focused on toddler-aged children, and it is possible that different findings could emerge for preschool-aged children or those with more years of intervention. Finally, because this is the first study to address these questions in a controlled fashion, replication is necessary before practice alterations might be addressed. Replications should also address the weaknesses in this study that are associated with lack of power to conduct multiple within-site tests, as well as the lack of a no-treatment group, needed to ensure that both treatments are responsible for child gains that occur. It is also important for future studies to examine the question of similarities and/or differences of underlying mechanisms of child learning involved in these 2 approaches. Further studies are needed to help us understand the relationships among individual differences in child characteristics, type of intervention delivery, and child outcomes.

Recommendations for Clinicians

Clinicians are challenged by caregivers’ questions about their young ASD children’s potential and the course of treatment to pursue. One recommendation from our data addresses caregiver questions about predicting outcomes for young children. Children receiving intensive early intervention can change substantially in their developmental rates and autism severity over time, and a better prediction of future disability will result from examining child progress and status after a few years of early childhood intervention, rather than at the time of diagnosis.

A second recommendation has to do with caregivers’ questions about recommendations for 35 to 40 hours per week of behavioral treatment. It may be reassuring for caregivers, clinicians, and service providers to know that the field lacks high-quality evidence that such a high number of hours provides greater gain for children, even those with more severe difficulties.

Finally, the “brand name” of interventions may be less important than more general characteristics of high-quality interventions, which both interventions tested in this study met: (1) uses a manualized, evidence-based approach that assesses children frequently and teaches to developmental and behavioral needs across all domains; (2) incorporates developmental and behavioral science in treatment content and delivery; (3) delivers treatment consistently at fidelity across multiple environments; and (4) integrates treatment needs into everyday activities by coaching caregivers regularly.

Perhaps the most important message that clinicians can provide early on is that, for both people with ASD and people without disability, quality of life across the lifespan is not determined by scores on tests, years of education, or income, but rather by quality of relationships with others and of engagement in personally satisfying activities, both at work and in play.