A randomized controlled trial of Hanen’s ‘More Than Words’ in toddlers with early autism symptoms

Abstract

Background

This randomized controlled trial compared Hanen’s ‘More than Words’ (HMTW), a parent-implemented intervention, to a ‘business as usual’ control group.

Methods

Sixty-two children (51 boys and 11 girls; M age = 20 months; SD = 2.6) who met criteria for autism spectrum disorders (ASD) and their parents participated in the study. The HMTW intervention was provided over 3.5 months. There were three measurement periods: prior to randomization (Time 1) and at 5 and 9 months post enrollment (Times 2 and 3). Children’s communication and parental responsivity were measured at each time point. Children’s object interest, a putative moderator, was measured at Time 1.

Results

There were no main effects of the HMTW intervention on either parental responsivity or children’s communication. However, the effects on residualized gains in parental responsivity from Time 1 to both Times 2 and 3 yielded noteworthy effect sizes (Glass’s Δ = .71, .50 respectively). In contrast, there were treatment effects on child communication gains to Time 3 that were moderated by children’s Time 1 object interest. Children with lower levels of Time 1 object interest exhibited facilitated growth in communication; children with higher levels of object interest exhibited growth attenuation.

Conclusions

The HMTW intervention showed differential effects on child communication depending on a baseline child factor. HMTW facilitated communication in children with lower levels of Time 1 object interest. Parents of children who evidence higher object interest may require greater support to implement the HMTW strategies, or may require different strategies than those provided by the HMTW curriculum.

There is increasing evidence that early intervention improves outcomes for children with autism spectrum disorders (ASD) (Rogers, 1998; Smith, Groen, & Wynn, 2000; Vismara & Rogers, 2010). To date, however, very few intervention studies have focused on the toddler period, with positive findings for the efficacy of the Early Start Denver Model (ESDM) providing a notable exception (Dawson et al., 2010; Vismara & Rogers, 2008). Given the intensity of the ESDM (15 hours a week of professional services for 2 years), less expensive alternatives, specifically those focusing on parent-implemented early interventions, are needed. Moreover, given the variability in treatment response across studies of children with ASD (Vismara & Rogers, 2010; Yoder & Stone, 2006a, 2006b), it is critical to determine the characteristics of subgroups of children who benefit most from specific interventions.

In contrast to interventions for preschool-aged and older children with ASD, there is consensus that parents should be involved in interventions designed for infants and toddlers (Meadan, Ostrosky, Zaghlawan, & Yu, 2009; Zwaigenbaum et al., 2009). This view is consistent with broader best practice for early intervention, which involves working with children in their natural environments (Dunst, Hamby, Trivette, Raab, & Bruder, 2002). Specifically, there is recognition that parent involvement can capitalize on teachable moments as they occur, provide learning opportunities during naturally occurring routines, and facilitate the generalization of child learning across contexts. Additionally, parent-mediated intervention has the potential to be relatively inexpensive and to increase parents’ sense of efficacy and empowerment.

One putative outcome of parent-mediated intervention is improvement in children’s communication. Communicative and social deficits are not only core to ASD, but are among the first symptoms observed (Zwaigenbaum et al., 2009). Further, because communicative competence in early childhood is associated with positive long-term outcomes (e.g., Billstedt, Gillberg, & Gillberg, 2007), we were particularly interested in an intervention that was focused on communication.

Hanen’s ‘More than Words’ (HMTW) is a parent training program that provides support, education, and practical skills for enhancing communication in children with ASD. The HMTW program consists of eight weekly group sessions and three individual family sessions, designed to increase the frequency of playful parent–child interactions and facilitate child communicative development. The curriculum, administered by a speech and language therapist, is designed to teach parents to (a) structure everyday routines in a manner that is sensitive to the child’s developmental level and provides opportunities for the child to initiate or respond, and (b) provide linguistic and nonlinguistic responses to children’s communication. Two prior studies have evaluated this program. The first was a clinical case series with parents of 3 preschool-aged children with ASD (Girolametto, Sussman, & Weitzman, 2007) and the second a quasi-experimental study of 51 preschoolers with language delay and suspicion of ASD (McConachie, Randle, Hammal, & Le Couteur, 2005). Both studies reported positive gains in parenting behaviors. The case series reported a range of positive outcomes for children and the quasi-experimental study reported positive gains in observed parenting behaviors and parents’ reports of the number of words children used.

The goals of the present study were to test whether participation in the HMTW program: (a) enhanced parental responsivity to toddler’s actions, their focus of attention, and their communication (i.e., behaviors that are emphasized in the HMTW intervention); and (b) increased communication in toddlers with symptoms consistent with ASD. In addition, given the importance of determining which treatments may be most effective for subgroups of children with ASD exhibiting particular characteristics, we explored the potential moderating role of two pretreatment child characteristics, communication and object interest, both of which have been found to be moderators in a previous child-centered treatment with preschool children with autism (Yoder & Stone, 2006a, 2006b).

Given that HMTW is a parent-implemented intervention, we anticipated that at the end of the intervention period (Time 2), parents assigned to HMTW would show greater responsivity relative to controls, and that this difference would be maintained at Time 3. Moreover, we hypothesized that HMTW would facilitate children’s communication at the Time 3 assessment, after children had had time to benefit from parents’ use of HMTW strategies.

Methods

Participants

Sixty-two children (51 boys and 11 girls) and their parents participated in the study. The mean child age at enrollment was 20.25 months (SD = 2.6; range = 15–25). Seven children (11.3%) had older siblings previously diagnosed with ASD. As shown in Figure 1, 165 families were screened for study inclusion. The study was conducted at three project sites (cities in the south, southeast, and northeast United States) and families were recruited from ASD specialty clinics, early intervention programs, pediatric and neurology practices, and the online Interactive Autism Network. Initial phone screening information determined 68 families to be ineligible (e.g., child older than 24 months of age, having a genetic disorder). The remaining 97 families were scheduled for an initial in-person evaluation in a clinic setting. The initial evaluation began with the Screening Tool for Autism in Two-year-olds (STAT; Stone, Coonrod, & Ousley, 2000; Stone, Coonrod, Turner, & Pozdol, 2004; Stone et al., 2008). Families of children who did not obtain a predetermined ‘at-risk’ score on the STAT and/or did not meet symptom criteria for an ASD based on expert clinical impression were excluded (n = 32). One child was excluded after the in-person evaluation because of a late disclosure of a Fragile X diagnosis and two families whose children met enrollment criteria were not randomized and did not participate due to family choice. Thus, 62 families were randomized and those retained were included in ‘partial’ intent to treat analyses. Families were randomized within each site using software that utilized a random number generator to assign children to the intervention or control group. The software weighted the probability of assigning the family to groups based on the relative proportion of the intervention and control group sample sizes at the time of assignment. All participants with pre-treatment data and data from at least one of the follow-up periods were included in analyses, irrespective of HMTW treatment attendance. However, we use the term ‘partial’ intent to treat because ‘full’ intent to treat requires imputation of all missing data. We elected not to carry forward values from earlier periods to later periods when participants were missing Time 1 data for a particular procedure or to use multiple imputation. Because a large minority of participants had missing scores for several variables, estimation would have increased the risk of producing non-replicable results.

Figure 1.

Participant recruitment, enrollment, randomization, and retention

As shown in Table 1, there was considerable heterogeneity with respect to developmental functioning on the Mullen Scales of Early Learning (MSEL; Mullen, 1995) and Vineland Adaptive Behavior Scales Second Edition (Vineland II; Sparrow, Cicchetti, & Balla, 2005) at Times 1 and 3 as well as on the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2000) at Time 3. The sample was also diverse with respect to sociodemographic characteristics. Sixteen percent of parents had no more than a high school education, 33% had some college coursework, an associate’s degree, or vocational/trade degree, 35% had a college degree, and 16% had advanced degrees. Parents identified 47.4% of children as White, 38.6% as Hispanic or Latino, 3.5% as Black, 5.3% as Asian/White, 3.5% as American Indian/Alaskan Native/White and 1.8% as American Indian/Alaskan Native/Hispanic. Ninety-two percent of children who received Time 3 evaluations (46/50) met criteria for an ASD based on both the ADOS and DSM-IV-based clinical impression.

Table 1.

Clinical characteristics of ‘partial’ intent to treat sample

	HMTW intervention		Control
	Time 1 n = 32	Time 3 n = 25	Time 1 n = 30	Time 3 n = 25
	M ± SD (Range)	M ± SD (Range)	M ± SD (Range)	M ± SD (Range)
CA (months)	21.11 ± 2.71 (15.47–24.84)	29.98 ± 2.74 (24.44–34.60)	21.51 ± 2.82 (16.30–24.97)	30.72 ± 2.80 (25.79–34.73)
Mullen Expressive Language Age (mos)	8.22 ± 6.01 (1.00–29.00)	16.20 ± 7.23 (5.00–35.00)	7.33 ± 3.71 (1.00–16.00)	16.68 ± 7.88 (5.00–29.00)
Mullen Receptive Language Age (mos)	8.41 ± 5.42 (1.00–27.00)	15.52 ± 6.93 (7.00–30.00)	8.17 ± 4.44 (1.00–24.00)	17.48 ± 8.33 (4.00–33.00)
Mullen Visual Reception Age (mos)		22.42 ± 5.75a (7.00–31.00)		21.64 ± 6.53 (5.00–33.00)
Mullen Fine Motor Age (mos)		22.00 ± 3.50 (16.00–31.00)		21.92 ± 4.09 (14.00–30.00)
Mullen Early Learning Composite (ELC)		63.88 ± 18.41a (49.00–129.00)		64.88 ± 13.94 (49.00–91.00)
Vineland Socialization SS	73.95 ± 6.46b (63.00–94.00)	71.42 ± 7.07c (61.00–87.00)	72.42 ± 6.59a (56.00–88.00)	70.70 ± 6.89d (61.00–89.00)
Vineland Communication SS	66.61 ± 12.87e (50.0–101.0)	76.14 ± 13.85f (56.0–107.0)	63.21 ± 9.13a (44.00–84.00)	76.43 ± 14.05f (50.00–97.00)
Vineland Motor SS		83.16 ± 7.36g (72.00–96.00)		81.55 ± 9.26d (59.00–93.00)
Vineland Daily Living SS		77.84 ± 7.07g (64.00–95.00)		72.95 ± 10.11d (53.00–93.00)
ADOS Social-Communication Total		15.56 ± 4.56 (7.00–21.00)		13.60 ± 4.89 (4.00–21.00)

Note:

^an = 24,

^bn = 22,

^cn = 19,

^dn = 20,

^en = 23,

^fn = 21,

^gn = 19, SS = Standard Score.

Procedures

All parents provided informed consent. Children meeting enrollment criteria participated in a Time 1 visit consisting of a Developmental Play Assessment (DPA; Lifter, 2000), the Early Social Communication Scales (ESCS; Mundy et al., 2003) and a Parent–Child Free Play procedure (PCFP), all of which were filmed for later coding. The MSEL Expressive and Receptive Language Scales were also administered. Parents were interviewed with the Vineland II Communication and Socialization domains and asked to complete a packet of questionnaires that included questions about family demographics, participation in ‘business as usual’ interventions, and the Parent Interview for Autism – Clinical Version (PIA-CV; Stone, Coonrod, Pozdol, & Turner, 2003). The PCFP procedure was completed again at Times 2 and 3, which occurred approximately 5 (M = 5.3, SD = .47 months) and 9 months (M = 9.3, SD = .56 months) after the Time 1 visit, respectively. The ESCS and PIA-CV were repeated at Times 2 and 3. Additional measures administered at the Time 3 visit included the full MSEL, full Vineland II, and the ADOS. Finally, a clinical impression that the child met DSM-IV symptom criteria for autistic disorder or pervasive developmental disorder – not otherwise specified (PDD-NOS) was made at Times 1 and 3 by a clinical psychologist familiar with ASD in early childhood. The clinical psychologist administered or observed the STAT and other portions of the Time 1 evaluation session and often spoke to parents about observed symptoms and experiences of their child at home. Parents assigned to the HMTW intervention completed questionnaires rating their group leader as well as cohesion and support within their treatment group following the treatment phase. Finally, parents in both groups reported on the number of hours their child attended non-project interventions at all three periods.

As seen in Figure 1, of the 62 participants randomized, 32 were assigned to the intervention group (29 of whom had at least one outcome measure at Time 2 or 3), and 30 were assigned to the no treatment group (26 of whom had at least one outcome measure at Time 2 or 3). Thus a maximum of 55 participants with follow-up data were included in analyses (89% retention). Of the 29 participants retained in the intervention group, 5 participated in fewer than 9 of the 11 individual and group HMTW sessions. Following intent to treat analysis principles, these 5 participants were included in analyses when their outcome data were available.

Hanen’s ‘More Than Words’ intervention: goals and content

Hanen’s ‘More Than Words’ (HMTW) is a parent training program designed to teach parents of young children with ASD practical strategies to use during everyday routines to increase children’s communication. HMTW involves eight group sessions with parents only and three in-home individualized parent–child sessions interspersed; missed group sessions were not made up. Group and individual sessions are provided by a speech/language pathologist certified by the Hanen Centre. The strategies parents are taught are drawn from current research on enhancing social communication in children with ASD and focus on helping children reach the following four goals: (a) improved two-way interaction, (b) more mature and conventional ways of communicating, (c) better skills in communicating for social purposes, and (d) improved understanding of language. HMTW incorporates current best practice guidelines, highlighting the importance of affect, predictability, structure and the use of visual supports to enhance learning in children with ASD. Sessions cover early child communication development and parental interaction styles thought to enhance child communication, including responding to the child’s communicative attempts, following the child’s lead, building and participating in joint action routines in play, enhancing interaction during caregiving routines, using books and play as contexts for communication elicitation and reward, using visual supports to help children understand expectations, supporting peer interactions, and scaffolding peer play dates. Overall, many of these strategies involve enhancing responsivity to children’s attention and communication attempts. The individual sessions incorporate video-feedback and are designed to help the parent implement HMTW strategies with their children.

Measures

The Screening Tool for Autism in Two-Year-Olds (STAT; Stone et al., 2000, 2004, 2008) consists of 12 interactive items which assess the behavioral domains of Play, Requesting, Directing Attention, and Motor Imitation. Overall STAT scores range from 0 to 4, with lower scores indicating less impairment. A liberal threshold for classification as ‘high risk’ for autism (i.e., a score of 2.25) was used in this study for children under 24 months old and the standard STAT cutoff of 2 was used for children who were 24 months old (Stone et al., 2004).

The Mullen Scales of Early Learning (Mullen, 1995) provided an overall developmental composite score (Early Learning Composite) that was used at Time 3 to characterize the sample. To minimize participant burden, only the Expressive and Receptive Language scores were administered at Time 1. Expressive and Receptive scores were used to evaluate group comparability.

The Vineland Adaptive Behavior Scales, Second Edition (Vineland II; Sparrow et al., 2005) were used to assess adaptive skills. The Socialization and Communication domains were administered at Time 1 to evaluate group comparability. The full Vineland II was administered at Time 3 to characterize the sample.

The Autism Diagnostic Observational Schedule (ADOS; Lord et al., 2000) is a semi-structured, interactive observation designed to assess social and communicative functioning in individuals suspected of having an ASD. All children received Module 1 (preverbal or single words) at Time 3 to inform ASD diagnosis and provide clinical characterization.

Parent Interview for Autism–Clinical Version (PIA-CV; Stone et al., 2003) is a measure of autism symptom severity. The original PIA (Stone & Hogan, 1993) demonstrated strong psychometric properties for children between 20 months and 5 years, 11 months. The questionnaire version of the Nonverbal Communication domain was used in this study. Residualized gains from Time 1 to Time 2 and Time 1 to Time 3 on this domain (13 items) were included as child communication outcomes. The internal consistency of the Nonverbal Communication domain in this sample ranged from Cronbach’s alpha of .85 to .87, depending on the assessment time point.

The Early Social Communication Scales (ESCS) – Abridged (Mundy et al., 2003) are designed to assess nonverbal communicative behaviors in young children between 8 and 30 months of age. The ESCS was used to measure Initiating Joint Attention (IJA) and Initiating Behavior Requesting (IBR) (Mundy et al., 2007) at each time point. Residualized gains on these variables were analyzed as child communication outcomes. Time 1 IJA and IBR frequencies were analyzed as putative moderators.

IJA was coded when the child made eye contact with the examiner while holding a toy or watching an active toy, pointed to an object of interest (with or without eye contact), or showed a toy to the examiner by holding it up toward the examiner with eye contact. IBR was coded when the child made eye contact with the examiner while watching a distal and inactive toy, reached for a toy (with or without eye contact), pointed to a desired object (with or without eye contact), or gave a toy to the examiner (with or without eye contact).

Coding for all videotaped procedures was completed by coders who were blind to treatment group using software that enables computer control of digital recordings (ProcoderDV; Tapp & Walden, 1993). Interobserver reliability was estimated through blind, independent codings of a random selection of approximately 20% of the sessions at each time period. The intra-class correlation coefficients (ICCs) for number of IJA and IBR were .93 and .68 at Time 1, .95 and .96 at Time 2, and .95 and .99 at Time 3, respectively.

The Parent–Child Free Play Procedure (PCFP) was used at each time point to measure child and parent behaviors. The PCFP was divided into two parts: (a) a 10-minute toy play segment, during which caregivers were encouraged to offer the toys that their child would enjoy most from a basket of developmentally appropriate toys; and (b) a 5-minute book-sharing segment during which toys were removed and caregivers looked at three books with their child. Parents and children were seated adjacent to one another at a child-sized table positioned in the corner of the room to facilitate reliable coding of children’s nonverbal communication and caregivers’ responses.

PCFP sessions were coded for parental responsivity using a partial interval coding system with 5-second intervals. In partial interval coding, the occurrence of a behavior is coded if it occurs at any time during the interval. The sum of codable intervals, defined as an interval in which the child was visible and productively engaged with onscreen referents for at least 1 second, was computed. Nonverbal responsivity was coded when a parent aided the child in his/her play, performed the same action as the child with a similar object, expanded on the child’s play, or responded to a child’s request. Verbal responsivity was coded when a parent described or talked about the child’s current focus of attention, or verbally expanded upon a child’s communication act, without directing the child’s behavior. The responsivity variable was the proportion of codable intervals with parental nonverbal or verbal responsivity. Residualized gains on this variable from Time 1 to Time 2 and from Time 1 to Time 3 were analyzed as the parenting outcome. Reliability (ICC) was estimated through blind, independent codings of a random selection of approximately 20% of the sessions at each time period. The ICCs on proportion of codable intervals with a parental response were .46 at Time 1, .84 at Time 2, and .75 at Time 3. The Time 1 ICC was relatively low because between-subject variability was lower at Time 1 (SD = .05) compared to that at Time 2 (SD = .09) and Time 3 (SD = .09) and ICC reflects between-subject variability as well as within-subject agreement between observers (Yoder & Symons, 2010). Mean inter-observer occurrence agreement estimates on intervals with responsivity were .89 (SD = .047) at Time 1.

PCFP sessions were also coded for the weighted frequency of child intentional communication at Times 1, 2 and 3. Child intentional communication was based on the occurrence of one of three classes of behavior: 1) gestures or nonword vocalizations during which the child coordinated attention between the message recipient and an object or salient event; 2) conventional gestures (e.g., distal points, head nods, pantomime) with attention to an adult; and 3) symbols (i.e., spoken words or signs) that were used in a non-imitative manner. The Weighted Frequency of Intentional Communication was obtained by multiplying each intentional communication act by the following weights: nonverbal = 1; single symbol = 2; and multiple symbols = 3. Previous research has indicated that the weighted variable is more sensitive to change over time than the unweighted variable and that growth in the weighted variable (but not the unweighted variable) was predictive of later social impairment in younger siblings of children with ASD (Yoder, Stone, Walden, & Malesa, 2009). The ICCs for weighted triadic communication were based on blind, independent codings of a random selection of approximately 20% of the sessions at each time period and were .95, .97, and .99 for Times 1, 2, and 3, respectively.

The Developmental Play Assessment (DPA) was adapted from Lifter (2000) to measure children’s object interactions. An examiner presented two standard sets of toys within the child’s reach for equal time intervals within a 7-minute free-play session. To maintain child engagement, the examiner imitated the child’s actions and/or provided verbal description of the child’s actions. However, the examiner did not model new acts, expand upon the child’s current actions, or provide verbal prompts to perform new actions.

The number of toys with which children used differentiated play at Time 1 was our measure of object interest (Yoder & Stone, 2006b) and a putative moderator of treatment effects on child communication gains. In our past research with preschoolers with ASD, this measure of object interest moderated treatment effects on expressive communication (Yoder & Stone, 2006b). Differentiated play was coded when the child engaged in play action included in a list of anticipated actions that was developed for each toy prior to the study onset (Yoder & Stone, 2006a). Mouthing, banging, shaking, or close inspection was rated ‘Undifferentiated play.’ Interobserver reliability was estimated through blind, independent codings of a random selection of approximately 20% of the sessions at Time 1 (ICC = .87).

Fidelity of Treatment Implementation (FOT) was conceived as adherence to Hanen-recommended content, quality of teaching style, and group size. Checklists were developed for this study to rate the speech and language pathologists’ administration of the HMTW intervention, in consultation from the authors of Hanen’s ‘More Than Words.’ Checklists were completed by speech language pathologists for 97% of the group sessions and 78% of the individual sessions. In addition, a random sample of the checklists from the group (23%) and individual (34.5%) sessions were rated by a second observer to estimate inter-observer agreement. Mean item-by-item agreement between the group leaders and reliability observers was 92% (SD = 10) for the group sessions and 92% (SD = 11) for the individual sessions across sites, suggesting that group leaders were able to reliably assess session fidelity. Across sites, HMTW was implemented with 88% (SD = 4.7) of intended elements present in the group sessions and 89.9% (SD = 7.9) of intended elements in the individual sessions. To ‘fill out’ group sessions, families for whom HMTW seemed appropriate and who did not qualify for the study (e.g., children older than 24 months) were invited to join HMTW groups. Owing to recruitment challenges, the mean number of families participating in each session during this study was 3.57 (SD = 2.29; range: 1–10). Thus, while content fidelity of administration was high, group composition size did not adhere to the HMTW standard of a minimum of 8 families per group.

A Consumer Satisfaction Survey was developed for this study. Caregivers randomized to the HMTW intervention were asked to complete 11 questions (on a Likert scale ranging from 1 to 6) describing their experience with the HMTW group leader (e.g., pacing, useful exercises) and 17 questions (on a Likert scale ranging from 1 to 4) describing their experience of group’s climate (i.e., cohesion, support).

A Non-Project Treatment Questionnaire was developed to gather information about the number of non-HMTW intervention hours children received per month. To assist parents in remembering all of the interventions their children might have received, parents were asked about the number of hours of different types of therapies (e.g., occupational therapy, speech and language therapy, applied behavior analysis, physical therapy) their child had received during the previous month. These were administered all periods for both groups.

Results

Preliminary analyses

A number of preliminary analyses were conducted to rule out potential threats to internal validity. First, log10 or square root transformations were applied to normalize the data when appropriate (Tabachnick & Fidell, 2006). Second, t-tests and chi square analyses were computed for all Time 1 measures of primary outcome variables, putative moderators, sociodemographic characteristics, and clinical characteristics potentially associated with outcomes (e.g., parental education, STAT scores), to examine pre-treatment HMTW intervention and ‘business as usual’ control group equivalence. There were no differences between the HMTW and ‘business as usual’ groups on sociodemographic characteristics or on any Time 1 measures of the experimental or clinical child variables (all ps > .10) (see Table 2). Third, chi square analyses comparing the percentage of children and parents with analyzable data were conducted for each study outcome to determine whether differential attrition threatened the interpretation of any observed group differences. There was no evidence for differential attrition for any study outcome (all ps > .10). Owing to child non-compliance, technical problems, and human error, the number of participants with valid outcome data differs across the measures, influencing power (see Table 2). Finally, analyses of variance and multiple regression analyses were conducted to examine the potential influence of site and site-by-treatment interactions on the five primary outcome variables. No statistically significant main effects of site or siteby-treatment interactions were observed for any of the outcome variables (all ps > .20). There were no significant differences in attendance of non-project treatments between the HMTW and ‘business as usual’ groups at any measurement period (all ps > .10).

Table 2.

Means and standard deviations of study variables (raw and residualized gain scores)

Study variables	Time period	HMTW		Control		ES*	95% CI
		N	M (SD)	n	M (SD)
PCFP proportion of codable intervals with parental responsivity
Raw scores	T1	21	.32 (.06)	24	.29 (.08)	.41	[−.18, 1.00]
	T2	25	.37 (.10)	20	.33 (.07)	.57	[−.04, 1.18]
	T3	23	.34 (.07)	24	.30 (.10)	.40	[−.19, 1.00]
Residualized gain scores	T1 to T2	17	.02 (.09)	16	−.03 (.07)	.71	[−.01, 1.44]
	T1 to T3	17	.03 (.08)	20	−.02 (.10)	.50	[−.18, 1.18]
ESCS frequency of initiating joint attention
Raw scores	T1	30	5.90 (5.41)	29	5.59 (6.14)	.05	[−.46, .56]
	T2	28	8.11 (8.53)	23	9.26 (9.77)	−.12	[−.69, .44]
	T3	24	10.33 (9.82)	25	8.68 (9.26)	.17	[−.40, .75]
Residualized gain scores	T1 to T2	26	.00 (.38)	23	.00 (.39)	.00	[−.58, .58]
	T1 to T3	23	.06 (1.21)	25	−.06 (1.01)	.12	[−.46, .70]
ESCS frequency of initiating behavior requests
Raw scores	T1	30	11.87 (10.09)	29	9.00 (6.22)	.34	[−.17, .85]
	T2	28	14.32 (13.04)	23	12.22 (8.85)	.24	[−.33, .80]
	T3	24	16.50 (14.33)	25	15.48 (13.20)	.08	[−.50, .65]
Residualized gain scores	T1 to T2	26	.00 (1.58)	23	.00 (1.07)	.00	[−.58, .58]
	T1 to T3	23	.03 (.34)	25	−.03 (.37)	.16	[−.42, .74]
PCFP weighted frequency of intentional communication
Raw scores	T1	20	5.55 (6.29)	20	8.20 (12.63)	−.26	[−.88, .37]
	T2	24	16.96 (14.62)	19	21.26 (27.52)	−.16	[−.78, .47]
	T3	22	18.91 (20.50)	24	20.75 (21.14)	−.09	[−.69, .51]
Residualized gain scores	T1 to T2	15	.00 (1.48)	12	.00 (2.66)	.00	[−.80, .80]
	T1 to T3	15	.18 (1.69)	17	−.16 (2.21)	.15	[−.57, .88]
PIA-CV nonverbal communication
Raw scores	T1	31	2.30 (.64)	23	2.28 (.73)	.00	[−.54, .54]
	T2	27	2.78 (.60)	20	2.84 (.68)	−.15	[−.74, .45]
	T3	23	2.89 (.67)	24	2.92 (.65)	.00	[−.59, .59]
Residualized gain scores	T1 to T2	27	.00 (.49)	16	.00 (.58)	.00	[−.64, .64]
	T1 to T3	23	−.05 (.63)	20	.06 (.58)	−.19	[−.81, .43]

Note: PCFP = Parent–Child Free Play; ESCS = Early Social Communication Scales; PIA-CV = Parent Interview for Autism–Clinical Version; T1 = Time 1; T2 = Time 2; T3 = Time 3.

^*Hedge’s g is used at Time 1 and Glass’s delta is used at other times and for all gain scores (Glass, 1977).

Time effects on primary dependent variables

Overall (independent of treatment group), parents exhibited moderate increases in their responsivity from Time 1 to Time 2, d = .55, t(32) = 2.88, p = .007, and showed moderate decreases in responsivity during the follow-up period (from Time 2 to Time 3) d = −.44, t(38) = −2.4; p = .02. Overall, children’s increases in communication from Time 1 to Time 3 were moderate (IJA d = .43, IBR d = .55), large (weighted frequency of intentional communication, d = .77), and very large (PIA-CV nonverbal communication d = 1.15), ps < .05.

Consumer satisfaction with HMTW

Consumer satisfaction was extremely high, with mean ratings of 5.48 (out of 6) on the group experience questionnaire and 3.46 (out of 4) on the group leader experience questionnaire.

Primary hypothesis testing

The main goal of the primary analyses was to examine the effect of the HMTW intervention on gains in parental responsivity from Time 1 to Time 2 and from Time 1 to Time 3 and on four child communication outcomes from Time 1 to Time 3 (see Table 2 for parental responsivity and child communication outcomes at each time points). A second goal was to examine conditional effects of the HMTW intervention; specifically, we asked whether Time 1 child communication and object interest indices moderated the effect of treatment on gains in child communication from Time 1 to Time 3. Consistent with recommendations for randomized clinical trials (McCartney et al., 2010) and in an effort to conserve statistical power, gains in parental responsivity and child communication were quantified as residualized gain scores. These scores were obtained by regressing the Time 1 measure of each variable onto the later measure of the same variable. Analyses were conducted using SPSS version 17 and the web-based utility provided by Preacher, Curran, and Bauer (2006).

Independent-sample t-tests were used to test the main effect of treatment group assignment on parental responsivity residualized gain scores. Hierarchical linear regression was employed to assess the main effect of the HMTW intervention on child communication residualized gain outcomes and to determine whether Time 1 child IBR, IJA, or object interest moderated the effect of the HMTW intervention on child outcomes. As recommended by Aiken and West (1991), the interaction terms were created by grand-mean centering the Time 1 moderator variables and multiplying them by the dummy-coded treatment status variable. When moderation was observed, the method recommended by Preacher and colleagues (2006) was used to identify higher and lower regions of significance. These empirically derived regions of significance specify the upper and lower values of the moderator at which the intervention and control groups are significantly different from one another on the dependent variable of interest. As it is possible to observe an interaction in which no child’s value is in the region of significance, statistically significant interactions were interpreted only when there were children in both groups who had values in the region of significance. This method represents both a statistically conservative and practically meaningful approach to analyzing interactions, particularly within an RCT context where it is useful to determine the point along the moderator at which the treatment becomes facilitating or attenuating.

The main effects of the HMTW intervention on change in parental responsivity from Time 1 to Time 2 and from Time 1 to Time 3 did not reach conventional levels of statistical significance, t(1, 31) = 1.8, p = .08 and t(1, 35) = 1.8, p = .09, respectively. Effect sizes, however, were in the medium to large range at both time points, Glass’s (1977) Δ = .71 and .50, respectively. Table 2 contains effect size confidence intervals and descriptives.

Counter to our expectations, there were no main effects of the HMTW intervention on residualized gains in child communication from Time 1 to Time 3 (see Table 2). As expected, there were no effects of HMTW on gains from Time 1 to Time 2. In addition, Time 1 initiating joint attention and initiating behavioral requests (i.e., IJA and IBR) did not moderate the treatment effect on gains in any child communication outcomes from Time 1 to Time 3. However, object interest at Time 1 moderated the treatment effect on the residualized gain for several communication variables from Time 1 to Time 3, indicating that the impact of participating in HMTW on child communication depended on children’s level of object interest at study entry.

As shown in Table 3 and Figure 2, object interest moderated treatment effects on the residualized gain scores from Time 1 to Time 3 for IJA, t(45) = −3.38, p = .002, IBR, t(45) = −3.41, p < .01, PIA-CV non-verbal communication, t(42) = −2.39, p = .02, and PCFP weighted frequency of intentional communication, t(29) = −3.39, p = .003. In predicting residualized gains in IJA, IBR and PCFP weighted frequency of intentional communication, both higher and lower regions of significance were interpretable. Children who played with fewer than three toys at Time 1 exhibited greater gains in IJA, IBR and weighted frequency of intentional communication if they were randomized to the HMTW intervention group rather than to the control group. In contrast, those children who played with greater than five or six toys (depending on the outcome) at Time 1 showed lower gains in IJA, IBR and weighted frequency of intentional communication if they were randomized to the HMTW group rather than to the control group (see Figure 2.A, 2.B, and 2.C. for specific regions of significance).

Table 3.

Hierarchical linear regression analyses predicting child communication variables from treatment status and Time 1 object interest

	Dependent variables
	Residualized gain scores of initiating joint attention			Residualized gain scores of initiating behavioral requests			Residualized gain scores of PIA-CV nonverbal communication			Residualized gain scores of weighted frequency of intentional communication
Predictors	ΔR2 a	B	95% CI	ΔR2 a	B	95% CI	ΔR2 a	B	95% CI	ΔR2 a	B	95% CI
Intercept		.039	[−.39, .47]		.017	[−.12, .15]		.079	[−.19, .35]		.834	[−.27, 1.94]
Treatment Status	.01	.163	[−.45, .77]	.00	.004	[−.19, .20]	.01	−.086	[−.46, .28]	.01	−.499	[−1.99, .99]
DPA # Toys at T1	.03	.191	[−.11, .49]	.24***	.172	[.08, .27]	.05	.134	[−.06, .32]	.33**	1.514	[.65, 2.38]
Treatment Status × DPA # Toys at T1	.20**	−.664	[−1.06, −.27]	.21**	−.213	[−.34, −.09]	.13*	−.288	[−.53, −.04]	.29**	−1.718	[−2.76, −.67]
N	45			45			42			29

Note: The n’s displayed are smaller than those presented in Figure 1 and Table 2 when children were missing a specific outcome measure or the Time 1 measure to required to compute the residualized gain score; CI = confidence interval; PIA-CV = Parent Interview for Autism – Clinical Version; PCFP = Parent–Child Free Play; DPA = Developmental Play Assessment; T1 = 1st time point.

^aR squared change was calculated as the square of the part correlation coefficient.

^*p < .05.

^**p < .01.

^***p < .001.

Figure 2.

Graphs depicting treatment moderation by DPA number of toys predicting the residualized gains of (a) initiating joint attention; (b) initiating behavioral requests; (c) PCFP weighted frequency of intentional communication; (d) PIA-CV nonverbal communication. DPA = Developmental Play Assessment; PIA-CV = Parent Interview for Autism – Clinical Version; PCFP = Parent–Child Free Play; T1 = 1st time point; Max = maximum value of T1 moderator variable; Min = minimum value of T1 moderator variable; Higher RoS = higher region of significance; Lower RoS = lower region of significance

The pattern of results for residualized gain scores in nonverbal communication was more restricted in that only the higher region of significance was interpretable (see Figure 2.D). That is, parents of children who played with at least six toys at Time 1 reported more limited growth in nonverbal communication if they were randomized to the HMTW group rather than to the control group.

Discussion

To our knowledge, this is the first randomized controlled trial of the HMTW intervention with toddlers evidencing symptoms consistent with an ASD. Given that HMTW is a parent-implemented intervention, ascertaining treatment effects on parenting behaviors is essential. In addition, measuring the endurance of such effects several months after treatment ends is important because children with ASD are unlikely to benefit unless such maintenance of enhanced parenting practices occurs. Although not statistically significant, the magnitudes of effect sizes of the intervention on parenting responsivity were consistent with the findings of McConachie and colleagues (2005). The effect sizes observed also appear to be consistent with Green et al.’s (2010) report of a treatment effect for parental synchronous response to child in an RCT of a similar parent-mediated, communication-focused, intervention of higher intensity. In the current study, the effect size immediately after treatment was medium to large (.71) and was moderate even at the follow-up period (.50). It has long been argued that effect sizes are more informative than statistical significance (Cohen, 1994; Denis, 2003; Stam & Pasay, 1998). Future meta-analyses will be needed to determine the population value of the effect size of low-intensity parent-implemented treatments such as HMTW.

In the current study, there were no main effects of treatment on child outcomes immediately after the parent-implemented treatment or 5 months after treatment. These findings, across multiple child outcomes, raise concerns about the general appropriateness of the HMTW intervention in very young toddlers with symptoms consistent with an ASD. The lack of a main effect of treatment on communication outcomes is consistent with Oosterling and colleagues’ randomized clinical trial of a different parent-mediated intervention with somewhat older preschoolers with ASD (Oosterling et al., 2010). A second recent study also assessed a parent-mediated communication-focused treatment in preschool-aged children with autism (Green et al., 2010), and found no main effects of the treatment on autism severity, clinical language measures, or teacher ratings of adaptive communication. There was, however, a medium effect size of the intervention on observed child social communication, similar to the behaviors coded in the current study as well as on parent-reported child communication. Green et al. noted, however, that the positive effects of treatment observed were (thus far) limited to the parent–child dyad, and did not appear to generalize to interactions with other adults.

Although no main effects of treatment were observed on child outcomes in the current study, exploratory analyses revealed several significant conditional effects. These effects indicated that the HMTW intervention was facilitative of communication for some children, but attenuated growth in communication for others. Time 1 object interest (i.e., the number of toys children played with in a differentiated, or functional, manner) moderated four communication outcomes: observed gains in initiating joint attention, initiating behavioral requests, weighted intentional communication, and parent-reported gains in nonverbal communication.

Facilitation was present for three observed outcomes at low levels of object interest. In another study on young children with autism, children with initially low object interest acquired superior communication skills during a responsivity-based treatment relative to a contrast treatment (Yoder & Stone, 2006b). The treatment that facilitated linguistic communication for children with low object interest also effectively taught object play skills; the contrasting treatment did not teach object play (McDuffie, Lieberman, & Yoder, in press). Object interest is important to responsivity-based treatments because teaching episodes tend to occur in object-oriented joint action routines and because providing access to objects contingent on child communication is frequently used as a functional reward.

In contrast, children randomized to the HMTW intervention who played with more than five to six toys (depending on the outcome examined) showed more limited growth in all three observed communication outcomes and in parent-reported nonverbal communication outcome. That is, children who entered the study with more limited object interest appear to have benefited, while children with greater object interest showed attenuated growth. These conditional effects were exploratory in nature. Additionally, empirically derived regions of significance vary according to the outcome assessed. Thus, replication is essential before using these results to select treatments for individual children. Although we had expected joint attention to moderate the effects of the HMTW intervention on child outcomes, none of the interactions were statistically significant. In previous work in which IJA did moderate the effect of treatment on outcome (Yoder & Stone, 2006a), IJA was assessed across two contexts and the children were somewhat older. Thus, it is possible that the current sample’s limited joint attention skills, which were sampled only in the ESCS, constrained this potential moderator’s variability.

All of our findings must be considered within the context of the study’s limitations and strengths. Although the mean fidelity of treatment implementation was high with respect to measured HMTW content and interventionist quality, and parents rated the HMTW intervention and group leaders extremely favorably, the size of the parent groups was not commensurate with Hanen recommendations. This circumstance may have compromised the potential learning opportunities and social supports that emerge in group sessions with a larger number of parent participants. In addition, although overall study attrition was low and there was no evidence of differential attrition across the treatment and control groups, statistical power was constrained for several outcome measures due to child non-compliance, technical problems, and experimenter error; due to the extent of missing data a full intent to treat design with imputation was not pursued.

The use of an experimental design and ‘partial’ intent to treat analyses are strengths of this study. Further, the majority of assessors and all observational coding of children’s behavior were blind to treatment condition. Finally, analyses documenting pre-treatment equivalence on measures of child, family and non-project interventions, along with a lack of differential attrition across the HMTW intervention and ‘business as usual’ control groups, raise confidence in the internal validity of the study. Moreover, despite conducting the study in three sites that varied significantly on service availability, there were no site-by-treatment interactions on study outcomes. Although nonsignificant effects can occur due to low within-site sample size, the findings of this randomized controlled trial of the HMTW intervention are compatible with the hypothesis that the current findings are generalizable to a relatively broad range of settings in the United States.

An additional methodological strength is the multi-method assessment of child communication that was employed. This study measured communication during parent–child interactions, examiner–child interactions, and via parental report. Observation of the child with both the parent and an unfamiliar examiner is critical for evaluating the child’s ability to generalize the use of new strategies beyond the parent–child context in which the parent is supporting and scaffolding the child’s developmental progress in a manner that is familiar to the child. The finding that the HMTW intervention led to increased frequency of initiating joint attention and behavioral requesting acts, albeit limited to children with low initial object interest, suggests that these children were able to transfer skills learned with their parents to the novel (i.e., untrained) context of interaction with an unfamiliar examiner in the ESCS. Replication of this finding is critically important, as children with ASD have more difficulty generalizing skills across contexts (Paul, 2008).

Children with ASD who have limited object interest may be particularly well suited to the HMTW approach, which emphasizes teaching parents to show their child what is interesting about toys and to prompt, wait for, and respond to requests. The lack of overall treatment effects on child outcomes may be due to the relatively low intensity of the HMTW intervention, or to the non-optimal implementation due to smaller than recommended group size. The emphasis of HMTW is on the group experience, and individual work with families occurs only during three sessions. It may be the case that additional individualized sessions in which parents receive feedback on implementing HMTW methods are needed for this unique sample of young children.

An unanticipated finding of this study was that children with relatively high levels of object interest who were randomized to the HMTW intervention showed attenuation of growth relative to those children randomized to the ‘business as usual’ group. It is important to note that, overall (i.e., across treatment groups), children showed moderate to large gains in all skills assessed. Further research is clearly warranted to aid in understanding these unanticipated negative outcomes for certain subgroups. Additionally, it should be noted that other studies of parent-implemented treatments have found similar unexpected effects (i.e., experimental group results lower than control group results). For example, parental sensitivity training (a part of which is responsivity training) resulted in attenuated growth on parental sensitivity and attenuated reduction on child aggression in certain subgroups of participants in a study by Stolk et al. (2008). Such findings highlight the importance of comparing an active intervention to a ‘business as usual’ control group, as was done in the current study. Had two active interventions been compared, the interpretation would likely favor the treatment with greater growth rather than recognizing the presence of growth attenuation. Such results remind us that low-intensity parent-implemented treatments may not be the best treatment for all children with ASD.

Determining clinical significance of effects is central to science that supports clinical practice. Although our use of measures that were closely linked to the intervention goals is a strength, the use of non-norm referenced instruments for both child and parent outcomes limits clinical interpretation. The metric that supports the clinical significance of these findings is the presence of moderate to large effect sizes for the parental responsivity outcome and the child outcome moderation effects, which meet or exceed the usual benchmark of .50 for determining clinically meaningful effects (Hill, Bloom, Black, & Lipsey, 2008; Wolf, 1986).

In conclusion, there were no main effects of the treatment on child outcomes. However, some children showed clear gains in communication that were associated with being randomized to the intervention while others showed attenuation in communication growth. The gains in communication, which were evident both with parents and an unfamiliar examiner, are heartening, and highlight the critical role that parents can play and the potential utility of relatively low-intensity, well-designed, developmentally sensitive interventions. However, the growth attenuation is of concern, and highlights the urgency for conducting additional intervention research focused on determining which interventions are most appropriate for which children with ASD and their families.

Key points.

• This randomized controlled trial compared Hanen’s ‘More than Words’ (HMTW), a parent-implemented intervention, to a ‘business as usual’ control group, in a sample of toddlers demonstrating symptoms of autism.

• The HTMW intervention did not benefit all children. Children entering the study with limited object interest benefited from HMTW. However, children with greater object interest showed more limited growth than those in the ‘business as usual’ group.

• The HTMW intervention does not seem to be effective for all families. Some families may need a more intensive or different type of intervention.