Abstract
Children with Autism Spectrum Disorder (ASD) often experience difficulty participating in everyday home routines, such as bed time or bath time. This randomized controlled trial examined the efficacy of an interactive, web-based parenting tutorial for improving children’s engagement in daily routines (i.e., proximal outcomes) as well improving children’s social communication and parenting efficacy and stress (i.e., broad outcomes). Parents of children with ASD between 18 and 60 months were randomly assigned to the Tutorial group (n = 52) or the Control group (n = 52). All parents completed questionnaires at baseline (T1), 1 month after T1 (T2; post-tutorial completion), and 2 months after T1 (T3). Relative to the Control group, parents in the Tutorial group reported significantly higher use of evidence-based instructional strategies and higher levels of child engagement during routines at T2 and T3. In addition, parents in the Tutorial group reported significantly lower parenting stress and higher parenting efficacy at T3, as well as higher ratings of child social communication at T2 and T3, compared to the Control group. Parents reported being highly satisfied with both the clinical content and technical aspects of the tutorial. These improvements in both proximal and broad parent-child outcomes suggest that this tutorial may be a promising and accessible way for empowering some parents and improving parent-child interactions.
Keywords: daily routines, randomized control trial, parent training/coaching, parenting stress, parenting efficacy, social communication
Introduction
As the prevalence of Autism Spectrum Disorders (ASD) has increased, the gap between the number of children receiving this diagnosis and the availability of specialized community-based services for them has widened [Chiri & Warfield, 2012]. State early intervention programs have reported critical shortages of providers that deliver ASD-related services, including behavioral therapists and speech-language pathologists [Bailey, 2008; Wise, Little, Holliman, Wise, & Wang, 2010], and most providers working with young children are not trained sufficiently to provide ASD-specialized services [Stahmer & Mandell, 2007; Wise et al., 2010]. Although recent years have seen a dramatic increase in the availability of evidence-based interventions for young children with ASD [i.e., “Naturalistic Developmental Behavioral Interventions”; Schreibman et al., 2015], their implementation in community settings has been limited. Importantly, the success of these interventions highlights the significance of teaching young children within the context of their daily routines, and of coaching parents in the use of strategies for promoting their child’s development.
Daily routines such as bath time, snack/meal time, bed time, and play time represent meaningful family rituals that provide context, structure, and frequent opportunities for rich social interactions that can support early social-emotional development [Spagnola & Fiese, 2007]. In fact, daily routines confer several unique advantages for learning new behaviors relative to other situations, such as the presence of natural reinforcers (e.g., preferred food during snack time) as opposed to arbitrary reinforcers [Schreibman et al., 2015]. For these reasons, embedding the delivery of intervention services within daily routines is emphasized in the guidelines and practices of state educational/intervention programs (e.g., IDEA Part C). However, completion of daily routines often presents significant challenges for children with ASD and has been described as a specific and significant stressor by parents [Marquenie, Rodger, Mangohig, & Cronin, 2011]. Therefore, providing parents with strategies for improving their child’s functioning during routines may help reduce parenting stress and improve parenting efficacy by mitigating a prominent and common stressor. Furthermore, teaching parents to use evidence-based strategies within the context of daily routines may increase the likelihood of their use, as the regularity of these routines offers multiple opportunities for repetition and practice [Schreibman, et al., 2015].
Numerous studies have shown that parents of children with ASD can be taught to successfully implement comprehensive behavioral interventions that lead to both improvements in child behavioral outcomes and parenting stress and parenting efficacy/empowerment [Ingersoll, Wainer, Berger, Pickard, & Bonter, 2016; Kasari et al., 2015; Minjarez, Mercier, Williams, & Hardan, 2013]. Several of these interventions have been embedded within naturalistic activities/routines and have examined child behaviors during a specific routine (i.e., snack time) as an outcome [Ingersoll & Wainer, 2013; Kasari, Gulsrud, Wong, Kwon, & Locke, 2010; Rogers et al., 2012; Schreibman et al., 2015; Wetherby et al., 2014]. However, none have evaluated the efficacy of a parent-implemented intervention specifically targeting child and parent behaviors during daily routines as the primary outcomes. The dissemination and scalability of these interventions is hindered by barriers such as a shortage of ASD interventionists/trainers, lack of child care or access to transportation, and high costs [Wainer & Ingersoll, 2015]. However, increasingly widespread access to the internet and the advent of new technologies such as web-based, distance learning may provide a potential solution for enhancing the knowledge and skills of parents.
Self-directed, web-based training (e.g., online course/tutorial) has the potential to: (a) increase the dissemination and quality of instruction for those parents who experience barriers to accessing services; and (b) act as supplemental training for families already receiving other intervention services. This approach offers several advantages to traditional instructional methods, including 24-hr accessibility, standardization of training, personalization/individualization (e.g., self-paced), widespread reach, risk free environment, and the opportunity for interactive exercises and multimedia components, which have been found to enhance knowledge retention [Ballew et al., 2013]. Web-based training is cost-effective and overcomes geographical, transportation, and child care constraints. Most importantly, it facilitates access to training that most likely would be unavailable and overcomes the shortage of ASD interventionists. Given these advantages, several web-based trainings aiming to improve parenting competencies (non-ASD specific/targeted) have been developed over that last two decades and been found to have a significant medium effect on parenting and child outcomes [Nieuwboer, Fukkink, Hermanns, 2013].
Several self-directed, web-based parent training programs that teach behavioral strategies specifically for ASD have been developed over the last few years and are effective at improving parent and child outcomes. Vismara and colleagues [2012, 2013] conducted pilot studies (n = 8 and n = 9) using a hybrid approach, which included an interactive DVD for didactic instruction on the Early Start Denver Model [Rogers & Dawson, 2010] that parents reviewed prior to 12 weekly videoconference sessions (web-based) for live parent coaching/feedback. Using a single-subject, multiple baseline design, they found improvements in parent engagement style, child social communication, and high levels of user satisfaction. Hamad et al. [2010] used a web-based course to teach behavioral intervention skills to both parents (n = 15) and professionals/paraprofessionals (n = 36), and found improvements in knowledge of these techniques for both cohorts. Using a waitlist control design with 28 parents, Jang and colleagues [2012] also found that a web-based course on applied behavior analysis improved participant knowledge and was well-received. Ingersoll and colleagues [2015, 2016] employed a RCT design to compare parents learning Project ImPACT online through a self-directed only approach (n = 13) versus a therapist-assisted approach (n = 15). While both groups demonstrated improvements in child language, parenting stress, parenting efficacy, parent perceptions of the child, and implementation fidelity, the therapist-assisted group exhibited gains in child social skills and had a higher program completion rate. While these studies provide evidence that web-based, self-directed training for parents can be effective, they used relatively small convenience samples.
Given the challenges that children with ASD can experience during daily routines, this study was designed to evaluate a web-based tutorial for parents that focuses specifically on the context of home routines. Our focus on routines is consistent with the principle that adults are motivated to learn to the extent that they perceive it will help them perform the tasks they confront in everyday life [Knowles, 1990]. This study builds upon and extends the pioneering work that has been conducted with families of children with ASD using web-based technology by being the first study to employ a RCT design with a relatively large sample. The specific goals of the present study were to examine whether the parenting tutorial is effective for improving outcomes that are: (a) routine-specific and directly targeted by its content (i.e., proximal outcomes); and (b) related to child social communication and the parent-child relationship across broader contexts (i.e., broad outcomes). Proximal outcomes included parents’ use of specific evidence-based strategies [Odom, Collet-Klingenberg, Rogers, & Hatton, 2010] and children’s engagement behaviors during routines. Broad outcomes included parenting stress, parenting efficacy, and child social communication. Outcomes were assessed using self/parent-report measures only because the study was conducted remotely.
Methods
Participants
This RCT comprised 104 parents of children with ASD (mothers = 94; fathers = 10), who were recruited from Vanderbilt University and the University of Washington using research databases, websites, and flyers. To be eligible to participate, families needed to: (a) have a child between 18 and 60 months with a diagnosis of ASD and no severe auditory, visual, or motor impairments; (b) provide a copy of child’s diagnostic report confirming an ASD diagnosis; (c) have access to the internet; and (d) be proficient in English and speak to the child in English at least 90% of the time. The Tutorial group and Control group were comparable on child age, parental age, and parental education, ps = 0.45–0.93, and the two study sites were also comparable on these demographics, ps = 0.07–0.37 (Table 1). The study was reviewed and approved by the Institutional Review Board (IRB) at each university and all participants provided informed consent.
Table 1.
Tutorial group (n = 52) | Control group (n = 52) | |
---|---|---|
Child age (in months) at entry M (SD) | 42.83 (13.39) | 44.77 (12.50) |
Parent age (in years) at entry M (SD) | 34.71 (6.24) | 34.61 (6.22) |
Infant sex # (%) | ||
Female | 6 (11.5) | 10 (19.2) |
Male | 37 (71.2) | 39 (75.0) |
No response | 9 (17.3) | 3 (5.8) |
Child race # (%) | ||
White | 47 (90.4) | 42 (80.8) |
Native American | 1 (1.9) | 0 |
Asian/Pacific Islander | 1 (1.9) | 3 (5.8) |
Black/African American | 1 (1.9) | 3 (5.8) |
Other | 2 (3.8) | 4 (7.7) |
Highest level of education attained # (%) | ||
No college degree | 20 (38.4) | 17 (32.7) |
College degree (two-year or Bachelor’s degree) | 24 (46.1) | 24 (46.1) |
Graduate degree | 8 (15.4) | 10 (19.2) |
No response | 0 (0) | 1 (1.9) |
Procedure
Parents completed online surveys describing their own behavior and their child’s behavior during everyday home routines at three time points: baseline (T1), post-tutorial completion (T2; 1 month after T1), and 1-month follow-up (T3; 2 months after T1). One-month measurement intervals were used to allow families in the Tutorial group sufficient time to review the tutorial and to examine short-term follow-up use. After completing T1 surveys, parents were randomized to the Tutorial group (n = 52) or the Control group (n = 52) using a random number sequence generator. Parents assigned to the tutorial condition were immediately provided with access to the web-based tutorial, which they were asked to go through at their own pace over the following month. All parents received the same online surveys at T2 and T3 regardless of whether or not the tutorial had been completed. At T2, parents in the Tutorial group also completed user satisfaction questionnaires regarding their experience with the tutorial. See Fig. 1 for recruitment, eligibility, and measure completion flow. The Control group received free access to the tutorial after completing T3 surveys; no additional data was collected from these families and their utilization of the tutorial was not tracked.
The Enhancing Interactions tutorial was developed using adult learning principles designed to foster engagement, participation, and interest (Vincent & Ross, 2001). The tutorial is highly interactive, making full use of the technology and principles of instructional design to enhance the learning experience. Interactive learning activities were incorporated to present new information as well as to test parents’ comprehension of the material and reinforce learning. The tutorial content emphasizes the importance of everyday routines as opportunities for learning, and focuses on teaching parents how to use evidence-based strategies (e.g., using simple instructions, using visual supports) and avoid less than optimal strategies (e.g., repeating verbal directions if the child fails to comply) to improve their child’s behavior and increase their child’s participation in daily home routines. Four levels of engagement/participation during the routines are defined (i.e., not tolerating, tolerating, cooperating, and engaging socially), and parents learn how to systematically move their child’s behavior from one level to the next. Videos of parents demonstrating the techniques with their children in actual home settings are incorporated to illustrate the material.
The tutorial comprises three main sections: (a) introductory material (e.g., definition of home routines; tips for establishing routines); (b) description and illustration of four daily routines—bath time, snack time, play time, and bed time—which includes both general information and individualized content; and (c) “toolbox” modules describing specific, evidence-based behavioral strategies for enhancing children’s cooperation and participation with routines (Table 2). It is structured so that parents are presented with a ‘menu’ of the four daily routines and are able to choose a specific activity within that routine that they would like to improve. For each routine, individualized content enables parents to: (a) identify the sequence of steps in their routine; (b) identify a specific step within the routine that they would like to improve (e.g., getting undressed for “bed time”); (c) identify the child’s current level of participation in that step of the routine (e.g., tolerates); and (d) learn behavioral strategies for improving the child’s participation during the selected step of the routine. Allowing parents to set individual goals for their child and then teaching them strategies to use during everyday situations has been found to be an effective way to train parents. This approach is consistent with Knowles’ principles of andragogy (adult learning)—that adults are motivated to learn to the extent that they perceive it will help them perform tasks they confront in their life situations [Knowles, 1990]. Select interactive sections from the tutorial can be viewed and tried out firsthand at: http://telepsychology.net/OnlineAssessmentTools/Resources/Autism2Demo/Story.html.
Table 2.
Section | Content |
---|---|
Introduction to home routines |
|
Individual routines: Bath time Snack time Play time Bed time |
|
Tool box modules |
|
Parents in the Tutorial group were instructed to review the general modules (i.e., the overview and “toolbox” modules), and to choose at least one specific daily routine to focus on. They were also encouraged to review the tutorial across at least 4 or 5 sessions, as the time to complete the entire tutorial is approximately 6 hours; parents did retain access to the tutorial after their one-month interval elapsed. Of the 52 parents in the Tutorial group, 46 (88.5%) viewed both the general modules and at least one specific daily routine.
Proximal Outcome Measures
At each of the three-time points, parents completed two routine-specific surveys that were developed specifically for this study: one describing the behavioral strategies they used, and the other describing the child’s engagement (or participation) behaviors. The items on these measures were selected to map onto the specific strategies taught in the tutorial. Parents rated each item on a four-point Likert scale indicating the frequency with which it occurred (ranging from “almost never [1]” to “almost always [4]”) during home routines over the previous 2 weeks.
Parent Behavior Survey
This survey comprised 15 specific parent behaviors that examined parents verbal instructions/directions (e.g., using simple instructions, repeating verbal directions if the child fails to comply [reverse coded], praising the child for completing all or part of a routine), use of visual supports (e.g., use of pictures and timer), and their approach to structuring the routine steps. After reverse coding the one undesired behavior, higher scores on each item indicated more optimal use of evidence-based strategies. The items demonstrated acceptable internal consistency reliability, α = 0.79. Items are averaged to obtain a mean total score that ranges from 1 to 4, with higher scores indicating increased use of desired strategies.
Child behavior survey
This survey focused on five specific child behaviors, reflecting child engagement during routines (i.e., actively participating, interacting and communicating while participating, difficulty transitioning when getting started with a routine and/or after completing routine, and resisting participation). After reverse coding the negative behaviors (i.e., difficulty transitioning when getting started with a routine and/or after completing routine, and resisting participation), higher scores on each item indicated more optimal child engagement. The items demonstrated acceptable internal consistency reliability, α = 0.71. A mean total score was derived as above, so that the score ranged from 1 to 4, with higher scores indicating higher levels of child engagement.
Broad Outcome Measures
Parent interview for autism–clinical version (PIA-CV)
Children’s social communication skills were assessed using the PIA-CV [Stone et al., 2003], a parent-report measure of autism symptom severity designed for young children. The PIA-CV was adapted from the original version (PIA), which has strong psychometric properties, including construct validity and discriminant validity for children between 20 months and 6 years [Stone & Hogan, 1993]. The PIA-CV consists of 41 behavioral items presented in questionnaire format that are rated on a 5-point Likert scale ranging from “almost never (1)” to “almost always (5)”. PIA-CV domain scores have demonstrated sensitivity to diagnostic changes (measured independently) between the ages of 2- and 4-years old [Stone et al., 2003] and has also been used with children under 24 months [Carter et al., 2011]. Given the strategies and goals of the tutorial, two PIA-CV domains were used: Social Relating (17 items) and Nonverbal Communication (13 items); the two domains demonstrated good internal consistency reliability (α = 0.85–0.88) with the current sample. Mean scores for each domain are calculated after reverse-coding negatively-worded items, such that scores range from 1 to 5, with higher scores reflecting more optimal levels of social communication.
Parental efficacy scale (PES)
Parenting efficacy was assessed through self-report using the PES [Teti & Gelfand, 1991]. Ten items measuring parenting-related efficacy (e.g., understanding what the child wants, knowing what the child enjoys) are rated on a four-point Likert scale ranging from “not good at all (1)” to “very good (4)”. The PES has strong psychometric properties and has been used in many studies involving children with ASD [Carter, Martinez-Pedraza, & Gray, 2009]; it demonstrated acceptable internal consistency reliability (α = 0.78) with the current sample. Items are summed to obtain a total score that ranges from 10 to 40, with higher scores reflecting higher levels of perceived parenting efficacy.
Parenting stress index/short form (PSI/SF)
The PSI/SF (Abidin, 1995) consists of 36 items measuring three domains of parental stress: (a) Parental Distress (PD); (b) Parent-Child Dysfunctional Interaction (PCDI); and (c) Difficult Child (DC). The PSI/SF has strong psychometric properties and has been used extensively with families of children with disabilities, including ASD [Tomanik, Harris, & Hawkins, 2004]; the three domains demonstrated good internal consistency reliability (α = 0.84–0.91) with the current sample. Each domain contains 12 items that are scored on a five-point Likert scale ranging from “strongly disagree (1)” to “strongly agree (5)”. Scores for each domain range from 12 to 60, with higher scores indicating higher levels of parenting-related stress.
Usability and Satisfaction Measures
System usability scale (SUS)
User satisfaction with the technical aspects of the web-based tutorial was assessed using the System Usability Scale [SUS; Bangor, Kortum, & Miller, 2008; Brooke, 1996]. The SUS is a reliable, well-validated 10-item scale designed to evaluate the usability and user satisfaction with web-based applications and other technologies. Each item is rated on a five-point Likert scale ranging from “strongly disagree (1)” to “strongly agree (5)”. An example item is: “I thought the tutorial was easy to use”. The SUS has good internal consistency reliability (coefficient α = 0.91) in assessing usability across diverse types of user interfaces [Bangor et al., 2008]; it demonstrated similar internal consistency reliability (α = 0.88) with the current sample. Due to experimenter error, item 9 of the scale (i.e., “I felt very confident using the tutorial”) was accidently omitted from the survey for all participants. The “person mean substitution” approach was used to replace the missing item score for each participant by averaging their values from the nine available items [Downey & King, 1998]. Satisfaction with technical usability was examined using a total score that ranges from 10 to 100 (total scores are multiplied by 2), with higher scores reflecting greater satisfaction with the usability of the tutorial.
User satisfaction questionnaire (USQ)
Satisfaction with the clinical content of the tutorial was evaluated using the User Satisfaction Questionnaire [Kobak, 2004; Kobak, Craske, & Wolitsky-Taylor, 2013; Kobak, Reyn-olds, & Griest, 1994]. This measure has good internal consistency [Cronbach’s alpha = 0.90; Kobak et al., 2013]; it demonstrated similar internal consistency reliability (α = 0.93) with the current sample. It contains 15 statements covering different dimensions of the user experience (e.g., “There were sufficient examples and illustrations”). Items are rated on a four-point Likert scale ranging from “strongly disagree (1)” to “strongly agree (4)”. Total scores range from 15 to 60, with higher scores reflecting greater satisfaction with the clinical content of the tutorial.
Results
Analytic Approach Overview
Intention-to-treat
The current study used the Intention-to-Treat model [ITT; Moher, Schulz, & Altman, 2001] in the analysis of the primary aims. The ITT approach requires that all participants who are randomly assigned be compared on outcomes regardless of their adherence to treatment, reasons for withdrawal, and the presence of missing responses. These guidelines (a) preserve the group comparability established by the randomization process at baseline; (b) reduce the bias introduced by participants dropping out of treatment; and (c) reflect how treatments will perform at the population level by ignoring degree of adherence to treatments when data are examined.
Data screening and attrition
All outcome variables at T1–T3 demonstrated acceptable levels of skewness and kurtosis and did not contain outliers as measured through leverage and Cohen’s distance [Tabachnick, Fidell, & Osterlind, 2001]. There were no site differences on the percent of participants with missing data, χ2 = 3.03, P < 0.08. There were significantly more participants missing post-treatment data (i.e., T2 and T3) in the Tutorial group (21%, n = 11) than the Control group (2%, n = 1), χ2 = 9.42, P < 0.01; of the 11 participants in the tutorial group with missing post-treatment data, 55% (n = 6) did not complete the tutorial. In addition, at T1 there were no significant differences between the participants in Tutorial group without post-treatment data and those in the Tutorial and Control groups with post-treatment data on the outcome variables, ps = 0.09–1.00, or other key demographics including child’s age, ps = 1.0, parental age, P = 0.58–1.00, and parental education, P = 0.88; the subgroup of the Control group without post-treatment data was not included in the analyses since it comprised only 1 participant. Overall, these attrition analyses appeared to indicate that the post-treatment data were missing at random and that it was appropriate to apply the ITT approach using an estimation method and avoid list-wise deletion [Bell, Kenward, Fairclough, & Horton, 2013].
Analyses and handling missing data
Hierarchical Linear Models [HLM; Singer & Willett, 2003] with Maximum Likelihood estimation were conducted using the HLM7 program [Raudenbush, Bryk, & Congdon, 2011] to apply the ITT approach and avoid list-wise deletion of participants with missing responses. Bayesian estimates of the intercept and slopes (i.e., time vectors) were derived from the final models for each participant using the Level-2 residual file from HLM7 in order to provide estimated means and standard deviations in Table 3, which also includes effect sizes (i.e., Cohen’s d).
Table 3.
Variables & time period | Tutorial group (n= 52) M (SD) | Control group (n = 52) M (SD) | Cohen’s d | |
---|---|---|---|---|
Proximal outcomes | ||||
Parent use of evidence-based strategies | ||||
T1 | 2.64 (0.29) | 2.69 (0.32) | – | |
T2* | 3.09 (0.21) | 2.72 (0.33) | 1.33 | |
T3* | 3.10 (0.21) | 2.74 (0.30) | 1.39 | |
T1 to T2 change* | 0.46 (0.37) | 0.03 (0.44) | 1.06 | |
T1 to T3 change* | 0.46 (0.32) | 0.05 (0.38) | 1.17 | |
Child engagement | ||||
T1 | 2.43 (0.58) | 2.64 (0.39) | – | |
T2* | 2.75 (0.37) | 2.51 (0.52) | 0.53 | |
T3* | 2.83 (0.34) | 2.53 (0.46) | 0.61 | |
T1 to T2 change* | 0.31 (0.77) | −0.14 (0.70) | 0.74 | |
T1 to T3 change* | 0.39 (0.59) | −0.12 (0.61) | 0.85 | |
Broad outcomes | ||||
Child social relating | ||||
T1* | 3.20 (0.60) | 3.46 (0.56) | – | |
T2* | 3.53 (0.41) | 3.23 (0.60) | 0.58 | |
T3* | 3.56 (0.47) | 3.24 (0.67) | 0.55 | |
T1 to T2 change* | 0.33 (0.77) | −0.23 (0.78) | 0.72 | |
T1 to T3 change* | 0.36 (0.81) | −0.22 (0.84) | 0.70 | |
Child nonverbal communication | ||||
T1* | 2.86 (0.63) | 3.20 (0.62) | – | |
T2* | 3.20 (0.45) | 2.98 (0.69) | 0.38 | |
T3* | 3.29 (0.47) | 3.13 (0.71) | 0.27 | |
T1 to T2 change* | 0.34 (0.78) | −0.21 (0.96) | 0.63 | |
T1 to T3 change* | 0.44 (0.78) | −0.06 (0.94) | 0.58 | |
Parenting efficacy | ||||
T1 | 28.77 (4.16) | 29.61 (3.76) | – | |
T2 | 30.77 (3.29) | 29.07 (4.15) | 0.45 | |
T3* | 31.21 (2.96) | 29.49 (3.83) | 0.50 | |
T1 to T2 change | 2.00 (6.04) | −0.54 (5.36) | 0.44 | |
T1 to T3 change* | 2.44 (5.88) | −0.12 (5.15) | 0.46 | |
Parent-child dysfunctional interaction | ||||
T1 | 30.50 (7.94) | 28.52 (7.43) | – | |
T2 | 27.96 (5.69) | 30.06 (7.85) | −0.31 | |
T3* | 26.49 (5.92) | 30.61 (8.03) | −0.58 | |
T1 to T2 change | −2.54 (11.14) | 1.54 (10.62) | −0.37 | |
T1 to T3 change* | −4.01 (11.40) | 2.10 (10.87) | −0.55 | |
Difficult child | ||||
T1 | 37.42 (10.14) | 36.43 (9.21) | – | |
T2 | 35.72 (7.69) | 39.38 (9.93) | −0.41 | |
T3 | 35.63 (7.65) | 39.61 (9.60) | −0.46 | |
T1 to T2 change | −1.70 (13.37) | 2.95 (12.74) | −0.36 | |
T1 to T3 change | −1.80 (13.67) | 3.18 (12.73) | −0.38 | |
Parental distress | ||||
T1 | 31.04 (11.17) | 30.12 (9.36) | – | |
T2 | 30.18 (8.66) | 31.51 (10.87) | −0.14 | |
T3 | 28.80 (8.06) | 32.52 (10.62) | −0.39 | |
T1 to T2 change | −0.86 (15.37) | 1.39 (15.20) | −0.15 | |
T1 to T3 change | −2.23 (14.30) | 2.40 (14.62) | −0.32 |
P < 0.05.
Analyses
HLM overview
Two-level HLMs were used to determine whether there were differences between the Tutorial and Control groups at pre- and post-treatment. At Level-1, time effects were modeled through two dummy vectors coding for T1 (i.e., vector 1 = 0 and vector 2 = 0), T2 (i.e., vector 1 = 1 and vector 2 = 0), and T3 (i.e., vector 1 = 0 and vector 2 = 1) to allow for direct comparisons between the groups at each time point at Level-2; these predictors were modeled as random effects. Age was examined as a covariate at Level-2, but it was not significantly associated with the outcomes at any of the three-time points, ps = 0.08–0.99, and as a result, was not retained in the final models. Tutorial group assignment was examined as a Level-2 predictor of random effects, including as a predictor at T1 to assess whether the groups were comparable at baseline.
Proximal outcomes
For parents’ use of evidence-based strategies: (a) the Tutorial group did not differ significantly from the Control group at T1 (β01 = −0.05, P = 0.55); (b) the Control group did not increase significantly from T1 to T2 (β10 = 0.03, P = 0.66) or T3 (β20 = 0.05, P = 0.46); and (c) the Tutorial group increased significantly from T1 to T2 and T3 and was significantly higher than the Control group at T2 (β11 = 0.42, P < 0.01) and T3 (β21 = 0.41, P < 0.01).
For children’s engagement behaviors: (a) the Tutorial group did not differ significantly from the Control group at T1 (β01 = −0.21 P = 0.09); (b) the Control group did not increase significantly from T1 to T2 (β10 = −0.14, P = 0.28) or T3 (β20 = −0.12, P = 0.30); and (c) the Tutorial group increased significantly from T1 to T2 and T3 and was significantly higher than the Control group at T2 (β11 = 0.45, P = 0.01) and T3 (β21 = 0.51, P < 0.01).
Broad outcomes
For PES parenting efficacy: (a) the Tutorial group did not differ significantly from the Control group at T1 (β01 = −0.84, P = 0.34); (b) the Control group did not increase significantly from T1 to T2 (β10 = −0.53, P = 0.56) or T3 (β20 = −0.12, P = 0.89); and (c) the Tutorial group increased significantly from T1 to T3, but not from T1 to T2 (β11 = 2.54, P = 0.06), and was significantly higher than the Control group at T3 (β21 = 2.57, P < 0.05).
For PSI/SF PCDI: (a) the Tutorial group did not differ significantly from the Control group at T1 (β01 = 1.98, P = 0.23); (b) the Control group did not decrease significantly from T1 to T2 (β10 = 1.54, P = 0.36) or T3 (β20 = 2.09, P = 0.23); and (2) the Tutorial group decreased significantly from T1 to T3, but not from T1 to T2 (β11 = −4.08, P = 0.10), and was significantly lower than the Control group at T3 (β21 = −6.11, P = 0.02). There were no significant differences across time points and/or between the groups on the other two PSI/SF subscales (ps = 0.10–0.68).
For PIA-CV social relating: (a) the Tutorial group was significantly lower than the Control group at T1 (β01 = −0.26, P = 0.04); (b) the Control group did not increase significantly from T1 to T2 (β10 = −0.23, P = 0.08) or T3 (β20 = −0.22, P = 0.09); and (c) the Tutorial group increased significantly from T1 to T2 and T3 and was higher than the Control group at T2 (β11= 0.55, P < 0.01) and T3 (β21 = 0.58, P < 0.01). For Nonverbal Communication: (a) the Tutorial group was significantly lower than the Control group at T1 (β01 = −0.33, P = 0.02); (b) the Control group did not increase significantly from T1 to T2 (β10 = −0.21, P = 0.13) or T3 (β20 = −0.06, P = 0.65); and (c) the Tutorial group increased significantly from T1 to T2 and T3 and was higher than the Control group at T2 (β11 = 0.55, P < 0.01) and T3 (β21 = 0.50 P = 0.02).
Parent satisfaction with tutorial
Parents in the Tutorial group indicated high levels of satisfaction with the: (a) technical aspects (M = 91.00, SD = 10.32) as the overall mean score reflected an average score of at least “4” (or “agree”) on all SUS items; and (b) clinical content (M = 53.90, SD = 5.65) as the overall mean score reflected an average score of at least “3” (or “agree”) on all USQ items.
Discussion
The current study is the first to employ a RCT design with a large sample to evaluate the effectiveness of a self-directed, web-based training aimed at improving parent-child interactions during daily routines. The parenting tutorial was effective in improving key parent and child outcomes immediately (T2) and one month after its completion (T3). Specifically, the Tutorial group exhibited: (a) increased parental use of evidence-based strategies at T2 and T3 (large effect sizes); (b) decreased parenting stress and increased parenting efficacy at T3 (medium effect sizes); and (c) improved child engagement during routines and broader social communication at T2 and T3 (medium to large effect sizes). The Control group did not exhibit any such gains. The parents in the Tutorial group also rated the tutorial highly on usability and were highly satisfied with content. Overall, the tutorial appears to be a promising and accessible way for empowering parents and improving parent-child interactions.
The tutorial had a direct effect on the immediate dynamics of the interactions during routines as it led to changes in the routine-specific strategies parents used and improvements in the behaviors exhibited by children. Parents in the Tutorial group, but not the Control group, reported a significant increase in the evidence-based strategies they were implementing during routines with their children at T2 and T3. These strategies incorporated a variety of evidence-based techniques, including providing simple verbal instructions, using visual schedules, and appropriately modifying routine steps. These strategies were heavily emphasized throughout the tutorial, as parents learned about their general use as well as specific ways to adapt them to meet the unique demands of a given routine. This study, along with a case study [Kashinath, 2006], indicates that after receiving training, parents increase their use of evidence-based teaching strategies during daily routines with their children. Children in the Tutorial group exhibited increased engagement during routines at T2, which was sustained at T3. This finding indicates that children improved in their ability to tolerate and transition during routines, which may include exhibiting fewer externalizing behaviors (e.g., physically struggling, leaving the area, fussing). Overall, it appears that children and their parents were able to advance through the lower levels of participation (i.e., tolerating) and reached the higher levels of participation (i.e., engaging socially).
In addition to changing routine-related behaviors, the tutorial led to improvements in broad outcomes for children and parents. Despite having lower levels on social communication than the Control group at T1, the children in the Tutorial group surpassed the Control group at T2 and T3. These social-communication improvements coincided temporally with children’s increased ability to engage during routines and may suggest that improvements in routine-specific behaviors generalized to broader contexts and interactions. These findings continue to highlight the potential benefits of implementing interventions that focus on routines, as routines can frequently provide opportunities for positive and frequent social interactions that can support early social-communication development [Spagnola & Fiese, 2007].
Furthermore, parenting stress related to challenges in the parent-child relationship (i.e., PSI/SF PCDI scale) declined significantly for parents in the Tutorial group, but not until T3; in contrast, there were no significant declines in stress as measured by the other two PSI/SF scales, which assess parental distress and how difficult the parent perceives the child to be. This pattern of findings suggests that parents’ stress was ameliorated because they, specifically, perceived that the quality of their relationship and interactions with their children improved. Parents in the Tutorial group also exhibited significant gains in parenting efficacy by T3. Along with learning new strategies to use during routines, parents received education on how to perceive and interpret some their children’s challenging behaviors and deconstruct routines into key stages. This new knowledge may have made parents feel that they were more effective managers of their children’s behaviors [Spagnola & Fiese, 2007]. These findings parallel those of other RCTs examining parent-mediated ASD interventions, which have also found declines in parental stress and increases in parenting efficacy/empowerment [Ingersoll et al., 2016; Kasari et al., 2015; Minjarez et al., 2013]. The present study suggests that teaching parents specific and contextualized behavioral strategies may be effective even when they are not part of a specific and comprehensive intervention program.
Given that daily routines represent a transactional pattern of parent-child behaviors, it is possible that parent strategies may have positively affected child behavior, which in turn influenced parents’ sense of efficacy and stress [Spagnola & Fiese, 2007]. There is evidence indicating that lower levels of social relating and higher levels of externalizing behaviors in children with ASD are associated concurrently with higher levels of parenting stress [Davis & Carter, 2008]. Within the current study, for example, it is possible that improvements in children’s participatory behaviors during routines and increases in social relatedness facilitated the later improvements seen in parenting stress for the Tutorial group.
While this study applied an innovative approach (i.e., self-directed, web-based learning) and had notable strengths, including a RCT design and relatively large sample, there are significant limitations that need to be considered carefully. First, there was no direct observation of the parent and child outcomes since the study was conducted remotely; all outcomes were based on parental reports from a single respondent for each parent-child dyad. This significant limitation introduced the possibility that improvements in outcomes may have reflected a social desirability response bias and/or a placebo effect. However, it is unlikely that the social desirability or placebo effect account for the entire pattern of results, given that not all outcomes (i.e., parenting efficacy and parenting stress) demonstrated immediate improvement at T2. Furthermore, the lack of direct observation measures is magnified for the proximal outcomes given that those routine-specific surveys were developed and used for the first time as part of this study. It will be imperative for future studies to examine child outcomes and the fidelity with which parents’ implement strategies with objective measures. Second, because there were no in-person visits, we were not able to confirm participants’ ASD diagnosis through a gold-standard evaluation conducted by the research team and, instead, had to rely on reviewing diagnostic reports from community-based diagnosticians. Third, the current sample was homogeneous with respect to key socioeconomic factors, as the majority of participants were White, Non-Hispanic mothers with a college education. As a consequence, we were not able to examine the extent to which SES factors: (a) functioned as moderators of the treatment effect; and/or (b) impacted parents’ preferential and optimal learning style. Relatedly, because 90% of the parents in the sample were mothers, we were unable to compare whether outcomes differed for families based on whether it was the father or mother who was learning and completing the tutorial. Fourth, we cannot speak to the long-term sustainability of the gains demonstrated by parents and children because the study employed a relatively short follow-up window (i.e., T1–T3 occurred during two-month period). Finally, participants were also receiving other community intervention services, which we were unable to quantify and examine in the current analyses.
In addition to using direct observation for measuring key outcomes, including a more diverse population, and applying a longer follow-up period, future studies should investigate: (a) whether gains in child behaviors generalize to different situations/contexts; and (b) the potential mechanisms by which the tutorial had an effect on broad outcomes. Furthermore, there needs to be additional empirical investigation into the extent to which the behaviors targeted and improved during routines generalize to other settings and contexts.
Overall, the parenting tutorial provides an effective, low-cost way for improving parent-child interactions during daily routines, which may improve children’s broader social communication and augment other types of treatment services. Because the tutorial employs self-directed, web-based training that can be completed in the home, it has the potential to circumvent prominent barriers to obtaining services such as shortages of service providers and logistical constraints. Empowering parents with knowledge and skills for teaching their young child with ASD provides resources to the people with the most vested interest and longest lasting influence on a child’s long-term growth and development.
Lay Summary.
This web-based tutorial helped parents of children with ASD use proven strategies to improve their child’s participation in daily routines at home. Parents who used the tutorial reported less parenting stress, felt better about their parenting skills, and reported better child social interactions compared to parents who did not use the tutorial. This tutorial may be especially helpful for families who have limited access to services, as it can be completed at home.
Acknowledgments
This study was funded by the National Institute of Mental Health, National Institutes of Health, and the Department of Health and Human Services, under Small Business Innovation Research (SBIR) Grant No. R44MH086936, which was awarded to the Center for Psychological Consultation (Kenneth Kobak, PI). The authors acknowledge the contributions of Shanni Geller, the primary research assistant, who managed recruitment, enrollment, and data collection at the University of Washington. We are grateful for the families who gave their valuable time to participate in this study. While the lead author (LVI) does not have a financial conflict of interest (FCOI), the supporting authors (KK, AS, LW, ZW, and WLS) have a FCOI to declare as they receive profits from the sales of the tutorial.
Contributor Information
Lisa V. Ibañez, From the Department of Psychology, University of Washington, Seattle, Washington
Kenneth Kobak, Center for Psychological Consultation, Madison, Wisconsin.
Amy Swanson, Vanderbilt Kennedy Center Treatment and Research Institute for Autism Spectrum Disorders, (TRIAD), Vanderbilt University Medical Center, Nashville, Tennessee.
Lisa Wallace, Vanderbilt Kennedy Center Treatment and Research Institute for Autism Spectrum Disorders, (TRIAD), Vanderbilt University Medical Center, Nashville, Tennessee.
Zachary Warren, Department of Pediatrics, Vanderbilt Kennedy Center Treatment and Research Institute for Autism Spectrum Disorders, (TRIAD), Vanderbilt University Medical Center, Nashville, Tennessee.
Wendy L. Stone, Department of Psychology, University of Washington, Seattle, Washington
References
- Abidin RR. Parenting stress index professional manual. 3rd. Odessa, FL: Psychological Assessment Resources, Inc; 1995. [Google Scholar]
- Bailey K. Supporting families. In: Chawarska K, Klin A, Volkmar FR, editors. Autism spectrum disorders in infants and toddlers: Diagnosis, assessment, and treatment. New York: Guilford; 2008. pp. 300–326. [Google Scholar]
- Ballew P, Castro S, Claus J, Kittur N, Brennan L, Brownson RC. Developing web-based training for public health practitioners: What can we learn from a review of five disciplines? Health Education Research. 2013;28:276–287. doi: 10.1093/her/cys098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bell ML, Kenward MG, Fairclough DL, Horton NJ. Differential dropout and bias in randomised controlled trials: When it matters and when it may not. BMJ. 2013;346:e8668. doi: 10.1136/bmj.e8668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bangor A, Kortum P, Miller JA. The system usability scale (SUS): An empirical evaluation. International Journal of Human-Computer Interaction. 2008;24:574–594. [Google Scholar]
- Brooke J. SUS: A quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL, editors. Usability evaluation in industry. London: Taylor and Francis; 1996. pp. 189–194. [Google Scholar]
- Carter AS, Martínez-Pedraza FDL, Gray SA. Stability and individual change in depressive symptoms among mothers raising young children with ASD: Maternal and child correlates. Journal of Clinical Psychology. 2009;65:1270–1280. doi: 10.1002/jclp.20634. [DOI] [PubMed] [Google Scholar]
- Carter AS, Messinger DS, Stone WL, Celimli S, Nahmias AS, Yoder P. A randomized controlled trial of Hanen’s more than words in toddlers with early autism symptoms. Journal of Child Psychology and Psychiatry. 2011;52:741–752. doi: 10.1111/j.1469-7610.2011.02395.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chiri G, Warfield ME. Unmet need and problems accessing core health care services for children with autism spectrum disorder. Maternal and Child Health Journal. 2012;16:1081–1091. doi: 10.1007/s10995-011-0833-6. [DOI] [PubMed] [Google Scholar]
- Davis NO, Carter AS. Parenting stress in mothers and fathers of toddlers with autism spectrum disorders: Associations with child characteristics. Journal of Autism and Developmental Disorders. 2008;38:1278–1291. doi: 10.1007/s10803-007-0512-z. [DOI] [PubMed] [Google Scholar]
- Downey RG, King CV. Missing data in Likert ratings: A comparison of replacement methods. Journal of General Psychology. 1998;125:175–191. doi: 10.1080/00221309809595542. [DOI] [PubMed] [Google Scholar]
- Hamad CD, Serna RW, Morrison L, Fleming R. Extending the reach of early intervention training for practitioners: A preliminary investigation of an online curriculum for teaching behavioral intervention knowledge in autism to families and service providers. Infants and Young Children. 2010;23:195. doi: 10.1097/IYC.0b013e3181e32d5e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ingersoll B, Berger N. Correction: Parent engagement with a telehealth-based parent-mediated intervention program for children with autism spectrum disorders: Predictors of program use and parent outcomes. Journal of Medical Internet Research. 2015;17:e257. doi: 10.2196/jmir.4913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ingersoll B, Wainer A. Initial efficacy of project ImPACT: A parent-mediated social communication intervention for young children with ASD. Journal of Autism and Developmental Disorders. 2013;43:2943–2952. doi: 10.1007/s10803-013-1840-9. [DOI] [PubMed] [Google Scholar]
- Ingersoll B, Wainer AL, Berger NI, Pickard KE, Bonter N. Comparison of a self-directed and therapist-assisted telehealth parent-mediated intervention for children with ASD: A pilot RCT. Journal of Autism and Developmental Disorders. 2016;46:2275–2284. doi: 10.1007/s10803-016-2755-z. [DOI] [PubMed] [Google Scholar]
- Jang J, Dixon DR, Tarbox J, Granpeesheh D, Kornack J, de Nocker Y. Randomized trial of an eLearning program for training family members of children with autism in the principles and procedures of applied behavior analysis. Research in Autism Spectrum Disorders. 2012;6:852–856. [Google Scholar]
- Kasari C, Gulsrud AC, Wong C, Kwon S, Locke J. Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism. Journal of Autism and Developmental Disorders. 2010;40:1045–1056. doi: 10.1007/s10803-010-0955-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kasari C, Gulsrud A, Paparella T, Hellemann G, Berry K. Randomized comparative efficacy study of parent-mediated interventions for toddlers with autism. Journal of Consulting and Clinical Psychology. 2015;83:554. doi: 10.1037/a0039080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kashinath S. Enhancing generalized teaching strategy use in daily routines by parents of children with autism. Journal of Speech, Language, and Hearing Research. 2006;49:466–485. doi: 10.1044/1092-4388(2006/036). [DOI] [PubMed] [Google Scholar]
- Knowles MS. The adult learner: A neglected species. 4th. Houston: Gulf Publishing; 1990. [Google Scholar]
- Kobak KA. A comparison of face-to-face and videocon-ference administration of the Hamilton depression rating scale. Journal of Telemedicine and Telecare. 2004;10:231–235. doi: 10.1258/1357633041424368. [DOI] [PubMed] [Google Scholar]
- Kobak KA, Craske MG, Rose RD, Wolitsky-Taylor K. Web-based therapist training on cognitive behavior therapy for anxiety disorders: A pilot study. Psychotherapy. 2013;50:235. doi: 10.1037/a0030568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kobak KA, Reynolds WM, Greist JH. Computerized and clinician assessment of depression and anxiety: Respondent evaluation and satisfaction. Journal of Personality Assessment. 1994;63:173–180. doi: 10.1207/s15327752jpa6301_14. [DOI] [PubMed] [Google Scholar]
- Marquenie K, Rodger S, Mangohig K, Cronin A. Dinnertime and bedtime routines and rituals in families with a young child with an autism spectrum disorder. Australian Occupational Therapy Journal. 2011;58:145–154. doi: 10.1111/j.1440-1630.2010.00896.x. [DOI] [PubMed] [Google Scholar]
- Minjarez MB, Mercier EM, Williams SE, Hardan AY. Impact of pivotal response training group therapy on stress and empowerment in parents of children with autism. Journal of Positive Behavior Interventions. 2013;15:71–78. [Google Scholar]
- Moher D, Schulz KF, Altman DG. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology. 2001;1:2. doi: 10.1186/1471-2288-1-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nieuwboer CC, Fukkink RG, Hermanns JM. Online programs as tools to improve parenting: A meta-analytic review. Children and Youth Services Review. 2013;35:1823–1829. [Google Scholar]
- Odom SL, Collet-Klingenberg L, Rogers SJ, Hatton DD. Evidence-based practices in interventions for children and youth with autism spectrum disorders. Preventing School Failure: Alternative Education for Children and Youth. 2010;54:275–282. [Google Scholar]
- Raudenbush SW, Bryk AS, Congdon R. HLM 7.00 for Windows [Computer software] Lincolnwood, IL: Scientific Software International, Inc; 2011. [Google Scholar]
- Rogers SJ, Dawson G. The early start Denver model for young children with autism: Promoting language, learning, and engagement. New York: Guilford Press; 2010. [Google Scholar]
- Rogers SJ, Estes A, Lord C, Vismara L, Winter J, Fitzpatrick A, Dawson G. Effects of a brief Early Start Denver Model (ESDM)–based parent intervention on toddlers at risk for autism spectrum disorders: A randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry. 2012;51:1052–1065. doi: 10.1016/j.jaac.2012.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schreibman L, Dawson G, Stahmer AC, Landa R, Rogers SJ, McGee GG, Halladay A. Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. Journal of Autism and Developmental Disorders. 2015;45:2411–2428. doi: 10.1007/s10803-015-2407-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Singer JD, Willett JB. Applied longitudinal data analysis: Modeling change and event occurrence. Oxford University Press; 2003. [Google Scholar]
- Spagnola M, Fiese BH. Family routines and rituals: A context for development in the lives of. Young Children. Infants & Young Children. 2007;20:284–299. [Google Scholar]
- Stahmer AC, Mandell DS. State infant/toddler program policies for eligibility and services provision for young children with autism. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:29–37. doi: 10.1007/s10488-006-0060-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stone WL, Coonrod EE, Pozdol SL, Turner LM. The parent interview for autism-clinical version (PIA-CV) a measure of behavioral change for young children with autism. Autism. 2003;7:9–30. doi: 10.1177/1362361303007001003. [DOI] [PubMed] [Google Scholar]
- Stone WL, Hogan KL. A structured parent interview for identifying young children with autism. Journal of Autism and Developmental Disorders. 1993;23:639–652. doi: 10.1007/BF01046106. [DOI] [PubMed] [Google Scholar]
- Tabachnick BG, Fidell LS, Osterlind SJ. Using multivariate statistics. 5th. Boston, MA: Allyn and Bacon; 2001. [Google Scholar]
- Teti DM, Gelfand DM. Behavioral competence among mothers of infants in the first year: The mediational role of maternal self-efficacy. Child Development. 1991;62:918–929. doi: 10.1111/j.1467-8624.1991.tb01580.x. [DOI] [PubMed] [Google Scholar]
- Tomanik S, Harris GE, Hawkins J. The relationship between behaviors exhibited by children with autism and maternal stress. Journal of Intellectual and Developmental Disability. 2004;29:16–26. [Google Scholar]
- Vincent A, Ross D. Personalize training: determine learning styles, personality types and multiple intelligences online. The Learning Organization. 2001;8:36–43. [Google Scholar]
- Vismara LA, McCormick C, Young GS, Nadhan A, Monlux K. Preliminary findings of a telehealth approach to parent training in autism. Journal of Autism and Developmental Disorders. 2013;43:2953–2969. doi: 10.1007/s10803-013-1841-8. [DOI] [PubMed] [Google Scholar]
- Vismara LA, Young GS, Rogers SJ. Telehealth for expanding the reach of early autism training to parents. Autism Research and Treatment. 2012;2012:12. doi: 10.1155/2012/121878. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wainer AL, Ingersoll BR. Increasing access to an ASD imitation intervention via a telehealth parent training program. Journal of Autism and Developmental Disorders. 2015;45:3877–3890. doi: 10.1007/s10803-014-2186-7. [DOI] [PubMed] [Google Scholar]
- Wetherby AM, Guthrie W, Woods J, Schatschneider C, Holland RD, Morgan L, Lord C. Parent-implemented social intervention for toddlers with autism: An RCT. Pediatrics. 2014;134:1084–1093. doi: 10.1542/peds.2014-0757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wise MD, Little AA, Holliman JB, Wise PH, Wang CJ. Can state early intervention programs meet the increased demand of children suspected of having autism spectrum disorders? Journal of Developmental & Behavioral Pediatrics. 2010;31:469–476. doi: 10.1097/DBP.0b013e3181e56db2. [DOI] [PubMed] [Google Scholar]