Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Apr 1.
Published in final edited form as: Neuropsychiatry (London). 2013 Apr;3(2):169–180. doi: 10.2217/npy.13.14

Toward an exportable parent training program for disruptive behaviors in autism spectrum disorders

Karen Bearss 1, Luc Lecavalier 2, Noha Minshawi 3, Cynthia Johnson 4, Tristram Smith 5, Benjamin Handen 4, Denis Sukhodolsky 6, Michael Aman 2, Naomi Swiezy 3, Eric Butter 2, Lawrence Scahill 1
PMCID: PMC3678377  NIHMSID: NIHMS476451  PMID: 23772233

Abstract

Autism spectrum disorders (ASD) are chronic conditions of early childhood onset characterized by profound deficits in social interaction, impaired communication, and repetitive behavior. The prevalence of ASD is now estimated to be 1 in 88 children. As the number of identified cases of ASD has grown, so have the challenges of serving these children and their families. Unfortunately, the empirical foundation for many interventions for this population is not firmly established. Thus, there is a pressing need to conduct trials that will expand the evidence base and guide clinical treatment. Investigators from the Research Units in Pediatric Psychopharmacology (RUPP; Indiana University, Ohio State University, University of Pittsburgh, Yale University) followed a treatment development model outlined by an NIMH ad hoc committee to develop and test a parent training (PT) treatment manual for children with ASD accompanied by disruptive behavior problems. This article describes the process of manual development and cross-site therapist training, establishment and maintenance of treatment integrity, assessment of treatment acceptance by families as well as primary outcomes of three trials. Results suggest the structured PT program can be delivered with a high degree of fidelity within and across therapists, is acceptable to parents and can produce significant reductions in disruptive behaviors in children with ASD.

Autism spectrum disorders (ASD) are chronic conditions of early childhood onset characterized by profound deficits in social interaction, impaired communication, and repetitive behavior. The prevalence of ASD is now estimated to be 1 in 88 children (CDC, 2012), a figure that is dramatically higher than previous estimates. The increase in the detected prevalence is due in large part to better community sampling methods, broadening of the case definition and improved diagnostic precision (King & Bearman, 2010).

Of the many available interventions, most psychosocial and psychopharmacological treatments for children with ASD lack a firm empirical foundation (AHRQ, 2011). Thus, the increase in the detected prevalence of ASD and subsequent increased demand for services occurs against the backdrop of insufficient evidence for treatments, posing a major public health issue. There is a pressing need to conduct trials that will expand the availability of empirically-supported, time-limited and cost-effective interventions for children with ASD.

Parent training (PT) for families of children with ASD warrants interest as a potential intervention model for several reasons. First, PT is considered an efficacious treatment for typically developing children with disruptive behavior (Kazdin, 2005; Lundahl et al., 2006; Reyno & McGrath, 2006; Webster-Stratton & Reid, 2010; Zisser & Eyberg, 2010). Second, it is traditionally a time-limited approach (typically 10–20 sessions) delivered during brief (1–1.5 hour) weekly sessions. As such, it may be feasible to administer in a wide range of service settings. Third, it highlights the role of parents as the change agent. It is parents, after all, who confront the daily struggles that often come with rearing a child with ASD (Kogan et al., 2008; Tonge et al., 2006). For example, as many as 50–70% children with ASD have challenging behaviors that require skillful responses from their parents. These behaviors may consist of tantrums, aggression, noncompliance with routine demands, self-injury, property destruction, and hyperactivity (Lecavalier, 2006). Most children with ASD also require assistance with accomplishing activities of daily living and many actively resist acquiring new skills or performing already acquired skills. Indeed, on measures of adaptive functioning, children with ASD are often a full standard deviation below their assessed cognitive ability (Carter et al., 1998; Paul et al, 2004). To reduce noncompliance and improve adaptive functioning, parents may need specific instruction in behavioral interventions and procedures.

Disruptive behaviors also interfere with family quality of life (Herring et al., 2006). Compared to parents of typically developing children, parents of children with ASD report a greater sense of helplessness and are more likely to avoid conflict when facing challenges of parenting (Pisula & Kossakowska, 2010). Although the number of young children with ASD receiving special educational services has steadily increased, most school-based programs focus on the child and do not include parent training (U. S. Government Accountability Office, 2005).

To date, most research on parent-mediated interventions in this population has focused on helping parents treat core features of ASD, such as socialization and communication (Aldred, Green, & Adams, 2004; Carter et al., 2011; Dawson et al., 2010; Drew et al., 2002; Green et al., 2010; Kasari et al., 2010; Oosterling et al., 2010) or imitation skills (Ingersoll & Gergans, 2007). Parents have been included in treatment and taught ways to enhance their child’s eye contact, joint attention and play skills (Kasari et al., 2010). Parent training also has been studied as an adjunct to school-based programs for children with ASD (Dawson et al., 2010; Landa et al., 2011). The study by Dawson et al. (2010) included twice-monthly parent training to augment intensive child-focused intervention targeting cognitive skills, adaptive behavior, and core symptoms of ASD. By contrast, PT for disruptive behavior in children with ASD provides instruction for parents to reduce problem behavior and increase compliance (Anderson & McMillan, 2001; Ducharme & Drain, 2004; Lerman et al., 2000; Moes & Frea, 2002; Wahler et al., 2004). Examples of commonly used behavioral strategies include the use of antecedent management (e.g., visual schedules, functional communication training, environmental manipulations), reinforcement procedures (e.g., differential reinforcement, contingency management), compliance training, teaching skills (e.g., task analysis, prompting procedures) and other approaches to consequences (e.g., time-out).

The empirical support for PT for reducing disruptive behavior in children with ASD comes largely from single-subject design studies (Campbell, 2003; Horner et al., 2002; Odom et al, 2003; Smith et al., 2007) and a handful of small randomized clinical trials (RCT; Laugesen et al., 2008; see McConachie & Diggle, 2007 review; Sofronoff et al., 2004; Whittingham et al., 2009). Although these studies offer proof of concept, drawbacks include small sample sizes, nonrandom treatment assignment and poorly characterized samples (Smith et al., 2007). In addition, few studies used structured manuals, which hinders replication and dissemination of PT in ASDs. Rigorous study of PT as a stand-alone intervention for disruptive behavior in children with ASD remains limited, which leaves clinicians and families with inadequate guidance on treatment options (Smith et al., 2007).

This inadequate state of treatment development was identified by a National Institute of Mental Health (NIMH) ad hoc committee (Lord et al., 2005; Smith et al., 2007). The committee cited two necessary prerequisites for conducting a large-scale RCT in ASD: the development of a treatment manual and a pilot study to confirm feasibility (Smith et al., 2007). A treatment manual assembles validated techniques into a standardized format that guides consistent delivery of the intervention (Johnson et al., 2007). Pilot feasibility trials are used to demonstrate that the treatment is acceptable to families and to show that the manual can be delivered consistently by therapists at different sites (Smith et al., 2007).

Investigators from the Research Units in Pediatric Psychopharmacology (RUPP; Indiana University, Ohio State University, University of Pittsburgh, Yale University) followed the model outlined by the NIMH ad hoc committee to develop and test a PT treatment manual for children (ages 4 to 13 years) with ASD accompanied by disruptive behavior problems (Johnson et al., 2007; RUPP Autism Network, 2007; Aman et al., 2009; Scahill et al., 2012). The development of the manual is described in Johnson et al. (2007). Results of the pilot feasibility trial were published in the same year (RUPP Autism Network, 2007). The manual was then used in a large-scale, multi-site randomized trial comparing risperidone only versus risperidone plus parent training for school-age children with ASDs accompanied by serious behavioral problems (Aman et al., 2009; Scahill et al., 2012). This treatment development process was replicated in order to test PT as a standalone intervention in younger children with ASDs (ages 3–6 years) (Bearss et al., in press). This article describes the development of a structured PT program for children with ASDs, therapist training, establishment and maintenance of treatment integrity and assessment of treatment acceptance by families. We also summarize primary outcomes of the three trials.

Methods

Settings and Subjects

All three trials were approved by their study site Human Investigation Committee and all participating parents provided written informed consent prior to inclusion in the study. The three trials conducted by our group used similar versions of the treatment manual, therapist training methods, process measures and outcome measures. The first trial was an open-label pilot study of 17 school-age subjects recruited from four sites (RUPP Autism Network, 2007). The second study, also a multi-site trial, enrolled 124 school-age children who were randomly assigned to medication only (risperidone) or risperidone plus PT (Aman et al., 2009; Scahill et al., 2012). The third trial was an open, single site pilot trial of 16 preschool-age subjects (Bearss et al., in press). In all three trials, children were required to have an ASD diagnosis, moderate or greater behavioral problems, receptive language age equivalents greater than 18 months, and treatment with stable medication or no medication prior to entry.

Manual Development

In the absence of an available “off the shelf” manual, RUPP investigators constructed a treatment manual based on the principles of applied behavior analysis (ABA) (Johnson et al., 2007). Effective ABA intervention rests on the principle that disruptive, noncompliant and explosive behaviors serve a function for the child. A key to understanding the function of a given behavior is to identify the situations that precede the behavior and the consequences that follow the behavior. The exploration of consequences in this context is used to identify how the behavior is being inadvertently reinforced. Another underlying assumption of the PT manual is that reduction of disruptive behavior is a prerequisite to participate in learning activities and to make gains in adaptive living skills. In order to get beyond blame for the child’s maladaptive behavior, therapists are encouraged to remind parents that raising children with ASD requires particular challenges for parents that require specialized parenting skills.

The manual consists of 11 core sessions, three optional sessions, three booster sessions (two via telephone, one face-to-face), as well as two home visits. Core and optional sessions are delivered to the child’s primary caregiver(s) over 16 weeks. This schedule is tolerant of cancellations and rescheduling in order to insure that the full “dose” of the PT program can be delivered. This flexible schedule also offers the opportunity for parents to practice the various skills provided in each session. The telephone and booster sessions focus on generalization and maintenance of skills.

The first few sessions focus on behavioral strategies designed to reduce the child’s disruptive behavior. Later sessions provide strategies for teaching the child new skills and generalization of gains over time. This sequence is intended to make the child’s disruptive behaviors more manageable, which then sets the stage for skill development.

Sessions are 60–90 minutes in duration and are delivered individually to the primary caregiver by masters- or doctoral-level clinicians. Each session includes a therapist script and session-specific parent activity sheets (e.g., having parents identify antecedents and consequences of a problem behavior, having parents develop a task analysis for a grooming skill). Sessions use direct instruction, modeling, and role-playing to promote parental skill acquisition. Video vignettes are also used to illustrate skill implementation (e.g., demonstrating proper use of specific techniques such as guided compliance) or to show the connection between the antecedent and the consequence (e.g., one vignette shows that giving the child a cookie to quell a tantrum, which reinforces explosive behavior). The video vignettes are also used to test parent knowledge of materials covered in the session (e.g., having parents identify which steps of ‘planned ignoring’ in the vignette were correctly or incorrectly implemented). Families are given homework assignments at the end of each session designed to practice new skills learned in the PT sessions. Although homework assignments parallel the session content, the targeted behaviors and the selection of strategies are individualized for each child. At the completion of each session, the therapist documents each of the targeted strategies in the child’s Behavior Support Plan (BSP). This becomes a cumulative record of the individually designed techniques and procedures introduced in the program.

Therapist training

A standardized therapist training program was developed to promote consistent delivery of the PT manual across therapists. All PT therapists were required to have a masters or doctoral degree in clinical psychology, behavior analysis, or related profession. Prior to working with study subjects, new therapists received didactic training on the manual and watched a full set of video-recorded sessions delivered by an expert therapist. They also provided treatment to a non-study “training” case and recorded all sessions from this case for review by the manual developers, who scored therapist’s fidelity to the manual. Therapists needed to deliver each session with a minimum of 80% fidelity in order to begin providing PT in the study. As a check on whether fidelity was maintained over the course of the trial, a 10% random sample of sessions from each therapist was reviewed. An individual remediation plan was created for any therapist who subsequently fell below the 80% criterion with a study patient.

Treatment fidelity was also supported through weekly therapist conference calls. On these calls, therapists could present new cases and discuss barriers to implementing PT. Problems with parental adherence or attendance were also discussed on these calls. Discussions on calls frequently produced workable solutions and built up a body of precedents to handle future problems.

Process Measures

In addition to commonly used outcome measures described below, our group developed new measures to assess therapist fidelity, parental adherence and parent satisfaction.

Therapist Fidelity Checklists were created for each PT session to be completed by the therapist after each session. The checklists indicate the therapist goals for each Core and Optional session. The therapist rates his or her adherence to each of the session’s goals on a scale of 0 to 2 (0 = goal not achieved; 1 = goal partially achieved; 2 = goal fully achieved). Therapist fidelity can be expressed as a percentage (e.g., a score of 2 on all goals for a given session equals 100% fidelity for that session.)

Parent Adherence to treatment was measured on a checklist specifically designed for each Core and Optional session. After each session, the therapist rated parental attainment of the session-specific objectives. Each objective is rated on a 0 to 2 scale (0 = objective not achieved; 1 = objective partially achieved; 2 = objective fully achieved) based on observed parent behaviors in the session. Parent attainment of session objectives can also be expressed as a percentage (e.g., a parent with a score of 2 on each of the session objectives would receive a score of 100% adherence.) Parent behaviors considered in the rating include evidence of homework completion (e.g., collecting data on a specific behavior), accurate responses to video vignettes and written scenarios, as well as demonstration of techniques such as use of effective commands in role-play.

To rate therapist fidelity and parent adherence in the aggregate for a given trial, a random 10% sample of video recordings was scored by an independent rater. We defined acceptable therapist fidelity as 80% and acceptable parent adherence as 65%. These thresholds were intended to account for variability across therapists and parents over the course of the 6-month program.

A Parent Satisfaction Questionnaire was developed for the initial RUPP-PT feasibility pilot with slightly modified versions used in the subsequent trials. The questionnaire was administered at the end of the PT program. Items administered across all three trials included questions on the content of PT sessions, the number and length of sessions, the teaching tools (videotape vignettes, in-session worksheets and homework), and the level of confidence for handling future behavioral problems. Parents were also asked to indicate how often they applied the behavior management principles in daily life, which aspects of the PT program were particularly valuable and which aspects were less useful.

Outcome Measures

Children were assessed at baseline, at specific time points throughout the trial and at endpoint (Week 24).

The Aberrant Behavior Checklist (ABC; Aman et al., 1985a, 1985b) is a parent- and teacher-report measure with 58 items, each rated on a four-point Likert scale (with higher scores being more severe) on five subscales: Irritability (tantrums, aggression and self-injury, 15 items); Social Withdrawal (16 items); Stereotypies (7 items); Hyperactivity (16 items); and Inappropriate Speech (4 items), (Aman et al., 1985a; Brown et al., 2002). The primary outcome measure for the three trials was the commonly-used parent-reported ABC Irritability subscale (RUPP Autism Network, 2002).

The Home Situations Questionnaire (HSQ) (Barkley & Murphy, 1998) is a parent-rated scale for child noncompliance across everyday situations. This version included 25 items that were rated “yes” or “no”; “yes” items were then scored from 1 (mild) to 9 (severe). The total severity score is divided by 25 to obtain a per item mean. This slightly modified version of the HSQ for children with ASDs was used in all three trials and has been found to be reliable (Chowdury et al., 2010).

The Vineland Adaptive Behavior Scales – Interview Format (VABS; Sparrow, Balla, & Cicchetti, 1984) assesses adaptive functioning across several domains and relies on the primary caretaker to describe what the child actually does in the course of daily living. The Vineland provides standard scores (against population norms of 100 ± 15) on four Core domains (Communication, Daily Living Skills, Socialization, Motor Skills) as well as an Adaptive Behavior Composite. Since its reintroduction in 1984, the VABS has become the most commonly used measure of adaptive functioning in the developmental disabilities field (Sparrow, Balla, & Cicchetti, 1984). The clinician-administered interview format was utilized across the trials.

Parent Target Problems (Arnold et al., 2003) is an individualized description of the child’s two most pressing problems nominated by the primary caregiver. After the parent identifies the two problems, the independent evaluator asks about the frequency (for episodic behaviors) or constancy (for problems such as hyperactivity that reflect more enduring patterns), intensity and impact of the behavior on the family. The descriptions are recorded in brief narratives. The target problem narrative at baseline is reviewed and revised at specific time points during the study. The independent evaluator uses this description along with all other available information to make the CGI-I rating.

The Clinical Global Impression - Improvement Scale (CGI-I; Guy, 1976) is a 7-point scale designed to measure overall improvement from baseline. Scores range from 1 (Very Much Improved) to 4 (Unchanged) to 7 (Very Much Worse). The CGI-I was used by the independent evaluator (IE) for each study. This is an individual who is uninvolved in the administration of the PT and does not discuss cases with the therapist. The IE’s role is to assess overall response to treatment based on all available information (HSQ, ABC, and Parent Target Problems). Subjects who received CGI-I scores of 1 (Very Much Improved) or 2 (Much Improved) at Week 24 are classified as positive responders; all other subjects are classified as non-responders.

Results

Table 2 summarizes demographic, clinical, and educational information for study subjects across the three trials. As shown, most children had a diagnosis of Autistic Disorder, most were male and approximately one-third had an IQ below 70.

Table 2.

Demographic data for the three trials

RUPP-PT Pilot (N=17) RUPP -PT RCT (N=75) PreK Pilot (N=16)
Age Range (in years) 4 – 13 4 – 13 3 – 6
Mean Age (SD) (in years) 7.7 (2.6) 7.4 (2.2) 4.9 (0.8)
N (%) N (%) N (%)
Male 14 (82.4) 65 (86.7) 16 (100)
Cognitive Functioning
 ≥70 * 46 (63.0) 10 (62.5)
 <70 * 27 (37.0) 6 (37.5)
Income
 <40,000 5 (29.4) 35 (46.7) 3 (18.8)
 40–60,000 8 (47.1) 11 (14.7) 3 (18.8)
 60–90,000 4 (23.5) 16 (21.3) 2 (12.5)
 >90,000 0 (0.0) 13 (17.3) 8 (50.0)
Ethnicitya
 Caucasian 15 (88.2) 59 (78.7) 13 (81.3)
 African-American 2 (11.8) 9 (12.1) 0 (0.0)
 Hispanic 2 (11.8) 4 (5.3) 2 (12.5)
 Other 1 (5.9) 3 (4.0) 1 (6.3)
Diagnosis
 Autistic Disorder 11 (64.7) 49 (65.3) 7 (43.8)
 Asperger’s Disorder 2 (11.8) 4 (5.3) 0 (0.0)
 PDD-NOS 3 (17.6) 22 (29.3) 9 (56.3)
 Missing 1 (5.6) 0 (0.0) 0 (0.0)
School Programming (%)
 Special Ed Class/School 1 (64.7) 27 (35.1) 7 (43.8)
 Regular Ed with Services 2 (11.8) 7 (9.1) 3 (18.8)
 Regular Ed Classroom 4 (23.5) 26 (33.8) 6 (37.5)
 Homeschool/No School 0 (0.0) 17 (22.1) 0 (0.0)
Open in a new tab
*

The RUPP pilot study reported mean IQ scores (X=55.9, SD=22.3)

a

In the RUPP-PT pilot, the total N is greater than 100% because 18% of the participants selected more than one race identification category

Table 3 summarizes therapist fidelity and parent acceptability across the three trials. Based on the independent review of the 10% random sample of therapy sessions, therapist fidelity to the manual was very high, ranging from 93–95%. Parent acceptance of the PT program across the three trials was excellent, with parents attending between 84–93% of the core sessions and adhering to session materials and homework assignments at a rate of 80–89%. Attrition varied across the three trials, with the highest drop-out rate in the RUPP-PT trial (27%). In comparison, the two pilot trials had lower rates of drop out, ranging from 13–18%. Families reported a high level of satisfaction, with most respondents indicating increased confidence in handling current and future behaviors and saying they would recommend the program to other parents who have children with similar problems.

Table 3.

Therapist fidelity and parent acceptability for the three trials

RUPP-PT Pilot (N=17) RUPP-PT RCT (N=75) Pre K Pilot (N=16)
Therapist Fidelity Total 94% 95% 93%
Parent
 Session Adherence 80% 89% 87%
 Core Session Attendance 93% 86% 84%
 Attrition 18% 27% 13%
Satisfaction Ratings (N=11) (N=63) (N=14)
  Increased Confidence in Handling Current Behaviors 100% 91% 100%
  Increased Confidence in Handling Future Behaviors 91% 83% 100%
  Recommend program to another parent 91% 91% 100%
Open in a new tab

Table 4 summarizes within group baseline and endpoint results across the three trials. Primary outcomes of interest included changes on the parent-rated Aberrant Behavior Checklist – Irritability (ABC-I) subscale and the Home Situations Questionnaire (HSQ). Secondary analyses included change on the Vineland Daily Living Skills subdomain as well as ratings of improvement on the CGI.

Table 4.

Parent Ratings on the Aberrant Behavior Checklist and Home Situations Questionnaire

Standard Scores RUPP-PT PILOT (N=17) ES2 RUPP-PT RCT (N=75) ES2 PRE K PILOT (N=16)1 ES2
Baseline Week 24 Baseline Week 24 Baseline Week 24
X (SD) X (SD) X (SD) X (SD) X (SD) X (SD)
ABC Irritability 24.3 (9.3) 16.1 (9.5) 0.88* 29.3 (7.0) 11.0 (6.6) 2.61** 16.0 (9.2) 7.4 (6.2) 0.94**
HSQ 3.6 (1.1) 2.2 (1.5) 1.27* 4.3 (1.7) 1.2 (1.4) 1.82** 3.0 (1.1) 1.4 (1.2) 1.44**
Vineland DLS 38.3 (16.4) 45.2 (18.2) 0.42 50.8 (18.5) 55.7 (21.9) 0.26 67.2 (9.7) 67.3 (11.3) 0.01
CGI-Improve N (%)3
 Much/Very Much 9 (52.9) 62 (82.7) 14 (87.5)
 Minimal 5 (29.4) 10 (13.3) 0 (0.0)
 Nonresponder 3 (17.6) 3 (4.0) 2 (12.5)
Open in a new tab
1

Baseline data from two drop outs carried forward

2

Within Subjects ES = (Mean at Baseline – Mean at Endpoint)/Baseline SD

3

Rating at last visit

*

p < 0.01

**

p < 0.001

Statistically significant reductions in disruptive behavior were observed in all three studies. Standard scores in the Daily Living Skills domain of the Vineland did not improve significantly on any of three trials but indicated that children were at least keeping pace with the passage of time (i.e., no reduction in standard scores were observed over the course of the six-month trials). Rates of treatment response, as measured by the CGI-I, varied across the three trials, with higher positive response found in the large scale RCT (83%) and in the pilot trial of younger children (88%) compared to 53% for the original pilot study.

Discussion

Results across the three trials suggest that this structured PT program can be delivered with a high degree of fidelity within and across therapists and that the intervention is acceptable to parents. Both therapist and parent scores from all trials were higher than the established benchmarks of 80% for therapist fidelity and 65% for parent adherence to treatment. The rate of dropouts ranged from 13% to 27% across the three trials, with the highest rate of attrition occurring in the RCT of medication alone versus medication plus PT. The rate of drop outs in this trial included reasons apparently unrelated to the PT intervention (e.g., adverse medication effects) (Aman et al., 2009; Scahill et al., 2012). Taken together, these results demonstrate the successful design and delivery of a structured PT manual that is acceptable to families and ready for testing as a stand-alone treatment.

Results also indicate that the treatment can produce significant reductions in disruptive, explosive and noncompliant behaviors in children with ASD. Given the relatively low cost and time-limited format of this PT intervention, this series of studies suggests that PT could be implemented on a larger scale than more intensive and costly interventions. In addition to efficacy testing of PT as a stand-alone treatment, future studies designed to evaluate the wider application of PT are needed to justify increased access to PT children with ASD and their families.

Although this PT program includes modules on teaching parents how to promote daily living skills in their children, improvement on the Vineland Daily Living Skills domain was limited. We note that the two Teaching Skills sessions fall at the end of the program (typically between Weeks 12 and 16) and focus on teaching one skill at a time (e.g., zipping zippers, brushing teeth). Thus, the full impact of teaching various daily living skills, based on systematic instructional techniques such as task analysis and prompting procedures may not become evident until after the 24-week assessment. Future studies should examine whether additional sessions on training new skills or placing the skill development sessions earlier in the treatment would improve outcomes on measures of adaptive behavior.

Limitations

Although this series of studies provides evidence for the feasibility and efficacy of PT in children with ASD and disruptive behavior, several limitations warrant mention. First, we do not have data from an RCT showing that PT as a stand-alone intervention is superior to a control condition. We have presented data on change from pre- to post-intervention within groups because two of the three trials did not include a comparison group. The third trial was an RCT that compared medication alone to medication plus PT. Although combined treatment was superior to medication alone, the study showed that medication also produced considerable benefit as well (Aman et al, 2009; Scahill et al., 2012). This RCT also enrolled children with serious behavioral problems for whom risperidone is an appropriate choice of treatment. In the absence of a rigorous test of PT as a stand-alone treatment, it is unknown whether it would be an effective intervention for subjects with moderate levels of disruptive behavior for whom medication may not be appropriate.

Future Directions

Expanding Target Populations

Outcomes from the three trials provide initial support for the utility of this treatment. The foundation is set, models for assessing both fidelity and acceptability have been created, and the manual is now ready to be applied in a wider population of children with ASD. Our group is now engaged in evaluating whether PT is superior to parent education in 180 children (ages 3 to 6) with ASD accompanied by at least moderate disruptive behavior. This large scale, multi-site RCT is currently underway at six centers (Emory University, Indiana University, Ohio State University, University of Pittsburgh, University of Rochester, Yale University). The current 24-week randomized trial does not include a medication arm. Families will be followed for up to 24 weeks post-treatment.

The control condition in this RCT is a structured psychoeducational program (PEP). PEP, which was created specifically for this RCT, provides parents with an up-to-date survey of topics related to ASD (e.g., differential diagnosis, genetics, available treatments, educational placement). It also includes strategies for parents to become effective advocates for their child with ASD. Given the age of the study sample, the diagnosis of ASD in the child may be new to many families. Thus PEP is likely to useful to families of affected children. However, PEP does not include any information on child behavior management. As with PT, PEP is delivered individually to parents by trained therapists in 13 sessions over 24 weeks. This comparison condition is designed to control for time and attention (i.e., parental contact with the therapist). Similar measures are being used to evaluate therapist fidelity and parent acceptance of the PEP program.

These three trials included children with disruptive behavior across a range of developmental delays and autism-related disability. Although the studies included children across the range of intellectual disability, children with receptive language skills below 18 months were excluded. The basis of this exclusionary criterion was twofold. First, the differential diagnosis of an autism spectrum disorder is increasingly more challenging as intellectual functioning goes down. Second, although children were not required to be verbal, PT in its present form requires at least rudimentary receptive language. Nonetheless, there is little doubt that children with receptive language below 18 months have disruptive behavior that diminishes the quality of life for the child and the family. As with revisions of the original PT program for school-age children (Johnson et al., 2007) for the pre-school age group (Bearss et al, in press), modifications to the PT program may also be required to serve low-functioning, nonverbal children. Future studies would be needed to evaluate the feasibility and efficacy of a modified PT program in this under-served population.

Expanding Outcome Measures

Paralleling the need to expand PT to a wider sampling frame of children is the need to measure the social disability and adaptive functioning in children with ASD. In all three studies, we used the survey edition of the Vineland as an outcome measure. The Vineland has a strong track record as a standardized measure of adaptive functioning that maps to intellectual ability. It has been used as an outcome measure in clinical trials of children with ASDs (Dawson et al., 2010; Scahill et al., 2012). However, it is relatively time consuming and the normative properties suggest that it is unlikely to be useful as a change measure over brief periods of time. Thus, development of a shorter measure of adaptive functioning that is sensitive to change over relatively brief periods of time would be a valuable contribution.

Behavioral observation is often recommended to complement the current standard approach of relying on parent- and clinician-report for assessing outcome. All three trials included a Standardized Observation Analogue Procedure (SOAP), which study investigators developed as a means to capture change in parent and child behaviors from baseline to Week 24 across a series of 4 standardized behavior observation conditions (Johnson et al., 2009; Handen et al., in press). While the SOAP captured changes in child and parent behavior from Baseline to Week 24 in the RUPP-PT trial, the observation generally failed to consistently capture the disruptive and noncompliant behaviors that were the target of intervention. For example, at baseline children were compliant to 75% of requests during the demand condition. As the expectation will likely remain that well-designed RCTs include behavioral observation measures to assess outcome, there remains a need to identify a paradigm that reliably produces the target behaviors at baseline and is sensitive to change in both child and parent behaviors at the end of treatment.

Future studies could also consider the impact of intervention on parental skill acquisition. In the three studies reviewed here, parent attainment of in-session objectives (based on therapist rating) and overall program participation (session attendance) was evaluated, we did not collect data from direct observations of parental proficiency in implementing the various techniques. Parent knowledge of skills is a necessary first step, but ‘doing’ not just ‘knowing’ the skills may be essential for short- and long-term behavioral change.

Another parental factor to be explored is the degree of parental engagement in therapy. Parents may have variable levels of interest, enthusiasm, and attention during sessions that may affect their ability to carry out interventions with their children. Future studies could examine whether the level of parent engagement affects observed behavior change in the child.

Moderators of Response to PT

Finally, although improvements in disruptive behavior were noted in all three studies, we know very little about the clinical characteristics that predict benefit (Farmer et al., in press). For example, the mean baseline ABC-I score in the preschool pilot study was 8–13 points lower than in the two RUPP trials (Aman et al., 2009; RUPP Autism Network, 2007). One question is whether administration of the PT in the preschool years could prevent the emergence of more serious behavioral problems (e.g., aggression, severe tantrums, self-injury) and postpone or eliminate the need for medication as the child ages.

Table 1.

The Original RUPP Parent Training Program1

11 CORE SESSIONS SKILLS/ACTIVITIES
A. Behavioral Model & Functional Assessment Introduce overall treatment goals and concepts of behavioral functions, antecedents and consequences of behavior.
B. Prevention I: Antecedent Management Strategies Discuss antecedents to behavior problems and develop preventive strategies.
HOME VISIT 1 (between Prevention Strategies and Daily Schedules Sessions)
C. Prevention II: Use of Schedules Develop a daily schedule and identify points of intervention (including use of visual schedules) to decrease behavior problems.
D. Reinforcement Introduce concept of reinforcers – to promote compliance, strengthen desired behaviors and teach new behaviors.
E. Teaching Compliance Introduce elements of effective parental requests and the use of guided compliance in order to enhance compliance and manage non-compliance.
F. Planned Ignoring Explore systematic use of extinction (via planned ignoring) to reduce behavioral problems.
G. Functional Communication Skills Through systematic reinforcement, teach alternative, communicative skills to replace problematic behaviors.
H. & I. Teaching Skills Using task analysis, chaining, and prompting, provide parents with tools on how to replace problem behaviors with appropriate behaviors, and promote new adaptive, coping and leisure skills.
J. & K. Generalization & Maintenance Generate strategies to consolidate positive behavior changes and establish new additional skills.
UP TO 3 OPTIONAL SESSIONS Review session material or provide instruction on the following optional topics: Toileting, Feeding, Sleep, Time Out, Imitation Skills, Contingency Contracting, Crisis Management
WEEK 16
Telephone Booster WEEK 18 - Review implementation of intervention strategies and troubleshoot new behaviors.
In Person Booster WEEK 20 - Review implementation of intervention strategies and troubleshoot new behaviors.
HOME VISIT 2
Telephone Booster WEEK 22 - Review implementation of intervention strategies and troubleshoot new behaviors.
Open in a new tab
1

Modified from Table 1 in Johnson et al., 2007

Acknowledgments

Acknowledgement of funding: Supported by the following cooperative agreement grants from the National Institute of Mental Health (NIMH): U10MH66768 (P.I.: M. Aman), U10MH66766 (P.I.: C. McDougle), and U10MH66764 (P.I.: L. Scahill). This publication also was made possible by Clinical and Translational Scholar Award (CTSA) Grant Number UL1 RR024139 and 5KL2RR024138 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH roadmap for Medical Research.

References

  1. Agency for Healthcare Research and Quality. Therapies for children with autism spectrum disorders: A review of the research for parents and caregivers. 2011 Retrieved November 20, 2012, at http://www.effectivehealthcare.ahrq.gov/ehc/products/106/709/aut_fin_to_post.pdf. [PubMed]
  2. Aldred C, Green J, Adams C. A new social communication intervention for children with autism: Pilot randomized controlled treatment study suggesting effectiveness. Journal of Child Psychology and Psychiatry. 2004;45:1420–1430. doi: 10.1111/j.1469-7610.2004.00848.x. [DOI] [PubMed] [Google Scholar]
  3. Aman MG, McDougle CJ, Scahill L, Handen B, Arnold LE, Johnson C, et al. Medication and parent training in children with pervasive developmental disorders and serious behavior problems: Results from a randomized clinical trial. Journal of the American Academy of Child & Adolescent Psychiatry. 2009;48:1143–1154. doi: 10.1097/CHI.0b013e3181bfd669. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aman MG, Singh NN, Stewart AW, Field CJ. The Aberrant Behavior Checklist: A behavior rating scale of the assessment of treatment effects. American Journal of Mental Deficiency. 1985a;89:485–491. [PubMed] [Google Scholar]
  5. Aman MG, Singh NN, Stewart AW, Field CJ. Psychometric characteristics of the Aberrant Behavior Checklist. American Journal of Mental Deficiency. 1985b;89:492–502. [PubMed] [Google Scholar]
  6. Anderson CM, McMillan K. Parental use of escape extinction and differential reinforcement to treat food selectivity. Journal of Applied Behavior Analysis. 2001;34:511–515. doi: 10.1901/jaba.2001.34-511. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Arnold, Eugene L, Vitiello B, McDougle C, Scahill L, Shah B, Tierney E. Parent Defined Target Symptoms Respond to Risperidone in RUPP Autism Study: Customer Approach to Clinical Trials. Journal of the American Academy of Child & Adolescent Psychiatry. 2003;42(12):1443–1450. doi: 10.1097/00004583-200312000-00011. [DOI] [PubMed] [Google Scholar]
  8. Barkley RA, Murphy KR. Attention Deficit Hyperactivity Disorder: A Clinical Workbook. 2. New York: Guilford; 1998. [Google Scholar]
  9. Bearss K, Johnson C, Handen B, Smith T, Scahill L. A pilot study of parent training in young children with autism spectrum disorders and disruptive behavior. Journal of Autism and Developmental Disorders. doi: 10.1007/s10803-012-1624-7. in press. [DOI] [PubMed] [Google Scholar]
  10. Brown EC, Aman MG, Havercamp SM. Factor analysis and norms for parent ratings on the Aberrant Behavior Checklist - Community for young people in special education. Research in Developmental Disabilities. 2002;23:45–60. doi: 10.1016/s0891-4222(01)00091-9. [DOI] [PubMed] [Google Scholar]
  11. Campbell JM. Efficacy of behavioral interventions for reducing problem behavior in persons with autism: a quantitative synthesis of single-subject research. Research in Developmental Disabilities. 2003;24:120–138. doi: 10.1016/s0891-4222(03)00014-3. [DOI] [PubMed] [Google Scholar]
  12. Carter AS, Messinger DS, Stone WL, Celimli S, Nehmias AS, Yoder P. A randomized controlled trial of Hanen’s ‘More Than Words’ in toddlers with early autism symptoms. Journal of Clinical Psychology and Psychiatry. 2011 doi: 10.1111/j.1469-7610.2011.02395.x. epub. Retrieved May 6, 2011, at http://onlinelibrary.wiley.com/doi/10.1111/j.1469-7610.2011.02395.x/abstract. [DOI] [PMC free article] [PubMed]
  13. Carter AS, Volkmar FR, Sparrow SS, Wang J, Lord C, Dawson G, et al. The Vineland Adaptive Behavior Scales: Supplementary norms for individuals with autism. Journal of Autism & Developmental Disorders. 1998;28:287–302. doi: 10.1023/a:1026056518470. [DOI] [PubMed] [Google Scholar]
  14. Centers for Disease Control and Prevention. Prevalence of autism spectrum disorders — autism and developmental disabilities monitoring network, 14 sites, United States, 2008. Morbidity and Mortality Weekly Report. 2012 Mar 30;61 Available online: http://www.cdc.gov/ncbddd/autism/documents/ADDM-2012-Community-Report.pdf. [PubMed] [Google Scholar]
  15. Chowdhury M, Aman MG, Scahill L, Swiezy N, Arnold LE, Lecavalier L, Johnson C, Handen B, Stigler K, Bearss K, Sukhodolsky D, McDougle CJ. The Home Situations Questionnaire-PDD version: Factor structure and psychometric properties. Journal of Intellectual Disabilities Research. 2010;54:281–91. doi: 10.1111/j.1365-2788.2010.01259.x. [DOI] [PubMed] [Google Scholar]
  16. Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, Donaldson A, Varley J. Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics. 2010;125:e17–23. doi: 10.1542/peds.2009-0958. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Drew A, Baird G, Baron-Cohen S, Cox A, Slonims V, Wheelwright S, et al. A pilot randomised control trial of a parent training intervention for pre-school children with autism: Preliminary findings and methodological challenges. European Child & Adolescent Psychiatry. 2002;11:266–272. doi: 10.1007/s00787-002-0299-6. [DOI] [PubMed] [Google Scholar]
  18. Ducharme JM, Drain TL. Errorless academic compliance training: Improving generalized cooperation with parental requests in children with autism. American Academy of Child & Adolescent Psychiatry. 2004;43:163–171. doi: 10.1097/00004583-200402000-00011. [DOI] [PubMed] [Google Scholar]
  19. Green J, Charman T, McConachie H, Aldred C, Slonims V, Howlin P, et al. Parent-mediated communication-focused treatment in children with autism (PACT): A randomised controlled trial. Lancet. 2010;375:2152–2160. doi: 10.1016/S0140-6736(10)60587-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Guy W. ECDEU Assessment Manual for Psychopharmacology: Publication ADM 76-338. Washington, DC: US Department of Health, Education, and Welfare; 1976. pp. 218–222. [Google Scholar]
  21. Handen BL, Johnson CR, Butter EM, Lecavalier L, Scahill L, Aman MG, Vitiello B. Use of a Direct Observational Measure in a Trial of Risperidone and Parent Training in Children with Pervasive Developmental Disorders. Journal of Developmental and Physical Disabilities. doi: 10.1007/s10882-012-9316-y. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Herring S, Gray K, Taffe J, Tonge B, Sweeney D, Einfeld S. Behaviour and emotional problems in toddlers with pervasive developmental disorders and developmental delay: Associations with parental mental health and family functioning. Journal of Intellectual Disabilities Research. 2006;50:874–82. doi: 10.1111/j.1365-2788.2006.00904.x. [DOI] [PubMed] [Google Scholar]
  23. Horner R, Carr EG, Strain PS, Todd AW, Reed HK. Problem behavior interventions for young children with autism: A research synthesis. Journal of Autism and Developmental Disorders. 2002;32:423–446. doi: 10.1023/a:1020593922901. [DOI] [PubMed] [Google Scholar]
  24. Ingersoll B, Gergans S. The effect of a parent-implemented imitation intervention on spontaneous imitation skills in young children with autism. Research in Developmental Disabilities. 2007;28:163–175. doi: 10.1016/j.ridd.2006.02.004. [DOI] [PubMed] [Google Scholar]
  25. Johnson CR, Handen BL, Butter E, Wagner A, Mulick J, Sukhodolsky DG, et al. Development of a parent training program for children with pervasive developmental disorders. Behavioral Interventions. 2007;22:201–221. [Google Scholar]
  26. Kasari C, Gulsrud AC, Wong C, Kwon S, Locke J. Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism. Journal of Autism and Developmental Disorders. 2010;40:1045–1056. doi: 10.1007/s10803-010-0955-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kazdin AE. Parent Management Training: Treatment for oppositional, aggressive, and antisocial behavior in children and adolescents. New York: Oxford University Press; 2005. [Google Scholar]
  28. King MD, Bearman PS. Socioeconomic status and the increased prevalence of autism in California. American Sociological Review. 2010;76(2):320–346. doi: 10.1177/0003122411399389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kogan MD, Strickland BB, Blumberg SJ, Singh GK, Perrin JM, Van Dyck PC. A National Profile of the Health Care Experiences and Family Impact of Autism Spectrum Disorder Among Children in the United States, 2005–2006. Pediatrics. 2008;122.6:1149–58. doi: 10.1542/peds.2008-1057. Print. [DOI] [PubMed] [Google Scholar]
  30. Landa RJ, Holman KC, O’Neill AH, Stuart EA. Intervention targeting development of socially synchronous engagement in toddlers with autism spectrum disorders: A randomized controlled trial. Journal of Child Psychology and Psychiatry. 2011;52:13–21. doi: 10.1111/j.1469-7610.2010.02288.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Laugesen EA, Frankel F, Mogil C, Dillon AR. Parent-assisted social skills training to improve friendships in teens with autism spectrum disorders. Journal of Autism and Developmental Disorders. 2008;39:596–606. doi: 10.1007/s10803-008-0664-5. [DOI] [PubMed] [Google Scholar]
  32. Lecavalier L. Behavioral and emotional problems in young people with pervasive developmental disorders: Relative prevalence, effects of subject characteristics, and empirical classification. Journal of Autism and Developmental Disorders. 2006;36:1101–1114. doi: 10.1007/s10803-006-0147-5. [DOI] [PubMed] [Google Scholar]
  33. Lerman DC, Swiezy N, Perkins-Parks S, Roane HS. Skill acquisition in parents of children with developmental disabilities: Interaction between skill type and instructional format. Research in Developmental Disabilities. 2000;21(3):183–196. doi: 10.1016/s0891-4222(00)00033-0. [DOI] [PubMed] [Google Scholar]
  34. Lord C, Wagner A, Rogers S, Szatmari P, Aman M, Charman, et al. Challenges in evaluating psychosocial interventions for autism spectrum disorders. Journal of Autism and Developmental Disorders. 2005;35:695–708. doi: 10.1007/s10803-005-0017-6. [DOI] [PubMed] [Google Scholar]
  35. Lundahl B, Risser HJ, Lovejoy MC. A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review. 2006;26:86–104. doi: 10.1016/j.cpr.2005.07.004. [DOI] [PubMed] [Google Scholar]
  36. McConachie H, Diggle T. Parent implemented early intervention for young children with autism spectrum disorder: A systematic review. Journal of Evaluation in Clinical Practice. 2007;13(1):120–129. doi: 10.1111/j.1365-2753.2006.00674.x. [DOI] [PubMed] [Google Scholar]
  37. Moes DR, Frea WD. Contextualized behavioral support in early intervention for children with autism and their families. Journal of Autism & Developmental Disorders. 2002;32(6):519–533. doi: 10.1023/a:1021298729297. [DOI] [PubMed] [Google Scholar]
  38. Odom SL, Brown WH, Frey T, Karasu N, Smith-Canter LL, Strain PS. Evidence-based practices for young children with autism: Contributions from single-subject design research. Focus on Autism and Other Developmental Disabilities. 2003;18:166–175. [Google Scholar]
  39. Oosterling I, Visser J, Swinkels S, Rommelse N, Donders R, Woudenberg T, Roos S, Jan van der Gaag R, Buitelaar J. Randomized controlled trial of the Focus Parent Training for Toddlers with Autism: 1-Year Outcome. Journal of Autism and Developmental Disorders. 2010;40:1447–1458. doi: 10.1007/s10803-010-1004-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Paul R, Miles S, Cicchetti D, Sparrow S, Klin A, Volkmar F, Coflin M, Booker S. Adaptive behavior in autism and pervasive developmental disorder-not otherwise specified: Microanalysis of scores on the Vineland Adaptive Behavior Scales. Journal of Autism and Developmental Disorders. 2004;34:223–228. doi: 10.1023/b:jadd.0000022612.18116.46. [DOI] [PubMed] [Google Scholar]
  41. Pisula E, Kossakowska Z. Sense of coherence and coping with stress among mothers and fathers of children with autism. Journal of Autism and Developmental Disorders. 2010;40:1485–1494. doi: 10.1007/s10803-010-1001-3. [DOI] [PubMed] [Google Scholar]
  42. Reyno SM, McGrath PJ. Predictors of parent training efficacy for child externalizing behavior problems – a meta-analytic review. The Journal of Child Psychology and Psychiatry. 2006;47:99–111. doi: 10.1111/j.1469-7610.2005.01544.x. [DOI] [PubMed] [Google Scholar]
  43. Research Units on Pediatric Psychopharmacology Autism Network. Risperidone in children with autism for serious behavioral problems. New England Journal of Medicine. 2002;347:314–321. doi: 10.1056/NEJMoa013171. [DOI] [PubMed] [Google Scholar]
  44. Research Units on Pediatric Psychopharmacology Autism Network. A pilot study of parent management training in children with pervasive developmental disorder. Behavioral Interventions. 2007:179–199. [Google Scholar]
  45. Scahill L, McDougle CJ, Aman MG, Johnson C, Handen B, Bearss K, Dziura J, Butter E, Swiezy NB, Arnold LE, Stigler KA, Sukhodolsky DG, Lecavalier L, Pozdol SL, Nikolov R, Hollway JA, Korzekwa P, Gavaletz A, Kohn AE, Koenig K, Grinnon S, Mulick JA, Yu S, Vitiello B for the Research Units on Pediatric Psychopharmacology Autism Network. Effects of risperidone and parent training on adaptive functioning in children with a pervasive developmental disorders and serious behavioral problems. Journal of the American Academy of Child and Adolescent Psychiatry. 2012;51:136–146. doi: 10.1016/j.jaac.2011.11.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Smith T, Scahill L, Dawson G, Guthrie D, Lord C, Odom S, Rogers S, Wagner A. Designing research studies on psychosocial interventions in autism. Journal of Autism and Developmental Disorders. 2007;37:354–66. doi: 10.1007/s10803-006-0173-3. [DOI] [PubMed] [Google Scholar]
  47. Sofronoff K, Leslie A, Brown W. Parent management training and Asperger syndrome: A randomized controlled trial to evaluate a parent based intervention. Autism. 2004;8:301–317. doi: 10.1177/1362361304045215. [DOI] [PubMed] [Google Scholar]
  48. Sparrow SS, Balla DA, Cicchetti DV. Vineland Adaptive Behavior Scales: Survey Form Manual. Circle Pines, MN: American Guidance Service; 1984. [Google Scholar]
  49. Tonge BJ, Brereton AV, Kiomall MC, Mackinnon AJ, King NJ, Rinehart NJ. Journal of the American Academy of Child and Adolescent Psychiatry. 5. Vol. 45. Lippincott Williams & Wilkins; USA: 2006. Effects on parental mental health of an education and skills training program for parents of young children with autism: a randomized controlled trial; pp. 561–569. [DOI] [PubMed] [Google Scholar]
  50. United States Government Accountability Office. Special education: Children with autism (GAO Publication No 05-220) Washington, DC: Author; 2005. [Google Scholar]
  51. Wahler RG, Vigilante VA, Strand PS. Generalization in a child’s oppositional behavior across home and school settings. Journal of Applied Behavior Analysis. 2004;37:43–51. doi: 10.1901/jaba.2004.37-43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Webster-Stratton C, Reid MJ. The Incredible Years Program for children from infancy to pre-adolescence: Prevention and treatment of behavior problems. In: Murrihy M, Kidman A, Ollendick T, editors. A Clinician’s Handbook for the Assessment and Treatment of Conduct Problems in Youth. New York: Springer; 2010. pp. 139–162. [Google Scholar]
  53. Whittingham K, Sofronoff K, Sheffield J, Sanders MR. Do parental attributions affect treatment outcome in a parenting program? An exploration of the effects of parental attributions in an RCT of Stepping Stones Triple P for the ASD population. Research in Autism Spectrum Disorders. 2009;3:129–144. [Google Scholar]
  54. Zisser A, Eyberg SM. Treating oppositional behavior in children using parent-child interaction therapy. In: Kazdin AE, Weisz JR, editors. Evidence-based psychotherapies for children and adolescents. 2. New York: Guilford; 2010. pp. 179–193. [Google Scholar]

RESOURCES