Parent training for preschool ADHD: A randomized controlled trial of specialized and generic programs

Abstract

Background

The ‘New Forest Parenting Package’ (NFPP), an 8-week home-based intervention for parents of preschoolers with ADHD, fosters constructive parenting to target ADHD-related dysfunctions in attention and impulse control.Although NFPP has improved parent and laboratory measures of ADHD in community samples of children with ADHD-like problems, its efficacy in a clinical sample, and relative to an active treatment comparator, is unknown. The aims are to evaluate the short and long-term efficacy and generalization effects of NFPP compared to an established clinic-based parenting intervention for treating noncompliant behavior (‘Helping the Noncompliant Child’ [HNC]) in young children with ADHD.

Methods

A randomized controlled trial with three parallel arms was the design for this study. 164 3-4-year-olds, 73.8% male, meeting DSM-IV ADHD diagnostic criteria were randomized to NFPP (N = 67), HNC (N = 63), or wait-list control (WL, N = 34). All participants were assessed at post-treatment. NFPP and HNC participants were assessed at follow-up in the next school year. Primary outcomes were ADHD ratings by teachers blind to and uninvolved in treatment, and by parents. Secondary ADHD outcomes included clinician assessments, and laboratory measures of on-task behavior and delay of gratification. Other outcomes included parent and teacher ratings of oppositional behavior, and parenting measures. (Trial name: Home-Based Parent Training in ADHD Preschoolers; Registry: ClinicalTrials.gov Identifier: NCT01320098; URL: http://www/clinicaltrials.gov/ct2/show/NCT01320098).

Results

In both treatment groups, children's ADHD and ODD behaviors, as well as aspects of parenting, were rated improved by parents at the end of treatment compared to controls. Most of these gains in the children's behavior and in some parenting practices were sustained at follow-up. However, these parent-reported improvements were not corroborated by teacher ratings or objective observations. NFPP was not significantly better, and on a few outcomes significantly less effective, than HNC.

Conclusions

The results do not support the claim that NFPP addresses putative dysfunctions underlying ADHD, bringing about generalized change in ADHD, and its underpinning self-regulatory processes. The findings support documented difficulties in achieving generalization across non-targeted settings, and the importance of using blinded measures to provide meaningful assessments of treatment effects.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) affects individuals across the life span and onset is typically during the preschool years (Sonuga-Barke & Halperin, 2010). Preschool ADHD, which shares the symptom structure and core clinical features of ADHD in later childhood (Greenhill, Posner, Vaughan & Kratochvil, 2008), is often associated with problematic family interactions (Daley, Sonuga-Barke, & Thompson, 2003) that contribute to high levels of familial stress and mental health problems (DeWolfe, Byrne, & Bawden, 2000). Longitudinal studies highlight the relative stability of ADHD from preschool to school age, especially in clinical samples (Riddle et al., 2013), although subtype designation varies considerably over time (Lahey, Pelham, Loney, Lee, & Willcutt, 2005).

Effective interventions for ADHD in early childhood are needed – both to reduce impairment during the preschool period itself, and as strategies that may alter the disorder's longer-term trajectory. Randomized controlled trials support the value of stimulant treatment, although effects on core ADHD symptoms and functional outcomes may be smaller and less consistent than in older children and side effects more frequent (Abikoff et al., 2007; Greenhill et al., 2006). Because of resistance to medicating young children (Volkow & Insel 2003), the development of effective non-pharmacological treatments for preschoolers represents an important health policy objective. Indeed, the preschool period may be especially favorable to target ADHD with psychosocial approaches, given neurodevelopmental evidence for early life brain plasticity and the fact that the coercive cycles of parent-child interactions characteristic of ADHD may not yet be firmly established, and therefore easier to alter than in later childhood (Sonuga-Barke, Thompson, Abikoff, Klein, & Brotman, 2006). This possibility is consistent with two recent meta-analyses. Charach et al.&s (2013) review concluded that there was evidence for the value of parent training for preschool ADHD. Sonuga-Barke et al. (2013) reported that trials with preschoolers had the largest effect sizes for behavioral interventions, albeit on outcome measures completed by raters aware of and involved in treatment.

Parent training approaches used in preschool ADHD, generally referred to as behavioral parent training (BPT), typically apply principles of positive and negative reinforcement to teach parents behavioral strategies to manage conduct problems, especially noncompliance and oppositionality. Examples include Webster-Stratton's ‘Incredible Years’ (Webster-Stratton, Reid, & Beauchraine, 2011) and Sanders' ‘Triple P’ (Bor, Sanders, & Markie-Dadds, 2002). BPT yields improvements in oppositional and disruptive behaviors, and enhances parenting skills (Charach et al., 2013). However, improvements in ADHD symptoms are less common and robust (Charach et al, 2013; Rajwan, Chacko, & Moeller, 2012). The common expectation that BPTs are indicated for ADHD symptoms has been attributed to the questionable (and often implicit) assumption that these symptoms will be as responsive as conduct problems to the modification of environmental contingencies through the effective use of rewards and punishments (Sonuga-Barke et al., 2006).

In contrast to BPT, the New Forest Parenting Package (NFPP; Thompson et al., 2009) was developed specifically to treat ADHD in preschool children. NFPP, an 8-week home-delivered intervention, combines behavior management techniques with procedures intended to foster constructive parenting to target impairments in underlying processes, including self-regulation, attention, impulse control, and working memory, which ostensibly lead to ADHD symptoms in young children. Parents are taught to take on the role of skilled tutor to create occasions and settings within a positive emotional climate that promote reciprocal, sensitive, positive and playful parent-child interactions, and to use scaffolding to facilitate the development of self-regulation in young children (Wacharasin, Barnard, & Spieker, 2003; Connell & Prinz, 2002).

The first study of NFPP (Sonuga-Barke, Daley, Thompson, Laver-Bradbury, & Weeks, 2001), delivered by highly skilled nurse therapists, compared it to an active attention control (Parent Counseling and Support), and a wait-list control, in a community-based sample of three-year-old children with ADHD-type problems. Compared to both control groups, NFPP yielded clinically significant reductions in parent-reported ADHD symptoms and conduct problems (oppositional, defiant behaviors), increases in observed on-task behavior, and improved maternal sense of well-being. The improvements in ADHD symptoms were in the range found with stimulants in preschoolers (Greenhill et. al., 2006) and persisted to 15 weeks follow-up. A second controlled study found no evidence of efficacy when NFPP was delivered by non-specialist primary care nurses (Sonuga-Barke, Thompson, Daley, & Laver-Bradbury, 2004). During the course of the current study, a small-scale trial of NFPP (revised to enhance the treatment's constructive parenting element) (Thompson et al., 2009), provided by trained nurse specialists to preschoolers meeting ADHD rating-scale criteria, replicated some of the initial NFPP findings. Improvements in parent-reported ADHD symptoms were confirmed and were sustained at 7 weeks follow-up. Conduct problems improved based on parental interview, but not on parent ratings. Treatment effects on on-task behavior and maternal well-being were not replicated, and there was no improvement in the observed quality of parent-child interactions.

While these trials suggest the potential efficacy of NFPP in improving ADHD symptoms in preschoolers, they had important limitations. First, although the children in these community samples met recognized cut-off levels for ADHD on validated instruments, they had not been diagnosed as having ADHD. Second, generalization to settings outside the home (i.e., school) was not tested. Third, NFPP was not compared to BPT, which has typically been recommended in ADHD guidelines as a cost-effective treatment (American Pediatric Association, 2011; National Institute for Health and Clinical Excellence [NICE], 2008).

NFPP's distinctive claim is that it targets core ADHD processes and therefore should have a greater, more wide-ranging impact on preschool ADHD symptoms than standard BPT. Evidence supporting this claim would have important clinical implications and provide justification for choosing this parenting approach in treating preschool ADHD. We addressed the issue of relative superiority of NFPP by comparing it to a generic parent training program that has established efficacy in treating oppositional behaviors in young children – problems common in ADHD: ‘Helping the Non-Compliant Child’ (HNC) (McMahon & Forehand, 2003). Both HNC and NFPP are delivered on a one-on-one basis and over a similar period, thereby facilitating a head-to-head comparison. Our goal was to test the specific and nonspecific therapeutic value of NFPP in clinically-diagnosed three- and four-year-olds with ADHD in an RCT. The primary aims address NFPP's effects on children's ADHD core symptoms relative to HNC, and to a wait list control (WL), generalization to school settings, and maintenance of treatment effects over time. Secondary, we tested NFPP's effects on children's self-regulatory and oppositional behaviors and on parenting.

Based on NFPP's theoretical rationale and prior findings, we predicted that: 1) NFPP would be superior to both HNC and WL with regard to ADHD symptom reduction, rated by parents, teachers and clinicians, and by direct observation; 2) positive effects for NFPP, relative to HNC, would persist during the next school year; 3) NFPP would reduce impulsive choice, and increase delay tolerance, relative to HNC and WL; and 4) NFPP would show significant reductions in parent- and teacher-rated oppositional symptoms compared to WL – with no specific prediction regarding its relative value compared to HNC on these measures. The trial also examined treatment effects on parenting; specifically: whether (i) parental responsiveness and scaffolding improve in NFPP only; and (ii) positive parenting practices and perceived stress improve in both NFPP and HNC.

Methods

Site

The study was conducted at New York University (NYU) Langone Medical Center between March 2008 and December 2012. NYU and NYC Department of Education institutional review boards approved the study. After complete description of the study to parents, parents provided signed informed consent.

Design

In a three-group parallel design, children were randomly assigned to (a) NFPP, (b) HNC, or (c) WL. The randomization was stratified by age (3 or 4 years old) and gender. Block randomization to the three treatment conditions (NFPP, HNC, WL) was in a ratio (2:2:1) and was carried out in blocks of random sizes (5 or 10). The randomization assignment was computer generated and automatically linked to a subject when the subject's data for eligibility were entered into the database and it was established that s/he met the study entry criteria. The randomization sequence was generated by the research organization responsible for data management. Participants were enrolled by research coordinators; eligibility for enrollment was determined based on exclusion/inclusion criteria which were checked when subjects' data were entered into the database. (Trial registry: Home-Based Parent Training in ADHD Preschoolers; Registration ID, ClinicalTrials.gov Identifier: NCT01320098; URL: http://www/clinicaltrials.gov/ct2/show/NCT01320098).

Participants and Procedures

Sample size determination

The sample size was selected to allow at least 80% power for two-sided tests with significance level 0.05/2=0.025 (to account for conducting 2 tests) to detect what were deemed, a priori, to be clinically meaningful effects of NFPP against WL and HNC with respect to the primary outcomes (teacher and parent ADHD ratings). The planned total sample size of 187 (75:75:37 for NFPP:HNC:WL) allowed detecting differences of magnitude Cohen's d= 0.62 to 0.68 (depending on dropout rate, from 0% to 15%) against WL. Against HNC, this sample size allowed detecting differences of magnitude d=0.51 to 0.55. Lower recruitment (e.g., N=159) would have sufficient power to detect negligibly larger effects -- d=0.68 to 0.74 against WL and d=0.55 to 0.60 against HNC.

Inclusion criteria

Participants were 3.0 - to 4.11-year-old boys and girls attending a preschool, daycare or nursery school at least 2 and-a-half days a week. Inclusion required that the primary caretaker be fluent in English and that the child have an IQ ≥ 70 on the Wechsler Preschool Primary Scale of Intelligence, 3^rd edition (WPPSI-III; Wechsler, 2002); elevated scores above age and gender norms on the DSM-IV Total, DSM-IV Hyperactive/Impulsive, or DSM-IV Inattentive subscales on both the Revised Conners Teacher (CTRS-R) (T-score ≥ 65) and Parent (CPRS-R) Rating Scales (T-score ≥ 60), (Conners, Sitarenios, Parker, & Epstein, 1998a, 1998b); a Diagnostic and Statistical Manual of Mental Disorders (4^th ed.; DSM-IV; American Psychiatric Association, 1994) diagnosis of ADHD (any type) on the Diagnostic Interview Schedule for Children-Parent Report Version 4, (Shaffer et al., 1998), modified Young Child Version (DISC-IV-YC) (Lucas, Fisher, & Luby, 1998), confirmed by clinical evaluation conducted by a psychologist with child and parent; standard score ≥ 7 on the Concepts and Following Directions subscale of the Clinical Evaluation of Language Fundamentals (CELF-2, Semel, Wiig, & Secord, 2004).

Recruitment relied on referrals from preschools, daycares, nursery schools, community resources (clinics, physicians, and agencies), parent mailings, newspaper ads, and website postings.

Exclusion criteria

Reasons for exclusion included current medication or behavioral treatment for ADHD; a diagnosis of pervasive developmental disorder, psychosis, or posttraumatic stress disorder; history of sexual or physical abuse; or any other psychiatric or medical condition judged to contraindicate participation. Children with common mental health diagnoses were not excluded.

Measures

Assessment schedule

All children were assessed at baseline (PRE) and post-treatment (POST). WL participants obtained their treatment of choice after the POST assessments. Children treated with NFPP and HNC were followed-up (F-UP) in October/November of the next school year, when they were in a new class. The duration between POST and F-UP assessments did not differ between the two treatments (M = 6.8 months, SD =2.01; range = 2.76 – 10.57 mo.).

All families were compensated $70 for each assessment visit. HNC parents received $15 per session to cover travel costs to the clinic.

Assessment

Before treatment, assessments were made of ADHD and general psychopathology in the primary caretaker, family characteristics and demographics. Outcome measures were obtained at all three assessment points. (Study measures' psychometric properties are acceptable and are detailed in the online supplemental appendix S1.) Blinded outcome assessors included clinicians, observers and teachers. To help assure that teachers were masked to condition, parents were informed of the rationale and importance of not discussing their child's treatment with the teacher.

ADHD

Primary outcomes were teacher and parent ADHD ratings (Total, H/I and IN subscale scores) on the Conners scales, applying published norms (T-scores) for 3-4-year-old children. Both versions use a 4-point Likert scale. Non-primary ADHD outcomes were (i) clinician ADHD ratings using the ADHD-Rating Scale-IV (Zhang, Faries, Vowles, & Michelson, 2005), which assessed the 18 DSM-IV ADHD symptoms following a parent interview. The number of symptoms with frequency and impairment ratings of at least ‘often’ and ‘moderate’, respectively, served as the outcome measure; and (ii) children's levels of sustained and focused attention and activity during a videotaped five-minute period while playing with a standard multi-domain toy (‘Play Park’) coded by observers using a validated observational coding system. As per Sonuga-Barke et al. (2001), an ‘index of attention/engagement’ was calculated (time on task/total number of switches from zone to zone).

Oppositional and defiant symptoms

These were assessed with the preschool version of the New York Teacher and Parent Rating Scales (NYTRS, Miller et al., 1995; NYPRS, Brotman, Kamboukos & Theise, 2008). Both scales assess symptoms of oppositional defiant disorder and conduct disorder, and include identical Defiance and Physical Aggression subscales.

Delay of Gratification

This was assessed with the Delay of Gratification-Cookies Delay Task (Campbell, Pierce, March, Ewing, & Szumowski, 1994). Each child was asked to wait for a signal (clap) before taking a treat (an M&M candy) placed under one of three upturned cups. Eight trials were given in a pseudo-random order with delays of between 5 and 30 s. The experimenter's hands are raised at the midpoint (i.e., after 10s if the delay was 20s) ready to clap. A child's score on each trial indicates his level of inhibition (0 = not inhibited: the child lifts the cup and takes the M&M; 1 = partially inhibited: during the delay, the child makes a movement toward the cup but makes no attempt to take the M&M; 2 = fully inhibited: the child is able to wait and gets the M&M at the end of the delay, after the clap). Based on parental requests, for some children (n=37), a nonedible reward (a small ‘fuzzy bear’) was used instead of M&Ms (see appendix S1 for details).

Parenting Practices and Functioning

A composite score reflecting parent-reported positive parenting practices (e.g., clear expectations, appropriate discipline) was generated from three subscales of the Parenting Practice Interview (PPI; Webster-Stratton, 1998).

The Global Impressions of Parent Child Interactions–Revised (GIPCI–R; Brotman, Calzada, & Dawson-McClure, 2003) was used to make global ratings of parenting behavior during a 15-min semi-structured play interaction that increased in structure and parent directedness (i.e., free play, 7-min; puzzle task, 5-min; clean-up, 3-min). At the end of each segment, independent observers masked to condition assessed seven aspects of parenting (i.e., ‘valence’, ‘responsiveness’, ‘warmth’, ‘use of praise’, ‘enjoyment’, ‘ use of scaffolding’ and ‘effectiveness’) on the GIPCI-R 5-point rating scale. Reliability analyses of the individual ratings indicated that the ‘responsiveness’ and ‘use of scaffolding’ codes were insufficiently reliable in themselves. However, the reliability of the total score across all seven codes was acceptable (ICC =.54) (see appendix S1 for additional details). Therefore, following procedures used in previous trials, a composite score reflecting the average rating across the seven subscales and three tasks served as the outcome measure (Brotman et al., 2011).

Parents completed the 16-item Parenting Stress Index-Short Form Revised (PSI-R; Abidin, 1995) to evaluate parental distress, parent-child dysfunction and parent competence, and four items from the Parent Perceptions of Parent Efficacy Scale (Hoover-Dempsey, Bassler, & Brissie, 1992) to assess parental distress related to helping children succeed in school. The total score from these 20 items served as the outcome measure.

Treatment satisfaction

Parents assigned to NFPP and HNC completed the Consumer Satisfaction Questionnaire (CSQ; McMahon & Forehand, 2003) at POST. Parents indicated 1) their overall satisfaction with treatment (1 = Very dissatisfied to 7 =Very satisfied), 2) the perceived quality of services (1 = Poor to 7 = Superior), and 3) whether they would recommend the treatment to a friend or relative with a preschooler with ADHD (1 = Strongly not recommend to 7 = Strongly recommend). The sum of the three ratings served as the overall satisfaction index.

Interventions

New Forest Parenting Package

(see appendix S1 for detailed description). NFPP, a manualized intervention for preschoolers with ADHD, involves 8 weekly 1-to-1.5-hour sessions, delivered in the family home by trained clinicians. NFPP focuses on key issues related to ADHD children's functioning, and relies on the parent as the primary agent of change. While it shares a number of features with standard BPT (i.e., management of problematic behavior using behavioral techniques; promotion of authoritative parenting; increasing the quality and quantity of positive and reciprocal parent-child interaction; reduction of parental negative reactivity; and between-session ‘homework tasks’ to facilitate improvement in specific parenting techniques), it has a number of distinctive features. First, its home-based nature enables the therapist to model play and behavioral strategies for the mother in the setting where the behaviors are problematic. It also enables the therapist to address naturally occurring instances of problematic child behaviors (e.g., difficulty waiting, inattention, dysregulation, etc.) that call for the use of the parenting (and child) skills being taught. Sensitizing parents to the importance of these ‘teachable moments’ and of identifying and exposing their child to relevant real-world situations where skills can be used provides numerous opportunities for skills development and generalization.

Second, NFPP directly aims to improve four elements of constructive parenting: (i) Scoping-- learning how to observe their child's current level of competencies so as to promote realistic expectations and performance goals for their child regarding self-control, attention, and memory; (ii) Extending -- establishing new goals based on their child's performance and progress; (iii) Scaffolding -- using game-like activities to facilitate their child's skills development and goal achievement; and (iv) Consolidation—promoting their child's skill use across settings and situations to facilitate generalization.

Third, NFPP educates parents to alter their views of ADHD, avoid blaming their child for ADHD symptoms, and increase parental tolerance with the ultimate goal of improving the quality of the parent-child relationship.

Helping the Noncompliant Child

HNC is a manualized BPT intervention (McMahon & Forehand, 2003) for treating young children with noncompliance and oppositional problems. The individualized, clinic-based, treatment is delivered by therapists, with the parent and child jointly, in each session. The clinical provision of HNC typically averages 8-10 intervention sessions (McMahon & Forehand, 2003). To ensure that NFPP and HNC were equated for length and amount of therapist contact, HNC was delivered in 8 weekly sessions, lasting approximately one hour. HNC was provided according to the details specified in the McMahon and Forehand (2003) treatment manual, except that a fixed number of sessions was conducted and meeting behavioral criteria for advancement from one parenting skill to the next was not required.

HNC is based on social-learning theory and behavior modification principles and methods and incorporates characteristics of the BPT model developed by Hanf (1969). Treatment focuses on reducing noncompliance using a variety of methods to teach parents how to change their maladaptive interaction patterns with their child. Specific program components include: 1) modeling and parent role play, along with didactic instruction and discussion, to teach parents the skills of attending, rewarding, ignoring, clear instructions and time out; and 2) home practice, assignments and exercises, throughout the program.

The program includes two phases. Phase I focuses on differential attention. Parents are taught how to attend to and describe their child's appropriate behavior to the child (rather than give commands, or teach), to provide rewards through positive physical attention (e.g., hugs) and specific verbal praise, and to ignore their child's minor, inappropriate attention-seeking behaviors by not providing eye contact, nonverbal cues, verbal contact, or physical contact. Phase II focuses on compliance training. Parents learn the importance of clear and simple instructions, using a sequential approach to get their child's attention to instructions and provide positive rewards for compliance and negative consequences for non-compliance (i.e., Time-Out).

Treatment Delivery

Therapists (n=5) were clinical psychologists with at least two years of behavior therapy experience with children and families. To control for possible therapist effects, each therapist provided both treatments (Anderson, Ogles, Patterson, Lambert, & Vermeersch, 2009). Therapists participated in face-to-face group training with the developers of each treatment, and were supervised by them throughout the study through weekly group conference calls. A modified Latin Square design was used to select videotapes for quality review. All treatment sessions were videotaped and twelve percent were checked by an independent evaluator for treatment fidelity and integrity, based on procedures developed and used previously by Abikoff et al. (2013). The percentage of required therapeutic items covered served as an index of fidelity. The integrity checklist assessed treatment contamination -- treatment spillover due to therapist actions during a session (discussions, elaboration, or recommendations regarding ‘prohibited’ clinical elements that are specific to the non-assigned treatment).

Statistical Analyses

Linear mixed-effect models suitable for longitudinal data (Laird & Ware, 1982) were implemented with R software (Bates, Maechler, Bolker, & Walker, 2013; R Core Team). The outcome of interest at POST or F-UP was regressed on (1) baseline value of the same outcome and (2) a factor encoding both time point and treatment condition (WL at POST, HNC at POST, NFPP at POST, HNC at F-UP, or NFPP at F-UP), with a random subject effect to take within-subject correlation into account. Overall treatment effect (difference among the three conditions) at POST was tested by a likelihood ratio test, an approximate chi-square test with 2 df (Pinheiro & Bates, 2000). All pairwise contrasts among the three treatments at POST, and the difference between NFPP and HNC at F-UP were tested by approximate t-tests with a Satterthwaite approximation to the denominator degrees of freedom (Verbeke & Molenberghs, 2009). Maintenance of treatment effects was assessed by estimating change (F-UP minus POST) averaged over NFPP and HNC (overall ‘time effect’); and difference in change between the two treatments (time-by-treatment interaction).

In addition to missing data from 12 dropouts (described below), 15 children did not participate in scheduled assessments, including one at POST (NFPP) and 14 at F-UP (4 NFPP, 10 HNC). In accordance with the intent-to-treat principle, data from all randomized participants were analyzed according to their treatment assignment. Linear mixed-effect model inferences remain valid under the ‘missing at random’ assumption (Little & Rubin, 2002), meaning that given the observed outcome values, dropout is conditionally independent of the subsequent unobserved outcomes.

P-values reported here are unadjusted for multiple comparisons because (i) our aim was to test a number of pre-specified hypotheses that relate to different aspects of the relative efficacy of HNC and NFPP and (ii) we were especially concerned to minimize Type II error. Given the dearth of information regarding NFPP's effects in a clinical sample of preschool-aged children, we deemed it important not to miss the opportunity to detect clinically meaningful treatment effects.

Results

Sample

Recruitment occurred between 11/29/07 and 3/20/12. 164 preschoolers were randomized (NFPP, n = 67; HNC, n = 63; WL, n = 34); 153 participants (93.3%) completed the study; eight dropped out from NFPP (11.9%) and 4 from HNC (6.3%). Figure 1 is the participant flow chart (CONSORT diagram).

Figure 1. Participant flow chart.

CPRS = Conners Parent Rating Scale; CTRS = Conners Teacher Rating Scale; CELF = Clinical Evaluation of Language Fundamentals; NFPP = ‘New Forest Parenting Package’; HNC = ‘Helping the Noncompliant Child’; ITT = intent-to-treat; Y2 = year two; F-UP = follow-up

The study sample was 73.8% male; 69.2% Caucasian, 16.4% African-American, 8.8% Asian and 5.6% other; 25.6% of the participants were Hispanic. Children's mean IQ was 101.8 (±14.8). DSM-IV ADHD subtype diagnoses were 50.6% Combined, 33.5% Hyperactive/Impulsive, and 15.2% Inattentive; 41.5 % had a diagnosis of oppositional-defiant disorder and 6.7% had an anxiety disorder. 76.4% of mothers and 60.3% of fathers were college graduates. The primary caregivers and informants were predominantly mothers (92.7%). No child started medication from PRE to POST. At F-UP, three children in NFPP and three in HNC were reported to have started medication. There were no significant group differences on any demographic and clinical variables. The appendix S1 contains complete demographic and clinical characteristics.

Treatment Attendance, Fidelity, Satisfaction and Harms

Attendance was equally high for each treatment (NFPP, M = 7.40 [SD ±1.88]; HNC, M = 7.73 [± 1.11]). Treatment fidelity was high (NFPP: 96.3%; HNC: 96.9%) and contamination was low (0.6% of assessed NFPP and HNC sessions). Parents' treatment satisfaction was equally high for NFPP (19.98 ±1.36) and HNC (19.78 ±1.63).

It is conceivable that psychosocial treatment might be associated with iatrogenic effects (e.g., tantrums, sad mood, irritability, anxiety) (Abikoff et al., 2013). As such, during training and throughout the study, therapists were reminded to detect such occurrences and instructed to report any concerns regarding possible AEs during weekly supervision. There were no adverse effects with either NFPP or HNC. One WL participant dropped out because of worsening in ADHD symptoms.

Treatment Differences at POST and F-UP

The treatment groups' scores on all outcome measures at PRE, POST and F-UP are presented in Table 1. Table 2 presents comparisons of adjusted outcomes at POST and F-UP.

Table 1. Raw scores of study outcome measures at pre, post, and follow-up.

	NFPP (N = 67)						HNC (N = 63)						WL (N = 34)
	PRE		POST		F-UP		PRE		POST		F-UP		PRE		POST
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
ADHD Ratings
CPRS-Total-	75.42	11.43	65.26	12.15	68.01	11.69	75.21	9.49	62.62	11.05	63.44	10.13	78.01	9.17	76.44	9.84
CPRS-IN-	72.91	12.08	63.6	11.60	65.60	13.53	72.75	11.50	60.47	11.57	61.74	10.04	76.39	9.68	75.31	10.38
CPRS-H/I-	74.56	8.38	64.62	12.00	68.08	10.69	74.36	9.45	62.32	10.34	63.39	10.24	75.88	9.10	74.45	10.67
CTRS-Total-	76.73	9.26	68.46	11.41	64.27	12.27	74.67	7.92	68.10	9.95	62.06	11.39	75.96	9.94	70.65	11.22
CTRS-IN-	73.97	11.35	65.12	12.26	61.39	13.58	72.55	9.21	64.93	11.50	60.48	11.79	71.94	12.28	68.22	11.81
CTRS-H/I-	75.63	8.87	68.31	11.17	64.25	11.64	73.74	10.36	67.57	10.32	62.01	12.06	75.78	10.26	70.26	11.98
Clinician-Rated, Total Sx-	12.4	2.79	8.28	4.22	8.00	4.89	12.7	2.47	6.92	4.41	7.58	3.65	13.03	2.38	12.85	2.92
Clinician-Rated, IN Sx-	5.63	1.99	3.39	2.38	3.61	2.79	5.89	2.24	3.02	2.67	3.30	2.34	6.15	1.52	6.12	2.25
Clinician-Rated, H/I Sx-	6.78	1.63	4.89	2.43	4.39	2.66	6.81	1.63	3.90	2.61	4.28	2.51	6.88	1.67	6.73	1.68
ADHD Tasks
Candy Delay- Total+	13.36	3.72	14.07	3.83	14.26	3.16	13.30	3.70	14.31	3.12	14.52	3.28	12.75	4.14	13.45	3.97
Play Park- Time on Task+	4.78	.55	4.77	.65	4.79	0.56	4.80	.61	4.49	1.17	4.92	0.36	4.73	.54	4.54	1.15
Play Park- # of Switches-	7.41	3.48	7.83	3.05	8.12	3.64	7.96	4.33	8.38	3.31	8.02	3.91	8.28	4.28	7.58	3.28
Play Park- Engagement Index+	.92	.90	.71	.35	0.85	0.82	.79	.46	.66	.39	0.81	0.52	.75	.56	.75	.43
Opposition/Aggression
NYPRS-Defiance-	1.31	.65	.84	.52	.95	.57	1.24	.62	.75	.49	.82	.60	1.17	.80	1.10	.72
NYPRS-Aggression-	.57	.63	.43	.55	.30	.37	.61	.81	.34	.46	.32	.39	.63	.88	.58	.76
NYTRS-Defiance-	1.39	.81	.90	.69	.79	.81	1.21	.86	.98	.76	.62	.63	1.00	.75	.82	.77
NYTRS-Aggression-	.94	.99	.61	.76	.61	.90	.74	.90	.47	.67	.35	.72	.65	.89	.47	.60
Parenting Ratings
PPI- Total+	3.69	0.41	3.97	0.37	3.98	0.39	3.67	0.33	4.01	0.33	4.01	0.36	3.90	0.35	3.65	0.52
PSI-R Total-	2.31	0.58	2.12	0.53	2.10	0.54	2.52	0.54	2.08	0.44	2.11	0.62.	2.51	0.63	2.44	0.70
Parenting Observation
GIPCI-R Total+	3.44	0.44	3.61	0.43	3.52	0.47	3.52	0.44	3.89	0.44	3.66	0.55	3.51	0.43	3.45	0.47

Abbreviations: CTRS-R, CPRS-R= Conners Rating Scale-Revised (P= parent, T= teacher); NYPRS, NYTRS= New York Rating Scale (P= parent, T= teacher); PPI= Parenting Practices Interview; PSI= Parenting Stress Index; GIPCI= Global Impression of Parent Child Interactions;

^-lower score is better

⁺higher score is better

Table 2. Comparison of adjusted outcomes at post-treatment and follow-up.

	Post								Follow-Up
	Treatment Effect		NFPP v WL		HNC v WL		NFPP v HNC		Treatment Effect		Time		Treatment X Time
	${\chi }_2^2$	p	p	db	p	db	p	db	p	db	p	db	p	db
ADHD Ratings
CPRS- Total-	32.79	.001	.001	-1.01	.001	-1.24	ns	0.23	.047	0.38	.023	0.20	ns	0.15
CPRS- IN-	30.85	.001	.001	-0.89	.001	-1.09	ns	0.19	ns	0.30	ns	0.14	ns	0.11
CPRS- H/I-	31.40	.001	.001	-0.97	.001	-1.21	ns	0.24	.033	0.41	.012	0.23	ns	0.17
CTRS- Total-	1.47	ns	ns	-0.28	ns	-0.23	ns	-0.05	ns	0.12	.001	-0.53	ns	0.16
CTRS- IN-	2.48	ns	ns	-0.33	ns	-0.25	ns	-0.07	ns	0.00	.005	-0.39	ns	0.07
CTRS- H/I-	0.76	ns	ns	-0.18	ns	-0.15	ns	-0.03	ns	0.08	.001	-0.48	ns	0.10
Clinician-Rated, Total Sx-	41.97	.001	.001	-1.66	.001	-2.20	.051	0.54	ns	0.19	ns	0.03	ns	-0.36
Clinician-Rated, IN Sx-	35.11	.001	.001	-1.27	.001	-1.47	ns	0.20	ns	0.21	ns	0.12	ns	0.01
Clinician-Rated, H/I Sx-	31.23	.001	.001	-1.07	.001	-1.68	.014	0.61	ns	0.02	ns	-0.08	.024	-0.59
ADHD Tasks
Candy Delay- Total+	1.35	ns	ns	0.18	ns	0.19	ns	-0.01	ns	-0.09	ns	0.03	ns	-0.08
Play Park- Time on Task+	1.60	ns	ns	0.46	ns	0.04	ns	0.42	ns	-0.17	.040	0.44	ns	-0.59
Play Park- Number of Switches-	2.44	ns	ns	0.07	ns	0.24	ns	-0.17	ns	0.14	ns	-0.03	ns	0.31
Play Park- Engagement Index+	0.48	ns	ns	-0.08	ns	-0.14	ns	0.05	ns	0.02	ns	0.23	ns	-0.04
Opposition/Aggression
NYPRS- Defiance-	23.24	.001	.001	-0.59	.001	-0.69	ns	0.10	ns	0.06	ns	0.11	ns	-0.05
NYPRS- Aggression-	9.18	.010	ns	-0.17	.003	-0.37	ns	0.20	ns	-0.01	ns	-0.05	ns	-0.21
NYTRS- Defiance-	2.27	ns	ns	-0.17	ns	0.01	ns	-0.19	ns	-0.05	.009	-0.28	ns	0.14
NYTRS- Aggression-	0.12	ns	ns	-0.01	ns	-0.05	ns	0.03	ns	0.14	ns	-0.08	ns	0.11
Parent Ratings
PPI- Total+	45.71	.001	.001	1.20	.001	1.37	ns	-0.17	ns	-0.27	ns	0.00	ns	-0.10
PSI-R Total-	12.84	.002	ns	-0.28	.001	-0.58	.026	0.30	ns	.10	ns	-0.01	ns	-0.20
Parenting Practices-Observed
GIPCI-R Total+	19.34	.001	ns	0.40	.001	0.97	.003	-0.57	ns	-0.26	.005	-0.40	ns	0.31

^-Note. lower score is better

⁺higher score is better. Abbreviations: CTRS-R, CPRS-R= Conners Rating Scale-Revised (P= parent, T= teacher); NYPRS, NYTRS= New York Rating Scale (P= parent, T= teacher); PPI= Parenting Practices Interview; PSI-R= Parenting Stress Index; GIPCI-R= Global Impression of Parent Child Interactions-Revised

^aTwo-tailed tests

^bEffect sizes (Cohen's d) are the differences between adjusted scores at post-treatment, divided by the standard deviation at baseline, p-values are model-based

We hypothesized that the specialized NFPP intervention would be superior to HNC, a traditional behavioral program, in reducing ADHD symptoms.

ADHD Ratings

Parent ratings

POST

Contrary to expectations, parent ratings of inattention and hyperactivity/impulsivity at POST were not significantly better for children who received NFPP than HNC. However, as predicted for NFPP, and found as well for HNC, children in both treatment groups were rated as less inattentive and hyperactive than those on the waiting list, with large effects (all p < .001); (NFPP: Total, d = -1.01; IN, d = -.89; H/I, d = -.97; HNC: Total, d = -1.24; IN, d = -1.09; H/I, d = -1.21) (Table 2).

F-UP

Improvement in attention was maintained by both treatment groups. In contrast, Total ADHD and hyperactivity/impulsivity ratings worsened significantly in both treatments (p < .03, and < .02, respectively), as reflected by the significant Time effects. Findings were contrary to the hypotheses insofar as NFPP was not superior to HNC on any ADHD parental ratings at F-UP; unexpectedly, children who received NFPP were rated worse on Total ADHD (p < .05, d = .38) and hyperactivity/impulsivity (p < .04, d = .41) than children treated with HNC.

Teacher ratings

POST

At POST, no ADHD teacher rating yielded significant advantage for NFPP over HNC or over Waitlist. All groups improved significantly over time.

F-UP

Outcomes at F-UP were identical to those at POST (Table 2).

Clinician ratings

POST

At the end of treatment, clinicians, who relied on parent reports to evaluate children, did not consider children in NFPP to have fewer ADHD symptoms than children in HNC. To the contrary, clinicians viewed children in NFPP as having significantly more hyperactive/impulsive symptoms than children in HNC (p < .02, d = .61), but both treatments were superior to Waitlist (all p <.001); (NFPP: Total, d = -1.66; IN, d = -1.27; H/I, d = -1.07; HNC: Total, d = -2.20; IN, d = -1.47; H/I, d = -1.68) (Table 2).

F-UP

Clinician ratings showed no significant treatment differences in Total ADHD or inattention symptoms; these remained stable in both groups from POST to F-UP. In contrast, clinician ratings of hyperactivity/impulsivity symptoms decreased in the NFPP group and increased in the HNC group, accounting for the significant Treatment x Time interaction (p < .03, d = -.59) from POST to follow-up.

Observed Attention and Ability to Delay

Sustained attention

There were no significant treatment differences in children's Attention/Engagement Index, or its components (Time on Task and Number of Switches) at POST or F-UP (Table 2). Controlling for parental ‘interference’ during task performance did not alter the results.

Ability to delay

Edible and nonedible reward conditions yielded the same results and were combined. No significant differences between treatments were obtained at POST or F-UP (Table 2).

Oppositional and aggressive behaviors

Parent ratings

POST

On the NYPRS, parent ratings of Defiance did not differentiate children who received NFPP from those in HNC, but both treatment groups were significantly superior to the Waitlist (p < .001) (NFPP, d = -.59; HNC, d = -.69). A different pattern emerged for ratings of Physical Aggression; while NFPP and HNC were indistinguishable, children in the HNC group, but not those in NFPP, were rated better than children on the Waitlist (p < .004, d = -.37) (Table 2).

F-UP

At F-UP the groups' Defiance and Physical Aggression scores were not significantly different, and remained stable from POST to F-UP.

Teacher ratings

POST

Teacher ratings of Defiance and Physical Aggression on the NYTRS did not differ significantly as a function of treatment condition.

F-UP

Children in the two treatments did not obtain significantly different Defiance or Physical Aggression ratings at F-UP. The significant Time effect for Defiance (p <.01) reflects that, over time, ratings for children in both groups became significantly lower.

Parenting behaviors

Parenting practices

POST

On the PPI, parents in NFPP did not show superior parenting practices than parents in the HNC group, and compared to WL, parents in both treatments had significantly improved practices (p < .001) (NFPP, d = 1.20; HNC, d = 1.37) (Table 2).

F-UP

At F-UP, parenting practices did not differ as a function of treatment. Parenting practices did not deteriorate over time, but remained stable.

Observed Parenting

POST

Here again, NFPP did not lead to a significant advantage in parenting behaviors. The significant Treatment effect (p< .001) reflects that parents in the HNC group had significantly better GIPCI-R scores than parents in both the NFPP (p < .003, d = -.57) and WL (p < .001, d = .97) groups, which did not differ from each other (Table 2).

F-UP

At F-UP, parenting behavior scores for the NFPP and HNC treatment groups did not differ significantly. The significant Time effect (p < .005) is due to an overall worsening of GIPCI-R observed parenting behaviors over time, in both treatment groups.

Parenting stress

POST

Treatment with NFPP did not differentially reduce stress, as rated by parents. Rather, NFPP and WL parents did not differ significantly, and the significant Treatment effect (p <.002) reflects that parents in the HNC condition improved significantly more than parents in NFPP (p < .03, d = .30), and Waitlist (p < .001, d = -.58).

F-UP

At F-UP, the advantage for HNC was no longer significant.

Discussion

Replicating findings from previous trials using community samples (Sonuga-Barke et al., 2001; Thompson et al., 2010), the current study found that, compared to controls, NFPP improved preschoolers' symptoms of ADHD and oppositionality when outcomes were based on parent reports (either directly or via clinicians). Notably, although the magnitude of the effect sizes for NFPP for ADHD symptoms was large, the impact of these effects on functioning is unclear given that the group's mean ADHD T-scores on the CPRS-R were still in the clinical range at the end of treatment. More important, these parent-reported effects were not corroborated by direct observations, nor by teachers, both masked to condition. By integrating constructive parenting approaches, behavior management procedures and scaffolding techniques, NFPP was designed to enhance the child's development of attention and impulse control, particularly the ability to delay. The results from preschoolers meeting ADHD diagnostic criteria do not support the claim that NFPP addresses the dysfunctions underlying ADHD, and in turn, alters the underpinning processes that lead to change in ADHD symptoms. The failure to demonstrate differential improvements on the delay task is especially telling in this regard.

In contrast to parental reports, there were no differential improvements on any teacher-rated outcome. This failure to corroborate parent ratings of therapeutic effects on core ADHD symptoms with independent measures is consistent with a recent meta-analysis that found no effects of behavioral interventions when analyses were restricted to studies using blinded measures of ADHD symptom change (Sonuga-Barke et al., 2013), and to the extensive ADHD psychosocial treatment literature documenting difficulties in achieving setting generalization (Abikoff, 2009; Hinshaw, Klein, & Abikoff, 2007).

There are a number of non-exclusive possible explanations for the discrepant effects across home and school. First, that parents, aware of treatment allocation and invested heavily in its delivery, over-estimated the effects of treatment. Second, that the intervention changed parents' perceptions of ADHD symptoms, perhaps making parents more tolerant and thus causing them to provide less severe ratings, in spite of there being no real improvement in the child's behavior itself. Third, that NFPP had real effects at home, but these did not transfer to school. It is commonly accepted that to facilitate generalization, behaviors learned in one setting (e.g., home) have to be supported and contingently rewarded in non-treatment settings at the ‘point of performance’ (e.g., school) (Rajwan et al., 2012); and neither NFPP nor HNC targets school behavior.

Although HNC has been used clinically with children with ADHD (McMahon & Forehand, 2003), to our knowledge this is the first RCT to evaluate HNC in preschoolers who met diagnostic criteria for ADHD. Overall the pattern of effects was similar to, but somewhat more effective, than those seen for NFPP. As was the case for NFPP, reductions in parent-reported ADHD and conduct problems indicate that HNC may be useful in addressing disruptive behaviors at home in preschoolers with ADHD. However, once again, these positive effects did not transfer to school settings. Parental reinforcement of compliance may have indirectly facilitated children's attention to parental instructions. However, HNC addressed non-compliance but did not specifically target ADHD behaviors, yet it had effects on parent ratings of ADHD. This pattern of treatment effects for HNC suggests that improved ratings of ADHD were due to rater bias effects, and to changes in parental perception, perhaps linked to halo-effects (e.g., Abikoff et al., 1993), and/or to the reductions in parental stress reported by HNC parents (Oord, Prins, Oosterlaan, & Emmelkamp, 2006).

Some behavioral gains were sustained after treatment. The stability in clinician-rated ADHD symptoms from POST to F-UP replicates Sonuga-Barke et al.'s (2001) report of maintenance of NFPP treatment effects on clinicians' ADHD symptom ratings, based on information provided by parents. Contrary to prediction, however, with the exception of clinician ratings of hyperactivity/impulsivity symptoms, no behavioral outcome at follow-up favored NFPP over HNC, which suggests that NFPP is not associated with sleeper effects, and does not result in a subsequent consolidation of behavioral skills that leads to differential development over time. In fact, HNC was superior to NFPP on several ADHD parent outcomes at follow-up. Because HNC did not target ADHD behaviors, this finding seems difficult to explain. It is possible that HNC was more effective at bringing about fundamental changes in parents' views of their children's difficult behavior. However, a more straightforward explanation is that HNC's relative superiority reflects a fall-off in gains with NFPP, as post-hoc analyses of parents' ADHD ratings indicated a significant increase in the severity of NFPP children's symptom scores from POST to F-UP.

The additional improvements in teacher ratings of ADHD symptoms and Defiance during follow-up are difficult to interpret in light of the non-significant treatment effect at POST on these measures and the lack of follow-up data on controls. Given the decrease in teacher-rated problems in all three groups from PRE to POST, a parsimonious explanation is that the improvement in NFPP and HNC at follow-up represents a general decrease in symptoms over time in school, rather than a sleeper effect.

Parents' self-ratings of positive parenting practices improved for both interventions. This finding is in accord with a recent meta-analysis indicating that behavioral treatments for children with ADHD have beneficial effects on parent functioning (Daley et al., 2014), and suggests that NFPP and HNC may play a role in supporting parents of preschool children with ADHD. However, HNC reduced parenting stress and improved observed parenting relative to both WL controls and NFPP, but NFPP did not impact either domain. The absence of improvement in the NFPP group of observed parenting, by raters masked to condition, replicates findings from a previous NFPP trial (Thompson et al., 2009). This result is especially noteworthy given that the observation protocol was sensitive to HNC intervention effects in the current trial and to two other group-based BPT interventions with preschoolers evaluated in RCTs (Brotman, et al., 2008; Brotman et al., 2011). This pattern of findings calls into question the ability of NFPP to significantly alter parenting behaviors, especially relative to other BPT interventions.

Objective improvements in parenting with HNC have been reported in RCTs with oppositional children based on observations of parent-child interactions at home (Peed, Roberts, & Forehand, 1977) and in the clinic (Wells & Egan, 1988). It is possible that HNC's emphasis on differential attention and the use of time out procedures contribute to positive parenting effects, not only for parents of oppositional preschoolers, but also for those of children with ADHD.

The trial had several limitations. Interpretation of maintenance effects is not definitive because of the lack of a control group during follow-up. Non-adjustment increases the risk that at least one of the significant findings represents Type I error. In contrast to Sonuga-Barke et al. (2001), but consistent with Thompson et al. (2009), there was no improvement on observed attention during the Play Park task, a laboratory assessment of on-task behavior. It is possible that differences in task duration (5 minutes in the current and Thompson et al. studies vs. 10-minutes in the Sonuga-Barke et al. trial) contributed to the failure to replicate the initial findings. Indeed, at baseline, approximately 75% of the children in the current study were on-task for the entire task, suggesting that ceiling effects may have diminished the opportunity to detect possible treatment differences. The ecological validity and clinical relevance of the study's ‘objective’ laboratory measures are uncertain. Because high behavioral variability is a hallmark of ADHD, a single laboratory assessment may not capture the child's typical behavior. Moreover, the relationship between performance on laboratory analog tasks and functioning in real-world settings is often poor (Toplak, West, & Stanovich, 2013). The inclusion of multiple, blinded observations of the child's behavior at home might address some of the limitations associated with the use of parent ratings and laboratory measures. The percentage of treatment sessions that should be reviewed to ascertain treatment integrity and inform on a study's internal validity has not been established. However, the 12% independently reviewed in the current study is lower than that recommended by Perepletchikova (2014). Approximately two-thirds of the parents had a college or graduate degree. Whether the results reported here apply to children with less well-educated parents is uncertain. Finally, the 8-week treatment duration, corresponding to the original NFPP manualized protocol, is relatively brief. It is unknown whether a longer treatment period, with a more intensive focus on parental scaffolding would yield greater benefits.

In summary, notwithstanding its limitations, the strengths of this trial, including a clinical sample of preschoolers meeting DSM-IV criteria for ADHD, the collection of teacher ratings, high treatment fidelity and integrity, and the inclusion of an active treatment comparator, increase confidence in the study results.

Key points.

• Traditional behavioral parent training (BPT) programs have limited effects on ADHD symptoms of preschool children with ADHD.

• An RCT evaluated the New Forest Parenting Package (NFPP) (a home-based program reported to improve ADHD-related behaviors in community preschoolers) in a clinical sample using parent, teacher and laboratory measures, compared to ‘Helping the Noncompliant Child’ (HNC), an evidence-based BPT comparator, and Wait-list controls.

• NFPP was not superior to, and in some cases less effective than, HNC. Both treatments improved non-blind parent ratings of ADHD but not objective teacher and laboratory measures of ADHD.

• Maintenance effects were obtained for some non-blind parent-reported outcomes, particularly for HNC.

• The findings indicate that NFPP does not result in generalized change in ADHD and document difficulties in achieving generalization across non-targeted settings.