Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Dec 7.
Published in final edited form as: Behav Disord. 2016 Feb 1;41(2):95–106. doi: 10.17988/0198-7429-41.2.95

Early Intervention for Preschoolers at Risk for Attention-Deficit/Hyperactivity Disorder: Preschool First Step to Success

Edward G Feil 1, Jason W Small 2, John R Seeley 3, Hill M Walker 4, Annemieke Golly 5, Andy Frey 6, Steven R Forness 7
PMCID: PMC5720167  NIHMSID: NIHMS898389  PMID: 29225391

Abstract

This study evaluated the efficacy of the Preschool First Step (PFS) to Success early intervention for children at risk for attention deficit hyperactivity disorder (ADHD). PFS is a targeted intervention for children 3–5 years old with externalizing behavior problems and addresses secondary prevention goals and objectives. As part of a larger multisite, randomized controlled trial, the efficacy of the PFS program was evaluated on a subsample of 45 children who also had elevated comorbid ADHD symptoms as rated by parents and teachers. The PFS program was found to produce significantly higher social skills, and significantly fewer behavior problems across a variety of teacher-and parent-reported measures at postintervention. Effect sizes for teacher-reported effects were large across a variety of social competency indicators, including those specific to ADHD. Effect sizes for parent-reported social skills and problem behaviors were medium. Although not specifically designed for preschoolers at risk for comorbid ADHD, this generic behavioral intervention appeared to be successful for this population. Implications and limitations of the study are discussed.

Most children with attention-deficit/hyper-activity disorder (ADHD) tend to have a pre-school onset of at least some of their symptoms. Recent surveillance research on a longitudinal birth cohort suggested that pediatric predictors of ADHD were evident by 17 months of age and that children with the highest levels of hyperactive behaviors at 17 months still had very stable high levels of hyperactive behaviors at age 6 (Galéra et al., 2011). Using six longitudinal data sets in a meta-analysis, Duncan and colleagues (2007) found that the strongest predictors of school readiness and later achievement were much more likely to be preschool attention skills than externalizing or internalizing disorders. They also concluded that changes in attention skills during the preschool years were especially strong predictors. In another longitudinal study of real-life dyadic friendships of young children with ADHD and comparison children over time, Normand et al. (2013) reported significantly poorer friendship quality, more friendship conflict, and less friendship satisfaction for children with ADHD over the same period. Thus, early interventions that target children at risk for ADHD in preschool appear to be a potential avenue for improving both educational attainment and social functioning.

Evidence-based treatment for ADHD usually involves stimulant medication and/or behavioral programs, such as behavioral parent training or teacher consultation (Konopasek & Forness, 2014). Charach and colleagues (2013) recently conducted a comparative effectiveness review of 55 studies on preschoolers with ADHD involving medication, behavioral intervention, or their combination. Behavioral interventions produced a mean effect size of 0.75 on child behaviors but only 0.55 on parent skill levels. The findings on stimulant medication were more equivocal because many of the studies were quite small. An exception was the Preschool ADHD Treatment Study (PATS) that involved 304 preschoolers across six different sites (Greenhill et al., 2006). Core symptoms of ADHD improved significantly with stimulant medication treatment over the initial phase of the study, as rated by both teachers and parents, but long-term follow-up suggested that most children with ADHD retained their diagnosis 6 years later and that this was especially true of children with comorbid conduct or oppositional defiant disorder (Riddle et al., 2013). Authors suggested that more emphasis on classroom intervention, rather than just parent training, might have improved outcomes. It was also noted that preschool children were twice as sensitive, compared to elementary school children, to side effects of methylphenidate even though the dose was kept relatively low (March, 2011). Thus, behavioral interventions remain the treatment of choice for most preschoolers with ADHD. For young children, however, most classroom-based early interventions do not target specific diagnoses, such as ADHD, but focus on broader goals to improve social skills or academic readiness and decrease risk behaviors (Maag & Katsiyannis, 2010; Rajwan, Chako, & Moeller, 2012). Such generic interventions typically cast a wider net and focus on clusters of age-appropriate risk behaviors rather than on one specific disorder at a time, the usual target of most mental health treatments (Forness, 2012).

Dunlap and Fox (2014) described half a dozen such generic behavioral interventions that have met relatively strict standards for evidence-based practice for preschool children at risk for behavioral problems in the classroom setting. One of these is First Step to Success (Walker et al., 1998). First Step is designed to be used in the secondary tier of a multitiered model of prevention; the intervention forges a home and school partnership in which the teacher, the child’s parents, and the First Step behavioral coach work together in teaching the target child school success skills (Walker et al., 2014). Although originally developed for application with a range of at risk externalizing behaviors, a previous investigation of a subset of children at risk for ADHD in a larger randomized controlled trial of elementary children showed considerable promise (Seeley et al., 2009), but such findings may not necessarily generalize to younger preschool-age children at risk for ADHD. A preschool adaptation of the First Step intervention has recently been developed (Feil, Small et al., 2009; Frey et al., 2013). In this study, we attempted to assess the extent to which such a broad-based intervention might also potentially benefit a subsample of preschool children at risk for ADHD who were identified retrospectively as being at risk for ADHD in a large randomized controlled trial of Preschool First Step (PFS; Feil et al., 2014).

Such subgroup analyses are somewhat controversial but are frequently used when there is compelling evidence that a subgroup may be at particular risk and thus might respond differentially to a generic intervention (Sun, Ioannidis, Agoritsas, Alba, & Guyatt, 2014). There has long been evidence that children with externalizing disorders, the intended target for First Step intervention, are at greater risk when they have comorbid ADHD (Gresham, MacMillan, Bocian, Ward, & Forness, 1988). Recent evidence has suggested that this comorbidity is also a factor in long-term nonresponse of preschoolers to stimulant medication treatment as well (Vitiello et al., 2015).

Our objectives for this subgroup analysis were (a) to determine whether PFS significantly improves the general behavioral symptoms and social functioning of preschoolers with externalizing disorders who are also at risk for ADHD, and (b) to determine if this intervention likewise significantly improves their behavioral symptoms and social functioning specific to ADHD. A secondary objective was to examine the clinical impact of PFS on preschoolers with this comorbidity.

Method

Participants

Participants for this study included a subsample of 45 of the original 126 child–parent–teacher triads from Head Start and preschool programs in Oregon, Kentucky, and Indiana who were participants in the original PFS randomized control trial (Feil et al., 2014). Project staff recruited these triads across three cohorts between 2009 and 2012. Random assignment to the PFS intervention condition or usual-care control condition occurred at the classroom level. For each classroom, one child who exhibited elevated externalizing behavior based on teacher report was recruited and consented to participate, as described below. Note that although the classroom was the unit of randomization in our original randomized control trial, this may be misleading in that teachers screened their entire classroom using a multiple gating procedure (described below) to select one target child for intervention. Thereafter the PFS intervention focused on just that one child–parent–teacher triad per classroom.

Procedures for the Original Full Sample

More detail on these procedures is available in Feil et al. (2014), but a brief overview is presented here. We obtained Institutional Review Board approval for the original study and then recruited and consented participating teachers and parents. Teachers used an adapted version of the Early Screening Project (ESP; Walker, Severson, & Feil, 1995) to identify children in their classroom exhibiting externalizing behavior. The ESP is a gated screening procedure in which teachers nominated and rank-ordered five children in their classroom who matched most closely a description of externalizing behavior. For each child identified, teachers completed three Stage 2 rating scales from the ESP: the Adaptive Behavior Index, Maladaptive Behavior Index, and Aggressive Behavior Scale. Project staff scored the rating scales, converted raw scores to severity scores, and rank-ordered the five children within each classroom to identify the highest ranked child. If we could not recruit the highest ranked child, we attempted to recruit the next ranked child in the classroom. The process continued until we obtained parent consent for one eligible child or the parents of all eligible children had declined participation. Project staff collected baseline data and then randomized each child–parent–teacher triad to a PFS intervention condition or usual-care control condition.

The 126 children in the full trial had a mean age of 4 (SD = 0.4) and were predominantly male (65%). Most children were African American (31%) or Caucasian (44%). Nearly all teachers were female (99%) and most were either African American (18%) or Caucasian (72%). Teachers had taught for approximately 14 years, and 22% had a high school diploma, 33% an associate’s degree, 23% a bachelor’s degree, and 22% a master’s degree or higher.

Teachers in classrooms randomized to the intervention condition received (a) 4 hr of training in general education classroom management and positive behavior support; (b) 4 hr of training in PFS (described in detail below); and (c) ongoing, one-on-one consultation and support from a behavioral coach who worked with the teacher to implement the program. Teachers randomized to the usual-care control condition received only the same 4 hr of training in general education classroom management strategies. For parents randomized to the intervention condition, a behavioral coach worked with them for 6 weeks in 1-hr sessions to promote school success skills via reading, discussion, role play, and demonstrations. Coaches were employees of Oregon Research Institute or University of Louisville (eight coaches at each site) and held a bachelor’s degree or higher. Coaches attended a 2-day intensive training session on PFS implementation. Lead implementers met weekly with coaches to discuss and troubleshoot cases and were closely monitored via frequent fidelity checks during implementation. Postintervention data were collected roughly 4 months later, at completion of the intervention, and were yoked in timing to usual-care control triads for equivalency between groups.

Procedures for ADHD subsample

Although we did not report ADHD symptomology or functioning in our original main effects study (Feil et al., 2014), we had nonetheless collected data on such measures in anticipation of a subgroup or other moderator analysis. Consistent with recommendations by Rothwell (2005) for subgroup construction and analysis, we used parent and teacher report on the Conners’ ADHD Scales (CADS; Conners, 1999) to identify retrospectively the subsample of children at risk for ADHD in this study. The CADS scale has excellent evidence of reliability and validity and includes 18 items assessing the presence of ADHD symptomatology in the last month. Parents and teachers rate items on a 4-point rating scale ranging from not true at all to very much true. We used CADS normative data to identify the ADHD sample of children, with both parent-reported and teacher-reported ADHD total scores needing to be 1 SD or more above the mean to meet inclusion criteria. Applying this cutoff, 45 of the 126 children participating in the efficacy trial (35%) met criteria for being at risk of developing ADHD. We compared the ADHD sample identified for this report (n = 45) to the remaining nonidentified children from the sample (n = 81) on baseline outcome measures. As expected and as depicted in Table 1, the identified and nonidentified children differed significantly on five of 10 baseline measures and approached significance (p < .10) on three other baseline measures.

TABLE 1.

Baseline Means and Standard Deviations by Identified and Nonidentified Samples for Outcome Measures

Variable Identified Sample (n = 45)
Nonidentified Sample (n = 81)
Test Statistic
M (SD) M (SD) t p Value
Symptoms
 SSiS Hyperactive/Inattentive 14.2 (3.0) 12.3 (3.6) −2.93 .004
 C-TRF ODD 7.9 (3.3) 6.6 (3.4) −1.93 .056
 ESP MBI 30.4 (6.1) 28.7 (6.8) −1.44 .152
 ESP ABS 22.4 (6.7) 19.1 (5.8) −2.88 .005
Functioning
 ESP ABI 21.6 (4.5) 23.2 (4.9) 1.79 .076
 SSiS Cooperation 6.4 (2.1) 7.4 (2.8) 2.27 .025
 SSiS Engagement 10.9 (4.0) 10.7 (3.6) −0.24 .814
 SSiS Self-Control 6.6 (3.1) 7.7 (3.5) 1.66 .099
Home-based
 SSiS-PB 125.9 (12.7) 113.3 (17.2) −4.28 ,.001
 SSiS-SS 88.1 (12.4) 94.9 (13.6) 2.73 .007
Open in a new tab

Note. SSiS = Social Skills Improvement System Rating Scales; C-TRF ODD = Caregiver-Teacher Report Form Oppositional Defiant Disorder; ESP MBI = Early Screening Project Maladaptive Behavior Index; ESP ABS = Early Screening Project Aggressive Behavior Scale; ESP ABI = Early Screening Project Adaptive Behavior Index; SSiS-PB = Social Skills Improvement System Rating Scales Problem Behaviors; SSiS-SS = Social Skills Improvement System Rating Scales - Social Skills.

Of the 45 children at risk for ADHD, 19 had been randomized to the usual-care condition and 26 children had been randomized to the PFS condition. To establish baseline equivalency between intervention and control conditions, we examined child, parent, and teacher baseline demographics and baseline outcome measures for the two conditions. There were no statistically significant differences between the two groups on basic child demographic and screening characteristics as reported in Table 2. Not shown in Table 2, the control and intervention conditions also did not differ on other examined demographic variables including age of participating parent (M[SD] = 30.5[7.9] vs. 30.3[6.3]), percent of parents who were African American (37% vs. 35%), or Caucasian (53% vs. 58%), percent of parents with a bachelor’s degree or higher (5% vs. 7%), or the number of children in the household (M[SD] = 2.4[1.5] vs. 2.4[0.9]). Across conditions, all teachers were female. The majority of teachers in the control and intervention groups was Caucasian (74% and 77%, respectively). A disproportionate percentage of teachers in the control condition had reportedly attained a bachelor’s degree or higher (63%) as compared to the intervention condition (31%; χ2 = 4.66, p = .030). Although nonsignificant, teachers in usual-care condition reported teaching for more years (M[SD] = 18.1[9.5]) than teachers in intervention condition (M[SD] = 13.6[8.6]; t[42] = 1.63, p =.11).

TABLE 2.

Baseline Equivalence of Demographic Characteristics and Screening Measures

Variable Total (n = 45) Control/Usual Care (n = 19) Intervention (n = 26) Test Statistic p Value
Demographic characteristic
 Age M (SD) 4.1 (0.4) 4.1 (0.4) 4.1 (0.4) −0.20 .840
 Female (%) 16 (35.6) 6 (31.6) 10 (38.5) 0.23 .634
 African American (%) 16 (35.6) 7 (36.8) 9 (34.6) 0.02 .878
 Caucasian (%) 22 (48.9) 8 (42.1) 14 (53.8) 0.61 .436
 IFSP (%) 12 (26.7) 4 (21.1) 8 (30.8) 0.53 .467
Screening measures
 ESP rank 0.36 .833
  Ranked first (%) 36 (80.0) 16 (84.2) 20 (76.9)
  Ranked second (%) 6 (13.3) 2 (10.5) 4 (15.4)
  Ranked third (%) 3 (6.7) 1 (5.3) 2 (7.7)
 Aggressive Behavior Scale M (SD) 23.4 (5.5) 23.6 (7.0) 23.2 (4.3) 0.26 .795
 Adaptive Behavior Index M (SD) 20.8 (4.0) 21.3 (3.6) 20.5 (4.3) 0.66 .514
 Maladaptive Behavior Index M (SD) 32.2 (5.3) 31.8 (5.9) 32.5 (5.0) −0.46 .646
Open in a new tab

Note. Reported test statistics are t for continuous and χ2 for dichotomous measures. IFSP is percent of children having an individualized family services plan, indicating receipt of special education services. ESP = Early Screening Project.

Preschool First Step to Success Intervention

The PFS intervention included a day-long workshop for teachers in the universal principles of classroom management (Sprague & Golly, 2013) and also training in the PFS intervention (Walker et al., 1998). A behavioral coach was then assigned to each participating teacher and was available for one-on-one consultation in his or her respective classroom during instructional hours. PFS is described very briefly below (for more detailed descriptions, see Feil et al., 2014; Frey et al., 2013).

Classroom-Based Component

Implementation of the classroom component had three phases of 10 days each: (a) the coach phase, (b) the teacher phase, and (c) maintenance. The PFS program involves providing feedback initially from the coach and then the teachers, using a green and red card —green for positive classroom behavior and red for off-task behavior. The participating child receives points and praise for engaging in appropriate classroom behavior (e.g., following classroom rules, cooperating, sharing, sitting quietly during circle time). Group dependent contingencies are used not only to motivate the participating child but also to provide incentives for the child’s classmates to invest in his/her success; individual contingencies through a home-school reward system provide additional incentives to support the child’s mastery of school success skills.

Home-Based Component

Over this same period, caregivers meet for six home visitation sessions with the First Step coach to learn how to teach school success skills via reading, discussion, role play, and demonstrations. Specific skills taught are communication and sharing, cooperation, limit setting, problem solving, friendship making, and self-confidence. Parents are also provided with a manual containing all the information and accompanying materials needed to implement this home component.

School-Based Outcome Measures

As part of the larger assessment battery for the full study, teachers and parents completed the Social Skills Improvement System Rating Scales (SSiS; Gresham & Elliott, 2008). Teachers completed three ESP Stage 2 rating scales (Feil, Severson, & Walker, 1998) and the Achenbach Caregiver-Teacher Report Form DSM (Diagnostic and Statistical Manual of Mental Disorders)-oriented subscale for Oppositional Defiant Disorder (ODD; Achenbach & Rescorla, 2001). We used these measures divided into two domains for this study as described below: (a) ADHD and disruptive behavior symptoms, and (b) social functioning.

ADHD and Disruptive Behavior Symptoms

For this study, symptom scales included a measure specific to ADHD hyperactive and inattentive behavior, along with a general measure of maladaptive behavior, a general measure of aggressive behavior, and one measure of ODD symptoms (mean intercorrelation = .62). To examine ADHD-specific variables, we used inattentive and hyperactive behavior scores on the SSiS Hyperactivity/Inattention subscale. The seven-item scale (α = .76) examines the teacher’s perceived frequency of hyperactive or inattentive behaviors such as fidgeting, impulsivity, distractibility, difficulty waiting for turns, and disrupting group activities (Gresham & Elliott, 2008). SSiS items are reported on a 4-point frequency scale (never, seldom, often, almost always). Scores can range from 0 to 21 with higher scores indicating higher levels of impairment. The ESP Maladaptive Behavior Index is a nine-item scale (α = .81) assessing the child’s teacher-related and peer-to-peer maladaptive behavioral adjustments. The ESP Aggressive Behavior Scale is a nine-item scale (α= .79) measuring the frequency of aggressive behavior. The Maladaptive Behavior Index and Aggressive Behavior Scale are rated on a 5-point frequency scale ranging from never to frequently. Raw scale scores (ranging from 9 to 45) were computed for each measure. Higher scores indicate higher levels of problem behavior. The Care-giver-Teacher Report Form ODD measure is a seven-item (α= .82) DSM-oriented subscale assessing ODD symptoms. Caregiver-Teacher Report Form items are rated on a 3-point scale from not true to very true. Raw scores range from 0 to 14 with higher scores indicating higher levels of ODD symptomatology.

Social Functioning

Measures of social functioning included ADHD-related measures of cooperation, engagement, and self-control, as well as a general measure of adaptive behavior (mean intercorrelation = .36). The SSiS cooperation subscale is a six-item scale (α = .79) assessing behaviors such as following rules and directions, completing tasks, and participating appropriately. The SSiS engagement subscale is a seven-item measure (α = .83) assessing the child’s ability to join activities, make friends, and interact with others. The SSiS self-control subscale consists of seven items (α = .76) assessing the child’s ability to resolve disagreements, accept criticism, and respond calmly to teasing or bullying. These three SSiS subscales are rated on the same 4-point frequency scale described above and were used as measures of particular concern in functioning of children with ADHD. Higher scores on the three scales indicate higher levels of school-based functioning. The ESP Adaptive Behavior Index is a more general eight-item scale (α = .77) assessing teacher-related and peer-to-peer adaptive behavioral adjustments. Adaptive Behavior Index items are scored on a 5-point frequency scale. Raw scores range from 8 to 40 with higher scores indicating higher levels of adaptive functioning.

Home-Based Outcome Measures

Home-based outcomes included parent report for the SSiS social skills and problem behavior scales. The social skills scale targets learned behaviors that promote pro-social interactions with adults and peers (Gresham & Elliott, 2008). It includes 46 items (α = .95) assessing the child’s development of social skills pertaining to day-to-day interaction and activities in the home setting. The 33-item (α = .92) problem behavior scale assesses internalizing and externalizing behaviors that may interfere with the child’s social functioning. Informants report their perceived frequency of the child’s problem behaviors on a 4-point scale.

Process Measures

We collected PFS coach and teacher implementation fidelity data related to the school component and data to identify dosage, parent fidelity, and parent compliance. We also collected indicators of teacher and coach alliance, as well as parent and teacher satisfaction data for participants randomized to the intervention condition. The implementation fidelity checklist assessed the adherence and quality of classroom implementation and was collected once during the coach phase and twice during the teacher phase with an interrater reliability of .82. The classroom monitoring form was used to record the target child’s compliance during the classroom component such as daily points earned, whether the daily criterion was met, or if the program day was repeated because the child did not meet the daily reward criterion. Classroom dosage is the proportion of program days out of 30 completed. The home monitoring form was used to compute dosage, parent fidelity, and parent compliance for the home component of First Step and is completed by the First Step coach after each session. The alliance survey was completed by the coach and teacher at postintervention to assess various aspects of their working relationship. Coefficient alpha was .94 for the coach version and .95 for the teacher version. The satisfaction survey was collected by teachers and parents at postintervention, and assessed the perception of training and support received, program usability, and program effectiveness, with alphas of .91 and .94, respectively. Further details on all of the above measures are available in Sumi et al. (2013), Walker et al. (2009), and Woodbridge et al. (2014).

Statistical Analysis

For each outcome, we specified a linear regression model in Mplus 6.0 (Muthèn & Muthèn, 1998–2010). We utilized the full information maximum likelihood estimator to account for missing data. Full information maximum likelihood utilizes all available data to calculate unbiased parameter estimates and standard errors (Schafer & Graham, 2002). Each regression model included one covariate (i.e., the baseline value of the outcome) and a dichotomous predictor for intervention condition (1 = PFS intervention, 0 = control).

We applied the Benjamini-Hochberg correction (B-H; Benjamini & Hochberg, 1995) to correct for multiple comparisons. To calculate a B-H correction, statistically significant outcomes are ranked in ascending order based on p value and a new critical cutoff for each is calculated using the formula px′ = .05x/n, where x equals the rank for each outcome and n equals the number of ranked outcomes (What Works Clearinghouse [WWC], 2014). Applying this formula to the 10 outcomes reported in this manuscript, the first outcome would still be significant at p < .005 and the 10th outcome would still be significant at p < .05.

We report Hedges’ g as a measure of effect size and the WWC (2011) improvement index as a measure of practical significance. Hedges’ g is the difference between the mean outcome score of each group divided by the pooled within-group standard deviation (WWC, 2011). Effect sizes of 0.25, 0.5, and 0.8 are considered small, medium, and large effects, respectively.

We utilized Jacobson and Truax’s (1991) Reliable Change Index (RCI) to identify clinically significant postintervention response. Change is based on a two-step criterion that accounts for magnitude of change, the RCI, and change in functioning or movement across a specified cutoff (Jacobson, Roberts, Berns, & McGlinchey, 1999). To assess change in functioning or, more specifically, reduction in ADHD symptomatology, we set a cutoff at 1 SD above the normative mean (T = 59). We then classified children as (a) responded if at postintervention they moved into the normative range on the CADS screening measures and had a RCI less than −1.96; (b) improved if they did not move into the normative range but had an RCI less than −1.9; (c) unchanged if they did not meet the RCI criterion (i.e., RCI > −1.96 and < 1.96); or (d) deteriorated if the RCI criterion worsened (i.e., RCI > 1.96). See Small et al. (2015) for more detail.

Results

Attrition and Missing Data

Of the 45 children in the ADHD sample, baseline questionnaire data were available for all teachers and parents. At postintervention, we obtained questionnaire data from 100% of teachers and 89% of parents. Missing item-level data precluded subscale scoring for two additional cases. One child was missing parent-reported data for one of two outcomes at baseline and another child was missing teacher-reported data for four of eight outcomes. We utilized Little’s Missing Completely At-Random test (Little, 1988) to examine the assumption that data were missing completely at random. The test was non-significant (χ2 = 48.72, df =1, p =.641), suggesting data were indeed missing completely at random.

Process Measures

The summary of descriptive statistics for child, parent, and teacher process measures were as follows. Coaches adhered to 96%, and teachers to 94%, of observed core program components. Classroom implementation quality was excellent for coaches (.92; range = .77–1.00) and acceptable for teachers (.78; range = .41–1.00). Children in the ADHD sample received 97% (range = 67%–100%) of program days. Families received 87% (range = 17%–100%) of home sessions. Child compliance was excellent (.89), but parent compliance (.54) was low and parent fidelity (.78) was moderate. Informants rated highly their working relationship with one another. Mean coach ratings were 4.3 on a 5-point scale and mean teacher ratings were 4.9. Teachers and parents also reported high levels of program satisfaction (4.4 and 4.3, respectively, on a 5-point scale).

Posttest Differences on Outcome Measures

Table 3 summarizes results from the covariate-adjusted regression models. Even after a B-H correction, children who received the PFS intervention had statistically significant improvements from baseline to postintervention on all teacher-reported and parent-reported outcomes. For teacher-reported reductions in symptoms, Hedges’ g effect sizes ranged from −1.02 to −1.14. For teacher-reported improvements in functioning, effect sizes ranged from 0.80 to 1.40. Hedges’ g effect sizes for parent-reported outcomes were in the medium range for reduction in symptoms (−0.77) and improvements in functioning (0.65) in the home setting.

TABLE 3.

Baseline and Postintervention Means and Standard Deviation for Outcome Measures by Condition and Regression Results

Domain/Measure Baseline
Usual Care/Control (n = 19)
Baseline
Intervention (n = 26)
Condition Effect
Effect Size
Postintervention
Postintervention
M (SD) M (SD) MAdj M (SD) M (SD) MAdj Test Statistic p Value Hedges’ g
Symptoms
 SSiS Hyperactive/Inattentive 14.7 (2.9) 13.4 (2.9) 13.1 13.7 (3.1) 9.2 (3.8) 9.4 −3.95 <.001 −1.10
 C-TRF ODD 8.6 (2.8) 7.8 (4.0) 7.6 7.3 (3.7) 3.8 (3.0) 4.0 −3.34 .001 −1.02
 ESP MBI 31.7 (5.7) 30.4 (6.3) 29.9 29.5 (6.4) 22.4 (6.7) 22.7 −3.90 <.001 −1.09
 ESP ABS 24.1 (8.0) 22.3 (8.3) 21.9 21.1 (5.3) 14.5 (4.4) 14.7 −3.38 .001 −1.14
Functioning
 ESP ABI 20.8 (4.2) 23.9 (5.6) 24.0 22.3 (4.7) 30.5 (5.4) 30.4 3.88 <.001 1.16
 SSiS Cooperation 6.3 (1.9) 7.8 (2.3) 7.9 6.5 (2.2) 11.4 (2.6) 11.4 4.73 <.001 1.40
 SSiS Engagement 11.0 (4.3) 11.8 (3.5) 11.8 10.8 (3.8) 14.8 (3.9) 14.8 3.92 <.001 0.80
 SSiS Self-Control 6.1 (2.8) 7.5 (4.2) 7.6 7.0 (3.2) 11.3 (3.6) 11.2 3.29 .001 0.92
Home-based
 SSiS-PB 126.7 (14.1) 126.8 (15.4) 127.1 125.3 (11.9) 114.7 (15.9) 115.0 −2.98 .003 −0.77
 SSiS-SS 86.8 (14.0) 88.2 (12.3) 89.2 89.0 (11.2) 97.6 (12.4) 97.3 2.42 .015 0.65
Open in a new tab

Note. SSiS = Social Skills Improvement System Rating Scales; C-TRF ODD = Caregiver-Teacher Report Form Oppositional Defiant Disorder; ESP MBI = Early Screening Project Maladaptive Behavior Index; ESP ABS = Early Screening Project Aggressive Behavior Scale; ESP ABI = Early Screening Project Adaptive Behavior Index; SSiS-PB = Social Skills Improvement System Rating Scales -Problem Behaviors; SSiS-SS = Social Skills Improvement System Rating Scales - Social Skills.

Responder Analysis

Based on teacher report, seven children in the intervention condition responded (27%), seven children improved (27%), and 12 children remained unchanged (46%). In comparison, one child in the control group responded (5%), one child improved (5%), 15 remained unchanged (79%), and two children in the control group deteriorated (11%). Overall, 14 intervention children (54%) either responded or improved after participating in the PFS intervention compared to two children (11%) in the control condition (χ2 = 8.99, p = .002, OR[95% CI] = 9.9[1.9, 51.9]). The seven intervention children classified as responded had on average a 2 SD improvement in postinter-vention CADS scores (M[SD] = −20.7[4.6]). For the seven intervention children classified as improved, postintervention CADS scores decreased by more than 1.5 SDs on average (M[SD] = −16.3[5.2]). Eight of the 12 children classified as unchanged improved by more than a .5 SD (M[SD] = −5.9[3.1]) but did not demonstrate clinically meaningful improvement. Scores for the remaining four children classified as unchanged remained stable from baseline to post intervention (M[SD] = 0.8[1.0]).

Based on parent report, 10 children in the intervention condition responded (39%), six children improved (23%), and six remained unchanged (23%). The remaining four children did not have parent-reported CADS data at postintervention. In comparison, two children in the control group (11%) responded, two children improved (11%), 11 remained unchanged (58%), and two deteriorated (11%). We did not have parent-reported CADS data at postintervention for the remaining two children in the control condition. Overall, 16 children (62%) in the intervention condition responded or improved based on parent report, whereas only four children (21%) in the control condition responded or improved (χ2 = 9.29, p = .002, OR[95% CI] = 8.7[2.0, 37.4]). Scores for the 10 intervention children classified as responded decreased, on average, by more than 1.5 SDs (M[SD] = −17.9[10.0]). Scores for the six children classified as improved decreased by almost 1.5 SDs (M[SD] = −13.3[5.3]).

Discussion

This study suggests that PFS seems particularly effective with preschool-age children with externalizing behavior who are also exhibiting elevated comorbid ADHD symptoms. The PFS intervention was implemented with relatively high levels of fidelity and program compliance for this subsample at risk for ADHD. These preschoolers, furthermore, showed highly significant improvement on a broad range of teacher- and parent-reported outcome measures, with eight of 10 effect sizes in the large range. Such effect sizes were also higher relative to those found in the original study in which only one of seven outcomes was large (Feil et al., 2014). Additionally, although PFS was not designed to target specific disorders such as ADHD, it nonetheless appears to improve both the specific symptoms and social functioning associated with this disorder, and its effect sizes also compare favorably with those for ADHD-specific interventions (Charach et al., 2013; Comer, Chow, Chan, Cooper-Vince, & Wilson, 2013).

This study has some noteworthy limitations as well. First, our interest in examining the potential efficacy of the PFS for children at risk for ADHD did not begin with an a priori hypothesis but was based on a relatively small subsample selected after the fact. Such subsample analyses can thus only be considered valid as hypothesis-generating studies, and evidence-based practice recommendations must await further confirmation from a priori randomized clinical trials (Rothwell, 2005; Sun et al., 2014). Second, our participants in this subsample can only be described as “at risk.” Our results might have been different had we selected preschoolers on a more thorough diagnostic assessment for ADHD involving more than one symptom-rating scale and concomitant measures of functional deficits or administration of a structured diagnostic interview. Third, our outcome measures were dependent on unblinded ratings by participants without further confirmation by direct observation or independent ratings. We might also note here the relatively more modest fidelity and compliance results from parents in this study, as well as lower home-base outcomes, suggesting a failure to fully engage and utilize families. For this reason, we have recently competed a pilot study (Frey et al., 2014) enhancing the PFS home component with motivational interviewing strategies and a more intensive individualized approach for each family, and these results have lead to a large randomized trial of First Step with these added modifications that is currently underway.

In the context of this subgroup analysis, however, our study nonetheless had several strengths derived from the original sample including the use of a randomized design to control for threats to internal validity, measures that are well established in preschool intervention research, simultaneous replication across three different sites, a relatively ethnically diverse sample of participants, substantial evidence that our intervention and control conditions were relatively well matched at baseline, use of not only generic outcome measures but also of measures specific to ADHD symptoms and functioning, and a responder analysis. This last analysis suggests a substantial clinical response such that on teacher ratings, more than a quarter of children moved well into the normative range with treatment and another quarter improved quite significantly (on average by 1.5 SDs). Such an improvement in preschool suggests a potentially favorable long-term outcome (Bussing et al., 2012).

Although we cannot recommend PFS as either a comprehensive or specific treatment for preschool ADHD at this point, it should be noted that most children with ADHD may not be recognized as such until well into their elementary years given that ADHD may often be comorbid with other externalizing or even internalizing disorders, which complicates its diagnosis (Forness & Kavale, 2002). These children may nonetheless demonstrate significantly disruptive behavior well before school entry, which may prompt a preschool referral for generic tier-two intervention. The selection of PFS as such an intervention might prove to be fortuitous in that several of the strategies employed in PFS seem quite consistent with approaches recommended in ADHD-specific interventions (Charach et al., 2013).

As for implications for practice, it is important to note that more than a third of the 126 preschoolers who were carefully screened for externalizing behavior in the original First Step trial also met criteria as being at risk for comorbid ADHD in this study. This is relatively consistent with epidemiologic studies of such comorbidity (Forness & Kavale, 2002). Thus, teachers should be aware that a substantial number of children with oppositional, defiant, aggressive, or other externalizing symptoms may also be at risk for ADHD. In this study, 62% of preschoolers with comorbid ADHD showed a significant clinical response to First Step (compared to 21% in the control group). This is impressive even in the context of response rates in more intensive ADHD interventions (Konopasek & Forness, 2014).

Preschool teachers should note, however, that a number of children with externalizing behaviors also seem to have an overlay of especially inattentive, impulsive, or hyperactive behaviors and may not always respond even to the best of such generic tier-two behavioral interventions. These nonresponders may indeed necessitate referral for a formal diagnostic workup for ADHD, possibly leading to more intensive tier-three intervention involving enhanced parent collaboration and classroom management strategies specific to ADHD. Teachers should also know that such evidence-based interventions for ADHD are characterized by multimodal treatment involving not only such enhanced psychosocial interventions but also possible medication management for nonresponders by qualified physicians (DuPaul & Stoner, 2014). Thus the best of tier-two pre-school interventions such as First Step serve not only to improve behavioral and social outcomes of most children at risk but also provide more compelling evidence of further need for more intensive and focused interdisciplinary treatment for nonresponders (Mitchell, Stormant, & Gage, 2011).

Acknowledgments

Research was supported by a grant from NICHD #R01HD055334. Drs. Feil, Walker, and Golly are three of the authors of the Preschool First Step to Success intervention.

Contributor Information

Edward G. Feil, Oregon Research Institute

Jason W. Small, Oregon Research Institute

John R. Seeley, Oregon Research Institute

Hill M. Walker, Oregon Research Institute

Annemieke Golly, Oregon Research Institute.

Andy Frey, University of Louisville.

Steven R. Forness, University of California–Los Angeles

References

  1. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont Research, Center for Children, Youth, & Families; 2001. [Google Scholar]
  2. Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B. 1995;57:289–300. [Google Scholar]
  3. Bussing R, Porter P, Zima BT, Mason D, Garvan C, Reid R. Academic outcome trajectories of students with ADHD: Does exceptional education status matter? Journal of Emotional and Behavioral Disorders. 2012;20:131–143. doi: 10.1177/1063426610388180. [DOI] [Google Scholar]
  4. Charach A, Carson P, Fox S, Ali MU, Beckett J, Lim CG. Interventions for preschool children at high risk for ADHD: A comparative effectiveness review. Pediatrics. 2013;131:e1584–e1604. doi: 10.1542/peds.2012-0974. [DOI] [PubMed] [Google Scholar]
  5. Comer JS, Chow C, Chan PT, Cooper-Vince C, Wilson LA. Psychosocial treatment efficacy for disruptive behavior problems in very young children: A meta-analytic examination. Journal of the American Academy of Child & Adolescent Psychiatry. 2013;52:26–36. doi: 10.1016/j.jaac.2012.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Conners CK. Conners attention deficit/hyperactivity disorder diagnostic scale. North Tonawanda, NY: Multi-Health Systems; 1999. [Google Scholar]
  7. Duncan GJ, Dowsett CJ, Claessens A, Magnuson K, Huston AC, Klebanov P, … Japel C. School readiness and later achievement. Developmental Psychology. 2007;43:1428–1446. doi: 10.1037/0012-1649.43.6.1428. [DOI] [PubMed] [Google Scholar]
  8. Dunlap G, Fox L. Supportive interventions for young children with social, emotional, and behavioral delays and disorders. In: Walker HM, Gresham FM, editors. Handbook of evidence-based practices for emotional and behavioral disorders: Applications in schools. New York: Guilford; 2014. pp. 503–517. [Google Scholar]
  9. DuPaul GJ, Stoner G. ADHD in the schools: Assessment and intervention strategies. 3. New York: Guilford; 2014. [Google Scholar]
  10. Feil EG, Frey AJ, Walker HM, Seeley J, Golly A, Small J. The efficacy of a home/school intervention for preschoolers with challenging behaviors: A randomized control trial of Preschool First Step to Success. Journal of Early Intervention. 2014;36:151–170. doi: 10.1177/1053815114566090. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Feil EG, Severson HH, Walker HM. Screening for emotional and behavioral delays: The Early Screening Project. Journal of Early Intervention. 1998;21:252–266. doi: 10.1177/105381519802100306. [DOI] [Google Scholar]
  12. Feil EG, Walker HM, Severson HH, Golly A, Seeley J, Small J. Using positive behavior support procedures in Head Start class rooms to improve school readiness: A group training and behavioral coaching model. NHSA Dialog. 2009;12:88–103. doi: 10.1080/15240750902774676. [DOI] [Google Scholar]
  13. Forness SR. Special education and the new mental health. Behavioral Disorders. 2012;37:41–46. [Google Scholar]
  14. Forness SR, Kavale KA. Impact of ADHD on school systems. In: Jensen PS, Cooper JR, editors. Attention deficit hyperactivity disorder: State of the science: Best practices. Kingston, NJ: Civic Research Institute; 2002. pp. 24-1–24-20. [Google Scholar]
  15. Frey AJ, Lee J, Small JW, Seeley JR, Walker HM, Feil EG. Transporting motivational interviewing to school settings to improve the engagement and fidelity of tier 2 interventions. Journal of Applied School Psychology. 2013;29:183–202. doi: 10.1080/15377903.2013.778774. [DOI] [Google Scholar]
  16. Frey AJ, Lee J, Small JW, Walker HM, Seeley JR, Ratcliffe P. Motivational interviewing training and support for school-based applications. 2014 Manuscript submitted for publication. [PMC free article] [PubMed] [Google Scholar]
  17. Frey AJ, Small JW, Feil E, Seeley JR, Walker HW, Golly A. The feasibility of First Step to Success with preschoolers. Children & Schools. 2013;35:171–188. doi: 10.1093/cs/cdt014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Galéra C, Côté SM, Bouvard MP, Pingault JB, Melchior M, Michel G, … Tremblay RE. Early risk factors for hyperactivity-impulsivity and inattention trajectories from age 17 months to 8 years. Archives of General Psychiatry. 2011;68:1267–1275. doi: 10.1001/archgenpsychiatry.2011.138. [DOI] [PubMed] [Google Scholar]
  19. Greenhill L, Kollins S, Abikoff H, McCracken J, Riddle M, Swanson J, … Cooper T. Efficacy and safety of immediate-release methyl-phenidate treatment for preschoolers with ADHD. Journal of the American Academy of Child & Adolescent Psychiatry. 2006;45:1284–1293. doi: 10.1097/01.chi.0000235077.32661.61. [DOI] [PubMed] [Google Scholar]
  20. Gresham FM, Elliott SN. Social Skills Improvement System. Bloomington, MN: Pearson Assessments; 2008. [Google Scholar]
  21. Gresham FM, MacMillan DL, Bocian KM, Ward SL, Forness SR. Comorbidity of hyperactivity-impulsivity-inattention and conduct problems: Risk factors in social, affective, and academic domains. Journal of Abnormal Child Psychology. 1998;26:393–406. doi: 10.1023/A:1021908024028. [DOI] [PubMed] [Google Scholar]
  22. Jacobson NS, Truax P. Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology. 1991;59:12–19. doi: 10.1037/0022-006X.59.1.12. [DOI] [PubMed] [Google Scholar]
  23. Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology. 1999;67:300–307. doi: 10.1037/0022-006X.67.3.300. [DOI] [PubMed] [Google Scholar]
  24. Konopasek DE, Forness SR. Issues and criteria for the effective use of psychopharmacological interventions in schooling. In: Walker HM, Gresham FM, editors. Handbook of evidence-based practices for emotional and behavioral disorders: Applications in schools. New York: Guilford; 2014. pp. 457–472. [Google Scholar]
  25. Little RJ. A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association. 1988;83:1198–1202. doi: 10.1080/01621459.1988.10478722. [DOI] [Google Scholar]
  26. Maag JW, Katsiyannis A. Early intervention programs for children with behavior problems and at risk for developing antisocial behaviors: Evidence-and research-based practices. Remedial and Special Education. 2010;31:464–475. doi: 10.1177/0741932509355992. [DOI] [Google Scholar]
  27. March JS. The preschool ADHD treatment study (PATS) as the culmination of twenty years of clinical trials in pediatric psychopharmacology. Journal of the American Academy of Child & Adolescent Psychiatry. 2011;90:427–429. doi: 10.1016/j.jaac.2010.09.018. [DOI] [PubMed] [Google Scholar]
  28. Mitchell BS, Stormont M, Gage NA. Tier two interventions implemented within the context of a tiered prevention framework. Behavioral Disorders. 2011;36:241–261. [Google Scholar]
  29. Muthèn LK, Muthèn BO. Mplus user’s guide. 6. Los Angeles: Psychological Assessment Resources; 2010. [Google Scholar]
  30. Normand S, Schneider BH, Lee MD, Maisonneuve MF, Chupetlovska-Anastasova A, Kuehn SM, Robaey P. Continuities and changes in the friendships of children with and without ADHD: A longitudinal, observational study. Journal of Abnormal Child Psychology. 2013;41:1161–1175. doi: 10.1007/s10802-013-9753-9. [DOI] [PubMed] [Google Scholar]
  31. Rajwan E, Chako A, Moeller M. Non-pharmacological intervention for preschool ADHD: State of the evidence and implications for practice. Professional Psychology. 2012;43:520–526. [Google Scholar]
  32. Riddle MA, Yershova K, Lazzaretto D, Paykina N, Yenokyan G, Greenhill L, … Posner K. The preschool attention-deficit/hyperactivity disorder treatment study (PATS) 6-year follow-up. Journal of the American Academy of Child & Adolescent Psychiatry. 2013;52:264–278. doi: 10.1016/j.jaac.2012.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Rothwell PM. Subgroup analysis in randomized controlled trials: Importance, indications, and interpretation. The Lancet. 2005;365:176–186. doi: 10.1016/S0140-6736(05)17709-5. [DOI] [PubMed] [Google Scholar]
  34. Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 2002;7:147–177. [PubMed] [Google Scholar]
  35. Seeley JR, Small JW, Walker HM, Feil EG, Severson HH, Golly AM, Forness SR. Efficacy of the First Step to Success intervention for students with ADHD. School Mental Health. 2009;1:37–48. doi: 10.1007/s12310-008-9003-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Small JW, Seeley JR, Feil EG, Frey A, Walker HM, Forness SR. Utilizing measures of clinical significance to classify responsiveness to preventive behavioral interventions. 2015 Manuscript submitted for publication. [Google Scholar]
  37. Sprague J, Golly A. BEST behavior: Building positive behavior support in schools. 2. Longmont, CO: Sopris Learning; 2013. [Google Scholar]
  38. Sumi WC, Woodbridge MW, Javitz HS, Thornton SP, Wagner M, Rouspil K, … Severson HH. Assessing the effectiveness of First Step to Success: Are short-term results the first step to long-term behavioral improvements? Journal of Emotional and Behavioral Disorders. 2013;21:66–78. doi: 10.1177/1063426611429571. [DOI] [Google Scholar]
  39. Sun X, Ioannidis JA, Agoritsas T, Alba AC, Guyatt G. How to use a subgroup analysis: Users’ guide to the medical literature. Journal of American Medical Association. 2014;311:405–411. doi: 10.1001/jama.2013.285063. [DOI] [PubMed] [Google Scholar]
  40. Vitiello B, Lazzaretto D, Yershova K, Abikoff H, Paykina N, McCracken JT, … Riddle MA. Pharmacotherapy of the Preschool ADHD Treatment Study (PATS) children growing up. Journal of the American Academy of Child & Adolescent Psychiatry. 2015;54:550–556. doi: 10.1016/j.jaac.2015.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Walker HM, Kavanagh K, Stiller B, Golly A, Severson HH, Feil EG. First Step: An early intervention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders. 1998;6:66–80. doi: 10.1177/106342669800600201. [DOI] [Google Scholar]
  42. Walker HM, Seeley JR, Small J, Severson HH, Graham B, Feil EG, … Forness SR. A randomized controlled trial of the First Step to Success early intervention: Demonstration of program efficacy outcomes in a diverse, urban school district. Journal of Emotional and Behavioral Disorders. 2009;17:197–212. doi: 10.1177/1063426609341645. [DOI] [Google Scholar]
  43. Walker HM, Severson HH, Feil EG. Early screening project: A proven child-find process. Longmont, CO: Sopris West; 1995. [Google Scholar]
  44. Walker H, Severson HH, Seeley J, Feil EG, Small J, Golly A, … Forness S. The evidence base of the First Step to Success early intervention for preventing emerging antisocial behavior patterns. In: Walker HM, Gresham FM, editors. Handbook of evidence-based practices for students having emotional and behavioral disorders. New York: Guilford; 2014. pp. 518–544. [Google Scholar]
  45. What Works Clearinghouse. Procedures and standards handbook (Version 2.1) Washington, DC: U.S. Department of Education, Institute of Education Sciences; 2011. Retrieved from http://ies.ed.gov/ncee/wwc. [Google Scholar]
  46. Woodbridge MW, Sumi WC, Yu J, Rouspil K, Javitz WS, Seeley J, Walker HM. Implementation and sustainability of an evidence-based program: Lessons learned from the PRISM applied to First Step to Success. Journal of Emotional and Behavioral Disorders. 2014;22:95–106. doi: 10.1177/1063426613520456. [DOI] [Google Scholar]

RESOURCES