Treatment Sequencing for Childhood ADHD: A Multiple-Randomization Study of Adaptive Medication and Behavioral Interventions

Abstract

Objective

Behavioral and pharmacological treatments for children with ADHD were evaluated to address whether endpoint outcomes are better depending on which treatment is initiated first, and, in case of insufficient response to initial treatment, whether increasing dose of initial treatment or adding the other treatment modality is superior.

Methods

Children with ADHD (ages 5–12, N = 146, 76% male) were treated for one school year. Children were randomized to initiate treatment with low doses of either (a) behavioral parent training (8 group sessions) and brief teacher consultation to establish a Daily Report Card or (b) extended-release methylphenidate (equivalent to .15 mg/kg/dose bid). After 8 weeks or at later monthly intervals as necessary, insufficient responders were rerandomized to secondary interventions that either increased the dose/intensity of the initial treatment or added the other treatment modality, with adaptive adjustments monthly as needed to these secondary treatments.

Results

The group beginning with behavioral treatment displayed significantly lower rates of observed classroom rule violations (the primary outcome) and parent/teacher ratings of oppositional behavior at study endpoint and tended to have fewer out-of-class disciplinary events. Further, adding medication secondary to initial behavior modification resulted in better outcomes on the primary outcomes and other measures than adding behavior modification to initial medication. Normalization rates on teacher and parent ratings were generally high. Parents who began treatment with behavioral parent training had substantially better attendance than those assigned to receive training following medication.

Conclusions

Beginning treatment with behavioral intervention produced better outcomes overall than beginning treatment with medication.

It is well established that evidence-based treatment for attention-deficit/hyperactivity disorder (ADHD) includes medication with psychostimulants (Conners, 2002; Greenhill, Pliszka, Dulcan, & the Work Group on Quality Issues, 2002) and behavioral interventions (Pelham & Fabiano, 2008; Evans, Owens & Bunford, 2014; Fabiano et al., 2009). These two modalities of treatment have been studied for decades, both separately and in combination. Even so, disagreements remain among professionals regarding which treatment modality is preferable, as well as how treatment for ADHD should begin. Some recommend beginning medication immediately and supplementing with additional medication when necessary (AACAP Work Group on Quality Issues, 2007). Others recommend beginning with psychosocial treatments and adding medication if those treatments are insufficient (APA Working Group on Psychoactive Medications for Children and Adolescents, 2006). Others recommend starting with both treatments simultaneously (http://www.chadd.org). Most recently, the American Academy of Pediatrics recommended each of the above strategies for different ages of children (Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management, 2011). However, the research base upon which these recommendations have been made is scant and limited in important ways (see for example Fabiano, Schatz, Aloe, Chacko, & Chronis-Tuscano, 2015).

In contrast to the hundreds of studies evaluating stimulants and behavioral interventions separately, only a handful of randomized controlled trials (RCT) have compared medication, behavioral treatment, and their combination, and each of these trials has limitations. A common feature in the existing studies is that they have used fixed doses—typically relatively high doses—of each treatment. For example, the largest and best-known RCT of comparative treatments for ADHD is the MTA (MTA Cooperative Group, 1999a), which used “optimal” dosing of medication (e.g., medication at school, evenings, and weekends) compared with a package of intensive behavioral treatments (parent training, summer treatment program, extensive teacher consultation, a classroom aide in school), and a combined condition that added the two high-dose treatments and began them simultaneously. The high-dose behavioral treatment was complex and costly, whereas the high-dose medication treatment had adverse effects on growth. The results of the MTA vary considerably based on the measure, individual differences, setting, timing of assessments, length of follow-up, and interpretation (e.g., MTA Cooperative Group, 1999a, 1999b, 2004; Molina et al., 2009; Pelham, 1999; Pelham et al., 2000; Owens et al., 2003; Swanson et al., 2007), suggesting that additional finely-tuned investigations with different doses and sequences of treatments are necessary to clarify relative effects of the two major, evidence-based treatment modalities.

More recent research has used both within-subject and RCT designs to evaluate multiple doses of medication in different combinations with varying doses of behavioral treatments (Fabiano et al. 2007; Pelham et al., 2005; Pelham et al., 2014; Pelham et al., under review). These studies have consistently found that intensive behavior modification produces acute effects similar to relatively high doses of medication, but that low doses of both treatments also maximize response in some but not all children. Further, these studies show that combining low-dose medication with low-intensity behavioral interventions produces equivalent effects to those of high-dose/high-intensity unimodal treatments for the majority of children but with lower side effects, high parental satisfaction, and less complex behavioral interventions. Side effects of stimulants increase with escalating dose and duration of exposure (Barkley et al. 1990; Pelham et al, 1999; Stein et al, 2003, Swanson et al., 2007). Therefore, adding behavioral interventions that reduce medication dose should improve the tolerability of medication treatments. These studies have provided much-needed information regarding the relative effects of different doses of medication and behavior modification. However, they were implemented in an analogue summer treatment program setting and thus do not directly address whether low doses of either modality or their combination would be sufficient for many children in community settings.

Moreover, no studies in the literature have systematically varied and compared the sequence in which the two evidence-based modalities for ADHD are implemented. Medication is the most commonly employed intervention and often the only intervention used in practice (Epstein, et al., 2014; Visser et al. 2014) even in young children where professional guidelines recommend starting with behavioral treatments (Subcommittee on Attention-Deficit Hyperactivity Disorder, 2011). Psychiatric guidelines endorse optimizing dose at home and school and using multiple medications prior to adding behavioral treatments (AACAP Work Group on Quality Issues, 2007). When medication is implemented at this high intensity level, there is less need for behavioral interventions, so the opportunity does not exist to discover whether some or most children would do well with behavioral interventions alone. For example, 75% of the individuals in the MTA behavioral treatment group remained without medication during the year of treatment, and, for the majority of those, for years afterward (MTA Cooperative Group, 1999a, 2004, Molina et al., 2009). This implies that many children might not need medication if behavioral treatments were employed first. Further, the majority of children in the medication management group needed additional treatment during the 14-month treatment period, but only medication could be used in this condition, and maintaining the initial medication effect required a 25% increase in dose during the year of treatment (Vitiello et al., 2001). In the combined treatment group, an adjustment to the classroom intervention—most often the Daily Report Card (DRC)—had to be made before medication dose could be increased, and that procedure reduced the need for increased doses of medication (Vitiello et al., 2001). The simultaneous introduction of conditions in the combined treatment group in the MTA means that it is not possible to evaluate whether a behavioral intervention employed before medication would have prevented the need for medication or reduced the dosage needed.

Thus, a significant limitation of existing ADHD treatment studies is that questions regarding sequencing, dosing, and combining treatments in natural settings have not been systematically explored. In contrast to this body of research, treatment decisions in practice are ongoing, based on the child’s impairment and response to intervention, and typically provided initially at low “doses” that are escalated only if necessary. There are two crucial decision points in treating a child with ADHD: (1) which treatment should be implemented first? and (2) what should be done if the child does not respond adequately to that initial treatment? For example, if a child begins treatment with medication and fails to respond, there are two possibilities – increase the medication dose or add behavioral treatment. These decision points have many implications with regard to tolerability/side effects, treatment cost, and treatment efficacy, yet no studies have systematically evaluated such treatment-sequencing questions for ADHD.

Adaptive treatment strategies have been gaining recognition as a strategy for preventive interventions and management of chronic disorders (Murphy, 2005; Collins, Murphy, & Bierman, 2004). In an adaptive approach, different dosages of treatment are provided differentially to individuals across time in response to decision rules that are based on individual characteristics. The major advantage of adaptive treatment designs is that they mimic what happens in typical practice where treatments are often modified or enhanced, but they retain controlled procedures, dosages, and rules to ensure replicability. Adaptive approaches have previously been used with comprehensive services of the type that are used for children with ADHD (for examples see Conduct Problems Prevention Research Group, 1999a, 1999b). Thus, in the present investigation, we employed a research design that has been recommended for developing and comparing adaptive strategies, a sequential multiple assignment randomized trial (SMART; Murphy, 2005; Lavori & Dawson, 2000). In such trials, individuals are randomized at multiple decision points to produce each treatment strategy, combinations of which can be analyzed because they have been assigned by randomization.

The current study was undertaken to address the limitations in the existing treatment literature for children with ADHD with regard to treatment decisions and sequencing. A SMART design was used to compare the results of various treatment decisions that included behavioral and/or pharmacological interventions and their combination that can be widely applied in clinical practice. Starting with low doses, treatments were conducted over an entire school year in children’s school and home settings and adapted monthly within setting based on response and need for additional intervention. End-of-study outcomes were measured on objective classroom observations of behavior and parent/teacher ratings to determine the relative benefits of the treatments and their sequences.

Within this design we were able to examine three important clinical questions/aims. First (Aim 1): does it produce better outcomes on endpoint objective classroom measures and parent/teacher ratings to initiate treatment with a low dose of (a) pharmacological intervention with a stimulant drug or (b) behavioral intervention (group parent training and a DRC at school)? Second (Aim 2), what is the most effective treatment protocol, or pattern of initial treatment and conditional secondary/adaptive treatment (e.g., BM: behavioral followed by medication in the event of insufficient response) among the four that we employed (BM, Behavioral-Behavioral (BB), Medication-Behavioral (MB), and Medication-Medication (MM)? Third (Aim 3), in the event of insufficient response to one of the initial treatments, are endpoint results improved more by increasing the dose of that modality (e.g., adding secondary/adaptive B to initial B (B then B) when necessary) or adding treatment with the other modality (e.g., adding secondary/adaptive M to initial B (B then M)?

Methods

Participants

One hundred, fifty-two children with ADHD, between the ages of 5 and 12, participated. Participants were recruited in three cohorts of approximately 50 each via radio advertisement; direct mail; and referrals from schools, physicians and mental health providers. Recruitment occurred during the spring and summer of 2006, 2007, and 2008, with treatment commencing in September of each year and continuing throughout the school year.

Exclusionary criteria included: (1) Full Scale IQ below 70; (2) history of seizures or other neurological problems and/or medication to prevent seizures; (3) history of other medical problems for which psychostimulant treatment may involve considerable risk; (4) childhood history or concurrent diagnosis of pervasive developmental disorder, schizophrenia or other psychotic disorders, sexual disorder, organic mental disorder, or eating disorder; (5) lack of functional impairment; and (6) placement in special education classrooms.

After screening and informed consent, parents and teachers completed a number of instruments to determine diagnosis and study eligibility. To determine ADHD diagnosis, parents and teachers completed the Disruptive Behavior Disorders (DBD) Rating Scale (Pelham, Gnagy, Greenslade, & Milich, 1992). The DBD RS is a list of the DSM symptoms of ADHD, oppositional-defiant disorder (ODD) and conduct disorder (CD), updated for DSM-IV, and rated as not at all, just a little, pretty much, or very much. In addition, parents completed a semi-structured DBD interview consisting of DSM-IV symptoms of ADHD, ODD, and CD with supplemental situational probes (available from the first author). Parents and teachers completed the Impairment Rating Scale (IRS: Fabiano et al., 2006), which asks parents and teachers to evaluate on a six point Likert scale the degree to which a child is like a typical child and needs no treatment or has extreme problems that definitely require treatment or special services in five areas of function—relationship with parents/teachers, relationships with peers/siblings, academic progress, general classroom/family functioning, and overall functioning. Two clinicians independently reviewed all screening instruments and made diagnoses based on the DSM-IV rules, counting a symptom as present when endorsed by either teacher or parent (pretty much or very much on the DBD or parent interview). Impairment also had to be present in any domain, as indicated by cutpoints on the Impairment Rating Scale (IRS; Fabiano et al., 2006). In case of disagreement, a third clinician reviewed the file to determine final diagnosis. Eighty percent of the children met criteria for ADHD-Combined Type, with 15% Predominately Inattentive and 5% Predominately Hyperactive/Impulsive. Comorbid rates of ODD and CD are shown in Table 1, along with demographic and descriptive information. None of the ADHD diagnoses and only 2% of the ODD and CD cases required a third reviewer to confirm diagnosis.

Table 1.

Sample Characteristics

Variable	Medication First	Behavioral First
Number of participants	74	72
Child Age in Years	8.3 (2.0)	8.5 (1.8)
Child Gender (% Male)	77%	75%
Child Race
White	76%	84%
Black/African American	17%	7%
Other	7%	8%
Child IQ	99.9 (16.2)	99.2 (12.5)
Other Diagnoses
Oppositional/Defiant Disorder	60%	54%
Conduct Disorder	17%	14%
ADHD Symptoms Endorsed
Inattention	7.6 (1.9)	8.1 (1.5)
Hyperactivity/Impulsivity	7.1 (2.2)	6.8 (2.1)
Parent Disruptive Behavior Disorders Rating
ADHD	1.89 (0.61)	1.99 (0.50)
ODD	1.32 (0.67)	1.29 (0.57)
CD	0.26 (0.28)	0.21 (0.20)
Teacher Disruptive Behavior Disorder Rating
ADHD	1.84 (0.62)	1.78 (0.60)
ODD	1.17 (0.84)	0.95 (0.73)
CD	0.45 (0.59)	0.31 (0.44)
Parental Marital Status (% Single Parent)	11%	7%
Highest Parental Education Level
High School Diploma or Less	10%	10%
Partial college or technical training	17%	14%
2-year degree	25%	19%
4-year degree	24%	31%
Graduate training	25%	26%
Previous Medication Treatment	27%	31%

Note. Groups did not differ significantly on any demographic measure.

The sample size was determined using data from our previous study in a controlled setting (Fabiano et al., 2007; Pelham et al., 2014) to estimate effect sizes for the first-stage treatments used in this study. These calculations determined that a sample size of 150 would result in at least 80% power for testing first-stage differences of at least 0.5 standard deviations when testing at a 0.05 level of significance. Recruitment and participant flow are illustrated in Figure 1. Six participants withdrew prior to the end of the study; 146 children completed the study assessments (96%). Three families withdrew because they did not wish to use medication and three withdrew because teachers refused participation upon initial contact after the family had been randomized. In the context of this multiple-randomization design, early withdrawal results in missing data on the group membership variable (i.e., responder versus non-responder). Although there are methods that can address this particular challenge (e.g., Shortreed, Laber, Scott-Stroup, Pineau, & Murphy, 2014), the subsequent analyses utilized only the 146 completers despite the use of multiple imputation to address missing data.

Figure 1.

Participant Flow

Design

Figure 2 illustrates the study design. Participants were randomly assigned to one of two initial treatments that were initiated at the beginning of the school year: low-dose medication for school hours only—Medication First (MedFirst) —or low-intensity clinical behavioral intervention consisting of weekly behavioral parent training groups (BPT) and a school consultation to establish a Daily Report Card (DRC: Jacob & Pelham, 2000; available at http://ccf.fiu.edu; Volpe & Fabiano, 2013)—Behavior First (BehFirst). Eight weeks of treatment were then provided to allow for sufficient time to implement behavioral treatments and medication and to measure their impact, after which each participant’s response was measured according to the procedures described below. If a child experienced continued impairment in the school and/or home setting—that is, insufficient response to the initial treatment—then a second randomization occurred. At this point, one of two treatment strategies was employed in the setting(s) where impairment was present: (1) increase the dose/intensity of the initial treatment or (2) add the other treatment for a combined treatment modality. Children who responded to the initial treatment condition were maintained on that condition and monitored monthly; if their performance deteriorated at any time during the school year, then the second treatment randomization occurred at that time. Children’s progress continued to be monitored and their secondary treatment condition was tailored adaptively (initial treatment was not tailored). For example, a child who began treatment with a 10-mg dose of medication and was re-randomized to receive behavioral treatment stayed on the initial 10-mg dose for the remainder of the school year, and subsequent changes were made only to the adaptive behavioral part of the treatment. Treatments, evaluations of response, and treatment adjustments were made independently for the home and school settings, which afforded independent evaluations of need for treatments, adherence, uptake, and effectiveness at home and at school.

Figure 2.

Study Design

Assessing need for additional treatment

Each month, parents and teachers completed ratings on a study-specific version of the IRS. The IRS was modified to ask whether, given the treatment currently in place, the child needed additional treatment, with responses ranging from 1 (definitely not) to 5 (definitely yes). If a rater responded probably yes or definitely yes in any domain, a study staff member called the rater to ask follow-up questions about the child’s impairment to ascertain whether the rating indicated true need for additional services, and to ensure that the impairment could be addressed with the available treatments (e.g., clinicians ruled out that a significant life event may have triggered a temporary increase in problem behavior or that comorbid learning problems may have accounted for impairment in academic progress).

As an objective measure of response to intervention in the school setting, teachers also kept records from an individualized target behavior evaluation (ITBE; Pelham, Fabiano & Massetti, 2005). The ITBE is sensitive to treatment effects, can be implemented by general education classroom teachers, and is individualized to children’s areas of impairment (Fabiano, Vujnovic, Naylor, Pariseau, & Robins, 2009; Fabiano et al., 2010). During the first few weeks of school, a study case manager met with each child’s teacher to establish target behaviors (e.g., work completion, complying with teacher directions, behavior toward peers) and criteria for what the teacher considered success on that target behavior. ITBE goal attainment percentages were computed across class periods each day, and weekly averages were calculated for evaluation. ITBE results were not shared with children or parents for children in the MedFirst group. For children in the BehFirst and adaptive behavioral groups, the ITBE doubled as a DRC and was sent home to parents, who provided contingent consequences at home.

At the 8-week point and monthly thereafter, the study team met to discuss each case. If monthly IRS ratings indicated impairment, the study team ensured that the impairment was related to an appropriate target of study treatments. Two clinicians who were not directly involved in the child’s treatment and were unaware of the initial treatment condition were required to agree that additional treatment was necessary based on the teacher or parent IRS before the child could be rerandomized. In the school setting, ITBE performance was simultaneously evaluated; if weekly averages consistently fell below 75%, and need for additional treatment was also indicated on the IRS, then additional treatment was considered. Finally, if a child was in immediate danger of class failure or school suspension, these factors were taken into account in treatment decisions.

For the children who were rerandomized, monthly treatment decisions were made regarding additional dose increases or adjustments to the behavioral treatment. These decisions were made using the same criteria as for initial response. Treatment recommendations were tailored to specific domains of impairment as described below. Parents were able to decline treatment recommendations for medication or additional behavioral services, but recommendations were reoffered monthly if indicated. All treatment recommendations, the reasons for them, and records of treatments received were documented.

Treatment Descriptions

Table 2 lists the components of the low and high dose medication and behavioral treatments. Children were initially randomized to a dose of behavioral or medication treatment, with additional treatment added, if indicated, based on a second randomization (see Figure 2).

Table 2.

Intervention Components

Modality	Initial Treatment	Secondary/Adaptive Treatment
Medication	8-hour stimulant equivalent to 0.15 mg/kg methylphenidate b.i.d.	Increased school dose Added evening/weekend doses
Behavioral Treatment	8 weekly sessions of group behavioral parent training (concurrent group social skills training for children) Monthly booster parent training sessions 3 consultation meetings with primary teacher to establish a school-home daily report card One individual parent training session to establish home rewards for daily report card	Group or individual classroom contingency management systems (Barrish, Sauders, & Wolf, 1969) Time-out in school Tutoring Organizational skills training (Schultz & Evans, 2015) School-based rewards Weekly Saturday social skills sessions (Pelham et al., 2008) Homework skills training (Power et al., 2001) Paraprofessional-implemented school rewards programs Home-based daily report card

Note. The adaptive components listed represent those offered or recommended as-needed based on individual areas of impairment. Not every child received every component of the adaptive treatment.

Initial Treatments

For children assigned to the BehFirst condition, parents received an 8-session, group parent training program using the Community Parent Education Program which has been extensively used with ADHD children (COPE; Cunningham, Bremner, & Secord-Gilbert, 1998); children participated in concurrent group social skills training sessions, modified after a recreational period in the Summer Treatment Program (STP; Pelham et al., 2010). Prior to the first group parenting session, parents received an individual session to establish a home reward system for the DRC. The case manager also conducted three brief consultation visits with the child’s teacher regarding standard classroom management strategies. This included an initial review of the teacher’s classroom management practices, discussion of basic classroom management, including praising appropriate behavior, planned ignoring, and appropriate commands, as well as procedures related to implementing a DRC. DRCs were sent home each day and parents provided daily and weekly rewards for good performance at school. Following the initial 8-week treatment period, monthly parent-training booster sessions with a focus on maintenance and problem-solving were offered for the remainder of the school year. The case manager also communicated with the teacher each month regarding adjustments to the DRC and the basic classroom management interventions that were in place.

For children assigned to the MedFirst condition, a dose equivalent to 0.15 mg/kg/dose b.i.d. of immediate release methylphenidate was calculated. In order to separate home and school settings for assessment and interventions, an 8-hour extended release preparation of methylphenidate (MPH) was used for the school setting only. For most children (92%), this was 10 mg per day of the extended release MPH preparation; for the remainder, their initial dose was 20 mg daily. School doses were administered by parents in the morning prior to school, and home meds were administered after school and on weekend mornings. The 0.15 mg/kg dose was selected based on data from controlled studies (Fabiano et al. 2007, Pelham et al., 2005; Pelham et al., 2014) showing significant effects over placebo that are similar to a low intensity behavioral intervention with very good tolerability. Side effects were monitored weekly for the first two weeks of medication administration and monthly thereafter; spontaneous reports of side effects were also collected. Any time a child experienced moderate or severe side effects, the study physician made dosing adjustments if necessary. The case manager also adjusted the ITBE for children in this group as needed monthly based on teacher report.

Secondary (Adaptive) Treatments

For children who began with behavioral treatment and were rerandomized to receive secondary/adaptive behavioral treatment (B-then-B), more intensive standard behavior management procedures were implemented first to address individualized areas of impairment. At school, these included school-based rewards for DRC performance, classwide reward contingency systems, intensive classroom-based contingency programs administered by the teacher or a paraprofessional, and time-out procedures. Home-based DRCs and individual parent-training sessions were introduced in the home setting. Other interventions were then added to address specific areas of child impairment (See Table 2).

For MedFirst children who began with medication and then were rerandomized to secondary/adaptive behavioral intervention (M-then-B), the standard initial behavioral treatments were implemented first (i.e., group parent training, DRC consultation). After eight weeks, the additional services described above were added according to the child’s continued impairments and need for tailored behavioral treatments.

For children who began with medication treatment and were rerandomized to receive secondary/adaptive medication (M-then-M), adjustments were made in two ways. First, the morning dose of the extended release MPH preparation given on school days could be increased if problems continued at school. Second, an after-school dose of immediate release MPH could be added to the child’s regimen if home behavior or homework completion were impaired (cf. Greenhill et al., 1996). In addition, MPH could be added for weekends. Parents also had the option of switching to a 12-hour formulation if the criteria for additional treatment were met in both settings.

For children who began with behavioral treatment and were rerandomized to receive secondary/adaptive medication (B-then-M), medication could be added as above for school, home, or both settings. Performance was evaluated monthly and adjustments were made taking into account impairment level and side effects.

Primary Dependent Measure

Classroom rule violations

As it is commonly regarded as the gold standard in assessments of treatment outcome for ADHD children in school settings, we used objective observation of student behavior in the classroom context as our primary dependent measure (Fabiano et al, 2009; Pelham, Fabiano, & Massetti, 2005). Every 4–6 weeks, independent observers visited the children’s classrooms and conducted 40-min. direct observations during academic tasks. Observers used the Student Behavior/Teacher Response code (available from first author), which includes observations of children’s rule-breaking behaviors (i.e., disrespect toward others, noncompliance with teacher requests, disrupting others, leaving seat without permission, inappropriate use of materials, speaking out without permission, and off-task behavior) and the teacher’s response to those behaviors (e.g., ignoring, providing a reprimand, providing a consequence; Vujnovic et al., 2014). Child behaviors were coded independently of teacher responses and were coded even if the teacher did not observe the behavior. Observers watched the entire class and coded behaviors exhibited by the target child and classmates. Classmates were observed anonymously and were not identified to the observer. The average number of behaviors exhibited by classmates was computed to produce a classroom comparison rate used as a covariate in analyses. The final observation of the school year was used in endpoint analyses because it corresponded with the time interval during which parent and teacher endpoint ratings were collected

In order to enhance reliability, observers were required to memorize operational definitions of behavior categories and completed a training session consisting of role-plays, practice observations, and classroom observations with an experienced observer. For 21% of the classroom observations, a second trained observer accompanied the primary observer and conducted an independent reliability observation of the same classroom. Reliability of the observations was high, with a correlation of 0.91 (p<0.01), and a mean difference of 2.3 (SD = 2.8; range = 0–17) for the total classroom rule violations. These figures are consistent with those from previous studies using the same observational code (e.g., Fabiano et al., 2010).

Secondary Dependent Measures

Number of out-of-class disciplinary events

Teachers kept daily records of out-of-class disciplinary events (e.g., being sent to the principal’s office). The number of reported events was summed over the length of the school year for subsequent analysis.

Parent and teacher ratings

At endpoint, parents and teachers completed the DBD RS and the Social Skills Rating System (SSRS; Gresham & Elliott, 1989). These measures have been widely used in studies of ADHD and have published psychometric information.

Tracking of Treatment Fidelity

Attendance records were kept for all treatment sessions, and clinicians recorded all meetings and contacts with teachers and parents. Medication dispensing records were kept. Parents returned all unused pills at each medication visit, and pill counts were conducted to determine the number of pills used.

To ensure fidelity with the behavioral treatments, all treatment components were manualized and procedural checklists were developed for all parent and teacher sessions. Clinicians met weekly with supervisors to review records of their sessions, and supervisors provided feedback as necessary. At each classroom observation, the observer recorded whether or not the teacher implemented the prescribed behavioral management procedures during the observation period.

Missing Data Handling

Missing values were minimal across all study variables and all participants. Outcomes ranged from 0 to 14% missing: classroom rules violations (98% complete), out-of-class disciplines (97%), teacher DBD rating (99%), teacher SSRS rating (99%), parent DBD rating (90%), parent SSRS rating (86%), and final medication doses (100%). At the participant level, 125 of 146 (86%) participants had complete data for all of the analyzed outcomes. We used multiple imputation to ensure unbiased estimates, assuming the data to be Missing at Random (MAR). Here MAR is a plausible assumption given the inclusion of a large number of covariates, including baseline measures of outcome variables; measures’ values at earlier waves are typically the best predictors of missing values at later waves in a longitudinal design.

Imputation

In order to accommodate the non-normal distributions of many relevant variables, we implemented a chained equations approach in R 3.1.3 (R Core Team, 2015) using the mice package (v2.22; van Buuren & Groothuis-Oudshoorn, 2011) extended by the countimp package for imputing count variables (v1.0; Kleinke & Reinecke, 2013). As methodologists recommend an inclusive strategy (Collins, Schafer, & Kam, 2001), the imputation model included approximately 50 variables: all the variables in the subsequent analyses, all the sample characteristics listed in Table 1, and baseline measures of outcomes wherever available. Distributions of all imputed variables were inspected and each was modeled using normal, predictive mean matching, negative binomial, logistic, and multinomial regressions, as indicated. Due to the large number of items and their high correlations (i.e., multiple items from the same measure), imputation occurred at the level of the scale rather than at the level of the item. One hundred imputed data sets were created, following recent recommendations that using larger number of imputations can minimize simulation error (White, Royston, & Wood, 2011).

Analysis and pooling

All subsequent analyses were conducted in SAS 9.3. Analysis was completed separately on each of the 100 imputations according to the procedures described in subsequent sections. SAS 9.3 PROC MIANALYZE was used to combine estimates across imputations; all reported estimates represent these combined (or pooled) estimates.

Analytic Plan

Our analyses largely parallel those described by Nahum-Shani and colleagues (2012); we direct readers to that article for more details about SMART design analyses. In the present study, the analysis of treatment outcome data included a series of comparisons to test different treatment decisions. Each comparison is described below.

Main effect of initial treatment assignment on endpoint outcomes (Aim 1)

End-of-treatment outcomes of those that started with medication (MedFirst group) and those that started with behavioral treatment (BehFirst group) were compared using regressions with group membership as a predictor in order to examine whether the initial treatment modality impacted outcome. In addition, survival analyses were conducted to determine whether the groups differed in the need for additional treatment and the length of time before children needed additional treatment.

Pairwise comparisons among SMART-embedded treatment protocols on endpoint outcomes (Aim 2)

Second, outcomes were compared across each of the four treatment protocols naturally embedded in the SMART design—BB, BM, MB, and MM. The first letter denotes that protocol’s initial treatment (first-stage treatment in Nahum-Shani et al, 2012) and the second letter denotes that protocol’s secondary/adaptive treatment (second stage treatment in Nahum-Shani et al, 2012), to be implemented in the event of insufficient response to the initial treatment. For example, the BM protocol entailed starting the participant with behavioral treatment and then adding medication if and only if there was insufficient response. It is important to note that the protocols do not reflect the actual treatment received, but rather the set of rules followed to assign treatment at both stages. For example, a child who responded to Behavior First and is therefore never rerandomized to receive secondary/adaptive treatment is included in analyses of the BM protocol even though he did not receive medication. This idea of being consistent with a particular embedded protocol is a subtle but important aspect of the SMART design that is discussed in detail elsewhere (Nahum-Shani et al., 2012). In the present analyses, we used an effects coding scheme and generalized estimating equations to achieve all the pairwise comparisons of protocols in a single model using SAS PROC GENMOD with robust standard errors, as described in the appendices of Nahum-Shani et al. (2012). We also gave weights of 2 to the responders to first-stage treatment and weights of 4 to the insufficient responders in order to account for the systematic undersampling of the latter in each protocol due to the second re-randomization (Nahum-Shani et al., 2012).

Comparison of endpoint outcomes for secondary/adaptive treatments given insufficient response to initial treatment (Aim 3)

Third, supplemental comparisons were performed within each of the initial treatment arms to determine whether it is better to augment (i.e., increase the dose of) that treatment or add the other treatment, given insufficient response to an initial intervention. Thus, responders to the initial treatments were excluded from these comparisons. These analyses consisted of regressions with group membership as a predictor that compared (1) B-then-B with B-then-M and (2) M-then-M with M-then-B.

Normalization rates

Finally, we used the procedure reported in Swanson et al. (2001) to evaluate normalization of functioning on teacher and parent ratings at the study endpoint. A score of 1.0 or lower on an aggregate of ADHD and ODD items from the DBD Rating Scale was used to define normalization. Teacher and parent reports were examined separately due to the separation of interventions across settings.

Count outcomes

Two dependent variables were counts: observed classroom rule violations and number of out-of-class disciplinary events. Count outcomes often violate the assumptions of linear (OLS) regression, so we adapted the SMART analysis procedure to incorporate negative binomial regression, a robust approach to modeling count outcome variables (Coxe, West, & Aiken, 2009). Negative binomial regression is related to the more well-known Poisson regression, but relaxes some assumptions of Poisson regression that are typically not met (i.e., equidispersion). As with the continuous outcomes, SAS PROC GENMOD was used for these analyses, with the addition of the negative binomial modeling and incorporating the OFFSET option to adjust for individual differences in the length of observation intervals. In the observed classroom rule violations analyses, average peer rule violations per hour was included as a covariate to control for the general level of disruptive behavior in each classroom.

Results

Need for Additional Treatment

In the school setting, 67% of the children who began treatment with behavioral interventions required additional treatment by the end of the school year compared with 47% of the children who began the school year receiving a low dose of medication (odds ratio or OR=2.23). Survival analyses indicated a significant group difference; Breslow χ²=7.4, p < .01.

In the home setting, there was no difference in rate of rerandomization for BehFirst (82%) and MedFirst (88%) groups; OR=0.63. Almost all children met criteria for additional treatment in the home setting regardless of initial treatment.

Endpoint Classroom Observations

Tables 3–6 display the results of analyses for classroom rules violations as well as subsequent outcomes. Comparisons of initial treatment strategy (BehFirst vs. MedFirst) revealed a significant difference on classroom rule violations, as illustrated in Figure 3. Children who began treatment with behavior management exhibited significantly fewer rule violations per hour than children who received MedFirst (incidence rate ratio or IRR=0.66, p<.01). Pairwise comparisons of the four treatment protocols revealed several significant differences (Figure 3). The BB protocol resulted in fewer rule violations than the BM protocol (IRR=0.78, p=.054), the MM protocol (IRR=0.56, p<.01), and the MB protocol (IRR=0.50, p<.001). In addition, the BM protocol resulted in fewer rules violations than the MB protocol (IRR=0.65, p<.01). For insufficient responders to first-stage behavioral treatment, increasing the dose with second-stage behavioral treatment resulted in significantly fewer violations than did adding medication (IRR=0.71, p<.05). For insufficient responders to first-stage medication treatment, there were no significant differences between second-stage treatments.

Table 3.

Outcomes at Endpoint by Initial Treatment Assignment

Outcome	Medication First	Behavioral First	Effect Size
Classroom rules violations per hour**	12.6 [10.5, 15.3]	8.4 [6.8, 10.3]	IRR = 0.66
Out-of-class disciplinary events per school year†	3.1 [1.8, 5.2]	1.6 [0.9, 2.7]	IRR = 0.52
Teacher DBD—ADHD	0.98 (.67)	1.00 (.64)	d = −0.02
Teacher DBD—ODD	0.59 (.66)	0.45 (.51)	d = 0.24
Teacher SSRS Social Skills Total Score	33.9 (9.5)	36.0 (10.5)	d = 0.21
Parent DBD—ADHD	1.44 (.64)	1.45 (.62)	d = −0.01
Parent DBD—ODD	1.09 (.71)	0.98 (.65)	d = 0.16
Parent SSRS Social Skills Total Score	45.2 (10.8)	45.3 (10.7)	d = 0.01

Note. IRR=incidence rate ratio, DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). The IRR is the ratio of the event (e.g., rule violation) incidence rate in one group (here, Behavioral First) to the incidence rate in another group (here, Medication First). The other effect sizes are Cohen’s D with pooled standard deviation (equations 2.5.1 and 2.5.2, pp. 66–67, Cohen, 1988), and are listed such that a positive d reflects an advantage of Behavioral First.

^†p<0.10,

^**p<0.01.

Table 6.

Outcomes at Endpoint by Secondary/adaptive Treatment Given Insufficient Response to Initial Medication Treatment

Outcome	M-then-M	M-then-B	Effect Size
Classroom rule violations per hour	14.5 [9.5, 22.1]	17.1 [10.9, 26.9]	IRR = 1.18
Out-of-class disciplinary events per school year†	2.2 [0.8, 6.6]	8.2 [3.5, 19.6]	IRR = 3.66
Teacher DBD—ADHD	1.21 (.63)	1.43 (.71)	d = −0.34
Teacher DBD—ODD†	0.70 (.52)	1.15 (.91)	d = −0.61
Teacher SSRS Social Skills Total Score	32.2 (6.2)	28.8 (11.0)	d = −0.39
Parent DBD—ADHD	1.38 (.60)	1.62 (.63)	d = −0.38
Parent DBD—ODD†	1.02 (.65)	1.33 (.73)	d = −0.46
Parent SSRS Social Skill Total Score	44.5 (11.2)	44.0 (9.6)	d = −0.05

Note. M-then-M=began with medication treatment and then received higher dose medication treatment, M-then-B=began with medication treatment and then added behavioral treatment, IRR=incidence rate ratio, DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). The IRR is the ratio of the event (e.g., rule violation) incidence rate in one group (here, M-then-B) to the incidence rate in another group (here, M-then-M). The other effect sizes are Cohen’s D with pooled standard deviation (equations 2.5.1 and 2.5.2, pp. 66–67, Cohen, 1988), and are listed such that a positive d reflects an advantage of M-then-B.

^†p<0.10.

Figure 3.

Means on Observed Classroom Rules Violations and Out-of-Class Disciplinary Events as a Function of Treatment Decisions

Out-of-Class Disciplinary Events

Comparisons of initial treatment strategy revealed a trend wherein the BehFirst group displayed fewer out-of-class disciplinary events than the MedFirst group (IRR=0.52, p<.10, Figure 3). Pairwise comparisons of the four treatment protocols indicated that (Figure 3): the BM protocol resulted in significantly fewer events than the MB protocol (IRR= 0.16, p<.001) and the BB protocol (IRR=0.34, p<.05), and the MM protocol resulted in significantly fewer events than the MB protocol (IRR=0.34, p<.10). For insufficient responders to initial behavioral treatment, adding medication trended toward resulting in significantly fewer events than increasing the dose of behavioral treatment (IRR=0.30, p<.10). For insufficient responders to first-stage medication treatment, increasing the dose with medication treatment trended toward resulting in fewer events than did adding behavioral intervention (IRR=0.27, p<.10).

Teacher Ratings

On teacher DBD ratings, no significant differences emerged for ADHD symptoms. For ratings of oppositional/defiant behavior, the pairwise comparisons of the four treatment protocols indicated a near significant advantage of the BM protocol over the MB protocol (d=0.40, p=.06). The supplemental comparisons indicated that for insufficient responders to first-stage medication, increasing the dose with second-stage medication trended toward resulting in lower ratings of oppositional/defiant behavior than did adding behavioral (d=0.61, p<.10).

For Total Social Skills score of the teacher SSRS, there was a trend toward advantage of the BM protocol over the MB protocol (d=0.35, p<.10). Other comparisons were nonsignificant.

With regard to normalization of combined ADHD/ODD teacher ratings at endpoint, similar numbers of the children assigned to MedFirst or BehFirst had mean DBD ratings of 1.0 or less—69% and 78% respectively. Eighty-four percent of those who responded to first-stage medication treatment met the normalization criterion, as did 92% of those who responded to first-stage behavioral treatment. For those needing additional treatment, 63% of the M-then-M group was normalized, compared to 61% of the B-then-B group, and 38% of the M-then-B group compared to 80% of the B-then-M group.

Parent Ratings

As with teachers, there were no significant differences on ADHD ratings in any comparisons. For ratings of oppositional/defiant behavior, pairwise comparisons of the four treatment protocols revealed a significant advantage of the BM protocol over the MB protocol (d=0.56, p<.05) and BB protocol (d=0.38, p<.10), as well as a trend advantage of the MM protocol over the MB protocol (d=0.40, p<.10). The supplemental comparisons indicated that for insufficient responders to first-stage behavioral, adding second-stage medication trended toward resulting in lower ratings of oppositional/defiant behavior than did increasing the intensity of behavioral treatment (d=0.45, p<.10). Likewise, for insufficient responders to first-stage medication, increasing the dose with second-stage medication trended toward resulting in lower ratings of oppositional/defiant behavior than did adding behavioral (d=0.46, p<.10). There were no significant differences in any comparisons of the Total Social Skills score of the parent SSRS.

With regard to normalization of ADHD/ODD parent ratings at endpoint, 31% of the MedFirst and 39% of the BehFirst, groups met criteria for normalization. Two-thirds of those who responded to first-stage medication treatment met the normalization criterion, as did 54% of those who responded to first-stage behavioral treatment. For those needing additional treatment, 34% of the M-then-M group was normalized, compared to 30% of the B-then-B group, and 18% of the M-then-B group compared to 40% of the B-then-M group.

Treatment Received

For those who began with BehFirst, 3% of the families declined parent training. Remaining parents attended an average of 6 of the 8 group sessions (median=7, mode=8), 69% attended an adequate dose of parent training (cf. MTA Cooperative Group, 1999), and 31% attended at least one booster session after the initial parent training (see Figure 4). All scheduled teacher meetings were completed, and DRCs were established for all but one child in the BehFirst condition. For children in B-then-M, 13 parents (21%) declined the initiation of medication.

Figure 4.

Parent Training Attendance by Initial Treatment Assignment

Note. Figures for the Medication First families consider only those that were rerandomized to behavioral treatment (M-then-B, N=35).

At the initial 8-week assessment point, 9% of the MedFirst families had declined medication. Of those who accepted medication, they were medicated on 97% of their school days. For those in MedFirst rerandomized to the M-then-B group, 60% of these parents did not attend any of the assigned group parent training sessions (mean=1.9, median=0, mode=0); only 11% of these families received an adequate dose of parent training, and only 11% attended at least one booster session (Figure 4). At school, all required teacher meetings were completed and school DRCs were established.

In the adaptive behavioral treatment arms (either M-then-B or B-then-B), 11 children attended Saturday Treatment Program sessions, 3 received extra academic tutoring, and 13 received additional intensive interventions at school (e.g., the good behavior game initiated as a classwide intervention) administered by the teacher or a paraprofessional as part of the adaptive behavioral treatment condition. The remainder received either additional standard teacher consultations to establish higher-intensity teacher-delivered consequences such as school-based rewards and class-wide contingencies or individual parent sessions to improve parenting skills or establish more intensive parent-delivered interventions at home. Eleven families assigned to behavioral treatment took medication outside of the protocol. Five of these families used the medication for only 1–2 months before stopping the medication.

To determine whether beginning treatment with behavioral intervention would decrease the dose of medication required for a child treated with medication in school, school-day dosing in mg/kg/dose equivalent was compared for the two groups involving adaptive medication: B-then-M and M-then-M at endpoint. At the end of the school year, 24% of the B-then-M group was unmedicated at school compared with 5% of the M-then-M group (that is, parents either did not start or elected to stop medication). Of those who were medicated, children in B-then-M were taking significantly lower doses at school (M=0.21 mg/kg/dose, SD=0.10) than M-then-M (M=0.28, SD=0.14), F(1, 70)=4.26, p<.05. At home, 39% of the M-then-M group and 35% of the B-then-M group were unmedicated (parents either did not start or elected to stop medication). For those who were medicated at home, doses were not significantly different: (B-then-M: M=0.22, SD=0.10; M-then-M: M=0.21, SD=0.12).

Discussion

This study addressed three key questions: first (Aim 1), does it produce better outcomes on objective classroom measures and teacher and parent ratings to begin treatment with a low dose of (a) medication treatment or (b) behavioral treatment? Second (Aim 2), what is the most effective treatment protocol, or pattern of first-stage treatment and conditional second-stage treatment among the four imbedded SMART treatment protocols? Third (Aim 3), in the event of insufficient response to a specific initial treatment, is it more effective to increase the dose of that modality or add treatment with the other modality? All groups were functioning relatively well at endpoint, as was expected given that two effective treatments were compared. However, there were important differences in outcomes, as a result of the initial treatment assignment and the protocol followed. Our findings provide the following answers to the three questions/Aims noted above as follows:

1. Beginning treatment with a low dose of behavior modification resulted in significantly lower rates of observed classroom rule violations and a trend for out-of-class disciplinary events relative to beginning with a low dose of medication.

2. The best of the four specific treatment protocols began with behavioral treatment and then added medication in the event of insufficient response (BM). The worst protocol began treatment with medication and added behavioral treatment in the event of insufficient response (MB). The BB and MM protocols produced outcomes in between these two and were often comparable, though BB was superior to MB and MM on the primary outcome variable.

3. In the event of insufficient response to initial behavioral treatment, increasing the intensity of behavioral treatment (B-then-B) was significantly superior on the primary outcome (classroom rule violations); adding medication (B-then-M) had nominal advantages on several other outcome variables, two of which were trends. In the event of insufficient response to medication, increasing dose of medication (M-then-M) was nominally superior to adding behavior modification (M-then-B) on every measure with small to moderate effect sizes, three of which were trends.

These results have clear implications for treatment for children with ADHD in mental health, primary care, and school settings, and we discuss each in turn.

With regard to our first question, beginning school-based treatment with a low dose of behavior modification (eight sessions of group parent training plus establishing a DRC at school with home rewards) resulted in functioning in the school setting on key outcome measures that was comparable to or better than beginning school-based treatment with a low dose of stimulant medication. Notably, the low dose of behavior modification was a superior starting strategy on the primary outcome measure, direct observations of classroom behavior (66% as many rule violations), as well as the frequency of out-of-class discipline (54% as many incidents). Although teacher ratings did not differentiate BehFirst from MedFirst, ratings of oppositional behavior decreased by more than 50% from baseline in the BehFirst group, and teachers rated 78% of children in the BehFirst group as normalized. For 33% of the BehFirst children, the low dose of behavioral intervention was sufficient treatment in school for the entire school year.

There were no differences between the groups in the numbers of children who needed additional treatment at home, with more than 80% of both groups meeting criteria for rerandomization (see discussion below). Similarly, there were no significant differences between initial treatments on parent ratings of symptoms, oppositional behavior, or social skills.

Interestingly, compared to the 33% who did not need additional school-based treatment in BehFirst, nearly two-thirds more, 53% of children in the MedFirst initial assignment did sufficiently well with the low dose of medication that they did not require additional treatment. Further, a substantial portion of the children assigned to the BM embedded protocol (24%) were not taking medication at endpoint—far more than the MB protocol. In other words, although the combined treatment protocol (MB) within the MedFirst arm contained more medicated children than did the combined treatment protocol (BM) within the BehFirst arm, and although there were more initial responders in the MedFirst group than in the BehFirst arm, MedFirst remained inferior to BehFirst as an initial treatment condition.

These differences in the effects of the initial intervention may be linked to differences in treatment uptake of parent training, as shown in Figure 4. Engagement in the parent training groups and the booster sessions was dramatically reduced in the MedFirst families relative to the BehFirst families. Indeed, most BehFirst parents attended the majority of BPT sessions and received an adequate “dose” of BPT, while only a small minority of MedFirst families who were assigned to BPT as a secondary/adaptive intervention attended BPT. In other words, the provision of medication before the initiation of parent training was associated with greatly reduced rates of engagement in parent training, and presumably worse functioning at school and home. This finding parallels those reported in the STAR*D study of antidepressants for adults with major depression: 71% of adults that were insufficient responders to SSRI treatment did not choose to pursue subsequent cognitive behavior therapy (Wisniewski et al., 2007). Perhaps parents who began with behavioral treatment were more motivated to engage because they had not already dealt with several of weeks of problem behavior at school without having received the parenting toolkit provided in BPT. Alternatively, perhaps parents who began with medication, which requires minimal effort and time, were reluctant to participate in more effortful and time-consuming parent training, a major portion of which is learning to provide home backup for the school DRC. It is possible that the teachers were less engaged in second-stage behavioral interventions following initial medication for the same reasons as parents. Other researchers have reported difficulties with engagement and attendance in behavioral treatments for ADHD (see for example Barkley et al., 2000). Additional research is necessary to elucidate the mechanisms of this problem with treatment engagement. Whatever the mechanism, the clinical implications are quite clear: if providers intend for the parents of ADHD children to receive parent training and for teachers to provide “extra” classroom management for the child (i.e., for the child to receive multimodal treatment) but start treatment with medication, they reduce the likelihood of engagement in behavioral treatment and thus negatively impact treatment outcome. Unfortunately, standard practice among physicians is to provide medication immediately rather than delay it until the completion of parent training. The results of the present study suggest this is a poor strategy.

With respect to our second question/Aim—which of the four embedded treatment protocols produces the best outcomes?—it should be noted that each initial assignment is associated with two embedded protocols, specifically those that began with that particular modality (i.e., for BehFirst, the BB and BM protocols). With this in mind, the comparisons of treatment protocols suggest that the primary driver of the first-stage treatment main effect was the discrepancy between the two combined protocols, BM and MB. As Table 4 shows, the two protocols involving increasing the initial intervention (BB and MM) produced generally comparable results (though BB was superior to MM on classroom observations), but the BM combined-treatment protocol was significantly more effective than the MB protocol. The former produced the best outcomes on all but two variables, while the latter produced the worst outcomes on all but two variables. Children following a combined treatment protocol that began with behavioral treatment were superior on measures of classroom observations, disciplinary actions, and teacher and parent ratings of ODD. Not surprisingly, the normalization rates on teacher ratings in the school setting in the children receiving multimodal treatment was double for the B-then-M (80%) versus M-then-B (38%) groups, whereas normalization rates for the B-then-B and M-then-M protocols were nearly identical (61% and 63%). This finding illustrating the superiority of BM over MB has substantial clinical implications, shedding light on how sequencing of intervention can enhance (or inhibit) engagement within an evidence-based intervention. For example, the results of the present study suggest that the failure to begin behavioral treatment before medication may have contributed to the relatively small advantage of combined treatment to medication alone in the MTA study.

Table 4.

Outcomes at Endpoint by Treatment Protocol Followed

Outcome	BB protocol	BM protocol	MB protocol	MM protocol
Classroom rules violations per hour	7.2† [5.8, 9.0]	9.3a† [7.6, 11.4]	14.3b [11.1, 18.5]	12.7ab [9.0, 18.0]
Out-of-class disciplinary events per school year	2.6ab [1.1, 6.1]	0.9c [0.5, 1.7]	5.5a† [2.4, 12.9]	1.9bc† [0.9, 4.2]
Teacher DBD— ADHD	1.09 (.65)a	0.91 (.61)a	1.02 (.71)a	0.94 (.63)a
Teacher DBD— ODD	0.48 (.55)ab	0.42 (.46)a†	0.69 (.79)b†	0.50 (.50)ab
Teacher SSRS Social Skills Total Score	35.0 (10.8)ab	36.8 (10.0)a†	33.2 (10.7)b†	34.5 (8.2)ab
Parent DBD— ADHD	1.52 (.63)a	1.37 (.59)a	1.54 (.65)a	1.34 (.60)a
Parent DBD—ODD	1.10 (.69)ab†	0.86 (.58)c†	1.23 (.74)a‡	0.95 (.64)bc‡
Parent SSRS Social Skills Total Score	44.2 (10.0)a	46.4 (11.2)a	45.0 (10.1)a	45.4 (11.3)a

Note. DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. The first letter of each protocol indicates its first-stage treatment and the second letter indicates its second-stage treatment, to be implemented in the event of insufficient response (‘B’ for behavioral, ‘M’ for medication). Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). They were calculated using the weighting method as described in Nahum-Shani et al. (2012). Within each row, means that have no superscript in common are significantly different from each other, p<.05. Cross or doublecross next to a pair of means indicates difference was only marginal, p<.10.

With respect to our third question/Aim, how to best augment treatment given insufficient response to initial treatment, Table 5 shows that a low dose of medication is a useful adjunctive intervention to add to initial behavioral treatment in the case of insufficient response, as is increasing the intensity of behavioral intervention. Both additions were helpful on a range of measures. In contrast, given insufficient response to medication, increasing the dose of medication was superior across measures to adding behavioral treatment (Table 6). This finding has important clinical implications. Once medication has been employed, it appears that only a higher medication dose results in continued improvement when more treatment is needed. As discussed above, the reasons for this may be related to failure of parents (or teachers) to engage in behavioral treatments when they follow medication. It should be noted that these comparisons involved small Ns, and power to detect differences was limited. Further, a treatment regimen that includes only medication is not a viable long-term treatment strategy for ADHD children, as it confers no long-term benefit (Molina et al., 2009).

Table 5.

Outcomes at Endpoint by Secondary/adaptive Treatment Given Insufficient Response to Initial Behavioral Treatment

Outcome	B-then-B	B-then-M	Effect Size
Classroom rule violations per hour*	6.6 [5.1, 8.6]	9.4 [7.5, 11.7]	IRR = 1.41
Out-of-class disciplinary events per school year†	3.2 [1.2, 8.3]	1.0 [0.4, 2.7]	IRR = 0.30
Teacher DBD—ADHD	1.28 (.65)	1.00 (.65)	d = 0.44
Teacher DBD—ODD	0.63 (.60)	0.52 (.49)	d = 0.19
Teacher SSRS Social Skills Total Score	32.0 (9.6)	35.0 (9.1)	d = 0.31
Parent DBD—ADHD	1.60 (.66)	1.43 (.63)	d = 0.26
Parent DBD—ODD†	1.20 (.69)	0.90 (.59)	d = 0.45
Parent SSRS Social Skills Total Score	41.8 (9.1)	44.4 (11.2)	d = 0.26

Note. B-then-B=began with behavioral treatment and then received higher dose behavioral treatment, B-then-M=began with behavioral treatment then added medication treatment, IRR=incidence rate ratio, DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). The IRR is the ratio of the event (e.g., rule violation) incidence rate in one group (here, B-then-M) to the incidence rate in another group (here, B-then-B). The other effect sizes are Cohen’s D with pooled standard deviation (equations 2.5.1 and 2.5.2, pp. 66–67, Cohen, 1988), and are listed such that a positive d reflects an advantage of B-then-M.

^†p<0.10,

^*p<0.05.

Furthermore, with regard to the secondary treatments involving adaptive medication, children in the B-then-M group were taking significantly lower doses of medication at school than children in the M-then-M condition. This finding indicates that beginning treatment with behavior modification serves to decrease the necessary dose when medication is used, which will result in lower levels of dose-related side effects (cf. Swanson et al, 2006).

Taken altogether, our results replicate and extend in the school-year environment what we have reported in earlier studies conducted in analogue summer program settings (Fabiano et al., 2007; Pelham et al., 2005; Pelham et al., 2014). Namely, a low dose of behavioral treatment—in this study eight sessions of large group BPT and establishing a DRC at school— is effective and sufficient for a substantial number of children with ADHD in school, recreational, and home settings. Further, a low dose of medication (.15 mg/kg/dose b.i.d.) was sufficient in the school setting for 53% of the children. Low doses of the two modalities in combination were very effective for insufficient responders, but only when the behavioral treatment came first. Neither our previous studies nor the MTA sequenced interventions, but these results provide clear guidance about which sequence should be followed when implementing combined treatment—BM rather than MB.

This is the first study to our knowledge that has addressed the effectiveness of such low-dose interventions as a starting treatment for ADHD implemented in a community/school/clinic setting. Low dose medication was sufficient in the school setting for a year for nearly half of the children, but providing it first limited the effectiveness of additional behavioral treatment when necessary. Further, the cost of the MedFirst condition and its protocols in the study were far higher than BehFirst and its associated protocols (Page et al, this issue). Thus, the study demonstrates that starting with a low dose of behavioral treatment and either enhancing behavioral treatment or adding medication when necessary produces better outcomes and is a far less costly approach to treatment for ADHD than starting with medication (Page et al, this issue). Others have found increased side effects and reduced tolerability as the dose and duration of medication increases (Barkley, McMurray, Edelbrock, & Robbins., 1990; Stein et al., 2003; Pelham, et al, 1999; Swanson et al. 2006). An adaptive approach in clinical practice that begins with low intensity behavior modification and increases intensity or adds medication adaptively would appear to be the treatment approach of choice for children with ADHD. The MTA had previously shown that medication dose escalations over time are less necessary when multimodal treatment is being implemented compared to medication alone (Vitiello et al., 2001), but the present results extend that finding to considerably lower and less costly (Page et al., this issue) doses of both medication and behavioral treatment than employed in the MTA and most other studies in the ADHD treatment field.

Importantly, the adaptive nature of the approach to behavioral intervention was effective in this study in producing very positive outcomes with relatively low intensity interventions for most children and enhanced interventions for a small subset. For example, the adaptive behavioral treatments employed in the school setting were carried out by the general education teacher without additional intervention staff in two-thirds of the cases, and in only six cases was an intensive paraprofessional-based program implemented. In contrast, in the MTA a half-day paraprofessional—a very costly intervention—was provided for nearly a full semester for all participants regardless of need. The present results suggest that that was unnecessary for the vast majority of the children. These findings illustrate the utility of the adaptive treatment approach, in which children only receive the types and levels of treatment they require based on individual impairment. As discussed in the companion to this paper (Page et al., this issue), the BB protocol was the least costly of the four protocols and the BM protocol a close second. Thus, our effectiveness results and costs show a far different picture than presented in the only other comparative study of cost-effectiveness in the ADHD literature (cf. Jensen et al, 2005). These findings have important implications for the public health system and insurance companies with regard to treatment costs for ADHD and call for a reassessment of federal, insurer, and medical society recommendations on treatments for ADHD, which currently prioritize medication and limit the extent to which behavioral treatments can be utilized.

Limitations

It is important to note that this was an effectiveness study carried out in the natural environment and therefore strict experimental control over the behavioral interventions could not be exerted. Given the prevalence of classroom management training in schools, teachers in the medication–only group were no doubt routinely implementing behavioral strategies to manage their classrooms, and parents in the medication-only group may have been implementing behavioral practices such as time out. Some of the lack of differences in behavioral treatments may thus be due to the natural presence of behavioral treatments in school and home settings. In addition, it was not possible to collect measures of parents’ in-home implementation of procedures such as rewarding the Daily Report Card. Furthermore, although observers completed checks of treatment integrity and fidelity during their observations, they were often unable to observe teachers’ implementation of specific procedures that were to be implemented (e.g., tracking DRC targets and giving feedback to the child). We therefore were unable to calculate specific data for the fidelity of teacher-delivered interventions. In contrast, medication was provided with greater experimental control, with dosing practices varying some from what is done in routine clinical practice in order to systematically assess sequencing effects across settings. For example, initial medication treatments focused on school only, with evidence of objective impairment required to be eligible for additional treatment at school or at home.

As noted above, many more children met criteria for additional treatment in the home setting than in the school setting. In part this may be due to the initial treatment conditions. For example, medication was initially provided only at school for MedFirst.. This approach exposed those families to the impact of medication on their child at school and may have led parents to rate their child as needing medication at home in order to obtain medication for the home setting. The BPT program for BehFirst families was a brief, group-based program--as opposed to the individually developed DRC for each child--and was sufficient for only a subset of families. Others needed more individually-focused BPT, although the amount required was relatively little except for a small subset of families. The lack of a parallel, home-based DRC criterion (like the one used in the school setting) for additional treatment also made it easier for children to meet criteria for additional treatment at home relative to the criteria for allocating additional treatment at school. That is, parents simply needed to indicate that their child was having problems and needed more treatment, whereas teacher indication of need and ITBE target goal attainment rate below 75% were required at school. This may also explain the somewhat contrary findings that BehFirst resulted in superior outcomes relative to MedFirst, but the majority of children in the study met criteria for rerandomization in the home setting. Future studies that utilize similar ITBE goal attainment strategies in the home in addition to parent ratings may yield different outcomes that are more similar to that obtained in the school setting in this study.

Another limitation relates to our study design, which included a maximum of two randomizations per child. Ideally additional decision points might be included. For example, in our protocol a child in need of adaptive second stage behavioral treatment could receive ad lib treatment, as opposed to systematically limited, incrementally larger “doses” of behavioral intervention, which might have been sufficient. Alternatively, for a child who is still doing poorly after the first rerandomization, another opportunity to cross over to the other treatment might be considered. For example, rather than a temporary classroom paraprofessional, medication might have been considered at a third randomization for the small number of children who require that level of assistance. This may be particularly true in the case of nonadherence to the assigned treatments. The sample size required for additional decision points precluded such considerations for the current study. Inclusion criteria also required attendance within general education classrooms, so whether these results generalize to self-contained special education classrooms is not known.

A final limitation was statistical power to address secondary aims. The study was fully powered only for examination of the main effect of first-stage treatment (question/Aim 1). We did not expect so many children to respond to first stage treatments and not need additional treatment, so we did not plan for a larger N. Thus our power was reduced for pairwise comparisons of the embedded treatment protocols (question/Aim 2), and then further reduced for the comparison of treatments among insufficient responders (question/Aim 3). Thus, numerous small to moderate effect sizes did not achieve statistical significance but might have with a larger sample.

Future Research

Finally, the adaptive methodology employed is promising for future studies of interventions for ADHD in the pursuit of treatment tailoring for individual differences in functional deficits. Replication of these results with a larger sample would afford better power and the opportunity to investigate mediators and moderators for the treatment results that we reported herein, including individual differences in comorbidity and impairment. For example, why did parents whose children received medication first have such dramatically reduced rates of uptake of parent training and associated poor outcomes relative to the other protocols? Why did so many more children meet criteria for rerandomization at home compared to school? Since combined treatment starting with behavior modification was so effective, a natural follow-up question is how psychologists and other psychosocial mental health and school-based providers can collaborate with M.D. prescribers in practice settings to implement the conjoint strategies that were shown in this paper to be effective. Finally, how might these interventions and approaches have to change to be effective with samples of ADHD children both younger and older than our elementary-aged sample?

Clinical Implications

The results have direct relevance for clinical practice. The relatively low-dose-treatment strategies that we employed are implementable in community mental health, primary care, and school settings. The results suggest that practitioners should initiate treatment with low doses of intervention, increasing intensity only when indicated. Many children will respond sufficiently to low doses of initial treatments. Further, practitioners who initiate treatment with behavioral intervention (group parent training and a school DRC) will produce better outcomes for their patients than those starting with medication. In the face of inadequate response to initial behavioral treatment, either more intensive behavioral intervention or the addition of a low dose of medication for school hours will produce improved patient and family outcomes. In contrast, if medication is the first stage treatment and is insufficient, adding behavioral treatment is not an effective treatment option--outcomes are worse than other strategies and parent engagement in subsequent parent training is very poor. Physicians in particular need to be aware of these facts—simply advising their patients who have started medication to go to a psychologist for parent training will not result in the desired outcome—that is, a multimodal intervention. This paper and our companion paper on the costs of these interventions strongly suggest a reconsideration of the current practice of relying on medication as first line and typically sole treatment for many ADHD children. These results suggest that a stepwise approach to treatment, starting with low doses of behavioral treatment and increasing in intensity or adding medication only if necessary, would be a cost-effective public health strategy for treatment of childhood ADHD in school and community settings.