Remediating Organizational Functioning in Children with ADHD: Immediate and Long-Term Effects from a Randomized Controlled Trial

Abstract

Objective

The study compared the efficacy of two behavioral interventions to ameliorate organization, time management and planning (OTMP) difficulties in 3^rd–5^th grade children with Attention-Deficit/Hyperactivity Disorder (ADHD).

Method

In a dual-site randomized controlled trial, 158 children were assigned to Organizational Skills Training (OST, N=64); “PATHKO”, a performance-based intervention that precluded skills training (N=61); or a wait-list control (WL, N=33). Treatments were 20 individual clinic-based sessions over 10–12 weeks. OST involved skills building provided primarily to the child. PATHKO trained parents and teachers to reinforce children contingently for meeting end-point target goals. Primary outcomes were the Children’s Organizational Skills Scales (COSS-Parent, COSS-Teacher). Other relevant functional outcomes were assessed. Percentage of participants no longer meeting inclusion criteria for OTMP impairments informed on clinical significance. Assessments occurred at post-treatment, 1-month post-treatment, and twice in the following school year.

Results

OST was superior to WL on the COSS-P (Cohen’s d = 2.77; p< .0001), COSS-T (d = 1.18; p< .0001), children’s COSS self-ratings, academic performance and proficiency, homework, and family functioning. OST was significantly better than PATHKO only on the COSS-P (d = .63; p< .005). PATHKO was superior to WL on most outcomes, but not in academic proficiency. 60% of OST and PATHKO participants no longer met OTMP inclusion criteria vs. 3% of controls. Significant maintenance effects were found for both treatments.

Conclusions

Two distinct treatments targeting OTMP problems in children with ADHD generated robust, sustained functional improvements. The interventions show promise of clinical utility in children with ADHD and organizational deficits.

Organization, time management and planning (OTMP) difficulties, features of Attention-Deficit/Hyperactivity Disorder (ADHD), adversely affect children’s functioning and persist through adulthood (Barkley & Fischer, 2011). During childhood, poor organizational skills, manifested by misplacing or losing materials, forgetting materials, failing to record assignments and due dates, can compromise school performance and scholastic attainment (Power, Werba, Watkins, Angelucci, & Eiraldi, 2006). Impairments in time management and inefficient planning lead to difficulties starting and completing daily and long-term assignments (Barkley, Koplowitz, Anderson, & McMurray, 1997), which can contribute to conflicts with parents and teachers around school performance. Family conflicts also occur when children are disorganized with their belongings at home. When OTMP difficulties result in problems such as misplacing or forgetting materials for games and sports, peer relations can also be adversely affected (Dimantopoulou, Rydell, Thorell & Bohlin, 2007).

Despite the negative impact of OTMP problems, few systematic efforts have targeted these problems in elementary-school aged children with ADHD, the age when youth first encounter expectations for many independent, organized behaviors at home and school. Two studies, which did not require quantified OTMP impairment for inclusion, reported gains in organizational functioning with behavioral treatment. An eight-week after-school intervention with 4^th–7^th graders (N = 37) “identified by school personnel as having ADHD-related problems and in need of academic intervention” (p.416), improved children’s organization of school materials and parent-rated homework problems. However, teacher ratings of academic performance did not differ for intervention and control (n = 13) students (Langberg, Epstein, Urbanowicz, Simon, & Graham, 2008). A randomized controlled trial evaluated a 12-week home-school intervention for youngsters with ADHD-Inattentive type (N = 69) consisting of teacher consultation, parent training, child training in social competence and independent living skills including organization skills (Pfiffner et al., 2007). Children’s organizational skills improved significantly with treatment and gains were maintained at a five month follow up; however, the groups did not differ in organizational skills at follow-up because of improvements in the control group. In a small placebo-controlled trial in children with ADHD and OTMP difficulties (N = 19), organizational functioning improved with stimulants, presumably because of improvements in ADHD symptoms; however, OTMP functioning for the majority of children remained in the clinical range (Abikoff et al., 2008).

OTMP problems may represent behavioral manifestations of poor executive function (EF) (Barkley, 2006; Pennington & Ozonoff, 1996). Impairments in arousal, inhibitory control, delay tolerance, working memory, and time perception likely impede self-regulatory behaviors and interfere with organizing actions and planning (Barkley, Koplowitz, Anderson, & McMurray, 1997; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Thus, one approach for ameliorating OTMP difficulties may be through interventions that address such EF deficits. However, to our knowledge, there is no evidence from controlled studies that efforts to improve executive processes in children with ADHD ameliorate their OTMP difficulties.

It is also conceivable that OTMP difficulties result primarily from specific skills deficits. Our clinical work with children with ADHD and OTMP problems suggested that many of these youngsters lack the knowledge and proficiency to effectively organize materials, track assignments, manage time and plan their work. If a skills deficit is operative, behavioral treatment that emphasizes skills instruction, practice, and breaking skills into sub-steps would be expected to be useful. Support for this notion was obtained in a pilot study of an Organizational Skills Training (OST) intervention in 3^rd–5^th graders with ADHD and OTMP difficulties (N = 20). OST significantly improved organizational behaviors at home and school and reduced homework problems and family conflicts (Abikoff & Gallagher, 2008).

Finally, OTMP difficulties may result principally from a performance deficit. Namely, children with ADHD and OTMP problems may have the organizational knowledge and skills, but fail to apply them due to insufficient motivation, lack of carry through, and/or task avoidance. Underlying this performance deficit are two factors: 1) a motivational or “delay aversion” deficit (Sonuga-Barke, 2003), which makes it difficult for children with ADHD to wait for desired outcomes and work effectively over extended periods of time without considerable structure and reinforcement; and 2) deficits in executive function that adversely affect the ability to connect knowledge with performance in a manner that results in efficient self-directed adaptive behavior. Barkley (2006), a proponent of the performance deficit model, has described ADHD as primarily “not a disorder of knowing what to do, but of doing what one knows” (p. 324), due to faulty executive functions. If OTMP problems reflect a performance deficit, then interventions should enhance motivation by rewarding goal (i.e., “end-point”) behaviors, which should increase the occurrence of these behaviors and ostensibly reinforce the behavior chain linked to goal attainment as well.

Uncertainty regarding whether children’s organizational difficulties result predominantly from skills or performance deficits is not unique to this functional domain; it reflects longstanding speculations about the presence and impact of skills and/or performance deficits on the functioning of children with ADHD (Douglas, 1983; Green & Ablon, 2001; Gresham, Sugai, & Horner, 2001; Stein, Szumowski, Blondis, & Roizen, 1995).

The current study was informed by several issues noted above; namely, clinical observations of knowledge gaps and organizational skills deficits in children with OTMP difficulties, pilot results suggesting utility of a skills-based treatment, and a dearth of controlled studies evaluating the impact of performance-based interventions on the OTMP functioning of elementary-school children with ADHD. The study was designed to test the specific and relative efficacy of a skills-based intervention in ameliorating the organizational difficulties of children with ADHD, and to evaluate long-term maintenance effects. We hypothesized that skills-based treatment (OST) would be superior to: 1) a wait-list control (specific efficacy); 2) a performance-based intervention (relative efficacy); and that 3) treatment gains would be maintained significantly better for skills-based than for performance-based treatment. The study also compared the performance-based intervention and controls, although no a priori hypotheses were made.

To test these hypotheses, we conducted a dual-site randomized controlled trial evaluating OST’s efficacy compared to (1) a wait-list control (WL) and (2) a performance-based, contingency management intervention (“Parents and Teachers Helping Kids Organize”, PATHKO), which targets OTMP “end-point” goals and precludes skill training, allowing for evaluation of interventions derived from different underlying deficit models on children’s OTMP functioning and key functional domains adversely affected by OTMP difficulties.

Difficulty achieving generalization across settings and time has been a long-standing issue in ADHD psychosocial research (Hinshaw, Klein & Abikoff, 2007) attributed to a range of factors such as insufficient learning of target behaviors and skills, a focus on inappropriate treatment targets, restricted settings where treatment procedures are implemented, and focusing on broad rather than circumscribed aspects of functioning (Abikoff, 2009; Abikoff & Gallagher, 2008). Maintenance problems are especially common in contingency management interventions. Once children are no longer systematically reinforced for achieving targeted goals, reductions in behavioral gains typically occur (e.g., Barkley et al., 2000; Carlson, Pelham, Milich, & Dixon, 1992). Because OST ostensibly helps children learn specific skills whereas children in PATHKO are contingently reinforced for meeting OTMP end-point behaviors, we hypothesized that OST would result in significantly better maintenance effects compared to PATHKO when children were in the next grade, confronted with new and more complex OTMP demands.

Method

Sites

The study was conducted at New York University Langone Medical Center (NYU) and Duke University Medical Center. Outcome data were collected between March 2007 and June 2010. Site institutional review boards approved the study. After complete description of the study to parents and children, parents provided written informed consent and children assented.

Design

In a 3-group parallel design, stratified for site and baseline medication status, children were randomly assigned to: (1) OST, (2) PATHKO, or (3) WL, in 2:2:1 ratio

Participants and Procedures

Inclusion criteria

Participants were 8–11 year-old boys and girls in 3^rd–5^th grade with ADHD and organizational deficits. Inclusion required IQ ≥ 85 on the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999), elevated ADHD ratings on the Revised Conners Parent and Teacher Rating Scales (Conners, Sitarenios, Parker, & Epstein, 1998a, 1998b), a DSM-IV diagnosis of ADHD (all types) on the Diagnostic Interview Schedule for Children Parent Report Version 4 (DISC-IV) (Shaffer et al., 1996), confirmed by clinical evaluation with parent and child. OTMP entry criteria required factor or Total scores > 1 SD above age and sex norms (Abikoff, Gallagher, & Alvir, 2009)¹ on the teacher-completed Children’s Organizational Skills Scale (COSS-T), or on the parent COSS (COSS-P) Total score, and the teacher or parent had to indicate that OTMP problems were functionally interfering. To enter the trial, a child’s teacher had to consent to study participation.

Recruitment relied on referrals from schools, community resources (clinics, physicians, agencies), parent mailings, and newspaper ads. Medicated children not meeting ADHD screening criteria had a one-week washout to confirm ADHD status off medication.

Exclusion criteria

Reasons for exclusion included attending an all-day special education classroom; having an Individualized Educational Plan or paraprofessional aide to help specifically with organization; diagnosis of PDD, psychosis, bipolar disorder, PTSD, or any other condition judged to contra-indicate participation; or currently receiving behavioral treatment for ADHD. Children with diagnoses of conduct disorder, oppositional defiant disorder, anxiety or mood disorders were not excluded.

Measures

Assessment schedule

All children were assessed at baseline, posttreatment and one-month after treatment had ended (Year 1, Month 1 [referred to as Y1M1]), while still with the same class and teacher. OST and PATHKO participants had additional follow-up assessments one-month (Year 2, Month 1 [Y2M1]) and four-months into the next school year (Year 2, Month 4 [Y2M4]), with teachers blind to the treatment the children had received. (The duration between posttreatment and Y2M4 assessments ranged from 7–12 months, depending on when the child completed treatment). Outcome measures were obtained at all assessment points, except for academic functioning and attitude to school measures, which were not assessed at Y1M1. Changes in medication status were assessed every two weeks during treatment and the waiting period. Changes in services were inquired about at posttreatment. Because it was deemed unfeasible to withhold treatment from wait-list participants for 7–12 months, and would have likely resulted in differential attrition, WL participants obtained their treatment of choice after the Y1M1 month follow-up assessment, and did not participate in the Y2 follow-up assessments.

All families were compensated $15–$25 per session to cover travel costs to the clinic and $50 for participating in each of the study’s assessment visits.

Outcome measures

Measures assessed organizational functioning and other key domains including academic functioning, homework behaviors, family functioning, and attitudes to school and teachers.

Organizational functioning at home and school was assessed with the COSS-P and COSS-T, respectively. The COSS-P and COSS-T Total scores, the primary study outcomes, have good discriminative validity and are sensitive to treatment effects (Abikoff & Gallagher, 2008; Pfiffner et al., 2007). Children rated their OTMP behaviors on the COSS-C. Each COSS version (Abikoff & Gallagher, 2009) uses a 4-point rating scale (“1 = Hardly ever or never, 2 = “Sometimes”, 3 = Much of the time, 4 = Just about all of the time”), and yields three subscale scores (Materials Management, Organized Actions, Task Planning) and a “Cross-factor cluster” encompassing behaviors that load on more than one factor (e.g., backpack neatness, adherence to schedules).

Academic functioning was evaluated using the Academic Performance Rating Scale (APRS; DuPaul, Rapport, & Perriello, 1991), which assesses quality and accuracy of classroom work, and the Academic Proficiency Scale (APS, developed for this study), which assesses proficiency in six academic subjects relative to standard expectations (1 = “Well below the standard expected at this time of year”; 3 = “At the standard ″; 5 = “Well above the standard”). The sum of ratings across the six academic subjects was the unit of analysis. (The coefficient alpha for the scale = .84). The APS was used in lieu of report cards because the latter are issued at fixed times and do not represent children’s academic standing when study assessments occurred, and because there was no common report card format and grading system across participants’ schools.

Homework behaviors were measured with the Homework Problems Checklist (HPCL) (Langberg et al., 2010), which assesses homework completion and management.

Family functioning was measured by the Family Environment Scale (FES) (Moos & Moos, 1994), which yields a Family Relations Index (FRI), comprised of Cohesiveness, Expressiveness, and Conflict subscales, and by COSS-P ratings of Conflict, which assesses family conflict due to the child’s OTMP difficulties.

Attitude to school and teachers was assessed with two of the three subscales (excluding the Sensation Seeking subscale) that comprise the School Maladjustment Composite on the Behavior Assessment System for Children (BASC, Reynolds & Kamphaus, 1992).

Global improvement was judged by parents and teachers (teacher ratings obtained at post-treatment and Y1M1) with the 7-point Clinical Global Impression-Improvement scale (CGI-I, Guy, 1976), which assesses children’s improvement in organizational functioning; responders are those rated 1 (”Very much improved”) or 2 (“Much improved”).

Treatment satisfaction was assessed by parents and teacher at posttreatment with the Consumer Satisfaction Questionnaire (CSQ, McMahon & Forehand, 2003), which provides ratings from 1 = “Very dissatisfied” to 7 = “Very satisfied”.

Global severity was assessed with a 7-point scale (CGI-S, Guy, 1976) completed by parents and teachers to assess severity of children’s OTMP difficulties at baseline. Severity scores served as covariates in analyses of CGI-Improvement ratings.

Adverse behavioral events were assessed to evaluate possible iatrogenic effects associated with treatment (Barlow, 2010; Dimidjian & Hollon, 2010). Parents were asked at baseline and every two weeks about the occurrence of adverse behavioral events (tantrums, sad mood, irritability, anxiety, arguing, noncompliance, other), including their start date, duration, and possible relationship to treatment.

Interventions

Both treatments are individual and target organizational functioning, but rely on different procedures consonant with their underlying theoretical models. Treatments total 20 hour-long inclinic sessions held twice-a-week after school. (Initial OST treatment development efforts using once-weekly sessions yielded limited benefits. Two sessions a week provided more opportunities for practice and feedback, increased children’s engagement and improved their OTMP functioning. PATHKO was designed to match the frequency of OST sessions.) The treatments are fully manualized with detailed instructions to therapists regarding session goals, scripts and procedures used to address these goals, and between-session “homework” assignments for participants. Distinguishing features of OST and PATHKO are described in Table 1.

Table 1.

Key Features of OST and PATHKO Interventions

Treatment Features	OST	PATHKO
Length of Treatment	Total of 20 sessions; scheduled 2 times a week for 10–12 weeks.	Total of 20 sessions; scheduled 2 times a week for 10–12 weeks.
Persons Involved in Sessions	Child with parent present at beginning and end of session.	Parent with child present at end of session. One special session with parent and teacher in school.
Targets for Intervention	A set of sub-steps and skills aimed at improving the child’s ability to track assignments, manage materials, manage time, and plan tasks, thereby enabling the child to meet organizational demands at school and home.	Development and implementation of strategies and systems to increase the child’s motivation to achieve organizational endpoint behaviors in completing assignments, materials management, time management, and planning.
Session Content Emphasis	Review previously learned skills with child, teach and practice new skill, and instruct child on when to use the new skill between sessions.	Review previously targeted end-point behaviors and reinforcement systems with parent; identify new end-point behaviors to target; modify reinforcement systems as needed.
Role of parent	Made aware of the skills to be used and is trained to prompt, praise, and reward their use.	Learn to develop and implement structure and reinforcement systems at home and in collaboration with teachers.
Role of teacher	Made aware of the skills child learns and is trained to prompt, monitor, and praise its use. Provide feedback to parents on child’s skills use on daily checklist.	Identify, monitor, and praise child for meeting specific behavioral goals. Provide feedback to parents on child’s goal achievement on a Daily Behavior Report Card.
Sample Tools and Routines or Strategies Learned by Children or Parents and Teachers	Children: Daily Assignment Record, Assignment & Test Calendar, Accordion Binder, Backpack Checklist, Personal Calendar, Time Planning Conference and Procedure, Task Planning Conference and Procedure. Parents: Simple behavior modification procedures. Teachers: Prompting, Praising, and Monitoring Skill Use.	Parents: Effective praising, Daily Report Cards, Reward Systems, Homework Structures and Monitoring, Token Economy System, Time out or privilege removal for noncompliance with systems, collaborating with teachers Teachers: Prompting, Praising, and Monitoring Goal Achievement. Children: Cooperating with behavior management system.
Maintenance Materials	An Owner’s Manual for child and parents that guides how and when parent can facilitate child’s use of organizational skills in future situations at home and school.	An Owner’s Manual for parents that guides how and when to implement contingency management strategies in future situations at home and school.

OST

OST assumes that OTMP difficulties primarily reflect skills deficits in children’s ability to organize materials, track assignments, manage time, and plan tasks. To improve these skills, children are taught to use new tools and routines to record assignments and due dates, organize school papers into binders and use checklists for materials needed, track time required for task completion, and break tasks into steps. Session time is spent working with the child, with parents joining during the last 10 minutes. Work with children is supported by brief training of parents and teachers to prompt, praise, and reward skill use. Children receive prizes for in-session application of sub-steps; parents and teachers monitor children’s implementation of sub-steps for home rewards. OST is facilitated through a playful orientation that guides children to use skills to overcome annoying “glitches” (reflections of executive function gaps) and to maximize the effectiveness of their “Mastermind”.

PATHKO

Consistent with a performance deficit model, PATHKO motivates children by training teachers and parents to establish specific, individualized goals for children on written charts completed daily and to prompt, monitor, and praise/reward children for achieving these goals. Sessions primarily involve parents, with children coming in briefly at the end of every session. The three core components of PATHKO are: 1) Daily Report Cards (DRC) targeting end-point OTMP behaviors e.g., “assignments completed on time”, “desk/cubby is neat and organized”, where teachers monitor the behaviors at school and parents provide points at home, 2) Token Economy System, in which children receive points for achieving goals at home (e.g., “brings home all materials needed to do homework, backpack packed by bedtime”) and on their DRC, and exchange the points for privileges and rewards on a daily and weekly basis, and 3) Homework Rules and Structures, in which parents establish and reward children’s adherence to rules regarding completing homework. Underlying each of these components is an emphasis on Parent-Teacher Collaboration and procedures to facilitate sustainability.

Fundamental distinctions regarding features common to both OST and PATHKO

Although contingency management and teacher involvement are components of both OST and PATHKO, there are fundamental differences in their implementation, emphasis and objectives. Contingency management is the pivotal, defining characteristic of PATHKO and is emphasized throughout the intervention, whereas OST provides relatively minimal instruction in the use of contingency management procedures for parent and teachers. OST uses positive reinforcement procedures only, whereas PATHKO also relies on response cost procedures. OST focuses on “process”, reinforcing the child’s practice and use of intermediate sub-steps necessary for skills acquisition and learning, whereas PATHKO was designed to focus exclusively on OTMP “end-point behaviors” and precludes reinforcement of sub-steps and skills practice. Both treatments address homework-related organizational difficulties, but in different ways. PATHKO provides instruction to parents in setting homework rules and in reinforcing “end-point” goals such as “brings home all needed homework materials” or “finishes homework on time”, whereas OST teaches skills to children such as recording homework assignments, packing papers and books needed for homework, estimating time to complete homework, prioritizing homework assignments, and reviewing that their work is done neatly and completely.

Regarding teacher involvement, manuals provide therapist guidelines for five scheduled teacher contacts. Both treatments begin with a teacher-therapist meeting at the school whenever possible, to orient the teacher to treatment and initiate the core school-based treatment components. For OST, this is the Daily Assignment Record (DAR), a form that teachers initial daily and indicate whether the child had written down assignments accurately and practiced the specific skill being targeted (e.g., putting all papers in binder, getting “ready to go” in the classroom, holding a planning conference for task completion). For PATHKO, this is the DRC in which end-point organizational behaviors are targeted for students (e.g., homework turned in, 85% of work completed, work completed neatly and accurately, desk or backpack appears organized). PATHKO provides extensive parent support for implementing home-based rewards for DRC success, whereas OST makes briefer and more limited suggestions for doing so. Therapists in both treatments contact teachers approximately every two weeks to facilitate teacher implementation of the DAR and DRC, particularly as modifications are made over the course of treatment. However, in PATHKO, parents are taught to work directly with teachers around DRC implementation with therapist support, including participating in the initial teacher-therapist meeting and holding two teacher meetings without the therapist. This approach is intended to facilitate maintenance of the intervention beyond the period of study treatment.

Therapists

Therapists (n = 14) were psychologists with at least two years of behavior therapy experience with children and families. To control for therapist effects (Anderson, Ogles, Patterson, Lambert, & Vermeersch, 2009), therapists provided both treatments. Therapists were trained initially to pre-established criteria and supervised weekly by the treatment developers.

Treatment fidelity and integrity

Treatment sessions were audio-taped and therapists completed treatment fidelity and integrity checklists after each session. Fidelity checklists specified each session’s item content and yielded the percentage of items covered in each session. Integrity checklists assessed whether treatment contamination occurred in the session. Contamination denotes therapist actions that resulted in the incorporation of features from the non-assigned treatment (e.g., OST) into the assigned treatment (e.g., PATHKO). Contamination could occur either by the therapist initiating discussion about and recommending use of prohibited (non-assigned) treatment methods, or by actively supporting and elaborating on the parent’s or teacher’s use of treatment features specific to the non-assigned treatment. Fifteen percent of the recordings were rated for fidelity and integrity by Independent Evaluators (IEs) at both sites. Any fidelity or treatment contamination issues were discussed at each site in weekly supervision meetings and in weekly cross-site supervision calls including all study therapists.

Sample size

The study sample size N = 180 (72 in OST, 72 in PATHKO, 36 in WL) was determined based on power calculations to detect a difference between OST and PATHKO on the COSS-T of 0.5 SD with 80% power using a two tailed t-test with Type I error rate set to .05. As planned, 180 children were enrolled, randomized, and participated in the study (97 at NYU, 83 at Duke). However, during the last year of the study, administrative errors at one site resulted in unusable data on 22 participants (8 OST, 10 PATHKO, and 4 WL). Thus, all baseline data and analyses of immediate post-treatment and maintenance effects, as well as corresponding tables and figures are based on N = 158 (OST = 64, PATHKO = 61, WL = 33).

Impact of unusable data

It is important to note that the missingness of these data was not related to the treatment conditions, or to any subject characteristics, and thus is missing completely at random (Little & Rubin, 2002).

Statistical analyses

Approach to missing data

No covariates of interest were missing for analyses. Multiple imputation (Rubin 1987; Schafer 1997) was used to deal with missing outcomes. Of the 158 randomized subjects only 4% dropped out during the treatment and were assumed to be missing at random (MAR) and 11% had missing year 2 data due to administrative error, i.e., their data were missing completely at random MCAR (Little & Rubin, 1987). Scores missing at baseline were imputed for all treatment groups together, while missing post-randomization values were imputed separately by treatment group. All missing outcomes assessing a similar construct were imputed together: for example, all 4 COSS factors (Materials Management, Organized Actions, Task Planning and the Cross-factor cluster) and the total score for all three informants (parent, teacher and child) were imputed together. Although these sets of outcomes were not used simultaneously in the analysis model, they were used together in the imputation model, thus potentially increasing the efficiency and reducing bias compared to analysis of the observed cases only (Sterne et al. 2009; Spratt et al. 2010). Age and sex of the child were also used in all imputation models. Single outcomes missing at a given time point, when other outcomes in the same set were available, were imputed by the predictive mean matching method (Heitjan & Little 1991; Schenker & Taylor 1996) with the available outcomes at this time point and the previous available values of the currently missing outcome as predictors. Intermittently missing outcomes were first imputed using MCMC to make the missing pattern monotone, after which predictive mean matching method was used to complete the data set. A total of 200 complete data sets were created. The inferences from these complete data sets were combined using guidelines from Rubin (1987). The imputations were performed in SAS (SAS Institute Inc., Yuan 2010).

Immediate treatment effects

Intervention effects were estimated based on ANCOVAs for end of treatment scores. Significant main effects were followed by pair-wise comparisons across groups. No adjustments were made for multiple comparisons. The main outcomes were the total or global scores on all measures. Secondary comparisons of subscales were based on multivariate analyses using mixed effects models, which included interactions between subscales and group (Diggle, Heagerty, Liang, & Zeger, 2002). Significant interaction effects were followed by estimation of treatment effects separately for each subscale. Results are reported only if there were differences in treatment effects between subscales.² Treatment comparisons with respect to categorical outcomes (responder status and clinical outcome) utilized logistic regression. Clinical outcome was measured by the percentage of participants no longer meeting inclusion criteria for OTMP impairments at posttreatment. We report the adjusted odds ratios (OR) and 95% confidence intervals (CI). All analyses adjusted for baseline levels of the outcomes, the two stratifying factors (site and medication) and IQ, for which there was a site difference.

The effect of site and medication on treatment efficacy was explored by fitting models similar to those described above that included interaction between site and treatment. Data were analyzed for all participants according to intent-to-treat principles. Significance of interactions between site and treatment and the main effect for site was judged at α=0.05. Following significant interaction effects, the p-values of post-hoc tests are reported unadjusted for multiple testing. The same analysis was done to investigate the effect of medication.

Short-term maintenance effects

The analysis described above compared the three groups at one month post-treatment while children were in their same class. We also assessed functioning in the wait-list group over the 16 week period between baseline, post-treatment and Y1M1 evaluations. Mixed effects models with random subject effect (Diggle et al., 2002) (similar to repeated measures analysis of variance) were used to provide a benchmark for changes without any study treatment.

Long-term Maintenance effects

Children randomized to OST and PATHKO were assessed one and four months after the beginning of the next school year. The course of functioning following treatment was studied using mixed effects models for longitudinal data (Diggle et al., 2002). Individual outcomes were graphically examined to evaluate whether their course from immediate post-treatment to year 2 month 4 (Y2M4) was linear over time. There was no evidence for non-linearity for any of the outcomes, therefore linear mixed effects models with main effects of time, treatment and time-by-treatment interactions were used to compare the two active treatments. As above, the effects of baseline symptoms, site and medication status were controlled for in all analyses. The time since posttreatment assessment was also covaried. The correlations between repeated observations on a subject were taken into account by including random subject intercepts and time slopes. Presence of a significant interaction between time and treatment indicates that the course of functioning posttreatment differed between groups. Lack of significant interaction would signify no change in the contrast between OST and PATHKO, indicating that differences between groups observed at immediate post persisted through the next school year. In this case, the model was re-fitted with only main effects for treatment and time and the effect of time was evaluated based on this model.

RESULTS

Sample

Participant flow, exclusions, and study disposition are depicted in Figure 1. Of the 158 randomized participants, 151(95.6%) completed the study as intended. Three OST participants dropped out (one before starting treatment and two after the first session), and four dropped out of PATHKO (one after the first session, and the others after attending 4, 6 and 10 sessions). During the last year of the study, because of administrative errors, the Y2 follow-up data were invalid for 17 children at Duke(8 OST, 9 PATHKO). In accord with the study’s intent-to-treat principles, these participants’ Y2 follow-up scores and those of the dropouts noted above were imputed and included in analyses of long-term maintenance effects.

Figure 1.

Flow Diagram of Treatment Study

Missing data was minimal with only five subjects having no posttreatment assessment and 10% of subjects (uniformly distributed across treatments) having at least one primary outcome missing. Differences in baseline scores were evaluated between subjects with and without posttreatment data on a given primary outcome, and the three groups were compared with respect to these differences; no differences were observed for any total or global scores.

Baseline characteristics

Baseline demographic and clinical characteristics of the sample are detailed in Table 2. At entry, 56 (35.42%) of the children were being treated with ADHD medication. Other than a significantly lower WIAT-II spelling score in the WL control compared to the OST and PATHKO groups (ps < . 05), the groups did not differ significantly on any other baseline characteristic.

Table 2.

Baseline Demographics and Clinical Characteristics by Treatment Group

Child Demographics
Variable		OST (N = 64)	PATHKO (N= 61)	WL (N = 33)
Age	Yr, mean (SD)	9.06 (.91)	9.01 (.79)	9.15 (.76)
Sex	Male, N (%)	37 (57.8)	42 (68.9)	23 (69.7)
Race/Ethnicity, N (%)	African American	11 (17.2)	8 (13.1)	4 (12.1)
	White	46 (71.9)	43 (70.5)	21 (63.6)
	Other	7 (11.0)	10 (16.4)	8 (24.3)
	Hispanic	12 (19.0)	6 (9.8)	4 (12.1)
Grade, N (%)	3	27 (42.2)	20 (32.8)	10 (30.3)
	4	24 (37.5)	28 (45.9)	17 (51.5)
	5	13 (20.3)	13 (21.3)	6 (18.2)
Parent/Family Variables
High school graduate N (%)a	Mother	1 (1.6)	0 (0.0)	0 (0.0)
	Father	3 (4.8)	1 (1.7)	0 (0.0)
Some collegea	Mother	2 (3.1)	2 (3.3)	3 (9.1)
	Father	7 (11.3)	7 (11.7)	6 (18.8)
College graduatea	Mother	9 (14.1)	10 (16.7)	6 (18.2)
	Father	11 (17.7)	14 (23.3)	8 (25.0)
Advanced graduate/Professional degreea	Mother	23 (35.9)	21 (35.0)	9 (27.3)
	Father	22 (33.87)	17 (28.3)	5 (15.6)
Employed, N (%)	Mother	49 (76.6)	45 (73.8)	29 (87.9)
	Father	51 (79.7)	50 (82.0)	30 (90.9)
Married	N (%)	47 (73.4)	48 (78.7)	27 (81.8)
Child Clinical Characteristics
WASI IQ, mean (SD)	Verbal	113.31 (15.0)	114.89 (14.5)	111.85(13.0)
	Performance	107.03 (16.1)	110.33 (15.7)	104.55 (17.9)
	Full	110.42 (15.2)	113.72 (13.9)	108.70 (15.5)
	Word Reading	108.63 (11.6)	109.18 (12.4)	105.48(11.3)
WIAT-II, mean (SD)	Reading comprehension	108.30 (11.6)	109.27(11.5)	109.42(15.1)
	Numerical operations	104.59 (11.98)	106.08 (14.05)	104.21 (15.08)
	Spellingb	106.89 (15.45)	106.00 (14.84)	100.42 (11.36)
CGI-Severity (range: 1–7)	Parent, mean (SD)	5.39 (1.0)	5.28 (1.0)	5.16 (0.8)
	Teacher, mean (SD)	5.02 (1.3)	4.95 (1.1)	4.71 (1.6)
Conners Rating Scale ADHD Total	Parent, mean (SD)	75.66 (9.2)	75.29 (9.1)	73.97 (9.3)
	Teacher, mean (SD)	69.49 (10.6)	68.26 (10.3)	66.77 (8.7)
Score ADHD Diagnosis, N (%)	Combined	25 (39.1)	29 (47.5)	16 (48.5)
	Inattentive	39 (60.9)	32 (52.5)	17 (51.5)
	Oppositional-defiant disorder	11 (17.2)	5 (8.2)	6(18.2)
	Anxiety disordersc	18 (28.1)	12 (19.7)	7 (21.2)
Comorbidity, N (%)	Enuresis-Encopresis	3 (4.7)	4 (6.6)	1 (3.0)
	Tic disorder	3 (4.7)	0 (0.0)	3 (9.1)
	Conduct disorder	0 (0.0)	0 (0.0)	0 (0.0)
	Other	2 (3.1)	1 (1.6)	0 (0.0)
ADHD medication	N (%)	23 (35.9)	21 (34.4)	12 (36.4)
Reading Disability	N (%)	2 (3.1)	2 (3.3)	1 (3.0)
Special Servicesd	N (%)	16 (25.0)	19 (31.1)	8 (25.0)

Abbreviations: WASI, Wechsler Abbreviated Scale of Intelligence; WIAT, Wechsler Individual Achievement Test; CGI, Clinical Global Impressions.

^aHighest Education Level completed (not all parents responded);

^bWL < OST, PATHKO, p<.05;

^cIncludes Separation Anxiety, General Anxiety, Social Phobia, and Specific Phobia;

^dIncludes 504 plans, special education services, tutoring, and accommodations.

Treatment attendance, fidelity and satisfaction

Attendance

Attendance was very high in both groups. The mean number of sessions attended in OST was 19.80 (± .70, range 17–20) and 19.54 (± .85, range 15–20) in PATHKO.³ At least three of five teacher contacts occurred for 94% of OST and 88% of PATHKO cases.

Treatment fidelity

A mean fidelity score for each treatment was generated by averaging all sessions’ percent fidelity scores across all participants. There was no significant difference in the percentage of treatment content covered (OST = 96.3%, PATHKO = 95.4%). The reliability (percentage agreement) of therapists’ and IE fidelity ratings was .93, and .93 between IEs. Contamination scores represent the number of sessions in which therapists indicated that contamination occurred relative to the total number of sessions held. Instances of treatment contamination were extremely low (OST: 1 occurrence in 1200 sessions = 0.08%; PATHKO: 18 occurrences in 1123 sessions = 1.60%). The agreement of therapists’ and IE contamination ratings was 99.4%, and 100% between IEs.

Treatment satisfaction

Teachers and parents were satisfied with treatments, with no significant differences by treatment type. Teachers: OST = 5.73 (1.31), PATHKO = 6.09 (1.01); Parents: OST = 6.62 (.52), PATHKO = 6.48 (.71).

Immediate Post-treatment effects on functioning

Results, including p-values for tests and effect sizes, are detailed in Table 3. Findings regarding OST’s efficacy compared to WL control and PATHKO are summarized below for OTMP and other functional domains, followed by comparison of PATHKO and WL control.

Table 3.

Outcomes at Post-Treatment

	Baseline and Post-treatment Raw Scores											Model Based Comparisons
	OST (A) (N = 64)				PATHKO (B) (N = 61)				WL (C) (N = 33)				Group Contrastsa
Domain and Measure	Pre		Post		Pre		Post		Pre		Post		Analysis		A vs. B		A vs. C		B vs. C
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	F(2,151)	P	p	db	p	db	p	db
OTMP
COSS-Tc	2.85	.44	2.24	.47	2.86	.48	2.24	.55	2.77	.39	2.72	.44	19.75	***	ns	.02	***	−1.18	***	−1.21
COSS-Pc	2.99	.31	2.00	.43	3.00	.32	2.20	.39	2.97	.31	2.86	.42	55.71	***	**	−.63	***	−2.77	***	−2.13
COSS-Cc	2.24	.34	1.95	.42	2.24	.36	2.01	.45	2.21	.37	2.18	.34	4.49	*	ns	−.22	**	−.69	ns	−.47
Academic
APRSd	53.45	10.34	62.16	10.52	54.45	11.12	63.96	11.90	54.06	8.58	54.53	9.74	11.97	***	ns	−.08	***	.76	***	.82
APSd	16.39	4.27	18.55	4.26	17.08	3.54	18.35	3.89	16.05	3.22	16.63	3.30	5.18	**	ns	.23	**	.42	ns	.19
Homework
HPCLc	52.01	9.79	34.60	10.10	52.01	11.23	33.12	6.31	55.33	9.59	50.24	13.52	37.96	***	ns	.14	***	−1.37	***	−1.51
Family
FESd	20.32	3.35	20.62	3.28	18.71	4.53	19.91	3.36	18.69	3.94	17.73	4.42	8.38	***	ns	−.07	***	.47	***	.54
COSS-P Conflictc	3.22	.74	1.93	.73	3.11	.75	2.05	.60	3.09	.84	2.87	.93	21.75	***	ns	−.22	***	−1.26	***	−1.03
Attitudes
BASCc	48.00	8.28	48.30	8.74	52.13	9.36	50.90	11.19	49.36	8.73	51.05	8.20	0.78	Ns	na		na		na

^aTwo-tailed tests;

^bEffect sizes (Cohen’s d) are the differences between adjustedscores at post-treatment, divided by thestandard deviation at baseline,p-values are model-based;

^clower score is better,

^dhigher score is better. Abbreviations: COSS= Children’s Organizational Skills Scale (P= parent, T= teacher, C= child); CTRS-R, CPRS-R= Conners Rating Scale-Revised (P= parent, T= teacher); APRS= Academic Performance Rating Scale; APS= Academic Proficiency Scale; BASC= Behavior Assessment System for Children; HPCL= Homework Problems Check List; FES= Family Environment Scale; A= OST; B= PATHKO; C= WL; ns= Not significant; na= Not applicable

^*p< .05;

^**p< .01;

^***p< .001

OST vs. WL

OTMP functioning

There were highly significant treatment effects on organizational functioning. OST was superior to WL on the COSS-T and COSS-P (ps<.0001) and on the COSS-C (p<.004). Significant treatment-by-subscale interactions were obtained on the COSS-T (χ²=19.0, df=6, p< .005) and COSS-P (χ²=26.2, df=6, p<.0003). OST was significantly better than WL on all COSS subscales. However, relative to WL, OST’s impact on Organized Actions (d = −1.74) was more than twice the effect size for the other COSS-T subscales. Similarly, the largest effect size for OST on the COSS-P was on the Organized Actions subscale (d = −3.13).

Other functional domains

Significant main effects for OST were obtained on all study outcomes except the BASC. OST was superior to WL on academic functioning (APRS, p<.0001; APS, p<.007), homework behaviors (p<.0001), and family relations (FES, p<.001; COSS-P Conflict, p<.0001). The treatment-by-subscale interaction was significant on the FES (χ²=16.2, df=4, p<.003).Compared to controls, OST resulted in significantly more Cohesiveness (d = .47) and less Conflict (d = −.34), but no difference in Expressiveness.

OST vs. PATHKO

OST was superior to PATHKO on the parent COSS Total score (p < .005). There was a significant treatment-by-subscale interaction on the parent COSS (χ²=26.2, df=6, p<.0003). OST’s effect relative to PATHKO was greatest for Organized Actions (d = −1.21); there were no significant differences on the other COSS-P subscales. OST and PATHKO did not differ significantly on other outcome measures (Table 3).

PATHKO vs. WL

PATHKO was superior to WL in OTMP functioning (COSS-P and COSS-T, ps<.0001; COSS-C, p<.05), academic functioning (APRS only, p<.0001), homework behaviors (p<.0001), and family functioning (FES, p<.001; COSS-P Conflict, p<.0001) (Table 3). Significant treatment-by-subscale interactions on the FES indicated that PATHKO was superior to WL in Cohesiveness (p<.01, d=.53) and Conflict (p<.05, d = −.37) but not in Expressiveness. PATHKO did not differ significantly from controls on the APS and the BASC.

Global improvement/responder status

Responder rates using parent CGI-I ratings (covarying baseline CGI-Severity) were significantly better for OST (85.3%) and PATHKO (86.9%) than WL (0%), overall p<0.0001; the odds ratios (ORs) for OST and PATHKO vs. WL are infinity, because the proportion of responders in WL=0, OR_OSTvsPATHKO=0.87 (ns). The same pattern was obtained for Teacher CGI-I ratings (covarying teachers’ baseline CGI-Severity scores): OST, 58.1%; PATHKO, 65.3%; and WL, 3.0% overall p<0.0001; OR_OSTvsWL=55.4 CI=(6.7, 459.3), OR_PATHKOvsWL=73.5, CI=(8.8, 610.9), OR_OSTvsPATHKO=0.75 (ns).

Clinically meaningful change

Based on COSS-T and COSS-P post-treatment scores, 60.1% of OST, 60.0% of PATHKO and 3.0% of WL children no longer met study inclusion criteria for organizational deficits, overall p<0.0001; OR_OSTvsWL=48.80 CI=(6.2, 381.9), OR_PATHKOvsWL=48.5, CI=(6.2, 381.2), OR_OSTvsPATHKO=1.0 (ns).

Adverse Behavioral Events

For each adverse behavioral event, we compared the percentage of children in each group who had at least one reported occurrence of the event during the 10 week treatment, or waiting period. Arguing occurred in significantly more WL children (45%) than in those in OST (15%; χ² =10.17, p< .002) and PATHKO (21%; χ² = 6.19, p<.02). There were no significant group differences for any other event. Moreover, for each type of adverse event, the percentage of children in OST and PATHKO with a reported event decreased compared to baseline.

Site and medication effects on treatment outcome

Six children (3.8%) had a change in medication status during the study: five initiated medication (1 in PATHKO, 4 in WL); one child in OST stopped medication. Site and medication were not significant in any of the primary analyses (models with main effects only). The exploration of interactions between treatment and site or medication indicated differential treatment effect in the two sites on three outcome measures. On all, PATHKO was significantly better than WL at NYU (COSS-T Total, p < .0001; COSS-P Conflict, p < .0001; APS, p < .04), but not at Duke. At Duke, OST children had significantly better APS scores than those in PATHKO (p< .002), whereas at NYU the OST and PATHKO groups did not differ.

Short-term maintenance effects

On all Y1M1 measures, the overall treatment effects and group contrasts paralleled those obtained at post-treatment, indicating constancy between post-treatment and one-month follow-up in the pattern of improvements and minimal change in the level of functioning on these outcome measures.⁴

Analyses comparing WL participants’ scores on measures collected at baseline and at Y1M1 indicated no significant change in COSS-C, COSS-T, BASC, FES and COSS-P Conflict scores. (The APRS and APS were not collected at Y1M1; however, controls did not change significantly on either of these academic measures between baseline and posttreatment). The WL group improved significantly from baseline to Y1M1on the HPCL (Pre M = 55.33, ±9.59; Y1M1 M = 49.06, ±13.90, Wald’s t = −4.01, p<.001) and COSS-P (Pre M = 2.97, ±.31; Y1M1 M = 2.83, ±.40, Wald’s t = −2.66, p<.01).

Long-term effects

Treatment by time interactions

A significant Time effect for parent-rated homework problems on the HPCL (Wald’s t = 2.90, p<.004) has to be interpreted in the context of a significant Treatment X Time interaction (Wald’s t=2.28, p<.03). Inspection of HPCL trajectories indicated that OST children’s homework problem scores were unchanged from immediate posttreatment to the end of follow-up, in contrast to a significant increase in homework problems in PATHKO from post-treatment through each of the follow-up assessments (see Table 4). There were no other significant Treatment X Time interactions.

Table 4.

Comparison of adjusted outcomes in OST and PATHKO during follow-up^a

	Follow-Up Raw Scores							Model Based Effects
		OST			PATHKO			Treatment X Time		Time Effect		Treatment Effect
Domain	Variable	Y1M1	Y2M1	Y2M4	Y1M 1	Y2M1	Y2M4	t	p	t	P	t	P
OTMP	COSS-Tb	2.27 (.48)	2.24 (.45)	2.37 (.60)	2.27 (.51)	2.37 (.46)	2.3 (.47)	−0.36	ns	1.84	Ns	0.01	Ns
	COSS-Pb	2.11 (.46)	2.10 (.45)	2.14 (.43)	2.24 (.29)	2.25 (.37)	2.24 (.38)	−1.35	ns	2.37	* †	2.49	<.02
	COSS-Cb	1.89 (.38)	1.81 (.33)	1.77 (.34)	1.94 (.43)	1.85 (.38)	1.82 (.38)	0.19	ns	−5.71	*** ‡	1.15	Ns
Academic	APSc	NA	18.11 (3.27)	16.99 (3.89)	NA	17.58 (4.05)	18.17 (4.24)	1.44	ns	−2.47	*	−0.88	Ns
	APRSc	NA	63.04 (12.54)	61.25 (12.32)	NA	60.59 (12.74)	62.87 (12.19)	−0.26	ns	−0.78	Ns	−0.41	Ns
Homework	HPCLb	35.84 (11.82)	36.35 (10.27)	35.28 (9.24)	34.54 (7.22)	36.68 (10.30)	37.32 (10.05)	2.28	*	2.90	** † §	−0.02	Ns
Family	FESc	20.66 (3.65)	20.81 (3.28)	21.21 (3.27)	20.24 (3.93)	20.09 (3.48)	19.96 (3.59)	−1.13	ns	1.22	Ns	−0.06	Ns
	COSS-P Conflictb	2.04 (.72)	2.11 (.76)	1.96 (.75)	2.22 (.63)	2.15 (.71)	2.13 (.72)	−0.01	ns	0.77	ns	1.75	ns
Attitudes	BASC b	47.52 (8.84)	46.10 (9.92)	45.92 (8.79)	49.38 (9.73)	47.53 (10.13)	45.82 (8.24)	−1.26	ns	−5.42	*** ‡	−0.56	Ns

Note.

^aValues are expressed as mean (SD),

^blower score is better,

^chigher score is better.

Abbreviations: ns=not significant; COSS-Chidren’s Organizational Skills Scales (P=parent, T=teacher, C=child); APRS=Academic Performance Rating Scale; APS=Academic Proficiency Scale; HPCL=Homework Problems Checklist; FES=Family Environment Scale

^*p< .05;

^**p<.01;

^***p< .001.

^†Outcome at follow-up was significantly better than at baseline.

^‡Outcome at follow up significantly better than at post treatment.

^§p value for PATHKO, Time effect for OST is ns.

Treatment effects

Children who received OST continued to have significantly better COSS-P Total scores than those in PATHKO (Wald’s t = 2.49, p<.02; Table 4). COSS-P subscale analyses indicated that OST children showed significantly better functioning during follow-up compared to those in PATHKO in Organized Actions (Wald’s t=−3.58, p< .0004) and on the Cross-factor cluster (Wald’s t=−2.20, p<.03). The groups did not differ significantly in any other outcome measures.

Time effects

The post-treatment improvements obtained on the COSS-T, APRS, COSS-P Conflict and FES were sustained during the next school year as indicated by the nonsignificant Time effect for each of these measures (Table 4). Significant Time effects on the COSS-C (Wald’s t = −5.71, p<.001) and BASC (Wald’s t = −5.41, p<.001) reflect further improvements in children’s OTMP self-ratings, and in their attitudes toward school and teachers during the follow-up period in both groups.

There was a significant fall-off in treatment gains in children’s OTMP functioning at home (COSS-P, Time effect, Wald’s t =2.37, p<.02) and in their academic proficiency scores (APS, Time effect, Wald’s t =2.47, p<.02). However, the COSS-P Y2M4 scores for OST and PATHKO when tested within each group, remained significantly better than baseline levels (OST, t=15.94, p<.0001; PATHKO, t= 13.61, p<.0001). On the APS, PATHKO’s Y2M4 scores remained significantly better than baseline (t= −2.55 p<.02), whereas the Y2M4 scores for OST were not significantly better than baseline (t=−1.11, ns).

DISCUSSION

Extensive support was obtained for the efficacy of organizational skills training, as hypothesized. Children treated with OST improved substantially at home and school in their organization, time management and planning compared to controls, evidenced by large effect sizes of 2.77 on the COSS-P and 1.18 on the COSS-T. OST also benefitted other areas of functioning, with magnitude of effects ranging from d =.42 to 1.37. The reductions in homework-related problems and improvement in homework management with OST are similar to reported effects of stimulants (Hechtman et al., 2004). Enhancements in the quality and accuracy of classwork as well as in academic proficiency are clinically meaningful. Children’s proficiency across all academic subjects, which was below the level of proficiency expected at study entry, reached the expected standard level at the end of treatment. OST also had emanative effects that extended beyond the child’s functioning, resulting in better family cohesiveness and less conflict, including substantial reductions in conflict specific to OTMP problems.

Evidence of efficacy for skills-based, child-focused interventions is relatively rare in ADHD psychosocial trials (Hinshaw et al., 2007). Two factors that may have contributed to the positive, sustained results reported here include the specific domain (OTMP functioning) targeted in treatment (this issue is discussed further below), and the application of a “needs-based” inclusion criterion, which, unlike most studies, required participants to have quantified impairments in the functional area the treatment is intended for.

Skills-based treatment was more efficacious than a performance-based intervention for only one of nine global measures: parent ratings of organizational functioning. (OST’s superiority to PATHKO in improving academic proficiency was found at only one site, suggesting that this is not a generalizable treatment effect). OST children’s overall OTMP functioning at home continued to be better than their PATHKO counterparts at follow-up. Thus, for one of the study’s primary outcomes, the hypothesis that a skills-based approach would result in better efficacy and maintenance was supported. The differential improvement on the COSS-P may reflect sub-skills targeted in OST that are not focused on in PATHKO. The COSS-P subscale analyses support this possibility. OST’s advantage was greatest for Organized Actions, which measures children’s use of practical routines and tools to stay organized (e.g., uses a calendar to know when assignments are due and tests are scheduled, writes down the order of steps to work on before starting a big project, uses separate folders for each subject, keeps a neat desk). It should be noted, however, that although PATHKO did not target this organizational skill set, PATHKO children also improved considerably in this area, suggesting that they may have independently increased these actions in their efforts to meet DRC and home-based goals.

It is unlikely that the treatment effect on parents’ ratings of organizational functioning is due to rater bias or allegiance effects, since OST included considerably less parental involvement than PATHKO. Alternatively, OST’s focus on individual treatment with the child may have been more consistent with parent beliefs about the nature of the child’s OTMP difficulties (Hoza et al., 2000). The WL participants’ treatment choice after the waiting period (98.3% chose OST) is in accord with this notion and may reflect the treatments’ relative acceptability to parents.

Notably, children treated with PATHKO improved more than controls on seven of nine outcomes. The magnitude of these effects ranged from d =.47 to 2.13, supporting the overall efficacy of this approach. The improvements in organizational functioning suggest that children treated with a performance-based, contingency management intervention that rewards them for meeting end-point behavioral goals may increase their use of existing organizational skills, or perhaps motivate them to implement organizational skills to facilitate goal attainment. Unlike OST, PATHKO children were not superior to controls in academic proficiency. Whether global differences in treatment approaches or specific features of OST resulted in differential academic proficiency relative to controls is unknown.

A review of ADHD treatment studies emphasized that “providing lasting benefit is, in fact, the key treatment issue in the field.” (Hinshaw, Klein & Abikoff, 2007, p.9). The positive follow-up results for both interventions are of special import and stand in contrast to the difficulty sustaining benefits frequently seen. During the 7–12 month follow-up period, there was no significant decrease in either group in the gains obtained at posttreatment in family relations and OTMP-related conflicts, or in children’s organizational functioning in school and academic performance. Given the increasing educational and organizational demands children confront as they move to a higher grade, the maintenance of improvements during the next school year is important. It is possible that these sustained gains, coupled with additional improvements during follow-up in the children’s self-ratings of their OTMP behaviors, contributed to improvements in children’s attitudes to school and teachers, which did not change initially with treatment, but did so during the subsequent school year.

Treatment-related gains in organizational functioning at home and in homework behaviors declined somewhat during follow-up; however, children’s functioning in these areas remained significantly improved over pretreatment levels in both groups. Both groups also experienced a decline in teacher-rated academic proficiency. Although there was no significant group difference between OST and PATHKO in academic proficiency at follow-up, it is not obvious why APS scores remained significantly better than baseline levels only in PATHKO.

We hypothesized that skills-based treatment would yield longer-lasting gains than a performance-based approach; it is unclear why differential maintenance effects were found only on the COSS-P. A recent study reported that children’s COSS-T factor scores were significantly associated with grade point average (Langberg et al., 2011). Perhaps the sustained improvements in OTMP functioning in both treatment groups were instrumental in enabling children to maintain gains in other functional domains during follow-up. Planned mediational analyses will examine this issue, and also explore whether continued use of treatment strategies targeted in OST and PATHKO during follow-up contributed to maintenance effects.

It is conceivable that sustained improvements during the follow-up period reflect maturational rather than maintenance effects. An untreated control group is needed to address this possibility. For reasons noted, the inclusion of such a control for an extended period of up to 12 months was not feasible, and is atypical in ADHD psychosocial intervention studies (Charach et al., 2011). However, the course of the WL group from baseline through Y1M1is informative. For the majority of outcomes, the WL group’s functioning remained basically unchanged over a period of slightly more than four months (M = 20.49 weeks, range 17.29 to 23.43 weeks). Although this interval is shorter than the average duration between post-treatment and Y2M4 follow-up, the stability in functioning suggests that repeated assessments and time alone are unlikely to result in gains on the outcomes assessed in this study. (Although the gains in COSS-P and HPCL scores in the WL group were significant, they were substantially less than the magnitude of effect associated with treatment).

Why were treatment gains maintained in this study? We have speculated elsewhere that generalization across settings and over time are much less likely to occur when treatments target broad domains, such as social functioning (Abikoff, 2009; Abikoff & Gallagher, 2008). In these instances, children have to learn, use and adapt behaviors across complex, multidimensional situations and contexts, increasing generalization difficulties. In contrast, OTMP functioning encompasses a fairly circumscribed set of behaviors that are called for in situations that recur with regularity. It may be that treatments that focus on OTMP and similar relatively restricted domains provide the child with opportunities to practice, refine and develop competence in a somewhat limited, specific area, which facilitates learning and the likelihood that these behaviors will be maintained.

Contrary to hypothesis, the two interventions, based on different theoretical models were largely comparable. Contingency management and teacher involvement were common to both interventions, and it is possible that this overlap contributed to the minimal treatment differences. However, as noted previously, there were fundamental, clinical distinctions between OST and PATHKO in the emphasis placed on, and implementation of these treatment components. More importantly, the results complicate a simple understanding of OTMP difficulties and suggest that both skills-deficit and performance-deficit models may have relevance in children with ADHD, as noted by Greene and Ablon (2001). In accord with the current emphasis on personalized treatment research, studies that focus on identifying whether there are subgroups of children who benefit more from one treatment than another could prove especially useful. Further, the systematic analysis of OTMP deficits could facilitate development of assessments that identify which components of a child’s OTMP difficulties reflect skills-deficits, and which are performance-based. For practical purposes, an equally or possibly more powerful intervention might incorporate components from both treatments. Adding OST elements to PATHKO may gain more child investment and adding PATHKO elements to OST may make the end-points more salient. Perhaps a combined approach would prove most beneficial for some youngsters, such as the 40% of those in OST and PATHKO who still experience OTMP difficulties at the end of treatment.

The three significant site X treatment interactions likely stem from site differences in WL participants (controls at Duke had lower baseline scores and better post-scores on these measures than controls at NYU), and not a result of site variations in PATHKO’s effectiveness.

Treatment response was similar in medicated and unmedicated children. The premise that medicated children with OTMP deficits may have skills deficits and would benefit more from OST than PATHKO (Abikoff et al., 2009) was not supported. However, 46 of the 56 medicated children (82%) met ADHD entry criteria while on medication, suggesting their treatment regimens were not optimized. Reports of sub-optimal ADHD pharmacotherapy practices by community practitioners support this possibility (Jensen et al., 2001). Whether outcomes would have differed had children been on optimal regimens is unknown (Vitiello et al., 2001).

There are some limitations regarding the study design and methodology. Most post-treatment measures were obtained from parents and teachers who were treatment agents in OST and PATHKO, increasing the risk of rater biases. Problematically, to our knowledge, there are no objective, ecologically valid, clinically relevant indices of OTMP functioning. Notably, however, follow-up ratings were obtained from teachers who were blind to, and had no involvement in, the children’s treatment. As such, their assessments were unbiased and provided objective evidence of maintenance effects regarding children’s organizational functioning and academic performance.

The children were above average in intellectual functioning, with a mean IQ of 111, and parents were predominantly highly educated. It is unknown if similar results would be obtained with youngsters with lower mean IQs or with families with lower levels of parental education. In addition, the feasibility of implementing treatments that involve twice-weekly clinic sessions needs to be established. Finally, we did not assess children’s ratings of treatment satisfaction and acceptability. Given the differences in the treatments’ focus and in the children’s involvement, it will be useful to obtain this information in future studies.

Several characteristics of the study sample should be highlighted. The rates of Predominantly Inattentive (IA) Type of ADHD (56%) and comorbid ODD diagnoses (14%) are appreciably higher and lower, respectively, than typical samples in ADHD clinical trials. However, we do not consider the study sample to be unique and unrepresentative of children with ADHD; rather, the sample’s characteristics are likely typical of the subgroup of children with ADHD who have impairing OTMP difficulties, an inclusion criterion for this study. Similar high rates of ADHD-IA diagnoses (58% – 60%) have been reported in two small studies of children with ADHD and OTMP deficits on the COSS (Abikoff & Gallagher, 2008; Abikoff et al. 2009), and ostensibly reflect the overlap between DSM-IV Inattentive symptoms (does not follow through on instructions and fails to finish school work, difficulty organizing tasks, loses things, forgetful in daily activities) and OTMP behaviors assessed on the COSS. Further, the high percentage of ADHD-IA likely contributed to the low ODD comorbidity rate (Bauermeister et al., 2005; Crystal et al., 2001; Milich et al., 2001).

The absence of an attention control (AC) precludes ruling out the possibility that nonspecific factors related to frequent contacts and attention from clinicians contributed to the significant treatment effects. However, we opted not to include an AC for several reasons. First, findings from the few ADHD psychosocial trials that included an AC suggest that children with ADHD do not derive any meaningful change from nonspecific interventions (Abikoff et al., 1988; Abikoff & Gittelman, 1985; Abikoff et al., 2004; Sonuga-Barke, Daley, Thompson, Laver-Bradbury, & Weeks, 2001). Second, to be viable and control for non-specific treatment effects, it is essential that an AC treatment be credible and acceptable, which requires systematic study. Given the study’s specific aim of improving children’s OTMP functioning, we were unable even to suggest a potentially credible, face-valid, attention control treatment. The inclusion of an AC that lacked empirical evidence of credibility and acceptability ran the risk of high dropout rates and relatively lower participation. If so, this would have compromised the test of OST’s efficacy and could have resulted in a failed experiment.

In summary, two distinct behavioral interventions were found to ameliorate children’s organizational difficulties and yielded robust, sustained improvements in other functional domains. There was also support for the treatments’ generalizability. Similar treatment outcomes were obtained at two sites, and the interventions were 1) acceptable to parents and teachers; 2) effective in medicated and unmedicated children; 3) resulted in 60% of children no longer meeting criteria for OTMP deficits; and 4) were not associated with adverse behavioral effects. The interventions show considerable promise of clinical utility with elementary school-aged children with ADHD and OTMP deficits.