Educational Outcomes of a Collaborative School–Home Behavioral Intervention for ADHD

Abstract

This study evaluated educationally relevant outcomes from a newly developed collaborative school–home intervention (Collaborative Life Skills Program [CLS]) for youth with attention and/or behavior problems. Participants included 17 girls and 40 boys in second through fifth grades (mean age = 8.1 years) from diverse ethnic backgrounds. CLS was implemented by 10 school-based mental health professionals at their schools and included 3 integrated components over 12 weeks: group behavioral parent training, classroom behavioral intervention, and a child social and independence skills group. Parent and teacher ratings of attention-deficit/hyperactivity disorder (ADHD) symptoms, organizational skills, and homework problems, and teacher-rated academic skills, report card grades, academic achievement, and classroom observations of student engagement were measured before and after treatment. Significant pre–post improvement was found for all measures, with large effect sizes for ADHD symptoms, organizational skills, and homework problems, and medium to large effects for teacher-rated academic skills, report card grades, academic achievement, and student engagement. Improvements in organizational skills mediated the relationship between improvement in ADHD symptoms and academic skills. Significant improvement in both ratings and objective measures (achievement testing, report cards, classroom observations) suggests that improvement exceeded what might be accounted for by expectancy or passage of time. Findings support the focus of CLS on both ADHD symptom reduction and organizational skill improvement and support the feasibility of a model which utilizes school-based mental health professionals as providers.

Academic problems, beginning in the elementary years, are typical among students with attention-deficit/hyperactivity disorder (ADHD; Wolraich & DuPaul, 2010). Most students with ADHD have the requisite academic knowledge but still perform below their skill level, as reflected in failure to complete assigned work and in poor test performance, report card grades, and academic achievement scores (Langberg et al., 2011; Schultz, Evans, & Serpell, 2009; Wolraich & DuPaul, 2010). Researchers hypothesize that the relation between ADHD and poor academic outcomes may be mediated, at least in part, by poor organizational skills or related executive functioning deficits common to ADHD (Volpe et al., 2006).

Existing school-based services for ADHD are generally not evidence based; do not systematically target the specific, multiple social and academic impairments associated with ADHD or important risk factors for adverse outcomes (e.g., parenting, social skills); and do not systematically apply strategies for generalizing treatment gains across settings (Abikoff, 2009). Consequently, school-based interventions for ADHD, as they currently stand, almost certainly generate only circumscribed improvements for participating students (Wolraich & DuPaul, 2010). In light of the high costs of ADHD to the educational system (Robb et al., 2011) and long-term risks for school drop-out, school failure, and delinquency (Barkley, 2006), it is essential to develop effective cross-domain interventions.

Results of a recent meta-analysis (Fabiano et al., 2009) of behavioral interventions (e.g., classroom behavior modification, parent training), as well as outcomes from newer interventions focused on skill building (Abikoff et al., in press; Evans et al., 2009; Langberg, Epstein, Urbanowicz, Simon, & Graham, 2008; Pfiffner et al., 2007), suggest that both classes of intervention have a positive impact on ADHD symptoms and academic performance deficits, including homework, classroom deportment, organizational skills, and academic functioning. However, these evidence-based treatments were developed in highly controlled classrooms or in university settings rather than in naturalistic school settings and typically have been administered by nonschool personnel. Consequently, significant questions exist about feasibility and sustainability within the school context and in the absence of research staff.

The Collaborative Life Skills Program (CLS) was developed to address this research-to-practice gap. CLS was adapted from a clinic-based intervention for ADHD-I and consists of simultaneous delivery of three empirically supported treatments over a 12-week intervention period: behavioral teacher consultation and use of daily report cards (DRC; Fabiano et al., 2010), behavioral parent training (Pelham & Fabiano, 2008), and child social and life skills training (Pfiffner & McBurnett, 1997; Pfiffner et al., 2007). Delivery of these interventions is coordinated such that parents, children, and teachers are trained in their aspects of the treatment using the same terminology at approximately the same time, via a combination of group and individualized methods. Reinforcement contingencies are set both within and across settings (e.g., parents reward behaviors that occur at home and school, school clinicians reward behaviors that occur at home, school, and group) in order to maximize intervention impact and generalization. The net effect is around-the-clock support of child behavior across impairment domains via an active partnership of parents, teachers, and school clinicians.

In adapting this intervention for school-based implementation, a collaborative iterative development process was implemented in which existing school-based mental health professionals (learning support professionals [LSPs]) were trained and implemented the program in successive schools, and data were collected regarding the program’s fidelity, feasibility, and acceptability. Initial published results describe this development and provide preliminary evidence supporting the positive impact of this methodology (Pfiffner et al., 2011). Significant pre- to posttreatment improvement in ADHD symptoms, problem behaviors, social skills, and organizational skills for participating students also support the utility of this adapted treatment in improving student outcomes.

The purpose of the current study was to extend previous findings by comparing pre- and posttreatment outcomes for the larger, complete sample of schools and LSPs and by evaluating impact on a broader array of educationally relevant outcomes. Specifically, in the current study, we examined more objective academic outcomes (teacher-rated subject grade level estimates and report card grades, academic achievement) and classroom observations of learning engagement in addition to parent and teacher ratings of ADHD symptoms, organizational skills, and homework problems. In light of our preliminary findings, we expected positive changes on all measures.

In addition, the current study aimed to evaluate potential mechanisms underlying treatment changes, with the goal of better understanding the role of ADHD symptom change. Theoretical models posit that ADHD and academic skills are related through the mediating effects of academic enablers, such as organizational/study skills (Volpe et al., 2006). In this study, we examined mediational models to evaluate links between changes in ADHD symptom severity, organizational skills, and educationally relevant outcomes at school and at home. We expected that reductions in ADHD symptom severity would contribute to changes at home and at school, and that these effects would be explained by improvements in organizational skills.

Method

Participant Characteristics

Participants included 57 children in Grades 2 through 5 (mean age = 8.1 years) across nine schools in a California urban public school district and 10 LSPs (one school participated twice with a different school clinician each time). Seventy percent were boys. Participant race/ethnicity distribution was as follows: Caucasian, 40%; mixed race/ethnicity, 21%; Asian or Pacific Islander, 14%; African American, 12%; Hispanic/Latino, 11%; American Indian, 2%. Twenty-six percent of children were from single-parent families. Education level of the primary parent participating in the treatment was as follows: 2% (1 parent) had not completed high school, 14% had a high school degree, 37% had some college, and 47% had a college degree.

All participants met screening criteria for ADHD as described in the next section. Approximately 49% of the sample (distributed similarly across the schools/cohorts) also met symptom count criteria for oppositional defiant disorder (ODD), by having four or more symptoms endorsed as often or very often per parent or teacher report on the Child Symptom Inventory (Gadow & Sprafkin, 1994). Seven percent (n = 4) of participants were taking medication for attention or behavior concerns. Participating schools averaged 372 students (range, 252 to 554) in Grades K through 5. Within these schools, 38% (range, 16.3 to 70.6%) of students qualified for free or reduced lunch.

Participant Recruitment and Screening Procedures

Students were identified by LSPs for the program due to concerns about academic and social problems related to inattention and/or hyperactivity-impulsivity. Children taking medication were eligible as long as their regimens were stable. Students with significant visual or hearing impairments, severe language delay, psychosis, or pervasive developmental disorder, or who were in full-day special classrooms, were excluded.

Families and teachers of identified students were contacted by the school’s LSP. University of California, San Francisco (UCSF), staff completed telephone screenings with parents and teachers about the child’s academic, social, and behavioral functioning. Participants met the following eligibility criteria: (a) presence of ADHD symptoms (i.e., six or more inattention symptoms and/or six or more hyperactive/impulsive symptoms endorsed on the Child Symptom Inventory (CSI) by either the parent or teacher as occurring often or very often); (b) cross-situational impairment (home and school), documented as a score of 3 or greater in at least one domain of functioning on both parent and teacher Impairment Rating Scales (Fabiano et al., 2006); (c) a Full Scale IQ equivalent of >79 on the Wechsler Abbreviated Scale of Intelligence (WASI; (Wechsler, 1999); (d) a caretaker available to participate in treatment; and (e) a primary classroom teacher who agreed to participate in the classroom component of the intervention.

Consent forms (parent and teacher) and an assent form (child), approved by the UCSF Committee on Human Research, were completed by parents, teachers, and children. Parents and teachers were paid $50 for completing measures at each time point. Teachers received $100 for participating in program development meetings.

A total of 67 students were referred to the program by LSPs. Seven students were excluded: two students initiated psychoactive medication at the start of the study, three students had ADHD symptoms below the eligibility threshold, and parents of two students declined participation due to scheduling. Three families discontinued early: one family moved away, one dropped out due to perceived lack of need, and one dropped out due to medical problems. Data are reported on the 57 participants who completed the program.

LSP Background

LSPs were masters-level mental health clinicians who led student support services at their respective schools. All but two worked half time and implemented study interventions as part of their San Francisco Unified School District (SFUSD) responsibilities. LSPs received extended calendar pay, at a rate similar to their SFUSD salary, for attending training and program development meetings that occurred outside of their salaried positions.

Study Design and Description of CLS Treatment Components

The CLS program was developed and implemented across staggered (fall and/or winter) cohorts of two schools each over a 3-year time period (see Pfiffner et al., 2011 for details about the development process). All CLS components (group behavioral parent training, classroom behavioral intervention, and child skills group) were led by LSPs and implemented concurrently.

Classroom component

The classroom intervention consisted of a school–home daily report card (Classroom Challenge [CC]) and a homework plan. Additional classroom accommodations were implemented on an individual basis as appropriate (e.g., preferential seating, targeted use of praise, providing prompts to improve student compliance). Each student’s CC included two to three target behaviors (e.g., academic work, classroom deportment, social interactions) rated up to three times per day. Stars earned for meeting target goals were exchanged for daily home rewards and also brought to the child group each week for group-based reinforcement. Target behaviors were refined throughout the 12-week intervention period during two or three individual 30-min CC meetings (attended by teacher, parent, student, and LSP).

Parent component

The parent component included ten 1-hr group sessions comprised of modules to teach skills covered in traditional parent training programs, including effective use of commands, rewards, and discipline, and strategies for managing areas of difficulty commonly associated with ADHD covered in the child group (e.g., homework time, organization, independence in completing daily routines, peer interactions and social skills, stress management for parents).

Child skills component

This component included ten 40-min group sessions held during the school day. Modules targeted social functioning and independence (Pfiffner & McBurnett, 1997). Social skills modules included good sportsmanship, accepting consequences, assertion, dealing with teasing, problem solving, self-control, and friendship making. Independence modules included homework skills, completing chores and tasks independently, and establishing and following routines. Skills were taught through didactic instruction, behavior rehearsal, and in vivo practice. A reward-based contingency management program was utilized to manage child behavior, encourage active group participation, and reinforce new skills. To facilitate generalization, children also earned stars and rewards for accomplishing target goals at home and school.

LSP Training

LSPs attended group training sessions during the summer and fall (Years 1 and 2, four to eight 2-hr meetings; Year 3, one-day 6-hr group meeting) preceding their implementation of the program. A UCSF clinician-trainer (PhD level with several years of experience administering the clinic-based intervention from which the CLS was adapted) attended each session to complete fidelity measures and to model the curriculum, if needed. Weekly group and individual supervision reviewed manual content, viewed selected videotapes of sessions, role-played key aspects of treatment implementation, and troubleshot emergent problems. LSP content knowledge was assessed through quizzes that accompanied each manual section.

Process and Fidelity Measures

UCSF clinician-trainers rated LSP adherence to session content (coverage of each content item rated as not at all, partially, or fully) and implementation quality (competence of delivery, rated 1 = not at all to 5 = great deal). LSPs at least partially covered 85.7% of parent session elements and 93% of child session elements with moderate to high levels of competence (mean 4.0 for parent group and 4.3 for child group). Fidelity of teacher implementation included the number of days the CC was completed (a behavioral product based on a count of the completed CC forms) and LSP ratings of teacher’s adherence (1 = not at all to 5 = great deal). Teachers used the CC an average of just over 4 days per 5-day week (M = 4.1) and their overall adherence to the program was rated as being high by the LSPs (M = 4.3). Parent implementation of strategies taught during groups was measured through weekly self-ratings of strategy utilization frequency (1 = no days to 5 = every day), parent signatures on the daily CC, and UCSF clinician-trainer ratings of parent overall adherence to the treatment program (1 = not at all to 5 = great deal). Parents reported using the strategies taught during the parenting group just over half the time (M = 3.7). They self-reported using strategies to support the CC on most days (M = 4.2), and parent signatures were obtained on more than 80% of the CCs collected. Trainer ratings of parent’s overall adherence to the program averaged 4.1.

Satisfaction Measures

At posttreatment, parents, teachers, children, and LSPs rated items intended to reflect their satisfaction with treatment on questionnaires that were developed by our research team (Pfiffner et al., 2011). Parent, teacher, and LSP satisfaction with CLS was high. The vast majority of parents and teachers (>90%) rated the program as appropriate or very appropriate for treating children’s attention, academic, and social skills problems; were satisfied or very satisfied with the services received; and would recommend or strongly recommend the program to others (all of these ratings are one of the two most favorable options on a 5-point scale). Students (98%) reported they liked the group and/or learned a lot (most favorable option on a 5-point scale). All LSPs rated the overall quality of the program as very high (on a 5-point scale from very low to very high).

Attendance

Parent attendance at groups averaged above 80% (range, 57% to 99%); child attendance averaged above 90% (range, 70% to 100%). Students averaged two to three teacher/family meetings.

Student Outcome Measures

The following measures were completed for all participants pre- and posttreatment:

ADHD symptoms

Teachers and parents completed the CSI (Gadow & Sprafkin, 1994). The ADHD scale items correspond to ADHD symptoms outlined in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) and are rated on a 4-point scale (never, sometimes, often, very often). The CSI contains normative data and acceptable test–retest reliability and predictive validity for ADHD (Gadow & Sprafkin, 1997). In our sample, internal consistency was high for both parent and teacher versions of the CSI (alphas above .8); total ADHD scale scores were used in analyses.

Academic functioning

Parents completed the 20-item (rated on 4-point scale) Homework Problems Checklist (HPC; Foley & Epstein, 1993); total scores were used in analyses. Teachers completed the Academic Competence Evaluation Scale (ACES; DiPerna & Elliott, 2000). The Academic Skills scale includes 33 items rated on a 5-point scale comparing the student’s performance with grade level expectations (far below, below, grade level, above, far above) in the areas of reading/language arts, math, and critical thinking. Total scores from this scale were used in analyses. Both the ACES and the HPC have adequate psychometric properties, including test–retest reliability and internal consistency (alpha at or above .94).

Organizational skills

Teachers and parents completed the Children’s Organizational Skills Scale (COSS; Abikoff & Gallagher, 2009). Items are rated on a 4-point scale (1 = hardly ever or never to 4 = just about all the time). Items assessing organizational skills, management of materials/supplies, and task planning skills (parent = 58 items, teacher = 38 items) were totaled for analyses, with lower scores indicating better organizational skills. Both parent and teacher versions have adequate psychometric properties, including internal consistency (α = .94 for both versions).

School grades

Grades were obtained from SFUSD report cards. SFUSD utilizes a Standards Based Report Card system that rates skills students are expected to learn in each subject per grade level, where 1 = needs more time/practice to develop, 2 = approaching the standard, 3 = meets the standard, and 4 = exceeds the standards. Complete report cards were not available from six participants due to school transfers or teachers not using the standard reporting system.

Academic achievement

Four subtests from the Woodcock Johnson Tests of Achievement (3rd ed.; WJ-III; Woodcock, McGrew, & Mather, 2007) were administered: Paragraph Comprehension, Reading Fluency, Math Fluency, and Math Calculation. All have adequate psychometric properties, including test–retest reliability and internal consistency (alpha at or above .86). Subtest raw and standard scores were used in analyses.

Behavioral Observation of Students in Schools (BOSS)

Students were observed on three occasions during academic instruction in their classroom for 15 min over a 2-week period before and after treatment. The BOSS (Shapiro, DuPaul, Bradley, & Bailey, 1996) was used to code the following classroom behaviors: Active Engaged Time, Passive Engaged Time, Off-Task Motor, Off-Task Verbal, and Off-Task Passive (Junod, DuPaul, Jitendra, Volpe, & Cleary, 2006). Active and passive engagement categories were combined to form an engagement composite that was used in these analyses, with higher scores indicating a greater percentage of time engaged (“on task”). Research assistants (RAs) trained in the BOSS obtained 80% or above agreement on the coding categories with the BOSS training videotape and in live observations in the classroom before actual data collection. RAs overlapped on 22% of their classroom observations; interobserver agreement (kappa) for the engagement composite was .86. RAs were blind to group assignment and the purpose of the study. BOSS data are missing from two students due to absences during the observational period.

Results

In order to account for the nonindependence that resulted from the nested data structure (i.e., students within schools) and provide more accurate estimates of standard errors, all regression models were estimated using sampling weights (as described by Asparouhov, 2005) in Mplus version 5.21 (Muthén & Muthén, 2010). The analyses were considered exploratory and are interpreted based on an uncorrected family wise error rate (α); exact p values are reported, where appropriate, in order to allow for more conservative interpretation of these results. Given the number of mean comparisons being performed, a conservative Bonferroni adjustment would yield a family wise error rate of α = .05/13 = .004. Cohen’s d was calculated as a measure of the magnitude of baseline and posttreatment mean differences and was adjusted for the correlation between baseline and posttreatment means that resulted from the repeated measures design using Equation 8 reported in Morris and DeShon (2002):

$${\mathrm{\mu }}_{\mathrm{D}}/\mathrm{\left(}\sigma \surd \left[2\left(1-\mathrm{\rho }\right)\right]\right)$$

where μ_D is the mean difference between postintervention and preintervention scores, σ is the standard deviation of μ_D, and ρ is the correlation between preintervention and postintervention scores. Change in baseline and post-treatment scores was also calculated using continuous change scores (e.g., the absolute value of the posttreatment score minus baseline score). Finally, the reliable change index (Jacobson & Truax, 1991) was calculated as an indicator of the clinical significance of ADHD symptom reduction. The index is calculated using the following formula:

$$\mathrm{RC}=\left({\mathrm{X}}_1-{\mathrm{X}}_2\right)/S{E}_{\mathrm{diff}}$$

where SE_diff represents the standard error of measurement of the difference between two scores, X₁ represents an individual’s pretreatment score, and X₂ represents an individual’s posttreatment score in which improvement is indicated by a decrease in scores. In the case of academic skills, in which increases in scores represent improvement, X₁ and X₂ would be in reverse order.

Pre–Post Student Outcomes

Baseline and posttreatment means and standard deviations for each measure are presented in Table 1, along with effect sizes, Z statistics, and p values for each mean comparison. Statistically significant improvement occurred on all measures from baseline to posttreatment. Effect sizes were large for parent and teacher ratings of ADHD symptom severity, parent ratings on the HPC and COSS, and medium to large for teacher ratings on the COSS. A substantial proportion of children showed reliable improvement following the intervention on parent (49%) and teacher (53%) reported ADHD severity. Moreover, 51% of children in the sample were in the nonclinical range for ADHD symptom count at posttreatment (i.e., had fewer than six symptoms in both the inattentive and hyperactive-impulsive symptom groups, as reported by parents and teachers). Measures of academic achievement (WJ-III), academic performance (language arts and math report card grades), and percentage of time spent in on-task behavior in the classroom (BOSS Engagement composite) all resulted in effects in the medium to large range. In addition, the standard score for the math calculation subtest showed significant pre–post improvement (p < .001); standard scores for the other WJ subtests did not show significant change. Effect sizes for the pre–post changes on teacher ratings of academic skills were small to medium and comparisons between decile levels showed that students’ mean scores improved a full decile (from 3 to 4), indicating that the scores increased relative to a normative population.

Table 1.

Baseline and Posttreatment Comparisons for Student Outcome Measures

	Baseline mean (SD)	Post mean (SD)	ES	Z value (95% CI)	p value
CSI ADHD symptom severity (P)	1.76 (.54)	1.19 (.56)	1.09	−7.69 (−9.70, −5.75)	p < .001
COSS (P)	159.63 (20.51)	139.12 (23.15)	.93	−6.80 (−8.76, −4.84)	p < .001
HPC (P)	52.77 (10.46)	41.54 (12.00)	.89	−5.87 (−7.83, −3.91)	p < .001
CSI ADHD symptom severity (T)	1.74 (.55)	1.14 (.52)	1.23	−11.90 (−13.92, −9.98)	p < .001
COSS (T)	95.23 (13.93)	85.45 (15.68)	.71	−5.50 (−7.46, −3.86)	p < .001
ACES-Academic skills (T)	82.48 (23.70)	87.35 (24.40)	.41	3.12 (1.16, 5.08)	p = .002
WJ-Reading fluency	33.19 (12.70)	36.35 (13.39)	.52	3.70 (1.74, 5.66)	p < .001
WJ-Passage comprehension	24.75 (4.79)	26.95 (4.83)	.63	2.76 (.59, 4.72)	p = .006
WJ-Calculations	13.49 (3.96)	15.74 (4.58)	.81	3.70 (1.74, 5.66)	p < .001
WJ-Math fluency	41.82 (16.11)	46.30 (19.37)	.51	6.97 (5.02, 8.94)	p < .001
BOSS-Engagement	64.43 (14.99)	74.77 (13.46)	.70	5.43 (3.47, 7.39)	p < .001
Language arts report card grades	2.07 (.59)	2.56 (.63)	1.24	5.90 (3.96, 7.89)	p < .001
Math report card grades	2.11 (.56)	2.47 (.68)	.77	3.58 (1.63, 5.56)	p < .001

Note. ACES = Academic Competence Evaluation Scale; ADHD = Attention-Deficit/Hyperactivity Disorder; COSS = Children’s Organizational Skills Scale; CSI = Child Symptom Inventory; ES = Cohen’s d effect sizes; HPC = Homework Problems Checklist; P = parent-completed; SD = standard deviation; T = teacher-completed; WJ = Woodcock-Johnson Test of Academic Achievement.

Mediation Analyses

Mediation analyses were conducted to examine the mediational role of improvement in organizational skills in the relationship between improvements in ADHD symptom severity and educational functioning, based on contemporary approaches to the Baron and Kenny (1986) method (Frazier, Tix, & Barron, 2004). This requires that paths a, b, and c are statistically significant, and that path c′ is reduced in magnitude when considered simultaneously with a mediating variable (see Figures 1 and 2). Paths a and c were computed using zero-order correlations. Paths b and c′ were estimated using multiple linear regression analyses. Finally, indirect effects were calculated (a*b) and tested for statistical significance using the following formula for the Sobel test, as described by MacKinnon and colleagues (MacKinnon, Lock-wood, Hoffman, West, & Sheets, 2002):

$$\mathrm{ab}/\left(b^2{s}_a^2+a^2{s}_b^2\right)$$

where s_a represents the standard error of regression coefficient a, and $s_b^2$ represents the standard error of regression coefficient b.

Figure 1.

Mediation model for teacher-reported variables.

*p < .05.

Figure 2.

Mediation model for parent-reported variables.

*p < .05.

The first model tested whether or not improvement in organizational skills (targeted in CLS) mediated the relationship between improvements in ADHD symptom severity and the school-based index of educational functioning, that is, overall academic skills per teacher reports. Table 2 displays zero-order correlations between each of the teacher-reported variables and indicates that improvement in ADHD symptom severity was strongly associated with improvement in organizational skills (path a), and significantly and moderately associated with academic skills (path c). Paths b and c′ (see Table 3 for simultaneous model results) showed significance, indicating that improvements in ADHD symptom severity and organizational skills accounted for approximately 18% of the variance in improvement in academic skills. Although improvement in organizational skills significantly predicted improvement in academic skills and had a moderate to large effect size (β), improvement in ADHD symptom severity was not significantly related to improvement in academic skills in this model. These results satisfy the criteria of the traditional Baron and Kenny (1986) approach to testing full mediation. Sobel testing supported this finding, resulting in a significant indirect effect indicating that the relationship between improvement in ADHD and improvement in academic skills was fully explained by improvement in organizational skills.

Table 2.

Zero-Order Correlations Among Parent and Teacher Variables

	ADHD symptom severity	Organizational skills	Academic skills
Teacher measures
ADHD symptom severity	1.00
Organizational skills	.53* (path a)	1.00
Academic skills	.26* (path c)	.42*	1.00
Parent measures
ADHD symptom severity	1.00
Organizational skills	.56* (path a)	1.00
Homework problems	.61* (path c)	.63*	1.00

Note. ADHD = Attention-Deficit/Hyperactivity Disorder.

^*p < .05.

Table 3.

Simultaneous Multiple Linear Regression Model Results

	Academic skills R2= .18	Homework problems R2= .51
Organizational skills (path b)	β = .39*	β = .42*
ADHD symptom severity (path c′)	β = .06	β = .38*
Indirect Effects (path a*b)	.21*	.24*

Note. ADHD = Attention-Deficit/Hyperactivity Disorder.

^*p < .05.

The second model tested whether improvement in organizational skills mediated the relationship between improvements in ADHD symptom severity and the home-based index of educational functioning (homework problems). Table 2 displays zero-order correlations between each of the parent-reported variables and indicates that improvement in ADHD symptom severity was strongly associated with improvements in organizational skills (path a) and homework problems (path c). Paths b and c′ showed significance (see Table 3 for simultaneous model results). Improvements in ADHD symptom severity and organizational skills accounted for approximately 51% of the variance in improvement in homework problems. Improvements in organizational skills (path b) and ADHD symptom severity (path c′) both significantly predicted improvement in homework problems, with moderate to large effects (βs). The effect of improvement in ADHD symptom severity on improvement in homework problems was attenuated in this model relative to the bivariate model. These results satisfy the criteria of the traditional Baron and Kenny (1986) approach to testing partial mediation. Sobel testing supported this finding, resulting in a significant indirect effect, indicating that the relationship between improvement in ADHD symptoms and in homework problems was partially explained by improvement in organizational skills.

Discussion

Results from this study show significant improvement in ADHD symptoms, organizational skills, and homework problems per parent and teacher report in a larger sample of participants and schools than previously reported (Pfiffner et al., 2011). Improvement was substantial; effect sizes for these outcomes uniformly were in the large range. Clinical significance was demonstrated by reliable improvement in ADHD symptoms as well as symptom counts below clinical cutoffs for just over half the sample at posttreatment. This study adds to the previous study by identifying significant pre- to post-treatment improvement on multiple and ecologically valid measures of academic outcomes, including teacher grade-level estimates across academic subjects, report card grades in language arts and math, as well as on objective measures of academic achievement in reading and math fluency, passage comprehension, and math calculation. Student engagement in classroom activities (objectively observed) also improved significantly. Taken together, these findings indicate that positive outcomes are not merely a function of parent or teacher reporting bias. The degree of positive impact (assessed via effect sizes) for the CLS intervention is in line with findings from a meta-analysis of behavioral treatments mostly implemented in clinics and/or by research study staff rather than nonschool personnel (Fabiano et al., 2009), suggesting that the CLS program succeeded in adapting an evidence-based psychosocial treatment for ADHD to the public school setting.

Effects on measures of academic achievement were not as large as effects for the other measures. This is not surprising, given that the measures that are more proximal to the treatment (e.g., parent and teacher behavior ratings) would be expected to be impacted first, and that gains on measures more distal to the treatment, such as academic achievement, would accrue over a longer period of time. We are encouraged by finding the gains in academic achievement that we did, particularly in the area of math calculation, which showed a significantly increased standard score (greater than would be expected for the time period). These findings, along with gains in decile scores for teacher estimates of academic skills, further support the real-world educational significance of the CLS effects.

Mediational analyses suggested that although decreases in ADHD symptom severity are associated with increases in teacher’s grade level estimates of academic skills, improvements in organizational skills fully account for this relationship. In the home setting, improvements in organizational skills partially explain the relationship between improvement in ADHD symptom severity and improvement in homework problems. In other words, it appears that, in both school and home settings, improvements in academic skills that result from improvements in ADHD symptom severity can be attributed, at least in part, to improvements in organization. These findings support the CLS focus on teaching and reinforcing independence and organizational skills, and support a model that ADHD affects academic outcomes via organizational skills and related executive functions (Volpe et al., 2006). These findings are consistent with those of other treatment approaches (Abikoff et al., in press; Evans et al., 2009; Langberg et al., 2008), which prioritize skill building and/or treatment for specific impairments rather than having a singular focus on ADHD symptom reduction. Reduction in ADHD symptom severity appears to have a partially direct link to reduction in homework problems. This supports the potential value of directly targeting ADHD symptoms in addition to organizational skills. In the CLS intervention, home programs commonly targeted ADHD symptoms specific to the homework setting, such as attending to tasks, carefully checking details to avoid careless mistakes, and getting started right away on effortful tasks. This intervention focus may well have contributed to our findings.

Limitations

As this study was designed to be a series of open trials aimed toward developing the treatment and evaluating feasibility and initial outcomes, rather than the most stringent possible test of efficacy, it is possible that changes in student outcomes were due to factors such as time, maturation, and/or nonspecific treatment effects rather than the CLS program. Improvement on objective measures reduces the likelihood that the treatment effects were simply due to treatment expectancies, and the magnitude of improvement is certainly more encouraging than the typical course for ADHD (which tends to worsen with time in this age range). A randomized controlled trial testing the efficacy of CLS against existing school practices or other treatment approaches will be an important next step in evaluating this intervention.

Our findings are limited to short-term effects. Further research on efficacious methods for programming maintenance across time and settings is crucial for addressing long-term needs of students with ADHD.

We did not find CLS effects to vary significantly by specific racial/ethnic backgrounds. However, our sample size was insufficient to definitively evaluate such differences. We also note that our sample includes a sizable percentage of college-educated parents. It is possible that this factor contributed to our high rates of parent attendance, treatment acceptability, and adherence to the program. However, our sample of noncollege-educated parents is insufficient to draw firm conclusions. Future studies are needed to determine if the program will yield similar results across a broad range of parent education levels.

Conclusions and Implications

These findings extend support for this school-implemented, collaborative school–home intervention for ADHD. CLS was associated with improvements in ADHD and multiple measures of important educational outcomes, including teacher ratings of academic skills, parent ratings of homework, objective classroom observations of student engagement, ecologically valid school-generated report cards, and tests of academic achievement. Our findings highlight the role of organizational skills in facilitating improvements in both teacher-assessed academic skills and parent-assessed homework outcomes. The underlying conceptualization of CLS is that teachers and parents learn strategies to promote children’s engagement, motivation, and self-control, and children learn social, organizational, and daily living skills, which are reinforced by teachers and parents. This approach addresses common limitations of single-setting behavioral interventions by directly programming generalization across settings.

The findings and acceptability of the study support the feasibility of CLS for school-based clinicians who were able to provide, with good fidelity, the 2 to 3 hr of direct clinical contact per week for the 12-week period required by the program. A complete cost analysis is not yet available and will be important for evaluating sustainability. Our goal has been to achieve a balance between treatment intensity and costs. To this end, clinician and teacher time commitment for implementing the program was not increased beyond their contracted school hours. Although training and supervision sessions for clinicians provided after school hours were paid through grant funds, this time could be incorporated into clinician schoolwork hours on a one-time basis during their initial implementation of the program.

Future efficacy studies are needed to compare CLS effects against typical school practices. It is hoped that these initial findings portend well for eventual studies of the cost effectiveness of CLS, especially considering the current high costs of ADHD to school systems, the ever-decreasing allocation of funds toward education, and the consequent need to prioritize services with a strong evidence base.