Additive Effects of Parent Adherence on Social and Behavioral Outcomes of a Collaborative School–Home Behavioral Intervention for ADHD

Abstract

The present study evaluated the impact of the Collaborative Life Skills Program (CLS), a novel school–home psychosocial intervention, on social and behavioral impairments among children with attention and behavior problems. Fifty-seven ethnically/racially diverse children (70 % boys) with attention and/or behavior problems in the second through fifth grades participated in a pilot study. Ten school-based mental health professionals were trained and then implemented the intervention at their respective schools. Children significantly improved from pre- to post-treatment on parent, teacher, and report card ratings of children’s social and behavioral functioning. Treatment improvements were consistent for children with and without co-occurring disruptive behavior problems. The impact of the intervention was enhanced when parents used the intervention strategies more regularly, according to both clinicians’ and parents’ reports. Findings support the emphasis of CLS on coordinating intervention strategies across contexts to facilitate the generalization of treatment-related improvements in social and behavioral functioning.

Introduction

Attention deficit/hyperactivity disorder (ADHD), is characterized by pervasive symptoms of inattention (e.g., difficulty completing tasks) and/or hyperactivity (e.g., difficulty sitting still) and impulsivity (e.g., difficulty waiting turn) that lead to a variety of social and behavioral problems for affected children across school, home, and peer contexts [1]. In the school context, these symptoms often result in more off-task, verbal and physical disruptive behavior, interruption of instruction, and non-compliance [2]. At home, children with ADHD often are reprimanded for non-compliance with household rules and experience elevated conflict with their parents relative to their non-ADHD peers [3]. Among peers, the often overbearing and impulsive behavior of children with ADHD may result in social alienation from classmates who are annoyed by both these behaviors and their consequences (e.g., teacher frustration, loss of group rewards, etc.).

These particular social and behavioral deficits are exacerbated by high rates of comorbidity with more severe disruptive behavior problems, such as oppositionality, aggression, bullying, and rule-breaking behaviors [4, 5]. In fact, comorbidity rates between ADHD and Oppositional Defiant Disorder (ODD) have been estimated to be approximately 60 % [1, 6, 7]. Moreover, children with ADHD accompanied by ODD behaviors have been distinguished as having increased levels of aggression, impaired functioning in their relationships with peers, and conflict with parents and teachers, in addition to a variety of other impairments [4, 5, 8]. The overlap in the development of these sets of problems is particularly concerning in light of findings that children with co-occurring ADHD and ODD problems are at an increased risk for persistent ADHD, early onset and persistent conduct problems, and substance use, among other consequences [9–12].

Psychosocial Interventions for Children with ADHD

A large body of research supports the efficacy of behavioral parent training and teacher consultation/classroom behavior modification for improving the behavioral functioning of children with ADHD in the home and school contexts, respectively [13–16]. Likewise, substantial evidence indicates that these approaches are effective for ameliorating symptoms and impairments associated with ODD, as well as co-occurring ADHD and ODD problems [13, 16, 17]. Despite the relative successes of these behavioral intervention strategies in their respective contexts, studies have reported that among children with ADHD, improvements from individual intervention approaches tend to be context specific [16, 18, 19]. For example, social-behavioral improvements that result from parent training interventions have limited effects at school [19] and similarly social-behavioral gains from interventions with a school emphasis often fail to generalize to the home [18]. Given the impairments that children with ADHD often manifest across contexts, it appears important for behavioral interventions to be implemented across the settings in which children spend the majority of their time (i.e., school and home).

Considerably fewer studies have evaluated cross-setting, multi-component behavioral interventions in comparison to studies of single setting interventions [13, 15, 16, 20]. One such multi-component study is the large-scale Multimodal Treatment Study of ADHD [21] which included a behavioral intervention having both school and home components. This intervention resulted in greater improvement in homework problems and negative parenting than a community control group [22, 23], but no differences on school-based measures between these two groups were found. The fact that the majority of youth in the community control condition were taking medication may have attenuated group differences. More recently, interventions with an explicit focus on coordinating and integrating home and school components have been developed and evaluated. In a study with children having disruptive behavior problems, Sheridan et al. [24] demonstrated that an intervention promoting a collaborative partnership between parents and teachers, conjoint behavioral consultation (CBC), resulted in greater improvements in children’s social skills at school and at home and in parent-teacher relationships relative to usual school services. Power et al. [25] combined CBC approaches with a homework intervention, daily report card and general parenting skills (referred to as Family-School Success, FSS). They found improvements in homework problems and the quality of parent-child and parent-teacher relationships relative to an educational and supportive intervention which did not include a collaborative home-school component. In further support of collaborative treatment approaches, both of these studies reported that improvements in the quality of parent-teacher relationships partially accounted for the improvements observed in children’s functioning [24, 25]. Despite these positive outcomes, effect sizes of these treatments on social-behavioral outcomes have been relatively modest, with little to no impact on ADHD symptoms and other externalizing problems. A family-school intervention developed by Owens et al. [26] which did find treatment-related improvements for ADHD symptoms in addition to functional impairment, found these effects only at school, not at home.

In addition, both parent training and classroom interventions have had limited impact on social functioning/ peer relations of youth with ADHD [14]. While some evidence of improvement in social functioning has resulted from several social skills training programs [27–29], improvements have been inconsistent across social domains and studies [19, 28, 30–33]. One explanation for the inconsistency is that children with ADHD may have deficits in their social performance, or the consistent application of social skills across situations, which are not addressed through didactic instruction in the absence of real-world reinforcement [34]. Indeed, there appear to be differences in the impact of interventions that reinforce the acquisition and application of social skills in the situations in which they will be applied, relative to interventions that only teach skills in a didactic setting [14, 35]. For example, studies of intensive Summer Treatment Programs, in which children’s behaviors are reinforced during interactions with peers across a variety of settings (e.g., classroom, recreational, etc.) have reported improvements in teacher, counselor, and observer ratings of children’s interactions with peers [36, 37]. In addition, studies that emphasize the role of parents as facilitators of children’s acquisition and implementation of social skills through coaching and reinforcement have shown improvements in social skills, peer interactions, and social acceptance [27, 29, 33]. Another source of inconsistent treatment effects appears to be related to the comorbidity of ADHD and ODD problems [28]. Although Frankel et al. [29] found that social functioning among children with ADHD improved as a result of a social skills intervention regardless of ODD comorbidity, Antshel and Remer found that children with comorbid ADHD and ODD had less improvement in their social functioning relative to children with ADHD and no ODD comorbidity.

An important issue with empirically-validated behavioral treatments for ADHD is that these treatments are almost exclusively clinic-based or administered by research teams in schools [14]. These implementation approaches usually rely on single contexts (home or school) and do not systematically apply strategies for generalizing treatment gains across settings [38, 39]. Moreover, both approaches have inherent problems with regard to accessibility and sustainability.

Collaborative Life Skills Program

In order to address longstanding problems of limited cross-setting generalizability and to improve accessibility and sustainability of treatments, the Collaborative Life Skills (CLS; [40, 41]) program was developed. This program was adapted from a clinic-based intervention for ADHD, Predominantly Inattentive type [42], to be implemented in schools, by existing school mental health personnel and to target children with the full range of ADHD behaviors. CLS includes simultaneous delivery of adaptations of three empirically-supported treatment approaches over a 12 week intervention period: behavioral teacher consultation and use of daily report cards (e.g., [13–15, 43]), behavioral parent training [13, 14], and child social and life skills training [27, 42]. Child skills specifically address areas of knowledge and/or performance deficit for ADHD and are coordinated to coincide with parent content covering strategies to address/support these skills and parents, children, and teachers are provided with common terminology to use across contexts in order to maximize support for the acquisition of all intervention skill modules across settings. To further reinforce the cross-contextual generalization of the intervention effects, reinforcement contingencies are set both within and across settings where possible (e.g., parents reward behaviors that occur at home and school, school clinicians reward behaviors that occur at home, school, and group, etc.). This approach facilitates an active partnership among parents, teachers, and children and is intended to motivate their active participation in the program. These specific ingredients of CLS, namely, (1) utilization of evidenced based parent and classroom interventions, (2) direct involvement of the child in a skills-based approach, and (3) careful coordination of all treatment components, was designed to address the limitations of existing single context parent and classroom interventions and to provide more potent treatment effects than existing multi-component interventions. Indeed, results of initial CLS evaluations have provided evidence of larger treatment effects on ADHD symptoms (Cohen’s ds = 1.09 for parent ratings and 1.23 for teachers; [41]) relative to the effect for pre-post behavioral intervention studies (d = .70) reported by Fabiano et al. [13]. Additionally the effect size of the clinic-based version of CLS on inattention symptoms from our controlled study was .97 which compares favorably to between-group effects of behavioral treatments, mostly from effects in the settings in which the treatment was administered (d = .83) (Fabiano et al. [50]).

The intervention was adapted for school-based implementation using a collaborative iterative development process, in which existing school-based mental health professionals (learning support professionals, LSPs) were trained to implement and coordinate all three components of the intervention under the supervision of trained clinicians. Initial published results describe the iterative development process in depth and provide evidence of the program’s fidelity, feasibility, and acceptability across multiple successive school cohorts [40, 41]. Preliminary findings from the first three cohorts of students (N = 34) showed improvement in problem behaviors and social skills [40]. Using these data, as well as data from two additional cohorts of subjects (Total N = 57) which represented the full sample of schools, Pfiffner et al. [41] reported significant pre- to post-treatment improvement in ADHD symptom severity, organizational skills, observations of classroom behavior and academic functioning across the school and home contexts. Pfiffner et al. [41] also reported evidence that parents and teachers were adherent to the treatment, attended group sessions, and reported being satisfied with the intervention.

The purpose of the current study was to provide additional evidence in support of the feasibility and impact of school mental health professionals implementing and coordinating CLS across the school and home contexts. The primary goal was to extend the findings of Pfiffner et al. [41], using data from the same five cohorts, to examine the impact of the CLS program on child social and behavioral functioning for the complete sample of CLS schools and LSPs, using parent and teacher ratings of children’s social and interpersonal functioning, behavior problems, severity of ODD symptoms, and impairment in important relationships (i.e., with peers, parents, and teachers) as outcome variables. In addition, improvements in social development as indicated by report card grades provide real-world outcome data. Given improvements in these areas documented by our preliminary data from the first three cohorts [40], as well as the pattern of improvement in other domains that have resulted from this intervention across all five cohorts [41], we expected that children would exhibit statistically and clinically significant improvement on all social and behavioral outcomes evaluated. More specifically, we expected effect sizes to be moderate to large in magnitude (e.g., >.50), as well as reliable improvement for a substantial proportion (≥50 %) of the sample and clinical recovery to the normal range for a statistically significant proportion of the sample on all parent and teacher reported social and behavioral outcomes. We also expected that a statistically significant proportion of children would no longer meet symptom count criteria for ODD following treatment.

A secondary aim of the present study was to determine whether or not the impact of the CLS program differed between children who had ODD problems in addition to ADHD problems and children who had ADHD problems only. Although previous evidence has generally supported the impact of psychosocial interventions for children with co-occurring ADHD and ODD problems, the impact of these interventions, and social skills training interventions in particular, on social outcomes in the peer context has been less consistent. Nevertheless, given the coordination of behavioral intervention strategies across treatment components of the CLS program, in addition to the emphasis on the acquisition of skills in a real-world setting, we anticipated that children with ADHD problems only and children with co-occurring ADHD and ODD problems would have statistically significant improvement in their social and behavioral functioning across all parent and teacher reported outcomes.

In addition, the present study sought to extend previous findings in support of the intervention fidelity by exploring the impact of parents’ adherence to the intervention on children’s social and behavioral improvements. Behavioral interventions are most effective when they are implemented consistently and as they were designed to be implemented (i.e., treatment integrity/fidelity; [44–47]). Indeed researchers have reported that better treatment adherence is associated with greater improvements in treatment outcomes for children [46, 47]. Given these findings, it was hypothesized that greater parent adherence to the intervention and use of the intervention strategies would be independently and moderately related to greater improvement in child ADHD and ODD symptom severities as well as social and behavioral functioning.

Methods

Participant Characteristics

Participants included 57 children in grades 2–5 (mean age = 8.1 years) across nine schools in a California urban public school district (see Table 1 for more detailed child and family demographic characteristics). Seventy percent were boys, which is consistent with recent estimates of the prevalence of ADHD [48]. Participants were racially/ethnically diverse and were generally representative of children from the school district from which they were recruited. Most children were from two parent homes, and the majority of parents had completed at least some college education. Across participating schools, an average of 38 % of students qualified for free or reduced lunch (range 16.3–70.6 %).

Table 1.

Participant demographic characteristics

Child characteristics	n(%)	Parent characteristics	n(%)
Boys	40 (70 %)	Single parent	15 (26 %)
Age	M= 8.1 (SD = 1.1)	Parent education
Race/ethnicity		Less than HS	1 (2 %)
Caucasian	23 (40 %)	HS diploma	8 (14 %)
African American	7 (12 %)	Some college	21 (37 %)
Hispanic/Latino	6(11 %)	College degree or higher	27 (47 %)
Asian American	8 (14 %)	Income
Native American	1 (2 %)	$20,000 or less	5 (9 %)
Mixed/other	12 (21 %)	$20,001 – $50,000	15 (26 %)
On medication	4 (7 %)	$50,001 – $80,000	11 (20 %)
Baseline number P/T symptoms		$80,001 – $100,000	3 (5 %)
Inattentive	M = 7.8 (SD = 1.5)	$100,001 or more	23 (40 %)
Hyperactive/impulsive	M = 6.1 (SD = 3.0)
ODD	M = 3.4 (SD = 2.6)

P/T symptoms refer to symptoms that were endorsed by either parents or teachers as occurring “often” or “very often” on the CSI. M mean, SD standard deviation

All participants met screening criteria for ADHD as described below. Approximately 51 % of the sample (similarly distributed across the schools/cohorts) also met symptom count criteria for ODD, by having four or more symptoms endorsed as often or very often per parent or teacher report on the Child Symptom Inventory (CSI; 49). Seven percent (n = 4) of participants were taking medication for attention or behavior concerns.

Participant Recruitment and Screening Procedures

All study procedures were approved by the University of California, San Francisco Institutional Review Board. Students were identified by LSPs for the program based on teacher or school concerns about academic and social problems related to inattention and/or hyperactivity-impulsivity. Children taking medication were eligible as long as their regimens were expected to be stable during their participation in the program. Students with significant visual or hearing impairments, severe language delay, psychosis, or pervasive developmental disorder were excluded, as were students enrolled in full day special day classrooms.

Families and teachers of identified students were contacted by the school’s LSP and referred to the university-based project coordinator, who completed telephone screenings with parents and teachers about the child’s academic, social, and behavioral functioning. All children ultimately participating in the program met the following eligibility criteria: (1) presence of clinically-elevated ADHD symptoms (i.e., six or more inattention symptoms and/or six or more hyperactive/impulsive symptoms endorsed on the CSI by either the parent or teacher as occurring often or very often), (2) significant cross-situational impairment (home and school), documented as a score of 3 or greater in at least one domain of functioning on both parent and teacher Impairment Rating Scales [50], (3) a Full Scale IQ equivalent of 80 or above on the Wechsler Abbreviated Scale of Intelligence (WASI; [51]), (4) a caretaker available to participate in treatment, and (5) a primary classroom teacher who agreed to participate in the classroom component of the intervention.

Ethics committee-approved consent forms (parent and teacher) and an assent form (child) were completed by parents, teachers, and children at treatment baseline. Parents and teachers were paid $50 for completing measures pre- and post-treatment. Teachers received $100 for participating in program development meetings.

A total of 67 students were referred to the program by participating LSPs. Seven children were excluded for the following reasons: 2 students initiated psychoactive medication at the start of the study, 3 students had ADHD symptoms below the eligibility threshold, and parents of 2 students declined participation due to scheduling. In addition, three families discontinued early: 1 family moved away, 1 dropped out due to perceived lack of need, and 1 dropped out due to parent medical problems. The data reported are for the 57 cases that completed the program.

LSP Background and Training

Ten masters-level mental health clinicians (LSPs) led student support services at their respective schools (one school participated twice with a different school clinician each time). All but two worked half-time and implemented study interventions as part of their school district responsibilities. LSPs received extended calendar pay at a rate similar to their school district salary for attending training and program development meetings that occurred outside the hours of their salaried positions.

Learning support professionals attended group-training sessions during the Summer/Fall (Years 1 and 2: between four and eight 2-h meetings; Year 3: one 6-h group meeting scheduled on a single day) preceding their implementation of the program. A project clinician-trainer (PhD level with several years of experience administering the clinic-based intervention from which the CLS program was adapted) attended each session to complete fidelity measures and to model the curriculum if needed. Weekly group and individual supervision involved review of selected video from prior sessions, discussion of upcoming treatment content, role-plays of key aspects of treatment implementation, and troubleshooting of any emergent problems. LSP content knowledge was assessed through quizzes that accompanied each manual section.

Study Design and Description of CLS Treatment Components

The CLS program was developed and implemented across staggered (Fall and/or Winter) cohorts of 2 schools each (6 children at each school) over a 3 year time period (see 40, for details about the development process). All CLS components (group behavioral parent training, classroom behavioral intervention, and child skills groups) were led by LSPs and implemented concurrently.

Classroom Component

The primary classroom intervention consisted of a school-home daily report card (herein called a Classroom Challenge, or CC). Each student’s Classroom Challenge (CC) included 2–3 teacher-identified target behaviors addressing academic work, classroom deportment, and/or social interactions that teachers rated up to three times per day. Skills taught during the child group were included as target behaviors on the CC as needed. Students exchanged points earned for meeting their classroom goals for daily home rewards as well as group-based positive reinforcement in the context of the child skills training group. In order to maximize progress, target behaviors were refined and adjusted throughout the 12-week intervention period during an additional two to three individual 30-min troubleshooting meetings (attended by teacher, parent, student and LSP). In addition to the CC, meetings covered skills taught in the child group and homework expectations and procedures in order to ensure congruent parent-teacher expectations for homework process and completion. Additional classroom accommodations were implemented on an individual basis as appropriate (e.g., preferential seating, targeted use of praise, modified workload, additional prompting to improve student compliance).

Parent Component

The parent treatment component included 10 1-h parent group sessions covering traditional behavioral parent training skills; parent content included topics such as effective use of commands, rewards, and discipline, as well as strategies for managing areas of difficulty commonly associated with ADHD covered in the child group (e.g., homework time, organization, independence in completing daily routines, peer interactions and social skills, stress management for parents). Parent groups were conducted at each school either in the morning, after children were dropped off, or in the afternoon, before children were picked up. At each school, at least one parent of each of the participating children attended the group. Parents who missed a group reviewed the content during the next group session or during an individual make-up session.

Child Skills Component

The child treatment component included 10 40-min child group sessions targeting both social functioning and independence (Pfiffner and McBurnett [27]). Groups were held at school, during the school day and included the participating 6 children (grades 2–5) from the school. Social skill modules included: good sportsmanship; accepting consequences; assertion; dealing with teasing; problem solving; self-control; and friendship making skills. Independence modules included: homework skills; completing chores and tasks independently; and establishing and following routines. Skills were taught through didactic instruction, behavior rehearsal, and in vivo practice. A reward-based contingency management program was utilized to manage child behavior, encourage active group participation and reinforce new skills. To facilitate generalization, children also earned points and rewards for display of relevant skills at home and in the classroom. Children absent from group were taught the missing content during the subsequent group session when the skill was reviewed with the other children or during an individual make-up.

Process and Fidelity: (See [41] For Details)

Learning support professionals were adherent to the intervention protocol based on weekly clinician ratings, indicating that they covered 85.7 and 93 % of parent and child session content, respectively with moderate to high levels of competence. Teachers completed CC forms an average of 4 out of 5 school days each week, which was consistent with clinicians ratings of teachers’ adherence to the intervention (mean rating of 4.3 on a scale from 1 = not at all to 5 = a great deal). In addition, supervising clinicians rated parents weekly on their overall perceived adherence to the treatment (based on weekly check-ins/progress reviews during group sessions) on a scale from 1 (not at all) to 5 (a great deal). Parent adherence was supported by signatures on 80 % of CCs that were collected and clinician ratings of parents’ adherence to the intervention protocol (mean rating of 4.1). Each week, parents reported the number of days that they used the strategies learned during the previous session on a scale from 1 (no days) to 5 (everyday). Following the conclusion of the intervention, parents completed a questionnaire in which they rated how often they used the various strategies learned in group (e.g., attending, praising, response cost, etc.) on a scale ranging from 1 (not at all) to 5 (everyday) and how often they “prompted or reinforced” skills learned in the child skills group (e.g., good sportsmanship, problem solving, self-control, etc.) on a scale ranging from 1 (not at all) to 5 (everyday). On average, parents reported using the strategies that they learned in the parent training group during the previous week more than half of the time (mean rating 3.7). At the end of the intervention, parents also reported using the various strategies more than half of the time, on average (mean rating of 3.3) and “prompting or reinforcing” the skills that their children learned in the child life skills groups more than half of the time (mean rating 3.6).

Attendance

Parent attendance at groups averaged above 80 % (range 57–99 %); child attendance averaged above 90 % (range 70–100 %).

Outcome Measures

ADHD and ODD Symptoms

Teachers and parents completed the Child Symptom Inventories [49]. The ADHD (18 items) and ODD (8 items) severity scales correspond to DSM-IV ADHD and ODD symptoms and are rated on a 4-point scale ranging from 0 (Never) to 3 (Very often). The CSI contains normative data and acceptable test–retest reliability and predictive validity for ADHD and ODD [52]. In our sample internal consistency was high for both parent and teacher versions of each scale (all alphas above .88); mean ADHD and ODD symptom severity scores were used in the primary pre to post analyses. In addition, ADHD inattentive and hyperactive/impulsive, as well as ODD symptom counts were calculated by summing the number of items that either the parent or the teacher rated as occurring often or very often. A cut-off score of 4 ODD symptoms occurring often or very often by either parent or teacher report was also used to categorize children as meeting symptom count criteria for ODD.

Social and Behavioral Functioning

Teachers and parents completed the Social Skills Improvement System as a measure of social functioning (SSIS; [53]). The SSIS yields two subscales assessing Social Skills (46 items for parent and teacher versions) and Problem Behaviors (33 and 30 items for parent and teacher versions, respectively). All items are scored on a 4-point scale ranging from 0 (Never) to 3 (Almost always). The SSIS has excellent psychometric properties, including high internal consistency and adequate test–retest reliability for the parent and teacher versions of the Social Skills (αs = .95 and .97, rs = .86 and .84, respectively) and Problem Behaviors (αs = .94 and .95, rs = .78 and .81, respectively) subscales. In the present study, the SSIS Social Skills and Problem Behaviors standard scores were reported.

Teachers completed the Academic Competence Evaluation Scale (ACES; [54]). The Interpersonal Skills scale consists of 10 items rated on a 5-point scale ranging from 1 (Never) to 5 (Almost Always); items indicate the frequency with which children interacted appropriately with peers and teachers at school (e.g., “Expresses dissatisfaction appropriately” and “Listens to what others have to say”). The Interpersonal Skills scale has excellent psychometric properties including high internal consistency (α = .97) and test–retest reliability (r = .92). The mean score for Interpersonal Skills was reported in the present study.

The participating school district utilizes a standards based report card system that requires teachers to rate students on grade-specific expectations for social development, such as “Respects self and others”, “Resolves conflicts”, and “Follows directions”. We obtained, with parent consent, quarterly report cards, which provided an ecologically valid, standards-based measure of social behavior and development. Children were graded on a three point scale: 1 = unsatisfactory, 2 = satisfactory, and 3 = outstanding. The present study examined social development grades from the first quarter (i.e., before children participated in the intervention) and the fourth quarter (i.e., after children participated in the intervention). Complete report cards were not available from eight participants due to school transfers or teachers who did not use the school district’s standard reporting system.

Impaired Relationships

Teachers and parents completed the Impairment Rating Scale (IRS; [50]), a measure of the degree to which a child’s problems cause impairment across a number of domains that are relevant to children with ADHD, such as the child’s relationship with peers, parents, and teachers. The teacher (6 items) and parent (7 items) versions are scored on a 7-point scale ranging from 0 (No problem) to 6 (Extreme problem) and have evidence of reliability and validity when used as individual items or as a total scale score [50]. Specifically, Fabiano et al. [50] reported evidence of temporal stability of the IRS items over 2, 4, 6, and 12 month periods for parent ratings (rs ranged from .54 to .96) and teacher ratings (rs ranged from .40 to .98, and from .57 to .98 when ratings were provided by the same teacher). They also reported evidence of interrater (parent-teacher) reliability for the peer relationships item (rs ranging from .59 to 64), as well as evidence of concurrent, convergent, discriminant and predictive validities. The present study used IRS items that assess the degree to which children’s impairments affect their peer relationships from the parent and teacher versions, their relationships with their parents from the parent version, as well as their relationships with their teacher from the teacher version, using the stem “How your/this child’s problems affect his or her relationship with…”, for each relationship.

Life Skills Knowledge

In order to test the degree to which children learned the content introduced in the child skills group, the Test of Life Skills Knowledge (TOLSK) was adapted from similar measures developed in previous intervention studies (e.g., [27, 42]) and was administered at baseline and post-treatment. The measure consisted of 10 items that were scored on a 3-point scale (1 = no or inaccurate response, 2 = partial response, 3 = full accurate response). Interviewers and raters were blind to the child’s group assignment at post-treatment and raters were very consistent (Kappa = .82).

Data Analysis

The primary goal of the present study was to evaluate the pre- and post-treatment mean differences on parent and teacher-reported social and behavioral measures. In order to account for the non-independence that resulted from the nested data structure (i.e., students within schools) and provide more accurate estimates of standard errors, these mean differences were evaluated using regression models with sampling weights (as described by Asparouhov [55]) in MPlus version 5.21 [56]. The present study utilized methods to control the false discovery rate that can result from performing multiple hypothesis tests, as described by Benjamini and Hochberg [57]. Specifically, using this method, each p value below the a priori family-wise alpha level of .05 (1) is ranked in ascending order, i thru M, where M is the rank of the largest p value. These p values are then compared iteratively to an adjusted alpha level of i(α)/M, until the one of the p values (k) is larger than the adjusted alpha level. When this occurs, k and all p values ranked after k are considered nonsignificant. Cohen’s d was calculated as a measure of the magnitude of baseline and post-treatment mean differences and was adjusted for the correlation between baseline and post-treatment means that resulted from the repeated measures design using equation 8 provided by Morris and DeShon [58]:

$${\mu }_{\mathrm{D}}/(\sigma \sqrt{(2[1-\rho ])})$$

where µ_D is the mean difference between post-intervention and pre-intervention scores, σ is the standard deviation of µ_D, and p is the correlation between pre-intervention and post-intervention scores.

In order to provide evidence of clinical significance, the reliable change index was calculated [59]. The index is calculated using the following formula:

$$\text{RC}=({\mathrm{X}}_1-{\mathrm{X}}_2)/{\text{SE}}_{\text{diff}}$$

where SE_diff represents the standard error of measurement of the difference between two scores, X₁ represents an individual’s pretreatment score and X₂ represents an individual’s post-treatment score when improvement is indicated by a decrease scores. In the case of academic skills, in which increases in scores represent improvement, X₁ and X₂ would be in the reverse order. Moreover, Jacobson and Traux [59] recommended examining the proportion of the sample that has recovered to the normal range, as defined by scores that are within two standard deviations of the normed mean. However, given the overlap in the normal and disordered populations in the current study, a more conservative definition of the normal range was used, requiring children’s post-treatment scores to be within one standard deviation of the normed mean in order to be considered recovered. McNemars (χ²) test was used to identify whether or not the proportion of the sample that was within the normal range at post-treatment statistically differed from the proportion of the sample that was within the normal range at baseline.

In order to investigate the secondary aim of the study, the sample was split into two groups: children who met symptom count criteria for ODD (N = 29) and those that did not (N = 28). The regression analyses described above were replicated in each group to evaluate whether or not children’s mean pre- and post-treatment social and behavioral functioning statistically significantly improved in each group separately. Finally, the third aim of the study, which concerned identifying the incremental impact of parent adherence to treatment on children’s improvements in social and behavioral functioning, was evaluated using multiple linear regression analyses with parent adherence to treatment predicting parent and teacher-rated social and behavioral outcomes after adjusting for children’s pretreatment scores.

Results

Improvement in Social and Behavioral Outcomes

Baseline and post-treatment means and standard deviations for each measure are presented in Table 2 along with effect sizes, Z statistics, and p values for each mean comparison. Statistically significant improvement occurred on all measures from baseline to post-treatment except the degree to which children’s problems caused impairment in their relationship with their teacher after adjusting for false discovery rate, as described above. Effect sizes were large for parent reported problem behavior standard scores, children’s TOLSK scores, and report card grades for social development. Effect sizes were in the medium to large range for parent ratings of ODD symptom severity, social skills standard scores, and the degree to which children’s problems caused impairment in their relationships with their parents, as well as teacher ratings of social skills standard scores and interpersonal skills. Effect sizes were small to medium for improvements on teacher ratings of ODD symptom severity, problem behavior standard scores, and parent and teacher reports of the degree to which children’s problems caused impairment in their peer relationships.

Table 2.

Mean changes from pre to post-treatment

Measure	Baseline M (SD)	Post-treatment M (SD)	ES	Z	P
TOLSK	1.36 (.27)	1.96 (.42)	1.55	15.02	p < .001
Parent measures
CSI ADHD symptom severity	1.76 (.54)	1.19 (.56)	1.09	6.74	p < .001
CSI ODD symptom severity	1.21 (.66)	.88 (.56)	.59	4.25	p < .001
SSIS social skills	79.65 (13.37)	88.11 (14.02)	.66	6.60	p < .001
SSIS problem behaviors	117.95 (15.30)	107.18 (15.15)	.87	6.62	p < .001
IRS peer impairment	2.84 (2.16)	1.96 (2.08)	.36	3.84	p < .001
IRS parent-child impairment	3.30 (2.08)	2.03 (1.88)	.64	7.91	p < .001
Teacher measures
CSI ADHD symptom severity	1.74 (.55)	1.14 (.52)	1.23	11.45	p < .001
CSI ODD symptom severity	.83 (.64)	.68 (.55)	.30	2.72	p = .006
SSIS social skills	82.44 (12.26)	89.30 (12.18)	.60	5.76	p < .001
SSIS problem behaviors	116.18 (12.39)	110.58 (12.23)	.47	2.62	p = .009
ACES interpersonal skills	3.18 (.74)	3.52 (.66)	.63	3.41	p = .001
IRS peer impairment	3.60 (1.92)	2.82 (2.08)	.47	2.59	p = .010
IRS child-teacher impairment	3.42 (1.78)	3.04 (1.95)	.20	1.68	p = .093
Report card ratings of social development	1.85 (.45)	2.13 (.46)	.82	9.18	p < .001

CSI child symptom inventories, SSIS social skills improvement system, IRS Impairment Rating Scale, ACES Academic Competence Evaluation Scale, TOLSK test of life skill knowledge

As previously reported by Pfiffner et al. [41], effect sizes for parent and teacher reported ADHD symptom severity were large and these measures reliably improved for a substantial proportion of children in the sample (49 % per parent report and 53 % per teacher report), based on the reliable change index. The current findings show similar impact on social and behavioral outcomes. Substantial proportions of the sample reliably improved on parent and teacher reported social skills and problem behaviors (49 and 54 %, respectively for social skills, and 47 and 44 %, respectively for problem behaviors). Based on the criterion for recovery to the normal range described above (i.e. post-treatment scores within 1 SD of the mean), a statistically significant proportion of the sample recovered to the normal range at post-treatment based on teacher reported social skills (47 vs. 28 % at baseline, χ²(1) = 5.26, p = .022) and a substantial, but not statistically significant proportion recovered to the normal range based on parent-reported social skills (54 vs. 38 % at baseline, χ²(1) = 3.37, p = .066). For problem behaviors, a statistically significant proportion of the sample recovered to the normal range at post-treatment based on parent (75 vs. 53 % at baseline, χ²(1) = 9.60, p = .002) and teacher (68 vs. 51 % at baseline, χ²(1) = 4.05, p = .044) reports. Pfiffner et al. [41] also reported that 51 % of the sample were in the nonclinical range for ADHD symptom counts at post-treatment. Similarly, 81 % of children were in the nonclinical range for ODD symptom counts at post treatment based on a cut off of four or more symptoms of ODD reported by parents or teachers, relative to 49 % at baseline, which was a statistically significant difference, χ²(1) = 16.06, p < .001).

The sample was split into two groups, children who met symptom count criteria for ODD (N = 29) and those who did not (N = 28). The regression analyses described above were replicated in each group and revealed the same pattern of significant improvements across outcome variables in both groups.

Impact of Fidelity on Outcomes

The incremental impact of measures of the fidelity of treatment implementation on children’s outcomes was examined using multiple linear regressions and adjusting for problems at baseline. Specifically, higher weekly clinician ratings of parent adherence to the intervention were associated with lower parent ratings of their children’s ADHD, Z = −3.15, p = .002, β = −.25, and ODD, Z = −2.26, p = .02, β = −.25, symptom severities, as well as their problem behaviors, Z = −2.31, p = 02, β = −.28, at post treatment. Moreover, higher mean parent ratings of their use of strategies taught in the parent group during the previous week were associated with lower teacher ratings of children’s ODD symptom severity, Z = −2.60, p = .009, β = −.31, and higher parent ratings of their children’s social skills, Z = 3.10, p = .002, β = .24, at post treatment. Parents who reported prompting and reinforcing the skills children learned in the child life skills groups at post treatment also rated their children’s social skills higher, Z=2.81, p = .005, β = .26, at post treatment.

Discussion

The results of this study provide additional support for the feasibility and impact of the school-based implementation of a coordinated behavioral intervention, CLS, by existing school mental health personnel and extend previously reported support [40, 41] to a larger sample of children and schools and additional measures in the domains of social and behavioral functioning. Moreover, these improvements in social and behavioral functioning and the decreased severity of ADHD and ODD symptoms that were associated with CLS treatment were consistent across school and home settings according to parent and teacher reports and for children with ADHD only and children with co-occurring ADHD and ODD problems. Improvements in ecologically valid measures of social behavior (i.e., report cards) and evidence of reliable change and clinically significant recovery further support these findings. Moreover, findings that greater parent treatment adherence and reinforcement and prompting of child skills were independently associated with decreased severity of children’s ADHD and ODD symptom severities and problem behaviors as well as improved social skills further support the impact and feasibility of the intervention approach.

It is notable that participating children improved across a broad range of social and behavioral domains. In addition to improvements in their ADHD and ODD severities, children improved in their social and interpersonal functioning at school and at home and the impact of these impairments on a number of important relationships decreased. This is an encouraging finding and may be a reflection of the emphasis of CLS on coordinating intervention components (such as those that targeted impairments in social functioning), across contexts in order to maximize the impact of the treatment effects. Specifically, the present study found that children’s social functioning improved more when parents more frequently used the strategies that they learned and prompted and reinforced the skills learned in the child skills group. In addition, children significantly improved in their understanding of the content introduced during the child skills groups. These findings provide support for the feasibility and potential benefits of the CLS approach to coordinating parent, teacher, and child intervention components across contexts. In addition to providing didactic instruction addressing specific social skills deficits, such as conversation skills and good sportsmanship, CLS facilitates and reinforces the acquisition of these skills by using real-world situations (e.g., games and activities), common terminology across parent, child, and teacher treatment components and by targeting skills taught in the child group using home and classroom-based reward systems.

Decreased impairments in relationships with parents and improvements in problematic, oppositional, and disruptive behaviors are consistent with findings from previous investigations of the impact of behavioral parent training and classroom behavioral interventions for ADHD on children’s behavior at home and at school, respectively [13–16]. In fact, the magnitude of the effects for externalizing problem behaviors found in the present study were higher than those reported by Fabiano et al. [13] in their meta-analysis of behavioral interventions (i.e., .76 for parent reported externalizing measures and .33 for teacher reported externalizing measures). Moreover, the consistent improvement of children with ADHD problems only and children with co-occurring ADHD and ODD problems across a variety of social and behavioral outcome measures and in the school and home contexts supports the impact of the coordinated, multi-component CLS intervention for these children. Although ODD problems were not an explicit focus of the CLS intervention, it is possible that the strong and consistent emphasis on cross-contextual coordination of behavioral management strategies (e.g., behavioral reinforcement of school behaviors at home) and reinforcement of children’s use of the skills they learned in the child skills component, facilitated the generalization of treatment gains to a variety of behavioral and social problems across situations.

Although it might seem logical that increased treatment integrity/fidelity, and specifically adherence, would produce more pronounced treatment outcomes, few studies have explicitly investigated this relationship [44, 45, 60]. The present study found that observer ratings of parents’ treatment adherence were associated with parent-reported improvements in ADHD and externalizing behavior problems and that parent ratings of their treatment adherence were associated with improvements in teacher-reported externalizing behavior problems and parent reported social skills. These results are consistent with previous studies that have found that better treatment integrity/fidelity is related to greater improvements in children’s treatment outcomes [44–47]. The magnitudes of the effects of treatment adherence on children’s outcomes in the present study were moderate, suggesting that treatment adherence is an important factor in children’s improvements following treatment.

Limitations

The data reported in this study were collected as part of a series of open trials designed to inform the development of the final intervention, while evaluating the feasibility and initial outcomes of the intervention protocol. As such, these results do not reflect the most rigorous evaluation of efficacy and a randomized controlled trial of the intervention is a crucial future direction. Although it is possible that the changes reported in this study could be attributed to other factors such as time, maturation, familiarity with peers, adjustment to classroom structure, or non-specific intervention effects, the rates of reliable and clinically significant recovery and magnitude of the effect sizes are larger and in the opposite direction (i.e., indicating improvement) of what would be expected of untreated children with ADHD during this age range based on previous research [61, 62]. Moreover, findings of improvements that resulted in part from higher quality implementation of the treatment strategies by parents and teachers further support the assertion that improvements in children’s problems were directly related to the intervention. An additional concern with regard to the present data is the lack of indicators of improved symptomatic and behavioral functioning from treatment blind raters. Although the effects on ecologically valid measures of social development (report cards) add to the real-world impact of the intervention, these report cards were ultimately completed by teachers who were involved in the child’s treatment. In addition, we do not provide evidence of extended treatment effects beyond the duration of the active intervention, other than report card data. Finally, the sample of families in the present study may not be representative of the larger population of children with ADHD. Specifically, children in the present sample were largely medication naïve and from mostly well-educated, two-parent families. These factors could contribute to a higher likelihood of parental treatment adherence. Nevertheless, there was substantial variability in clinician-reported, as well as parent-reported treatment adherence, which was independently associated with treatment outcomes.

Summary

Despite a growing evidence base in support of behavioral interventions for children with ADHD, current interventions are often inaccessible to many families and are limited to specific contexts. The present study provides support for the feasibility and utility of CLS, a collaborative school-home intervention, for improving social and behavioral impairments of children with significant ADHD-related problems. Existing school mental health personnel were able to implement the program with acceptable integrity/fidelity in the context of the school day and participation in the program was associated with improvements in behavior problems, social functioning, and relational impairments at school and at home, as well as decreases in the severity of ADHD and ODD symptoms. In addition to decreasing the severity of ODD problems in this sample, the intervention had an equally potent impact on the problems and functional impairments of a subgroup of children that met symptom count criteria for comorbid ADHD and ODD. In addition, parents’ adherence to the treatment was directly related to the magnitude of improvement on ADHD and ODD symptom severities, problem behaviors and social skills. These findings support the emphasis of CLS on the consistent application of analogous behavioral intervention strategies across contexts, using similar terminology, behavioral targets, and reinforcement contingencies.