Abstract
Objective
This study examined the relative contribution of two dimensions of parent engagement, attendance and homework adherence, to parent and child treatment response and explored whether early engagement was a stronger predictor of outcomes than later engagement.
Method
The sample consisted of parents of participants (n = 92; M age 9.4 years, SD = 1.27; 67% male; 69% White) in a 12-session evidence-based family-school intervention for children with ADHD. Attendance was assessed using clinician records, and homework adherence was measured by rating permanent products. Outcomes included parent and teacher ratings of family involvement in education, parenting practices, and child functioning.
Results
Accounting for the contributions of baseline scores and attendance, homework adherence was a significant predictor of parental self-efficacy, the parent-teacher relationship, parenting through positive involvement, and the child’s inattention to homework and homework productivity. Accounting for the contribution of baseline scores and homework adherence, attendance was a significant predictor of one outcome, the child’s academic productivity. Early homework adherence appeared to be more predictive of outcomes than later adherence, whereas attendance did not predict outcomes during either half of treatment.
Conclusions
These results indicate that, even in the context of evidence-based practice, it is the extent to which parents actively engage with treatment, rather than the number of sessions they attend, that is most important in predicting intervention response. Because attendance is limited as an index of engagement and a predictor of outcomes, increased efforts to develop interventions to promote parent adherence to behavioral interventions for children are warranted.
Keywords: parent engagement, attendance, homework adherence, intervention, ADHD
In response to calls to understand who benefits most from evidence-based interventions for youth (La Greca, Silverman, & Lochman, 2009), accumulating research attests to the importance of parent engagement in behavioral interventions for children (Khanna & Kendall, 2009; Nix, Bierman, McMahon, & The Conduct Problems Prevention Research Group, 2009). Several theoretical models propose that successfully engaging parents in mental health service for children has direct and indirect influences on child outcomes (Berkel, Mauricio, Shoenfelder, & Sandler, 2011; Staudt, 2007). Given the significance of parent engagement and the fact that it can be difficult to achieve, there are a growing number of interventions designed to increase parents’ treatment attendance (Ingoldsby, 2010). However, increasing parent attendance may not be sufficient to improve child outcomes because, arguably, “it is quality (not quantity) of parent involvement that is important” (La Greca et al., 2009, p. 376). In fact, a key mediator between treatment attendance and outcome may be the extent to which parents actually implement the skills that are theoretically linked to behavior change. This raises an important empirical question: In the context of evidence-based practice, is it the number of sessions a parent attends or the extent to which the parent actively engages with the intervention that is most important in predicting intervention response?
Understanding the relative impact of quantity versus quality of engagement requires a clear delineation of the distinct dimensions of parent engagement. Nock and Ferriter (2005) focused on attendance at and adherence to therapy as core components of parent participation, or engagement, and defined adherence as “involvement of the patient in a mutually acceptable course of behavior to produce a…therapeutic result” (p. 151). In two recent studies of behavioral parent training, parent engagement was assessed by measuring attendance and homework completion (Chacko et al., 2009; Fabiano et al., 2009). Engagement has also been viewed as a construct reflecting three dimensions: attendance, participation in discussions in session, and completion of between-session activities, referred to as homework adherence (Baydar, Reid, & Webster-Stratton, 2003; Kazantzis, Whittington, & Dattilio, 2010). The current study aims to advance our understanding of parent engagement by examining two of these dimensions – attendance and homework adherence.
To date there have been very few studies of multiple dimensions of parent engagement in relation to intervention response. Most studies have examined engagement exclusively in terms of attendance and found that higher attendance generally predicts improved parenting (e.g., Baydar et al., 2003). Research on adherence indicates that treatment gains are associated with adherence to homework between sessions (Kazantzis et al., 2010, p. 145). Currently, there is only one published study examining intervention response in relation to parent attendance and adherence to a behavioral intervention for children. In their study of parent training, Nix et al. (2009) measured attendance and the quality of parent participation, which included an assessment of adherence (i.e., homework completion and implementation of skills within sessions). Results revealed that the quality of parent participation contributed uniquely to the prediction of several parenting outcomes. However, like most research on parent engagement, Nix et al. did not examine the impact of engagement on child outcomes. Other limitations included the lack of data on the reliability of the quality of participation measure and the fact that the measure was non-specific, combining clinician-report items assessing homework adherence, in-session adherence, and parent interest during sessions. Clearly, replication of their work is warranted (La Greca et al., 2009, p. 376).
The current study extends previous research on parent engagement by being the first to examine the impact of parent engagement on both parent and child response to psychosocial intervention and only the second study to examine the relative contribution of parent attendance and homework adherence to outcomes. Two specific hypotheses were tested. First, we hypothesized that parent attendance and homework adherence would independently predict treatment response. Second, based upon the results of the Nix et al. (2009) study, we predicted that homework adherence generally would have a stronger association with outcomes than attendance. Finally, given evidence that session attendance decreases over the course of parent training (Baker, Arnold, & Meagher, 2011), we explored whether levels of parent engagement during the latter half of the intervention were lower than levels early in the intervention and whether early or late engagement was more predictive of outcomes.
Method
Participants
This study was conducted in the context of a clinical trial through an ADHD center within a metropolitan pediatric hospital. Participants were recruited from families that requested diagnostic evaluations through the center, and additional referrals were obtained from schools and community providers. Details regarding selection and recruitment procedures and inclusion and exclusion criteria for the clinical trial are presented elsewhere (authors blinded)1. Participants were randomly assigned to the experimental intervention, Family-School Success (FSS), or the control group. Only caregivers of children who participated in FSS (n = 92; M age 9.4 years, SD = 1.27; 67% male) were included in this study. Primary caregivers were primarily married, middle-to-upper class mothers. Demographic and diagnostic data for the sample are presented in Table 1.
Table 1.
Background Information about Participants in the Study (n=92)
| Primary caregivers (% mothers) | 89.1 |
| Single parent status (%) | 19.6 |
| SES (% Levels III, IV, V on Hollingshead) | 98.0 |
| Hispanic (%) | 6.0 |
| African American (%) | 19.0 |
| White (%) | 69.0 |
| Asian (%) | 3.0 |
| Multiracial (%) | 3.0 |
| ADHD, Combined Type (%) | 44.6 |
| ADHD, Inattentive Type (%) | 55.4 |
| Comorbid Learning Disability (% with LD) | 27.2 |
| Comorbid Disruptive Behavior Disorder (% with disorder) | 28.3 |
| Comorbid Internalizing Disorder (% with disorder) | 10.9 |
Note: SES refers to socioeconomic status, as assessed by the Hollingshead (1975) index of social status. Levels III, IV, and V reflect the middle to high levels of the scale. Clinical diagnoses were determined using the K-SADS-P IVR, which was completed by a licensed psychologist or advanced doctoral student supervised by a psychologist. Children diagnosed with both internalizing and externalizing disorders (3.3%) are included in both categories.
Intervention
FSS is a family-school intervention designed to improve parenting practices, family involvement in education, and child academic performance. FSS includes standard behavioral parent training components (e.g., McMahon & Forehand, 2003) and elements of educationally-focused interventions (Power, Karustis, & Habboushe, 2001; Sheridan & Kratochwill, 2008; Vannest, Davis, Davis, Mason, & Burke, 2010). FSS provides intervention over 12 weekly sessions using clinic-based parent group meetings, individualized family therapy sessions, and family-school consultations. Parenting groups include 7 families, on average, and the typical session length is 90 minutes for groups, 60 minutes for individual sessions, and 45 minutes for family-school meetings. Homework is assigned in session, and a record of what was practiced is documented on a worksheet and returned during the next session. Homework assignments are intended to encourage implementation of behavioral skills taught in session and enable parents to practice skills with their children (see Table 2).
Table 2.
Description of Sessions and Homework Assignments for Family School Success
| Session Title | Session Type | Session Content | Homework Worksheet Title and Description |
|---|---|---|---|
| 1 – Introduction to Family School Success | Group | Introduction to FSS Orientation to the CBC model Using attention to change child behavior |
Noticing Positive/Desired Behavior – On 3 occasions, the parent records the child’s positive behavior, what the parent did to explicitly recognize the behavior, and how the child responded |
| 2 – Preparing for Home-School Collaboration | Individual Family | Preparation for first school consultation Use of homework assignment books Use of DRC |
No homework assigned |
| 3 – Promoting Home-School Collaboration | School Meeting | Establishing collaborative home-school relationship Establishing use of assignment book & DRC |
The Home-School Note – The parent, in collaboration with the child and teacher, develops and fully implements a home-school note on at least 1 day |
| 4 – Understanding Basics of Behavior Management | Individual Family | Review school meeting Develop understanding of positive reinforcement and punishment |
Consequences: Rewards and Punishments – On 5 occasions, the parent records the child’s behavior, the consequence provided, whether the consequence was a reward or punishment, and how effective it was |
| 5 – Introducing the Token Economy | Group | Group discussion of school meetings Establishing a token economy |
Homework Rewards Worksheet – The parent and child initiate a list of homework behaviors with assigned point values; on at least 1 day, the parent records the homework behaviors and the total points earned |
| 6 – Understanding the Function of Behavior and Establishing the Homework Ritual | Group | Functional assessment to define homework problems (antecedents and consequences) Establishing the homework routine Guidelines for giving effective instructions |
Homework ABC – On at least 3 days, the parent monitors the child during homework and records the antecedents, behaviors, consequences provided, and the child’s response to the consequences |
| 7 – Managing Time and Goal Setting | Individual Family | Time management strategies for homework completion Goal setting approach to homework completion |
The Goal Setting Tool – The parent and child segment homework into subunits and negotiate goals for completing each subunit, including the amount of time permitted for the items, the number of items completed, and the number of items correct; the parent and child evaluate the extent to which the goals were met and assign points |
| 8 – Managing Time and Goal Setting -2 | Individual Family | Review goal setting strategies with clinician modeling and feedback | The Goal Setting Tool (cont’d) |
| 9 – Using Punishment Successfully | Group | Group discussion of experiences with goal setting Rationale for using punishment strategically Response cost and time-out Prepare for second school consultation |
The Goal Setting Tool (cont’d) |
| 10 – Collaborating to Refine Strategies | School Meeting | Review use of DRC and modify if needed Use of goal setting in the classroom |
The parent has the option of practicing several different study skills with the child |
| 11 – Developing Effective Study Skills | Group | Strategies for effective study skills, including incremental rehearsal | The parent has the option of practicing several different study skills with the child |
| 12 – Integrating Skills and Planning for the Future | Group | Review and problem solve implementation difficulties Develop individual family “Formulas for Success” End of program celebration |
No homework assigned |
Note: CBC = Conjoint Behavioral Consultation; DRC = Daily Report Card
Measures of Engagement
Attendance
Attendance was computed as the percentage of all sessions attended by a caregiver, based on the clinician’s written record and videotape records for group sessions.
Homework adherence
Homework adherence was assessed using a measure of compliance by rating the extent to which each homework worksheet was completed and calculating the sum of these ratings. This measure provides an index of parents’ implementation of skills at home. Each worksheet was rated using a 3-point scale (0 = homework was not attempted or submitted, 1 = submitted homework was attempted but incomplete, and 2 = submitted homework was completed). The highest possible total score for homework adherence was 12. Ratings were made by master’s- and doctoral-level members of the research team with clinical experience in behavioral parent training and FSS. Inter-rater agreement was calculated for 22.2% of the assignments selected at random. In 92.2% of the cases, there was exact agreement between raters. The homework for sessions 7–9 was counted as a single assignment, given the complexity of this assignment. Homework for sessions 10 and 11 was excluded because of inconsistency in how instructions were delivered. Homework assigned prior to session 6 and homework assigned on or after session 6 comprised the measures of early and late homework adherence, respectively.
Outcome Measures
Family involvement in education
Parental self-efficacy was assessed using a 10-item version of the Parent as Educator Scale (PES; Hoover-Dempsey, Bassler, & Brissie, 1992), which is a reliable and valid measure of parent-reported self-efficacy with regard to supporting their children’s education. Coefficient alpha in the present study was .83. The Quality of the Parent-Teacher Relationship factor derived from the Parent-Teacher Involvement Questionnaire (PTIQ; Kohl, Lengua, McMahon, & Conduct Problems Prevention Research Group, 2000), which has sound psychometric properties, was used to assess parents’ and teachers’ perspectives of the quality of the home-school relationship. Coefficient alpha for this study was .88.
Parenting practices
The Parent-Child Relationship Questionnaire (PCRQ; Furman & Giberson, 1995) was used to assess parent perceptions of their parenting practices. This measure assesses two dimensions – Positive Involvement and Negative/Ineffective Discipline. Validity for these factors has been supported in previous research (Hinshaw et al., 2000), and in this study coefficient alphas for the factors were .89 and .84, respectively.
Child functioning
Parents completed the Homework Problem Checklist (HPC; Anesko et al., 1987) by rating each of 20 homework problems. The HPC yields two factors, Inattention/Task Avoidance and Poor Productivity/Non-Adherence with Rules, which have adequate validity (Power, Werba, Watkins, Angelucci, & Eiraldi, 2006). Alpha coefficients for these factors in this study were .88 and .78, respectively. The Homework Performance Questionnaire – Teacher Version (HPQ-T; Power, Dombrowski, Watkins, Mautone, & Eagle, 2007) assesses teacher perceptions of students’ homework behavior and has established validity (Mautone, Marshall, Costigan, Clarke, & Power, 2012) for children with ADHD. The HPQ-T includes the 7-item Student Responsibility factor, which had an alpha coefficient of .91 in this study. The Academic Productivity factor of the psychometrically sound Academic Performance Rating Scale (APRS; DuPaul, Rapport, & Perriello, 1991) was used to assess teacher ratings of student productivity on academic assignments. The alpha coefficient for this factor in this study was .86. The composite score of the Swanson, Nolan, and Pelham Questionnaire (SNAP-IV; Swanson et al., 2001), which is completed by parents and teachers and has acceptable validity (Bussing, Fernandez, Harwood, Hou, Garvan, Eyberg, & Swanson, 2008), was used to assess ADHD and ODD symptoms. In this study, the alpha coefficients for the parent and teacher versions were .92 and .94, respectively.
Assessment Procedures
At the conclusion of each FSS session, clinicians recorded parent attendance on attendance logs and collected completed homework. Outcome measures for this study were collected at baseline (prior to session 1), mid-treatment (at session 6), and post-treatment (after session 12).
Statistical Analysis
Multiple regressions were conducted to analyze the data. For each regression model tested, the dependent variable was one of the 10 outcome measures assessed at post-treatment. A stepwise approach was used for variable selection. Baseline levels of the dependent variable were accounted for in each analysis. The predictor variables (attendance or homework adherence) were entered and retained in the model if they resulted in a p value < .15. Given that our purpose was to examine patterns of association between predictors and outcomes, and not to predict any particular outcome, predictors in the final model were considered statistically significant at p < .05 and no adjustments were made for multiple comparisons (Perneger, 1998). Effect sizes were determined using f2, a function of partial R2 divided by 1 minus partial R2. The magnitude of these values was evaluated in relation to standards established by Cohen (1992), with f2 of .02, .15, and .35 corresponding to small, medium, and large effects, respectively. Stepwise regression analyses were repeated to examine the relative contribution of attendance and homework adherence early in treatment, sessions 1–6 (predicting mid-treatment scores, controlling for baseline scores), and late in treatment, sessions 7–12 (predicting post-treatment scores, controlling for mid-treatment scores).
Results
Of the 92 families who participated in FSS, complete data sets across the 10 outcome measures were obtained for 86 to 88 participants2. On average, participants attended 10.4 (SD = 2.0, range = 1 – 12) of the 12 FSS sessions. The mean score for homework adherence was 4.75 (SD = 3.11, range = 0 – 12). Attendance and homework adherence data for each session is presented in Table 3. The correlation between attendance and homework adherence was .65 (p < .001).
Table 3.
Session Attendance and Homework Adherence Data for Participants in the Study (n=92)
| Session | Attendance | Completed Homework | Attempted Homework |
|---|---|---|---|
| 1 | 92.4% | 45.7% | 22.8% |
| 2 | 97.8% | - | - |
| 3 | 95.7% | 52.2% | 1.1% |
| 4 | 92.4% | 28.3% | 35.9% |
| 5 | 89.1% | 27.2% | 16.3% |
| 6 | 76.1% | 19.6% | 15.2% |
| 7 | 90.2% | - | - |
| 8 | 66.3% | - | - |
| 9 | 83.7% | 42.4% | 0.0% |
| 10 | 88.0% | - | - |
| 11 | 77.2% | - | - |
| 12 | 84.8% | - | - |
Note: Attempted homework refers to homework that was attempted but not completed. Homework was not assigned in sessions 2 or 12. Homework for sessions 7–9 was counted as a single assignment. Homework for sessions 10 and 11 were excluded from analyses.
Parental Attendance as Sole Predictor
Controlling for baseline scores, attendance was a significant predictor of post-treatment outcomes (p < .05) on 4 of 10 measures. Higher attendance predicted higher parental self-efficacy and better scores on three measures of child functioning: inattention to homework, poor homework productivity, and academic productivity. The magnitude of these effects were small (f2 = .03 to .09).
Parental Homework Adherence as Sole Predictor
Controlling for baseline scores, homework adherence was a significant predictor of post-treatment outcomes (p < .05) on 8 of 10 measures. Higher adherence predicted better scores for parental self-efficacy, the parent-teacher relationship, and positive involvement. Homework adherence also predicted better performance for five child outcomes: inattention to homework, poor homework productivity, responsibility with homework, academic productivity, and parent-rated ADHD/ODD symptoms. The magnitude of these effects ranged from small to medium (f2 =.03 to .23).
Relative Contribution of Parental Attendance and Homework Adherence as Predictors
The results of the stepwise regression analyses that incorporated both attendance and homework adherence in the model are provided in Table 4. Accounting for the contributions of baseline scores and session attendance, homework adherence made a significant contribution (p < .05) to the prediction of 5 of 10 outcomes, including parental self-efficacy, the parent-teacher relationship, parenting through positive involvement, inattention to homework, and poor homework productivity. The effect sizes (f2) ranged from .05 to .20; the contribution of homework adherence was medium for parental self-efficacy (f2 = .20) and inattention to homework (f2 = .18). Accounting for the contributions of baseline scores and homework adherence, attendance made a significant contribution (p < .05) to the prediction of only one outcome, academic productivity, and the effect size was small (f2 = .04). All results were in the expected direction. We considered including the interaction of attendance and homework adherence in the model, but the correlation between adherence and the interaction term was virtually perfect (r = .98), demonstrating that the interaction term did not add meaningful information to the model.
Table 4.
Results of stepwise regression analyses with attendance and adherence predicting outcomes at post-treatment, controlling for baseline scores (including only predictors with p < .15)
| Outcome Measure: | Predictor | df | F | p | Adj R2 | beta | F | p | Part R2 | f2 |
|---|---|---|---|---|---|---|---|---|---|---|
| PES | 2, 66 | 9.97 | 0.0002 | 0.23 | ||||||
| Intercept | 2.82 | 80.37 | < 0.0001 | |||||||
| Time 1 | 0.20 | 4.93 | 0.030 | 0.06 | ||||||
| Adherence | 0.07 | 13.86 | 0.0004 | 0.17 | 0.20 | |||||
| PTIQ | 2, 65 | 42.01 | < 0.0001 | 0.56 | ||||||
| Intercept | 0.80 | 9.50 | 0.003 | |||||||
| Time 1 | 0.70 | 62.26 | < 0.0001 | 0.52 | ||||||
| Adherence | 0.05 | 7.29 | 0.009 | 0.05 | 0.05 | |||||
| PCRQ - PI | 2, 64 | 30.07 | < 0.0001 | 0.48 | ||||||
| Intercept | 1.32 | 14.12 | 0.0004 | |||||||
| Time 1 | 0.64 | 52.57 | < 0.0001 | 0.44 | ||||||
| Adherence | 0.03 | 5.67 | 0.02 | 0.05 | 0.05 | |||||
| PCRQ - NI | 2, 64 | 5.79 | < 0.0001 | 0.33 | ||||||
| Intercept | 0.89 | 8.09 | 0.006 | |||||||
| Time 1 | 0.57 | 30.21 | < 0.0001 | 0.31 | ||||||
| Adherence | −0.03 | 2.23 | 0.14 | 0.02 | 0.02 | |||||
| HPC-IA | 2, 66 | 11.70 | < 0.0001 | 0.26 | ||||||
| Intercept | 0.92 | 19.13 | < 0.0001 | |||||||
| Adherence | 0.02 | 13.77 | 0.0004 | 0.15 | 0.18 | |||||
| Time 1 | 0.29 | 10.18 | 0.002 | 0.11 | ||||||
| HPC-PP | 2, 66 | 27.16 | < 0.0001 | 0.45 | ||||||
| Intercept | 0.32 | 7.62 | 0.008 | |||||||
| Time 1 | 0.44 | 37.77 | < 0.0001 | 0.4 | ||||||
| Adherence | −0.03 | 6.40 | 0.01 | 0.05 | 0.05 | |||||
| HPQ-T | 2, 67 | 26.13 | < 0.0001 | 0.44 | ||||||
| Intercept | 0.51 | 0.74 | > 0.15 | |||||||
| Time 1 | 0.57 | 49.14 | < 0.0001 | 0.41 | ||||||
| Attendance | 1.01 | 3.41 | 0.07 | 0.03 | 0.03 | |||||
| APRS | 2, 66 | 35.65 | < 0.0001 | 0.52 | ||||||
| Intercept | 0.66 | 2.63 | 0.11 | |||||||
| Time 1 | 0.61 | 59.84 | < 0.0001 | 0.48 | ||||||
| Attendance | 0.89 | 4.85 | 0.03 | 0.04 | 0.04 | |||||
| SNAP - P | 2, 64 | 27.01 | < 0.0001 | 0.46 | ||||||
| Intercept | 0.48 | 7.85 | 0.007 | |||||||
| Time 1 | 0.56 | 44.63 | < 0.0001 | 0.43 | ||||||
| Adherence | −0.03 | 3.83 | 0.054 | 0.03 | 0.03 | |||||
| SNAP - T | 1, 67 | 140.05 | < 0.0001 | 0.68 | ||||||
| Intercept | 0.09 | 1.02 | 0.32 | |||||||
| Time 1 | 0.82 | 140.05 | < 0.0001 |
Note. For each outcome measure, Time 1 scores accounted for more variance than adherence and attendance with the exception of HPC-IA. PES = Parent as Educator Scale; PTIQ = Parent-Teacher Involvement Questionnaire; PCRQ – PI = Parent Child Relationship Questionnaire – Positive Involvement; PCRQ – N/ID = Parent Child Relationship Questionnaire – Negative/Ineffective Discipline; HPC-IA = Homework Problem Checklist, Inattention Avoidance factor; HPC-PP = Homework Problem Checklist, Poor Productivity factor; HPQ-T = Homework Performance Questionnaire – Teacher; APRS = Academic Performance Rating Scale; SNAP = Swanson, Nolan, and Pelham Questionnaire.
Comparison of Early Versus Late Engagement
Attendance during the first half of treatment demonstrated a high correlation with attendance later in treatment (r = .76, p < .001), although the rate of attendance early in treatment (M = 83%, SD = 29%) was significantly higher (t = 3.84, p < .001) than attendance later in treatment (M = 75%, SD = 32%). Adherence on the four early homework assignments demonstrated a low to moderate correlation with adherence on the two later assignments (r = .39, p < .001), and the mean homework adherence score per assignment (range of 0 to 2) was significantly higher (t = 3.34, p < .001) in the first half of treatment (M = 0.89, SD = 0.57) than the second half (M = 0.65, SD = 0.72).
In general, early engagement appeared to be more predictive of treatment response than later engagement. During the first half of treatment, homework adherence made a significant contribution (p < .05) for 6 of 10 measures, including parenting outcomes (i.e., parental self-efficacy, positive involvement, and negative/ineffective discipline) and child outcomes (i.e., inattention to homework, poor homework productivity, and teacher-rated ADHD/ODD symptoms). The effect sizes were generally small (f2 range = .02 – .10). During the second half of treatment, homework adherence made a significant contribution (p < .05) for only two outcomes: parental self-efficacy and inattention to homework. The effect sizes were relatively small (.10 and .06, respectively). Attendance did not predict outcomes during either half of treatment.
Discussion
The results of this study affirm the importance of parent engagement in evidence-based intervention for childhood disorders and confirm previous research (Nix et al., 2009) indicating that attendance is limited as an index of engagement. Although the number of sessions attended, or the quantity of parent engagement, does appear to predict intervention outcome to some degree, the quality of parents’ engagement, as measured by their completion of written therapeutic assignments between sessions, is a much stronger predictor of intervention response, explaining more outcomes and greater variability in both parent and child response to treatment. In the context of a family-school intervention for children with ADHD, parent homework adherence made a unique contribution to the prediction of parental self-efficacy, the parent-teacher relationship, parenting through positive involvement, child inattention to homework and poor homework productivity. In contrast, parent attendance was uniquely predictive of academic productivity only.
This study raises important issues related to measurement in parent engagement research. First, the most important methodological contribution of this study involves the use of permanent products generated from parental homework assignments as a method of measurement. Across the parent engagement literature, there is wide variability in the measurement of adherence, including assessing parents’ reports of their use of newly acquired skills (e.g., Nock, Ferriter, & Holmberg, 2007) and rating their implementation of skills within sessions (e.g., Nix et al., 2009). Although rating homework completion fails to recognize adherence among parents who practice skills at home but do not turn in their homework, coding homework still has several advantages over alternative approaches to measuring adherence: coding homework compliance avoids the bias inherent in informant reports; it is easily accomplished because homework is a core feature of most behavioral parent training interventions; and it is an objective measure that can be scored with a relatively high level of reliability. Second, consistent with previous results (Baker et al., 2011), our findings suggested that rates of attendance and homework adherence declined from the early to the later phase of treatment. To account for this instability, it may be advisable to consider engagement as an aggregate over time or evaluate engagement at multiple time points.
There are a number of professional practice implications of this research. Perhaps the most significant implication relates to the development of interventions to increase parent engagement. Most existing interventions focus on enhancing enrollment and attendance; relatively little attention has been given to designing interventions to enhance parents’ adherence (Nock & Ferriter, 2005). Because homework adherence appears to be a stronger predictor of treatment effects than attendance, increased efforts to identify factors that promote homework adherence are warranted. Evidence from two evaluations of programs designed to increase parent attendance and adherence (Chacko et al., 2009; Fabiano et al., 2009) suggest that enhancing engagement by altering the format of treatment (e.g., embedding it in a sports activity) may have less of an impact on child outcomes than addressing parents’ treatment motivation and barriers to participation. In addition, the current findings suggested that homework adherence during the first half of treatment may be a better predictor of outcomes than adherence in the second half of treatment. As such, early indications of non-adherence to parental homework may be a sign that the family is at risk for poor outcomes and that additional strategies, such as coaching parents by telephone between sessions, may be needed to enhance parental implementation of new skills. Another implication for practice relates to the length of interventions for families of youth with ADHD. Given that attendance and homework adherence were greater early in treatment, practitioners should examine the potential effectiveness of briefer interventions (e.g., six sessions).
The methodological and practical implications of this research should be considered in light of the study limitations. Most importantly, the variability in levels of attendance was relatively low compared to the variability in homework adherence (see Table 3), which may have attenuated the association between attendance and outcomes and contributed to the differences in associations between the two indicators of engagement. It is also important to note that, although homework adherence did predict both parent-rated and teacher-rated outcomes in models that excluded attendance, once the contribution of attendance was accounted for in the models, parental homework adherence was a predictor only for those outcomes that relied on parent report. Therefore, it is possible that parents who put more effort into the intervention simply may have perceived more benefits from it. Additionally, parental ADHD was not examined in this study, despite the fact that parents of children with ADHD often have ADHD themselves (Takeda et al., 2010) and, therefore, may be likely to have particular difficulty with treatment engagement. Furthermore, although this sample was racially diverse, families were largely from middle- to upper-income groups, suggesting that our findings may not be generalizable to lower-income families who are also at increased risk for poor treatment engagement.
Another study limitation involves the measure of homework adherence. The index of adherence used in this study focused solely on degree of homework completed and did not account for more theoretically meaningful aspects of adherence, such as the extent to which parents viewed the assignments as useful, reasonable, and aligned with their treatment goals (Kazantzis, Deane, & Ronan, 2004). Also, the current study did not distinguish between parents’ mastery of new skills, skills practice, and their documentation of that practice on a homework worksheet. Furthermore, the impact of demand characteristics associated with homework assigned in group versus individual sessions was not examined. Addressing these limitations in future studies would enhance research on the role of parent engagement in child therapy. Another issue related to our measure of adherence is that four homework assignments were assessed early in treatment, whereas only two assignments were rated later in treatment. Further, later assignments may have been more complex than earlier assignments. These factors may have contributed to the decrease in homework adherence over time and the finding that early homework adherence was more predictive of treatment response. Alternatively, the early assignments may have had stronger conceptual links with treatment goals. Nonetheless, the findings related to homework adherence early versus later in treatment should be interpreted with caution.
A final area of potential concern is that parents had to attend a session in order to receive and submit homework, which meant that adherence was partially dependent upon attendance. We considered the possibility that multicollinearity had an effect on the findings, but ruled this out because the correlation between attendance and homework adherence (r=.65) was smaller than the high correlation often associated with the presence of multicollinearity (Kline, 2005, p. 56). Also, we examined the predictive models for each outcome and failed to find a case in which the overall model was significant but none of the predictors was significant, which is commonly observed when there is multicollinearity.
Evidence-based interventions for childhood disorders have proliferated in recent decades (Kazdin & Weisz, 2003), and, in order to maximize their impact, the next phase of research must identify for whom these interventions work best (La Greca et al., 2009). Taken together, the results of this study and the Nix et al. (2009) study suggest that evidence-based parent training programs are most efficacious for parents who not only attend regularly but who take the opportunity to enact the skills they learn in session when they are at home. Practicing and applying newly acquired skills in real life represents a higher-order level of parent engagement that surpasses simply logging time in session. One way of conceptualizing these different dimensions of parent engagement is to consider attendance as a passive form of engagement, as compared to between-session adherence and in-session responsiveness, which represent active forms of engagement. Parents who show up and put in the time must also put in the practice because time alone, whether measured in number of sessions attended or number of minutes in session3, is likely not enough to effect change. Therefore, research aimed at understanding the factors that promote parents’ active engagement in child intervention is an important direction for future study.
Footnotes
All participating children scored at or above 0.75 of a standard deviation above the mean on the Homework Problem Checklist (HPC; Anesko, Schoiock, Ramirez, & Levine, 1987), which was considered an indicator of educational impairment.
Missing data was primarily due to early termination. At mid-treatment, 4% of the families dropped out of FSS, resulting in 88 families being retained through post-treatment. Only participants with complete data were included in outcome analyses.
In the current study, the amount of clinical contact time received in group and family sessions was defined as the total number of recorded minutes that the parent was present in session based on videotape records. Because the amount of clinical contact was highly correlated with attendance (r = .82), clinical contact was not examined separately in this study.
Note. This study was supported by research grant MH068290 funded by the National Institute of Mental Health and the Department of Education awarded to the senior author (Thomas Power).
Contributor Information
Angela T. Clarke, West Chester University of Pennsylvania
Stephen A. Marshall, Ohio University
Jennifer A. Mautone, The Children’s Hospital of Philadelphia
Stephen L. Soffer, The Children’s Hospital of Philadelphia
Heather A. Jones, Virginia Commonwealth University
Tracy E. Costigan, American Institutes for Research, Washington, DC
Anwar Patterson, Drexel University.
Abbas F. Jawad, Perelman School of Medicine at the University of Pennsylvania /The Children’s Hospital of Philadelphia
Thomas J. Power, Perelman School of Medicine at the University of Pennsylvania /The Children’s Hospital of Philadelphia
References
- Anesko KM, Schoiock G, Ramirez R, Levine FM. The Homework Problem Checklist: Assessing children’s homework difficulties. Behavioral Assessment. 1987;9:179–185. [Google Scholar]
- Baker CN, Arnold DH, Meagher S. Enrollment and attendance in a parent training prevention program for conduct problems. Prevention Science. 2011;12:126–138. doi: 10.1007/s11121-010-0187-0. [DOI] [PubMed] [Google Scholar]
- Baydar N, Reid MJ, Webster-Stratton C. The role of mental health factors and program engagement in the effectiveness of a preventive parenting program for Head Start Mothers. Child Development. 2003;74:1433–1453. doi: 10.1111/1467-8624.00616. Retrieved from http://www.jstor.org/stable/3696187?origin=JSTOR-pdf. [DOI] [PubMed] [Google Scholar]
- Berkel C, Mauricio AM, Shoenfelder E, Sandler IN. Putting the pieces together: An integrated model of program implementation. Prevention Science. 2011;12:23–33. doi: 10.1007/s11121-010-0186-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bussing R, Fernandez M, Harwood M, Hou W, Garvan CW, Eyberg SM, Swanson JM. Parent and teacher SNAP-IV ratings of Attention Deficit/Hyperactivity Disorder symptoms: Psychometric properties and normative ratings from a school district sample. Assessment. 2008;15:317–328. doi: 10.1177/1073191107313888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chacko A, Wymbs BT, Wymbs FA, Pelham WE, Swanger-Gagne MS, Girio E, O’Connor B. Enhancing traditional behavioral parent training for single mothers of children with ADHD. Journal of Clinical Child and Adolescent Psychology. 2009;38:206–281. doi: 10.1080/15374410802698388. [DOI] [PubMed] [Google Scholar]
- Cohen J. A power primer. Psychological Bulletin. 1992;112:155–159. doi: 10.1037//0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
- DuPaul GJ, Rapport MD, Perriello LM. Teacher ratings of academic skills: The development of Academic Performance Rating Scale. School Psychology Review. 1991;20:284–300. [Google Scholar]
- Fabiano GA, Chacko A, Pelham WE, Robb J, Walker KS, Wymbs F, Pirvics L. A comparison of behavioral parent training programs for fathers of children with Attention-Deficit/Hyperactivity Disorder. Behavior Therapy. 2009;40:190–204. doi: 10.1016/j.beth.2008.05.002. [DOI] [PubMed] [Google Scholar]
- Furman W, Giberson RS. Identifying the links between parents and their children’s sibling relationships. In: Shulman S, editor. Close relationships in social–emotional development. Norwood, NJ: Ablex; 1995. pp. 95–108. [Google Scholar]
- Hinshaw SP, Owens EB, Wells KC, Kraemer HC, Abikoff HB, Arnold E, Wigal T. Family processes and treatment outcome in the MTA: Negative/ineffective parenting practices in relation to multimodal treatment. Journal of Abnormal Child Psychology. 2000;28:555–568. doi: 10.1023/A:1005183115230. [DOI] [PubMed] [Google Scholar]
- Hoover-Dempsey KV, Bassler OC, Brissie JS. Explorations in parent-school relations. Journal of Educational Research. 1992;85:287–294. doi: 10.1080/00220671.1992.9941128. [DOI] [Google Scholar]
- Hollingshead AB. Four factor index of social status. New Haven, CT: Yale University; 1975. [Google Scholar]
- Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies. 2010;19:629–645. doi: 10.1007/s10826-009-9350-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kazantzis N, Deane FP, Ronan KR. Assessing compliance with homework assignments: Review and recommendations for clinical practice. Journal of Clinical Psychology. 2004;60:627–641. doi: 10.1002/jclp.10239. [DOI] [PubMed] [Google Scholar]
- Kazantzis N, Whittington C, Dattilio F. Meta-analysis of homework effects in cognitive and behavioral therapy: A replication and extension. Clinical Psychology: Science and Practice. 2010;17:144–156. [Google Scholar]
- Kazdin AE, Weisz JR. Evidence-based psychotherapies for children and adolescents. New York: Guilford; 2003. [Google Scholar]
- Khanna MS, Kendall PC. Exploring the role of parent training in the treatment of childhood anxiety. Journal of Consulting and Clinical Psychology. 2009;77:981–986. doi: 10.1037/a0016920. [DOI] [PubMed] [Google Scholar]
- Kline RB. Principles and practice of structural equation modeling. 2. New York: Guilford; 2005. [Google Scholar]
- Kohl GO, Lengua LJ, McMahon RJ Conduct Problems Prevention Research Group. Parent involvement in school: Conceptualizing multiple dimensions and their relations with family demographic risk factors. Journal of School Psychology. 2000;38:501–524. doi: 10.1016/S0022-4405(00)00050-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- La Greca AM, Silverman WK, Lochman JE. Moving beyond efficacy and effectiveness in child and adolescent intervention research. Journal of Consulting and Clinical Psychology. 2009;77:373–382. doi: 10.1037/a0015954. [DOI] [PubMed] [Google Scholar]
- Mautone JA, Marshall SA, Costigan TE, Clarke AT, Power TJ. Multidimensional assessment of homework: An analysis of students with ADHD. Journal of Attention Disorders. 2012;16:600–609. doi: 10.1177/1087054711416795. [DOI] [PubMed] [Google Scholar]
- McMahon RJ, Forehand R. Helping the noncompliant child: A clinician’s guide to effective parent training. 2. New York: Guilford; 2003. [Google Scholar]
- Nix RL, Bierman KL, McMahon RJ The Conduct Problems Prevention Research Group. How attendance and quality of participation affect treatment response to parent management training. Journal of Consulting and Clinical Psychology. 2009;77:429–438. doi: 10.1037/a0015028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nock MK, Ferriter C. Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review. 2005;8:149–166. doi: 10.1007/s10567-005-4753-0. [DOI] [PubMed] [Google Scholar]
- Nock MK, Ferriter C, Holmberg E. Parent beliefs about treatment credibility and effectiveness: Assessment and relation to subsequent treatment participation. Journal of Child and Family Studies. 2007;16:27–38. doi: 10.1007/s10826-006-9064-7. [DOI] [Google Scholar]
- Perneger TV. What’s wrong with Bonferroni adjustments. British Medical Journal. 1998;316:1236–1238. doi: 10.1136/bmj.316.7139.1236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Power TJ, Dombrowski SC, Watkins MW, Mautone JA, Eagle JW. Assessing children’s homework performance: Development of multi-dimensional, multi-informant rating scales. Journal of School Psychology. 2007;45:333–348. doi: 10.1016/j.jsp.2007.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Power TJ, Karustis JL, Habboushe DF. Homework success for children with ADHD: A family-school intervention program. New York: Guilford; 2001. [Google Scholar]
- Power TJ, Werba BE, Watkins MW, Angelucci JG, Eiraldi RB. Patterns of homework problems among ADHD-referred and non-referred children. School Psychology Quarterly. 2006;21:13–33. doi: 10.1521/scpq.2006.21.1.13. [DOI] [Google Scholar]
- Sheridan SM, Kratochwill TR. Conjoint behavioral consultation: Promoting family-school connections and interventions. 2. New York: Springer; 2008. [Google Scholar]
- Staudt M. Treatment engagement with caregivers of at-risk children: Gaps in research and conceptualization. Journal of Child and Family Studies. 2007;16:183–196. doi: 10.1007/s10826-006-9077-2. [DOI] [Google Scholar]
- Swanson JM, Kraemer HC, Hinshaw SP, Arnold LE, Conners CK, Abikoff HB. Clinical relevance of the primary findings of the MTA: Success rates based on severity of ADHD and ODD symptoms at the end of treatment. American Academy of Child and Adolescent Psychiatry. 2001;40:168–179. doi: 10.1097/00004583-200102000-00011. [DOI] [PubMed] [Google Scholar]
- Takeda T, Stotebery K, Power TJ, Ambrosini P, Beretini W, Hakonarson H, Elia J. Parental ADHD status and its association with proband, ADHD subtype and severity. Journal of Pediatrics. 2010;157:995–1000. doi: 10.1016/j.jpeds.2010.05.053. [DOI] [PubMed] [Google Scholar]
- Vannest KJ, Davis JL, Davis CR, Mason BA, Burke MD. Effective intervention for behavior with a daily behavior report card: A meta-analysis. School Psychology Review. 2010;39(4):654–672. [Google Scholar]