“HOME PRACTICE IS THE PROGRAM”: PARENTS’ PRACTICE OF PROGRAM SKILLS AS PREDICTORS OF OUTCOMES IN THE NEW BEGINNINGS PROGRAM EFFECTIVENESS TRIAL

Abstract

An examination of the content and processes of evidence-based programs is critical for empirically evaluating theories about how programs work, the “action theory” of the program (West, Aiken, & Todd, 1993). The New Beginnings Program (NBP; Wolchik, Sandler, Weiss, & Winslow, 2007), a parenting-after-divorce preventive intervention, theorizes that program induced improvements in parenting across three domains: positive relationship quality, effective discipline, and protecting children from interparental conflict, will reduce the negative outcomes that are common among children from divorced families. The process theory is that home practice of program skills related to these parenting domains is the primary mechanism leading to positive change in parenting. This theory was tested using multi-rater data from 477 parents in the intervention condition of an effectiveness trial of the NBP (Sandler et al., in press; Sandler et al., 2016). Four research questions were addressed: Does home practice of skills predict change in the associated parenting outcomes targeted by the program? Is the effect above and beyond the influence of attendance at program sessions? What indicators of home practice (i.e., attempts, fidelity, efficacy, and competence) are most predictive of improvements in parenting? Do these indicators predict parenting improvements in underserved subpopulations (i.e., fathers and Latinos)? Structural Equation Modeling analyses indicated that parent-reported efficacy and provider-rated parent competence of home practice predicted improvements in the targeted parenting domains according to both parent and child reports. Moreover, indicators of home practice predicted improvements in parenting for fathers and Latinos, although patterns of effects varied by parenting outcome.

Decades of research have now demonstrated the efficacy of programs targeting parenting practices to prevent an array of negative developmental outcomes for children, including substance use, mental health problems, sexual risk behavior, delinquency, and school failure (NRC/IOM, 2009). These findings provide strong evidence for the potential of programs to have a significant public health impact if they can be implemented widely and effectively in the community. Unfortunately, the quality of implementation and subsequent effectiveness of programs can decline when programs are disseminated in community settings (Gottfredson et al., 2006). Further, although randomized trials have demonstrated program efficacy, studies that examine program content and processes are critical for empirically evaluating our theories about how programs work, the “action theory” of the program (West, et al., 1993). Without a strong grasp of how programs work, it is difficult to explain why they fail to meet expectations in community settings.

Mediational models linking program effects on proximal and distal outcomes can provide evidence for theories related to the program content (e.g., Wolchik, Tein, Sandler, & Kim, in press). Fewer studies have tested their theories of implementation processes. Beyond the theoretical knowledge this can provide, identifying dimensions of implementation that predict positive outcomes has important implications for developing systems to monitor and maintain high levels of program effectiveness (Berkel, Mauricio, Schoenfelder, & Sandler, 2011). These are essential tasks for advancing Type II translational research which seeks to span the divide between programs that have demonstrated efficacy in carefully controlled research settings and effective community implementation of those programs.

In-session implementation of evidence-based programs has been conceptualized to include four distinct dimensions: fidelity, quality, adaptation, and responsiveness (Berkel, et al., 2011). Responsiveness has been defined as the participant’s involvement and interest in the program (Dane & Schneider, 1998) and operationalized in four ways: attendance, active engagement during sessions, practice of program skills outside of the session, and satisfaction with the program (Berkel, et al., 2011). Several studies have found that composite measures of responsiveness predict program outcomes (e.g., Schoenfelder et al., 2012). However, rather than being considered as multiple indicators of one responsiveness construct, these may more appropriately be seen as unique and dynamically interrelated indicators of responsiveness. Further, the study of implementation of any program must be driven by the program’s action theory. While attendance is the most studied indicator of responsiveness (Durlak & DuPre, 2008), the theory of behaviorally-focused programs suggests that although attendance at the sessions is a necessary precursor, program effects depend on building skills during the program. Kaminski and colleagues (2008) found that the most effective parenting programs had components that taught specific skills and required parents to practice the skills with their children. The few studies that had directly tested the relations between home practice compliance with outcomes of behavioral parenting programs have found that higher compliance predicts better outcomes (e.g., Clarke et al., 2015; Riggs, Elfenbaum, & Pentz, 2006; Ros, Hernandez, Graziano, & Bagner, 2016). A number of unresolved issues have been identified by meta-analyses of home practice in cognitive behavior therapy (Kazantzis, Deane, & Ronan, 2004; Mausbach, Moore, Roesch, Cardenas, & Patterson, 2010). For example, while previous studies have focused on whether home practice was attempted, competence in practicing the skills may even more important (Kazantzis, et al., 2004). Efficacy is another important home practice dimension in that participants who feel efficacious may be more likely to continue skill use and show greater change (Coleman & Karraker, 1998). A methodological issue in the study of home practice concerns the reliance on participant self-report of both home practice and outcomes, which introduces the potential for bias due to shared method variance.

This study furthers our understanding of two critical aspects of implementation, program attendance and home practice of program skills, while testing the action theory of the New Beginnings Program (NBP; Wolchik, et al., 2007). Although we suggest that home practice is relevant across many types of programs, it may be particularly relevant for parenting programs because they rely on parents to take the skills home to children, who are the indirect, but primary targets of program benefits. The NBP is a 10-session parenting-after-divorce program that teaches parenting skills to protect children from negative outcomes of divorce. NBP’s theory states that program- induced improvements in parenting across three dimensions, i.e., positive relationship quality, effective discipline, and efforts to shield children from interparental conflict, will lead to reduced child adjustment problems following divorce (Wolchik, et al., 2007). Results of efficacy trials have confirmed these mediational pathways (e.g., Wolchik, et al., in press). In addition, NBP’s process theory suggests that while attendance is an important precursor for positive outcomes, home practice of skills related to the targeted parenting domains is the main process leading to positive change. Using data from an effectiveness trial of the NBP, the current study tests a theoretical model of the relations between attendance, home practice, and changes in the three targeted parenting constructs (see Figure 1) to test the effects of home practice on improvements in parenting, above and beyond the influence of attendance. We conceptualized multiple indicators of home practice that may lead to positive change: whether the parent attempted to do the skill (attempts); the degree to which they completed all of the components of the skill (fidelity); how well they felt the skill worked in their family (efficacy); and the providers’ perspective on how well the parent implemented the skill (competence). We tested models using parent and child report of change in parenting to rule out shared method variance.

Figure 1. Effects of Attendance and Home Practice on Improvements in Post-Divorce Parenting.

Notes. Each of these six models was tested separately with each of the four Home Practice indicators (i.e., attempts, fidelity, efficacy, and competence) for a total of 24 models.

FFT: Family Fun Time; OOT: One-on-One Time; CBG: Catch ‘em Being Good; GL: Good Listening; CP: Change Plan; KOM: Keeping Children out of the Middle

A universal issue in the study of implementation with important implications for Type 2 translational research concerns how implementation processes work in underserved subgroups. Most of the research on implementation of parenting programs has been conducted with non-Latina White (NLW) mothers. Little attention has been given to whether implementation processes are also relevant within minority populations (Berkel, Murry, Roulston, & Brody, 2013; St. George et al., 2016) or with fathers (Panter-Brick et al., 2014). In contrast to the efficacy trials of the NBP, which included a primarily NLW sample of divorced mothers, the effectiveness trial tested program effects with a diverse sample of divorcing fathers and mothers. To prepare for widespread dissemination, the NBP went through a cultural adaptation process to ensure the program content and materials were appropriate across ethnic subgroups. Although research supports the efficacy of parenting programs with fathers, in prior research, the effects have been found to be smaller for fathers as compared to mothers (e.g., Fletcher, Freeman, & Matthey, 2011). Similarly, although there is considerable evidence of the effectiveness of parenting programs for ethnic minority groups (Gonzales et al., 2012; Murry, Berkel, Brody, Gerrard, & Gibbons, 2007; Pantin et al., 2003), there is relatively little literature on how implementation processes predict changes in parenting within underrepresented ethnic minority groups. The current study will be the first to test the predictive effects of home practice on parenting for fathers and Latinos.

Methods

Participants

The study uses pre- and posttest data from the 477 families assigned to the intervention condition of the multi-site effectiveness trial of the NBP, including 103 families who never attended. Eligibility criteria for participating in the program were that the family was divorced or separated for a maximum of two years, had a child between 3 and 18 years, with whom there was weekly minimum contact (three hours or an overnight) and could complete the program in English. Children aged 9 and above (N = 297) also provided pre- and posttest data. Parents were recruited using a video shown during a 4-hour program mandated for all divorcing parents in Arizona and through media or referrals from community agents (e.g., judges, lawyers, etc). In terms of race/ethnicity: 32% (154) were Latino, 61% (283) were NLW, and 8% (38) represented other groups; 42% (201) were fathers and 58% (276) were mothers. Fifty percent of the 890 children were female.

Procedures

Assessments were conducted in English via telephone with the parents and children at Wave 1 (W1) and Wave 2 (W2). For all families, W1 occurred within two months prior to Session 1 and W2 occurred within two months after Session 10 (regardless of program condition or attendance), with an average of 20 weeks between W1 and W2. Parents provided consent for themselves and for children; children provided assent for themselves. Families received $100 for the completion of parent interviews and $50 for the completion of child interviews. They were not compensated for attending the program. Providers entered attendance and home practice data into a web-based data collection system following each session.

Twenty providers ran 50 groups over four cohorts, which occurred in the fall and spring of two consecutive years. Groups were single-gender (26 mother and 24 father groups). In 10 weekly sessions, NBP targets three parenting domains: relationship quality, discipline, and shielding children from interparental conflict. Parents learned new skills related to the targeted parenting domains and role played the skills during the session (see Table 1). They were assigned to practice the skills with their children each week, and complete a worksheet reporting on their home practice. Practice of skills was cumulative over the 10 weeks, such that after each skill was learned, parents continued to practice it weekly until the end of the program. The first 25 minutes of each session was devoted to a home practice review, in which parents reported on what they did and received feedback to reinforce positive skill use and constructive feedback to correct problems in skill use. If parents missed a session, they were called by the providers and invited to watch a make-up video immediately prior to the next session. While this meant they would not have the opportunity to practice the skill taught in that session, they would be able to follow the discussion with the group and practice when it was assigned again the following week.

Table 1.

Assessment Points

Interview Data	Parenting Domains	Reporter	Assessed
Wave 1 Interviews	Relationship Quality, Discipline, Conflict	Parent and Child	Within 2 months prior to Session 1
Wave 2 Interviews	Relationship Quality, Discipline, Conflict	Parent and Child	Within 2 months following Session 10
Program Skills	Parenting Domain	Skill Taught/Discussed in:	Home Practice Recorded in Sessions:
Family Fun Time (FFT)	Relationship Quality	10 sessions (1–10)	9 sessions (2–10)
One on One Time (OOT)	Relationship Quality	9 sessions (2–10)	8 sessions (3–10)
Catch ‘em Being Good (CBG)	Relationship Quality, Discipline	9 sessions (2–10)	8 sessions (3–10)
Good Listening (GL)	Relationship Quality	8 sessions (3–10)	7 sessions (4–10)
Keeping Children Out of the Middle (KOM)	Conflict	5 sessions (6–10)	4 sessions (7–10)
Change Plan (CP)	Discipline	4 sessions (7–10)	3 sessions (8–10)

Model	Home Practice is a composite of the following skills for the respective model:				For each model, attendance based on all sessions where the skill is taught/discussed:
Relationship Quality (latent construct with 4 skills)	FFT (averaged across 9 sessions)	OOT (averaged across 8 sessions)	CBG (averaged across 8 sessions)	GL ( averaged across 7 sessions)	10 sessions (1–10)
Discipline (2 skills averaged)	CBG (averaged across 8 sessions)		CP (averaged across 3 sessions)		9 sessions (2–10)
Conflict (assessed as 1 skill)	KOM (averaged across 4 sessions)				5 sessions (6–10)

Measures

Implementation Measures

Attendance

was recorded by the provider after the session each week and was calculated as a percentage of total possible sessions attended. The denominator for the attendance percentage varies across parenting domains depending on the number of sessions in which that domain was targeted (see Table 1). For example, while relationship quality topics were addressed in 10 sessions and the denominator was 10, protecting children from conflict was addressed in 5 sessions, thus the denominator was 5.

Home practice

was assessed via four indicators, attempts, fidelity, efficacy, and provider rated competence of each of the skills to improve relationship quality, discipline, and conflict. Once a skill was initially assigned, parents used a home practice worksheet to report on their attempts (dichotomous score of whether they tried the home practice) and efficacy (1–5 likert-type scale to rate “How did it go?” ranging from Awful to Great) for all of the subsequent weeks of the program. In addition, the first two weeks a new skill was assigned, parents were asked to report on their fidelity to the skill guidelines (i.e., a checklist of each of the behaviors involved in properly doing the skill). Based on responses to the home practice worksheet and the weekly home practice review, providers rated parents on their competence with each skill each week (1–5 likert-type scale to rate “How did parent do on [skill] this week?). If parents forgot their worksheets, they were provided with another copy to complete during the home practice review. Parents were able to turn in home practice late if they missed a session. If the parent had missing data for attempts and fidelity due to a lack of attendance on the date of assignment, these data were recoded as 0s. For efficacy and competence, missing scores were left as missing. Averages across sessions were taken based on the amount of complete data.

Parenting Measures

Relationship Quality

was assessed using parent and child report of the 16-item acceptance and 16-item rejection subscales of the CRPBI (Schaefer, 1965) and communication using the parent and child report of the 10-item Parent Adolescent Open Communication scale (Barnes & Olson, 1982). The response scale for acceptance and rejection was 1 (Never or almost never) to 5 (Always or almost always). The response scale for communication was 1 (Disagree a lot) to 5 (Agree a lot). For acceptance, Cronbach’s alphas were .88 and .85 for parent report at W1 and W2 and .95 for child report at both waves. For rejection, Cronbach’s alpha were .77 and .75 for parent report at W1 and W2, and .86 and .88 for child report at W1 and W2. For communication, Cronbach’s alphas were .80 and .79 for parent report at W1 and W2 and .92 and .90 for child report at W1 and W2.

Discipline

was assessed via four parent-report and one child-report measures. Using the Oregon Discipline scale (OSLC, 1991), parents responded to the 11-item Follow-Through subscale (α=.80 at W1 and .83 at W2), the 9-item Appropriate Use subscale (α=74 at W1 and .71 at W2), and the 5-item Inappropriate Use subscale (α=72 at W1 and .70 at W2). The response scale for Follow-Through was: 1 (Never or almost never) to 5 (Always or almost always) and for the Appropriate and Inappropriate use subscales was: 1 (Never Used) to 4 (Frequently Used). A ratio was calculated to determine the proportion of time appropriate strategies were used. Both parents and children responded to the 8-item Consistency of Discipline subscale from the CRPBI (parent: α=84 at W1 and .82 at W2; child: α=79 at W1 and .84 at W2). The response scale was 1 (Never or almost never) to 5 (Always or almost always).

Protecting Children from Interparental Conflict

was assessed using eight items from the parent report of the Children’s Perception of Interparental Conflict scale that assessed conflict in front of the child (CPIC; Grych, Seid, & Fincham, 1992). Cronbach’s alpha was .88 at W1 and .84 at W2. For child report, three indicators were used: all 15 items of the CPIC (α=87 at W1 and .90 at W2) and two measures from Buchanan and colleagues (1991): the 7-item Children in the Middle Scale (α=78 at W1 and .79 at W2), and a single item on badmouthing by the program parent (i.e., “In the past month, your [mom or dad] said bad things about your other parent”).

Analytic Strategy

Study hypotheses were tested in Mplus (Muthén & Muthén, 2012), using Full Information Maximum Likelihood (FIML) for missing data. The possible influence of clustering by intervention group was assessed by computing design effects; no variables had scores under 2 suggesting that clustering can be ignored. In the child report models, children were modeled as clustered within families. Associations between each indicator of home practice and improvements in parenting on that dimension were modeled separately for each outcome, according to the illustrations in Figure 1 (e.g., to test associations between home practice and improvements in parents’ report of relationship quality, four models were run: one with attempts, one with fidelity, one with efficacy, and one with competence). Models were analyzed using multiple raters of home practice (parent and provider) and of parenting (parent report and child report) to address the issue of within rater bias. Parenting and home practice variables were modeled as latent constructs whenever more than two indicators were available. Because of potential concerns about non-independence of models due to Catch ‘em Being Good being theoretically linked to both relationship quality and discipline, we present tests of individual skills in an appendix (available online). With models containing latent constructs, multiple practical fit indices were used to evaluate model fit (Hu & Bentler, 1999): a non-significant Χ² or a combination of SRMR ≤ .08, RMSEA ≤ .08, and CFI ≥ .90. Models with only manifest indicators were saturated. To examine pathways within underserved subgroups, multiple group analyses were conducted by gender and ethnicity. Sample size did not permit the study of gender within the Latino subsample, that is, for Latina mothers and Latino fathers.

Results

Descriptive Statistics

We conducted ANOVAs to ensure there were no significant differences by cohort in attendance, home practice indicators, pretest parenting, and posttest parenting; none were found. Means, standard deviations, and correlations for the relationship quality, discipline, and conflict models are presented in Appendix Tables 1–3, respectively (available online). Associations between attendance and attempts were high due to the fact that receiving assignments and turning in home practice sheets was dependent upon attendance (range = .88–.93). Although there was a pattern of significant correlations between attempts, fidelity, and efficacy, the magnitude of these relations is modest (median = .19), indicating they assess different aspects of home practice. Home practice indicators, especially efficacy and competence, were generally associated with W1 and W2 relationship quality and discipline, but not with conflict.

Measurement

All models demonstrated adequate fit statistics, reported in Table 2. Because of the high correlations between home practice attempts and attendance noted above, multicollinearity prevented the assessment of home practice attempts. Consequently, only models testing fidelity, efficacy, and competence are presented. For the latent construct of relationship quality, the loadings for all three indicators (i.e., acceptance, communication, and rejection) were above Β = .50, p ≤ .001 across all models. Loadings for discipline indicators (i.e., follow through, consistent discipline, and the appropriate discipline ratio) were above Β = .40, p ≤ .001 across all models. Loadings for child report of exposure to conflict indicators (i.e., Caught in the Middle, Badmouthing, and Children’s Perception of Interparental Conflict) were above Β = .60, p ≤ .001 across all models. For the relationship quality home practice construct, loadings for the four relevant skills (i.e., Family Fun Time, One-on-One Time, Catch ‘em Being Good, and Good Listening) were above Β = .40, p ≤ .001 for efficacy and competence. Two of the skills for fidelity did not load, so an average was used instead of a latent construct.

Table 2.

Standardized Path Coefficients for Models Linking Home Practice and Improvements in Parenting

Path€	1.Pretest →Posttest	2.Pretest→ Attendance	3.Pretest→ Home Practice	4.Attendance↔ Home Practice	5.Attendance →Posttest	6.Home Practice →Posttest	6.Home Practice→ Posttest by Subgroup
Total Sample							Gender			Ethnicity
Fidelity							Father	Mother	Wald	Latino	NLW	Wald
Relationship Quality
-parent reporta	.84***	−.10*	.16**	.19*	.08+	.05	.14+	.02	1.0	.06	.00	0.3
-child reportb	.79***	.08	.12+	.30+	−.03	.11+	.13	.07	0.5	.13	.06	0.1
Discipline
-parent reportc	.66***	.02	.05	.05	.17**	.17*	.15	.20+	0.6	.60***	.18*	7.9**
-child report	.75***	.24***	.03	.11	−.04	.04	.10	.01	0.0	−.03	.02	0.1
Conflict
-parent report	.46***	.06	.11	−.12	−.12*	.01	.03	.01	0.0	.40**	−.06	9.0**
-child reportd	.64***	−.10	.15	−.08	−.03	.02	−.05	.06	0.4	−.12	.04	1.4
Efficacy
Relationship Quality
-parent reporte	.73***	−.10*	.44***	.33**	.01	.25***	.25*	.24**	0.4	.14	.19**	0.1
-child reportf	.73***	.08	.20+	.16	−.04	.32***	.24*	.35**	0.0	.47*	.21**	0.8
Discipline
-parent reportg	.64***	.02	.26***	.61***	.13+	.09	.14	.07	0.1	.27	.06	0.6
-child report	.73***	.24***	.25***	.69***	−.22+	.26*	−.11	.36**	4.0*	−.03	.14+	1.2
Conflict
-parent report	.44***	.06	−.11	.19	−.08	−.20**	−.18+	−.25*	0.1	.11	−.29***	3.9*
-child reporth	.62***	−.11	−.15	.16	.00	−.21**	−.17	−.22*	0.0	−.04	−.23*	2.0
Competence
Relationship Quality
-parent reporti	.80***	−.10*	.18**	.53***	−.03	.21***	.34***	.14+	0.6	−.07	.17*	1.4
-child reportj	.78***	.08	.12	.49***	−.12+	.27**	.30*	.27*	0.0	.15	.20**	0.1
Discipline
-parent reportk	.61***	.02	.24***	.50***	.05	.27***	.27*	.31***	1.1	.56***	.21**	2.8+
-child report	.73***	.24***	.23**	.54**	−.15	.21*	−.08	.27**	4.1*	−.30	.14+	3.2+
Conflict
-parent report	.46***	.06	−.06	.27+	−.09	−.09	−.06	−.13	0.2	.13	−.14	1.6
-child reportl	.63***	−.11	−.17	.19	−.01	−.11	−.10	−.11	0.0	−.04	−.07	0.0

Notes. €Path numbers in each column map onto the labels in Figure 1;

^***p≤.001;

^**p≤.01;

^*p≤.05;

⁺p≤.10;

Fit Indices: Fidelity: ^aΧ²(14)=30.1**, RMSEA=.05(03;.07), SRMR=.03, CFI=.99;

^bΧ²(14)=21.7, p>.05;

^cΧ² (14) = 23.1, p >.05;

^dΧ ²(13)=15.2, p>.05;

Efficacy: ^eΧ²(35)=64.0**, RMSEA=.04(.03;.06), SRMR=.06, CFI=.98;

^fΧ²(36)=51.7*, RMSEA=.04(.01;.06), SRMR=.07, CFI=.98;

^gΧ² (14) = 19.8, p >.05;

^hΧ²(13)=11.2, p>.05;

Competence: ⁱΧ²(36)=64.3**, RMSEA=.04(.02;.06), SRMR=.06, CFI=.98;

^jΧ²(37)=53.4*, RMSEA=.04(.01;.06), SRMR=.10, CFI=.98;

^kΧ²(14) = 18.1, p >.05;

^lΧ²(13)=11.1, p>.05

Indicators of Home Practice and Improvements in Post-Divorce Parenting

As seen in Table 2 (Column 6), fidelity of home practice only predicted one of six measures of improved parenting (parent report of discipline). Efficacy predicted improvements in five of six measures of parenting, including both parent and child reports of relationship quality and conflict, as well as child report of discipline. Competence predicted improvements in parent and child report of relationship quality and discipline, but not exposure to conflict. In an appendix (available online), we present the results of individual skills to determine the impact of Catch ‘em Being Good, which is theoretically linked to both relationship quality and discipline. For relationship quality models, each of the skills is linked with outcomes, indicating that the effect is not driven solely by Catch ‘em Being Good. For the discipline models, both Catch ‘em Being Good and the Change Plan significantly predicted children’s report of improvements in discipline.

To assess the relative contribution of the home practice indicators to changes in parenting, we also tested the theoretical model for each parenting dimension with all three home practice indicators entered simultaneously. With the other home practice indicators in the model, only efficacy significantly predicted improvements in relationship quality (Β=.24, p≤.05 for parent report; Β=.21, p≤.05 for child report) and reductions in exposure to conflict (Β=-.25, p≤.05 for parent report; Β=-.24, p≤.05 for child report). For discipline, competence emerged as the best predictor of parent report (Β=.42, p≤.001), but with child report, none of the home practice indicators significantly predicted change.

Replication across Cultural Subgroups

Next, we tested our theoretical model for improvements in parenting within underserved subgroups for whom the program was culturally adapted: fathers and Latinos (see Table 2). For fathers, both efficacy and competence predicted improvements in relationship quality skills (both parent and child report) and competence predicted improvements in parent report of discipline skills, but there were no significant effects for home practice fidelity. For Latinos, fidelity predicted increased parent report discipline and conflict, efficacy was associated with child report of improvements in warmth, and competence predicted improvements in parent report of discipline.

Discussion

Findings from this study advance our understanding of the relations between indicators of home practice and changes in parenting following participation in NBP, an evidence-based parenting-after-divorce program. Two home practice indicators, parent report of efficacy and provider report of parent competence were significant predictors of improvements in each aspect of post-divorce parenting that were targeted by the program. These indicators of home practice predicted improvements in child and parent report of parenting, above and beyond the contribution of parental attendance. Somewhat different patterns of effects were found for two important understudied subgroups, fathers and Latinos. We discuss the contribution of these findings to evidence for NBP’s action theory, limitations of this study, and directions for future research on implementation of evidence-based programs.

According to NBP’s action theory, home practice is critical to improving parenting because it is the mechanism by which participants translate the skills they learn in the program into changes in their daily interactions with their children. The current study goes beyond prior literature by showing that home practice accounts for improvements in parenting above and beyond the effect of attendance. The fact that the effects of home practice efficacy and competence were found across raters of home practice (parent and provider) and across raters of parenting (parent and children) indicates that the findings cannot be explained by share reporter method variance. The replication of effects across program skills and outcomes is also noteworthy, especially given the fact that participants had many more opportunities to practice the relationship quality skills as compared to the discipline or conflict skills.

The findings provide support for the importance of measuring multiple aspects of home practice. Whether or not parents attempted the assignment is the most commonly used measure of home practice (Clarke, et al., 2015; Riggs, et al., 2006; Ros, et al., 2016). In the current study, home practice attempts were only modestly correlated with improvements in parenting. The very high correlation between home practice attempts and session attendance precluded studying an independent effect of home practice attempts on parenting improvement. Parents reported on two additional dimensions of their home practice each week: fidelity and efficacy, which were modestly correlated with each other and had different patterns of associations with parenting outcomes. Efficacy, or parents reports of how well the skills worked with their families, predicted improvements in relationship quality, discipline, and children’s exposure to conflict. Further, efficacy significantly predicted improvements in relationship quality and conflict after accounting for the effects of the other indicators of home practice. The effects of efficacy in predicting improvements in parenting is consistent with theoretical models of the critical role of parenting self-efficacy (Jones & Prinz, 2005). The NBP enhances efficacy by devoting the first 25 minutes of each session to troubleshoot problems in skill use and devise strategies for parents to successfully use the skills in the unique circumstances of their family. Other strategies for building efficacy include reinforcing successful use of skills and encouraging parents to attribute changes they see in their children to their use of the program skills. Future research is needed to assess how provider processing of home practice can facilitate home practice efficacy or competence.

Home practice fidelity assesses the degree to which parents applied each aspect of the skill in their home practice. In the current study, fidelity had limited impact on changes in parenting, which may be partially explained by fewer data points for fidelity, which was assessed only the first two times a skill was assigned. Fidelity may be more of a factor for discipline strategies because these are complex skills and if they are not done properly, the lack of success is most likely to be apparent to the parents in terms of continuing or escalating child misbehavior. The positive correlation between fidelity and efficacy is consistent with the theoretical expectation of feedback loops in which using the skills with fidelity leads to more positive outcomes, which in turn increases parents’ efficacy and encourages further skill use. The role of the provider in effectively coaching parents to apply these skills in a way that leads to the desired response from children may be key to developing the link between fidelity and efficacy to parenting improvements (Holtrop, Parra-Cardona, & Forgatch, 2014).

The finding that provider ratings of parents’ competence in doing the home practice predicted improvement in both parent and child report of relationship quality and discipline provides cross-informant support for the relations between home practice and parenting outcomes. It is important to note that the providers reviewed parent reports of home practice, observed role plays of skills in group, and provided feedback on skills use during the review of home practice. The high correlation between provider competence ratings and efficacy ratings (range of .59-.71) indicates that providers may be particularly influenced by parents’ reports of efficacy. However, the significant effect of provider-rated competence on parent report of discipline after accounting for the effects of efficacy argues for the added value of using provider judgments.

Evidence-based programs have the potential to reduce persistent health and social disparities (NRC/IOM, 2009). To accomplish this goal, we must understand the program experiences of underserved groups. Although prior research has examined dimensions of implementation as predictors of outcomes (Durlak & DuPre, 2008), there has been little research on implementation within minority populations (Berkel, et al., 2013; St. George, et al., 2016), and this is the first to examine home practice for fathers and Latinos. To better our understanding of what works with different groups, we must examine implementation processes within underserved subgroups, rather than simply reporting mean differences in participation or whether they differed from the majority population (Knight, Roosa, & Umaña-Taylor, 2009). This is the first study examining how several dimensions of home practice predict improvements in parenting for Latino parents. Fidelity and competence of home practice predicted parent report of improvements in discipline, few relations for relationship quality and conflict were significant. This is noteworthy given that the NBP was adapted to be appropriate across ethnic groups (Gonzales et al., 2006). Attendance was lower in Latino families, which could have resulted in less understanding and use of the program skills and more missing home practice data. The importance of familism in Latino families (e.g., Berkel et al., 2010) may be another explanatory factor in that Latino families may need less of this type of support for relating to their children. Alternatively, it may be that the relationship quality skills taught in NBP do not fit the way that positive relationships are typically enacted in Latino families, and consequently, they do not feel comfortable using them.

Divorced fathers are another important subgroup for implementation of NBP. Although fathers often experience diminished levels of contact with their children following divorce (Fabricius, Braver, Diaz, & Velez, 2010) and experience a heightened sense of role strain in the parenting role (Umberson & Williams, 1993), research has consistently found that the quality of paternal parenting following divorce is predictive of child well-being (Amato & Gilbreth, 1999; Sandler et al., 2012). The finding that fathers’ efficacy and competence of home practice predicted child and parent reports of improvement in relationship quality provides strong support that programs can improve father-child relationships through teaching and supporting use of these skills. Effects for discipline and conflict were more limited, perhaps due to fathers’ preexisting comfort with disciplinarian responsibilities based on traditional gender roles (Leve & Fagot, 1997) as well as the more limited contact with children for the fathers in this study, which gives these skills less time to translate into changes in parent-child interactions.

The findings have significant implications for monitoring and maintaining effectiveness of parenting programs when they are delivered as community-based services. The current findings showed that self-rated efficacy of home practice and provider-rated competence of home practice were significant predictors of improvements in parenting. Consistent with the theory of the program, they can be seen as good proximal indicators that the program is having the desired effects. Measures of efficacy and competence were constructed to be readily and inexpensively assessable through a web-based system of reporting. Such a system enables the monitoring of key indicators of implementation at scale and allows corrective and positive feedback to be provided shortly after program delivery. Future systems should capitalize on mobile technology to enable real-time troubleshooting of program skills and to reduce the correlations between attendance and submitting home practice reports (Gallo, Berkel, Brown, & Sandler, in preparation).

The limitations of this study point to directions for future research on program implementation in community settings. One limitation is that attendance and home practice indicators were averaged over the 10 weeks of the program, which obscures the direction of effects and dynamic processes between home practice and attendance as well as within the various aspects of home practice. Future studies should examine trajectories of home practice indicators and their relations with parenting outcomes to determine, for example, if efficacy is more important at initial attempts and competence or fidelity thereafter. There may also be associations between responsiveness indicators over time, such that efficacy in completing home practice may increase the likelihood that parents will attend and attempt additional home practice assignments in the future. Another critical area for future research is to identify aspects of provider behavior that influence home practice attempts, fidelity, efficacy, and competence. Berkel et al. (2011) proposed a theoretical model through which the quality of provider delivery predicts different aspects of parent responsiveness. Future research should consider how the quality of relationship between providers and parents, the fidelity of teaching the program skills, and the quality of coaching and feedback about use of the skills predict parents’ home practice. Finally, this study was limited in its ability to assess implementation processes for Latinos and fathers, but not within the Latino subgroup (i.e., Latina mothers and Latino fathers) and not for other disadvantaged groups. Future studies should seek to expand their sample sizes to examine the pathways within gender for Latinos, to replicate the effects observed for Latinos, and to examine pathways in additional underserved populations, including members of other racial/ethnic groups and sexual/gender minority groups.

In conclusion, this study found that, consistent with the NBP’s action theory, “home practice is the program,” and that home practice accounts for improvements in parenting above and beyond the effects of attendance. Further, in contrast to prior studies that have focused on quantity of home practice completed, we found that quality (i.e., efficacy, competence, and to a lesser extent, fidelity) predicted improvements in parenting. A methodological strength of the study is that multiple indicators of home practice, using parent and provider report, predicted improvements in parenting from the perspective of both parents and children. On the whole, efficacy and competence appeared to be important predictors of change for the general sample, Latinos, and fathers. These findings have both practical and theoretical implications with respect to ways to efficiently monitor programs delivered in the community.