Moving an Evidence-Based Parenting Program into the Community

Abstract

Attachment and Biobehavioral Catch-up (ABC) is a parenting program developed to enhance sensitivity among parents of infants who experience early adversity. In several randomized clinical trials, the intervention’s efficacy has been demonstrated. Moving interventions into the community with adequate fidelity is challenging, though, and intervention effects are often much smaller than when tested in randomized clinical trials. To enhance the likelihood that ABC is delivered with high fidelity, a micro-analytic fidelity assessment was developed. Using this fidelity tool as a central component of training, supervision, and certification, changes in parent sensitivity for 108 families with children ages 6 months to 2 years were as large as those seen in laboratory settings. These findings are discussed with regard to implications for moving other evidence-based interventions into the community.

Growing out of basic findings in attachment and stress neurobiology, Attachment and Biobehavioral Catch-up (ABC) is an intervention that targets sensitivity among parents of high-risk infants and toddlers, with the aim of enhancing children’s attachment behavior and self-regulatory capabilities. Through multiple randomized clinical trials, the efficacy of the intervention has been demonstrated in improving parent and child outcomes (e.g., Bernard et al., 2012; Bernard, Hostinar, & Dozier, 2015; Lewis-Morrarty, Dozier, Bernard, Moore, & Terracciano, 2012). Nonetheless, when evidence-based interventions are implemented in the community, fidelity can be compromised and effects are often much smaller than seen in lab settings (e.g., Hulleman & Cordray, 2009). To counter this threat, an ABC fidelity instrument was developed to allow careful monitoring of and tailored supervision to support fidelity. In this paper, we overview this fidelity assessment instrument, describe how it is used in community implementation procedures, and present pre- to post-intervention data testing the intervention’s outcomes in community sites.

Overview of ABC Intervention

Attachment and Biobehavioral Catch-up (ABC) is a 10-session intervention implemented in families’ homes with parents of children between 6 and 24 months of age. Session content is guided by a manual, and includes discussion to provide a rationale to the parent; structured practice activities to promote in-session practice; and video feedback to highlight parents’ strengths, point out areas for growth, and celebrate change. However, the most important component of the intervention is “in the moment” feedback in which clinicians, whom we call ‘parent coaches’, immediately discuss parent behaviors as they occur, described more below. ABC targets three specific parental behaviors related to sensitivity: nurturance when children are distressed, following children’s lead when they are not distressed, and avoiding frightening and intrusive behaviors at all times. Each of these behaviors was identified as important through research findings from our lab and other labs. First, nurturance is especially critical for children who have experienced adversity. When at-risk children do not have nurturing caregivers, they are particularly vulnerable to developing disorganized attachments (Dozier, Stovall, Albus, & Bates, 2001). Although children who have experienced adversity sometimes push their parents away (Stovall & Dozier, 2000; Stovall-McClough & Dozier, 2004), parents can over-ride these behavioral signals and provide nurturance even though it is not elicited. The first intervention target of ABC is to help parents provide nurturing care when children are distressed, even if children fail to signal their needs clearly. Second, children who have experienced adversity are often dysregulated biologically, as evidenced through a blunted diurnal pattern of cortisol production (Bernard, Dozier-Butzin, Rittenhouse, & Dozier, 2010). Relying on evidence that parents who are very responsive to their children have children who develop strong self-regulation (e.g., Raver, 1996), the second ABC intervention component was designed to help parents learn to be responsive to their children, i.e., follow their lead. Third, a number of studies have found that frightening and intrusive behaviors (e.g., yelling at or threatening children) undermine children’s ability to regulate behavior and biology (Carlson, 1998; Lyons-Ruth et al., 1993). The third component of ABC helps caregivers reduce their frightening and intrusive behaviors.

Intervention Efficacy

Attachment and Biobehavioral Catch-up (ABC) was designed to enhance parental nurturance and responsiveness, and to decrease parental frightening behavior, with the expectation that children would develop secure or organized attachments, and learn to regulate physiology, behavior, and emotions effectively. Through randomized clinical trials, we found that the ABC intervention enhanced both parental sensitivity (Bick & Dozier, 2013; Bernard, Simons, & Dozier, 2015) and children’s outcomes. Among CPS-involved children, more children in the ABC intervention formed secure attachments and fewer formed disorganized attachments than in the control intervention (Bernard et al., 2012). Children in the ABC intervention also showed a steeper (i.e., more normative) pattern of wake-up to bedtime cortisol production than children in the control group (Bernard et al., 2015a), with effects sustained three years post-intervention (Bernard et al., 2015b). Intervention effects also include stronger executive functioning among children in the ABC condition than in the control group (Lewis-Morrarty et al., 2012; Lind et al., 2017). Together these results provide a very strong evidence base for the intervention. Given that ABC proved to be efficacious, implementing with community-based parent coaches was the critical next step.

Intervention Dissemination

Efforts to disseminate interventions in the community are typically much less effective than results achieved through randomized clinical trials in labs (e.g., Durlak & DuPre, 2008). One of the contributing factors is that interventions are often implemented with inadequate fidelity to the model (e.g., Hulleman & Cordray, 2009). Understanding the core components of an intervention – that is, the ingredients necessary for intervention effectiveness – facilitates implementation efforts (Fixsen, Naoom, Blase, Friedman & Wallace, 2005). One way that knowledge of core components facilitates implementation is through guiding the development of fidelity assessment centered on active components (Fairburn & Cooper, 2011). Therefore, the ABC fidelity assessment was designed to focus on a core component of ABC.

Defining fidelity

The critical core component of the ABC intervention is making frequent “in the moment” comments on the basis of observed parent-child interactions (Caron, Bernard, & Dozier, 2016). These comments focus attention on, encourage practice of, and enhance understanding and valuing of the intervention-targeted behaviors. Comments can include one or more of three components: 1) Describe the behavior, making it clear to parents exactly what behavior is referred to; 2) link the behavior to an intervention target, such as following the lead or nurturance; and 3) link the behavior to a child outcome, making it clear why the behavior is important. For example, if a mother picks up her child after he falls, a parent coach might say, “He started crying and you picked him right up” (i.e., describing the behavior). “We’ve been talking about how important nurturing behavior is, and that’s such a great example” (i.e., linking to intervention target). “This will help him know that you are there for him” (i.e., linking to child outcome). Evidence for in the moment comments as a core intervention component is found in their association with ABC outcomes. Higher frequency and quality of in the moment comments are associated with greater parent behavior change, as seen in the results of a randomized clinical trial in the lab (Meade & Dozier, 2012) and an observational study in a community dissemination site (Caron et al., 2016).

To quantify in the moment commenting, a micro-analytic coding system was developed. In this coding system, fidelity is conceptualized as the quantity (number of comments per minute) and quality (number of information components as described above, percentage of comments that are “on target” or appropriate given the observed parent behavior) of coaches’ comments. This coding system has facilitated fidelity monitoring in community settings. The coding system has also informed the certification and rostering of ABC parent coaches; one requirement for certification is whether specific criteria for quantity and quality of comments are demonstrated on recent fidelity assessments. In addition to these evaluation functions, the fidelity assessment is also used to train and supervise parent coaches. Parent coaches learn to code their own sessions, which has been shown important in increasing the number of comments they make (Meade, Dozier, & Bernard, 2014). Each week, parent coaches code a 5-minute clip from one of their sessions, with the clip chosen randomly. Parent coaches then meet remotely with a fidelity supervisor at the University of Delaware who has coded the same clip to receive feedback on commenting fidelity and compare coding discrepancies. The supervising staff member helps parent coaches refine their coding and their frequency and quality of commenting. These implementation procedures are expected to focus coaches’ attention on the core component of ABC, in the moment commenting, and in doing so, produce strong implementation outcomes at both the parent coach and the client level.

Community Implementation of ABC

Previous research with one of our initial dissemination sites, a group of nine parent coaches in Hawaii, found significant improvements in parental behaviors from pre- to post-intervention (Caron, Weston-Lee, Haggerty, & Dozier, 2016). In this paper, we present new data regarding pre- to post-intervention outcomes across five community sites and nine training cohorts with a larger number of coaches and a wide range of implementation settings. Of particular interest are effect sizes in sensitivity.

Method

Data were available for 108 parents seen by 37 parent coaches at 5 sites (9 training cohorts) between July, 2013 and May, 2016. At one site, parent coaches were highly trained and educated (e.g., with Ph.D. degrees), at three sites parent coaches had social work and master’s degrees, and at one site the coaches had B.A. degrees.

All 37 parent coaches attended a 2–3 day training. Parent coaches then received two types of weekly remote supervision with supervisors at the University of Delaware over the course of a year in order to promote implementation fidelity. They received one hour of general clinical supervision with an advanced-degree supervisor in a small group format. These general clinical supervision sessions focused on case conceptualization and case-specific challenges. Parent coaches also received weekly supervision on an individual basis that focused on commenting fidelity and coding, using the fidelity coding system. A written summary of each week’s fidelity supervision is provided to supervision staff and site-specific supervisors in order to provide feedback to all supervisors concerning parent coach progress. The director of ABC dissemination at the University of Delaware also regularly facilitates communication concerning parent coach progress among all supervisors, including staff and supervisors at each dissemination site.

The data used to measure parent outcomes here were originally collected for program evaluation purposes. The University’s IRB considered the research exempt because we used archived data that had been collected and coded for purposes of program evaluation, and which were de-identified.

Procedure

Prior to beginning session 1, parent coaches filmed a semi-structured play assessment with the parent and child. Parent coaches asked the parent to place the child in a child seat, and provided three toys: a rattle, a squeaking toy, and a set of stacking cups. Parents were instructed to interact with their children as they normally would, and the interaction was filmed for 9 minutes. Parent coaches implemented ten sessions of ABC, typically conducting sessions on a weekly basis, and videotaping sessions for the purposes of video feedback and supervision. After the final session of intervention, the play assessment was administered again, using the same protocol, typically on the same day as session 10. Parent coaches sent videos of the play interaction to the University of Delaware, where the videos were coded by reliable undergraduate coders for program evaluation purposes.

Measure

Parent behavior

Parent behavior during the play assessment was coded using scales adapted from the NICHD Observational Record of the Caregiving Environment (ORCE; NICHD ECCRN, 1996). The 4-point ORCE scales were expanded to 5-point scales to allow greater variability in ratings. We created a composite construct of caregiver sensitivity, derived from the three ORCE scales that best aligned with our parent behavior targets of ABC: sensitivity (responsiveness to non-distress), intrusiveness, and positive regard (NICHD ECCRN, 1999). These scales were averaged (intrusiveness reversed) to create a composite of overall parental sensitivity. For this sample, three undergraduate coders at the University of Delaware completed ratings, with 17% of play assessments double-coded. Reliability was excellent across scales, with one-way, random effects, single-measures ICCs of .79 for sensitivity, .80 for intrusiveness, and .80 for positive regard.

Results

Multilevel analyses were used to account for the nested structure of families within parent coaches, and parent coaches within training sites using Hierarchical Linear Modeling software (HLM; Raudenbush & Bryk, 2002). To capture parent behavior change across the 10 sessions, we calculated change scores for each family by subtracting the pre-intervention ratings of sensitivity from the post-intervention ratings. Change scores were then entered into HLM equations that lacked any predictor variables, to test whether the average expected change score (i.e., model intercept) was significantly different from 0. The tested model used the following form:

$${\text{SensitivityChange}}_{\mathit{ijk}}={\gamma }_{\mathit{000}}+u_{\mathit{00}k}+r_{\mathit{0}jk}+e_{\mathit{ijk}}$$

in which γ₀₀₀, the intercept, represents the estimated average amount of sensitivity change in each parent; u_00k represents the dissemination site-level error term, r_0jk represents the parent coach-level error term; and e_ijk represents the case-level error term. Including site-level and parent coach-level error terms accounted for the nested structure of the data while testing whether estimated sensitivity change differed from 0.

Parents showed higher levels of sensitivity at post-intervention than at pre-intervention (β₀₁ = 0.81, p < .01). See Figure 1 for means and standard deviations. To estimate the effect size, Cohen’s d was computed by dividing the unstandardized coefficient for sensitivity change by the standard deviation. Effect sizes were computed using the raw data for sensitivity change, due to the lack of a generally agreed upon method of computing effect sizes in HLM (cf. Niehaus, Campbell, & Inkelas, 2014). The effect size for pre- to post-intervention change in sensitivity was large (d = 0.83).

Figure 1.

Pre- and post-intervention scores for parental sensitivity for dissemination sites.

Conclusion and Future Directions

These results provide support for the ABC intervention in community sites when implemented by parent coaches ranging in experience and training. The effect size for sensitivity was as large as seen in controlled studies conducted in the lab. For example, as reported by Caron and colleagues (2016b), pre- to post-intervention effect sizes for change in sensitivity were .75 and .49 for samples of foster parents and internationally adopting parents, respectively, in randomized controlled trials. Though large, these effect sizes should be interpreted with some caution, as they are pre-to-post and not treatment-control comparisons, and because the measure was collected by parent coaches, which may have increased experimenter expectancy and demand characteristics, compared to assessors in randomized trials.

We attribute the strength of these results partially to a fidelity assessment instrument that allows careful monitoring and supervision of parent coach fidelity to the intervention. The quantified nature of the measure allowed the development of certification criteria and also supports iterative developmental goals. Because this assessment is feasible to code on a weekly basis, parent coaches can receive feedback on their recent implementation of ABC, immediately implement supervisors’ recommendations in their next sessions, and then receive iterative feedback on adjustments in the following weeks. Prior to the use of the fidelity tool, we struggled to train coaches to make ITM comments similar to how they were being used in the laboratory. However, since beginning the use of fidelity coding, only 5% of trainees across all dissemination sites have failed to reach certification criteria (apart from those who dropped out due to site-specific reasons unrelated to coach skill-level). Moreover, when examining a large sample of certified parent coaches from the past two years (N=56), by the end of the supervision year, parent coaches are making on average 1.5 comments per minute. These comments are of high quality - on-target 94% of the time and with at least one component (average of 1.38). Further, because parent coaches are trained to code their own sessions, they develop the ability to evaluate their own performance, which is expected to promote sustained fidelity after the ABC training period ends. We plan to study the use of and effectiveness of this sustainability plan as data become available.

Deterioration in fidelity leads to reduced treatment effects in community settings (Hulleman & Cordray, 2009), and indeed, observed variation in ABC commenting fidelity has predicted differential outcomes in community settings (Caron et al., 2016a). Following from these findings, one of the most important strategies to move interventions into community settings while maintaining strong treatment effects is to focus on treatment fidelity. Measurements of fidelity should focus on active or core components of intervention (Fairburn & Cooper, 2011), and should be validated through links to treatment outcomes in both lab and community settings. Effective and efficient fidelity measures (Schoenwald et al., 2011) can then be used in training, supervisory, and sustainment contexts to promote both initial growth and subsequent maintenance of treatment fidelity.

Consistent with implementation research (Durlak & DuPre, 2008; Fixsen et al., 2005; Stith et al., 2006), the success of ABC in the community data presented here is likely attributed to parent coach training, supervision and fidelity monitoring, as well as additional strategies not described here, including careful attention to implementation site selection, staff selection, and organizational supports. In future work, we plan to investigate the specific effects of fidelity monitoring and supervision on coaches’ implementation fidelity with a multiple baseline design. It will also be important to provide further support for intervention effectiveness through randomized clinical trials conducted in community sites. Although implementation with such attention to fidelity has challenges, such as high demand on staff time, these results support a focus on the precise use of core intervention components for the effective translation of an intervention from the laboratory to the community.