Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Oct 30.
Published in final edited form as: J Clin Child Adolesc Psychol. 2016 Apr 4;47(SUP1):S35–S46. doi: 10.1080/15374416.2016.1141359

In Vivo Feedback Predicts Parent Behavior Change in the Attachment and Biobehavioral Catch-up Intervention

EB Caron a, Kristin Bernard b, Mary Dozier a
PMCID: PMC6205922  NIHMSID: NIHMS1500643  PMID: 27043449

Abstract

Objective

Understanding mechanisms and active ingredients of intervention is critical to training clinicians, particularly when interventions are transported from laboratories to communities. One promising active ingredient of parenting programs is clinicians’ in vivo feedback regarding parent-child interactions. The present study examined whether a form of in vivo feedback, in the moment commenting, predicted treatment retention and parent behavior change when the Attachment and Biobehavioral Catch-up (ABC) intervention was implemented in a community setting.

Method

Observational data were collected from 78 parent-child dyads (96% mothers; mean age = 29 years; 81% minority; infants’ mean age = 12 months; 90% minority) across 640 sessions conducted by 9 clinicians (100% female, mean age = 39; 67% minority) in Hawaii. Parental behavior was assessed with a semi-structured play task before and after intervention. Clinicians’ in the moment feedback to parents was assessed from intervention session videos.

Results

Clinicians’ frequency and quality of in the moment feedback predicted change in parental intrusiveness and sensitivity at post-treatment. Frequency of in the moment feedback also predicted likelihood of retention. Hierarchical linear modeling demonstrated strong support for these associations at the between-clinician level, and limited additional support at the within-clinician (i.e., between-case) level.

Conclusions

A hypothesized active ingredient of treatment, in the moment feedback, predicted community-based ABC outcomes. The results complement lab-based evidence to suggest that in vivo feedback may be a mechanism of change in parenting interventions. Helping clinicians to provide frequent, high quality in vivo feedback may improve parenting program outcomes in community settings.

Keywords: parent training, implementation, active ingredients, in vivo feedback, mechanisms of change

Mechanisms of change and active ingredients of intervention are neglected topics in child psychotherapy research (Kazdin & Nock, 2003), but could provide critical guidance for dissemination and implementation efforts. A wide gap exists between evidence-supported treatments studied in laboratories and the therapy services available in the community (National Advisory Mental Health Council, 2001; President’s New Freedom Commission on Mental Health, 2004). Further, on average, evidence-supported treatments perform only somewhat better than treatment as usual in community settings (Weisz, Jensen-Doss & Hawley, 2006). Community-based under performance of evidence-supported treatments may be the result of a lack of treatment fidelity, that is, the treatment not being implemented as competently as it was in the laboratory (Hulleman & Cordray, 2009). Alternatively, components of treatment may not operate in community settings as they were designed to in laboratories because the children and families in these settings differ so markedly from the research participants with whom treatments were tested (Weersing & Weisz, 2002). For this reason, investigation of how treatment components predict outcomes in community implementation settings is critical to inform dissemination efforts (Fixsen et al., 2005). Specifically, identification of active ingredients of intervention can guide decisions about adaptations to intervention that can be allowed, versus adaptations that change a critical component (Fixsen et al., 2005). Identification of core intervention components can also inform training practices, as focusing on these aspects of intervention when training new clinicians is likely to promote implementation fidelity and treatment success. In this study, we examined whether in vivo feedback, a core component of parent coaching interventions, predicted treatment outcomes in community settings.

In vivo feedback is a therapeutic technique used in parenting interventions, in which a clinician supports a parent’s engagement in intervention-targeted behaviors by making comments about relevant parent behavior. In vivo feedback can be both directive, scaffolding a parent’s engagement in targeted behavior, and responsive, responding to a parent’s behavior with praise or constructive criticism (Borrego & Urquiza, 1998). In the moment feedback, a component of the Attachment and Biobehavioral Catch-up(ABC) intervention, is one style of in vivo feedback, in which feedback is delivered in families’ homes, as parents and children interact during intervention sessions. In ABC, in the moment feedback is informally integrated with manual-guided discussion and video feedback. Live coaching is another approach to in vivo feedback, used in Parent-Child Interaction Therapy (PCIT; McNeil & Hembree-Kigin, 2010), an evidence-supported parent training treatment for children with behavioral problems. In PCIT, clinicians deliver feedback from behind a one-way mirror, via a “bug-in-the-ear” device, as parents interact with their children in the therapy room. In vivo feedback has also been used to coach intervention-targeted behavior with parents of children with autism (e.g., Robbins, Dunlap & Plienis, 1991) and traumatic brain injury (Antonini et al., 2014). Developments in technology have led clinicians to begin conducting in vivo coaching sessions with parents via computer-based video conferencing (e.g., Antonini et al., 2014). Thus, in vivo feedback is a component of several parenting interventions, although its style and delivery can differ across interventions.

Preliminary evidence supports in vivo feedback as a key ingredient of ABC, PCIT and other parenting interventions. Meta-analytic work finds that, of 18 identified components of parenting interventions, requiring in vivo practice with the parent’s own child is one of the strongest and most consistent predictors of improved parenting behavior (Kaminski, Valle, Filene & Boyle, 2008). Shanley and Niec (2010) experimentally manipulated receipt of in vivo coaching sessions in a group of mothers of non-referred children. Specifically, they examined the impact of two 15-minute PCIT-like coaching sessions on positive parenting skills, and found that mothers assigned to the coaching group increased in their frequency of the targeted skills, whereas the skills of the non-coached mothers tended to decline (Shanley & Niec, 2010).

Several studies have examined correlational relationships between frequency of in vivo feedback and improvement in parenting behaviors during intervention. Barnett, Niec and Acevedo-Polakovich (2014) found that PCIT therapists’ frequency of responsive feedback in the second or third coaching session of PCIT was associated with parents’ frequency of labeled praises with their children during the next coaching session. In a different sample,Barnett et al. (2015) found that PCIT therapists’ frequency of responsive feedback in the first coaching session predicted the number of sessions it took parents to attain skill mastery. Meade and Dozier (2012) found that ABC clinicians’ frequency of in the moment feedback in the third session of ABC predicted higher frequency of intervention-targeted parent behavior (i.e., following the child’s lead) in the ninth session, even when controlling for the parent’s prior level of this behavior. Thus, preliminary evidence suggests that in vivo coaching acts as an active ingredient in several parenting interventions.

However, limitations of these studies create a need for additional research on in vivo feedback. The Kaminski et al. (2008) meta-analysis examined the effect of in vivo practice, and not the specific effect of receiving feedback from a therapist during these practice sessions. The only experimental study of in vivo feedback (Shanley & Niec, 2010) was an analogue study, and did not test the effects of in vivo feedback with real clinical cases. The three correlational designs (Barnett et al., 2014; Barnett et al., 2015; Meade & Dozier, 2012), while coming from treatment studies, also have limitations. By not statistically addressing their nested research designs (i.e., families nested within clinicians),Barnett et al. (2014) and Meade and Dozier (2012) risked confounding therapist effects with true case-level effects of in vivo feedback. That is, the correlations between in vivo feedback and future parent behavior found in these studies may be better explained by therapist-level effects, with stronger therapists providing more in vivo feedback, and also producing more behavior change in parents, than by direct effects of in vivo feedback at the case level. Although Barnett et al. (2015) examined therapist nesting as a random effect, they eliminated the variable from the model because it was not significantly associated with parent skill mastery. To advance research on in vivo feedback, a more intensive multilevel modeling approach could examine effects at the case level while controlling for effects at the therapist level, to provide stronger evidence for associations with outcome.

The association between in vivo feedback and treatment retention is another area for further study. Barnett et al. (2015) found that parents who completed PCIT treatment received more frequent responsive/positive feedback in the first coaching session than parents who subsequently dropped out of treatment. No differences were found, however, in the frequency of directive or corrective feedback received by these parents (Barnett et al., 2015). As only one study to date has examined these associations, further investigation is critical.

Thus, although preliminary evidence for in vivo feedback as a potential mechanism of change is promising, additional research is needed. The lack of community-based research represents a particularly critical gap in the literature (Weersing & Weisz, 2002), as no evidence to date has demonstrated that in vivo feedback continues to predict parent behavior change or treatment retention when parenting programs are transported to communities. Examination of the role of in vivo feedback in a community-implemented parenting intervention would not only expand theoretical knowledge of this treatment component, it would also inform efforts to disseminate and implement evidence-based parenting interventions.

The present study examined the role of in vivo feedback in predicting parent behavior change when ABC was implemented in the community during a training period in which clinicians received ongoing supervision. In previous work with this community sample, we had found evidence of changes in parental behaviors after participation in ABC (Caron, Weston-Lee, Dozier & Haggerty, 2016). In this study, we examined whether the frequency and quality of clinicians’ in vivo feedback predicted parent behavior change and treatment retention. Additionally, we explored whether these characteristics of in vivo feedback had effects at the clinician level or at the case level.

Method

ABC Intervention

Attachment and Biobehavioral Catch-up (ABC) is a ten-session, manualized, home visiting intervention designed to address the attachment and regulatory problems of infants who have experienced early adversity, including maltreatment. Effects of ABC include decreased rates of disorganized attachment and increased rates of secure attachment (Bernard et al., 2012), normalized diurnal cortisol patterns (Bernard, Dozier, Bick & Gordon, 2015) that are sustained 2–3 years following intervention (Bernard, Hostinar & Dozier, 2015), improved executive functioning (Lewis-Morrarty, Dozier, Bernard, Terraciano & Moore, 2012) and enhanced emotion expression (Lind, Bernard, Ross, & Dozier, 2014).

ABC is thought to exert its effects on children by changing parents’ behaviors. Specifically, ABC targets three aspects of parent behavior: providing sensitive, nurturing care when children are distressed; contingently following children’s lead with delight (i.e., genuine positive affect/positive regard) when children are not distressed; and avoiding behavior that may frighten children. As clinicians implement ABC in families’ homes, they encourage targeted parent behaviors through manual-guided discussion, video feedback, and in the moment comments. ABC has been successful in improving these parent behaviors in both randomized clinical trials (Bernard, Simons & Dozier, 2015) and in community settings (Caron et al., 2016).

In the moment commenting

When implementing ABC, clinicians are expected to make in the moment comments about parent behavior at least once per minute. Clinicians use in the moment feedback to highlight times when parents engage in intervention-targeted behaviors; 1) describing the specific behaviors of the parent (so that the parent understands what behavior is referred to), 2) linking behaviors to the intervention targets (so that the parent understands how a specific behavior relates to global intervention targets), and 3) indicating the long-term outcomes of the parent’s behaviors (so that the parent learns how the behavior positively affects the child). An example of a comment that includes all of these components is, “He held the cup up to his mouth, and you pretended to drink too” (behavior description); “That’s following his lead” (intervention target); “He’s learning he has an effect on the world, and that’s going to help with his self-control in the future” (outcome).

Whereas the vast majority of in the moment feedback consists of positive (i.e., praising) responses to targeted behaviors, clinicians can also use in the moment comments to respond to negative parent behaviors. For example, when a parent is disengaged from her child, clinicians can make in the moment comments that scaffold targeted behaviors, such as, “Oh, he just picked up the drum. How can you follow his lead right now?” or “He’s tapping the blocks together… why don’t you try doing that too?” When a parent engages in intrusive or controlling play, clinicians can make comments that help the parent to recognize the behavior, such as “Hold on a second. Who’s leading right now?” Finally, with parents who overwhelmingly engage in negative behaviors with their children, clinicians can make shaping comments that highlight the positive aspects of an otherwise undesirable behavior sequence. For example, if the parent tickles the child in an over-stimulating way, the clinician could comment, “He was signaling that was too much, and you noticed his signals and stopped.”

Implementation of ABC

Ten clinicians from public and private agencies in Hawaii were invited to participate in ABC training. Clinicians were recruited and selected by an administrator at a foundation that funded the training. The administrator recruited clinicians by speaking with directors of social service agencies that were capable of reaching infants in their homes, to describe ABC and discuss the requirements of the training year. The administrator then collaborated with agency directors to determine which employee at each agency would be best suited for, and interested in, implementing ABC.

In March 2012, 10 clinicians were trained in a 3-day training seminar. Following the seminar, 9 clinicians began a year of ongoing supervision; one clinician moved to a supervisory position in her agency and did not continue ABC training. Although several cultural translations of ABC, such as incorporating local knowledge and culture, were observed to occur naturally, supervision ensured that these translations did not compromise essential aspects of ABC. Supervision was conducted weekly via video conferencing, and included clinical group supervision and commenting-focused individual supervision. Clinical group supervision was led by a Ph.D. level supervisor, and addressed case conceptualization, comfort with discussing manual content, and case-specific strategizing through video review of sessions. Commenting-focused individual sessions, described more below, were led by undergraduate supervisors who were experts in the in the moment comment coding system. Seven clinicians completed a year of follow-up supervision; one clinician discontinued training due to personal reasons, and another clinician discontinued due to limited time for ABC training and implementation. However, data from all 9 clinicians were used in analyses predicting parent retention in treatment, and data from 8 clinicians (all clinicians with at least one family that completed a post-intervention assessment) were used in analyses predicting parent behavior outcomes.

Commenting-focused supervision sessions were designed to promote adherence to and competence in the hypothesized active ingredient of ABC, in the moment commenting. In preparation for these sessions, clinicians and their supervisors coded commenting from 5-minute video clips of recent sessions. During the supervision session, clinicians received feedback on both their coding accuracy and their in the moment commenting. Feedback on coding was expected to help refine clinicians’ understanding of intervention-targeted parent behavior, whereas feedback on commenting was expected to provide clinicians with explicit evaluation of their performance, as well as guidance on how to improve performance. Specifically, clinicians were coached to make more frequent and high quality comments.

Participants

Clinicians

Nine clinicians from public and private agencies across several Hawaiian islands participated in the ABC training period. All clinicians were female. Most (8 of 9) had master’s degrees, with one clinician reporting attending but not graduating from college. Three clinicians (33%) were white/non-Hispanic, 3 (33%) were Asian American, 1 (11%) was Native Hawaiian, and 2 (22%) were multi-racial. Clinicians ranged in age from 27 to 56 (M = 39.1, SD= 10.2). On average, they reported 9.9 years (SD = 5.2) of experience in the social work field, and 4.3 years (SD = 3.6) in their current jobs.

Families

Seventy-eight families participated in ABC during the training period. Primary caregivers involved in the intervention were primarily birth mothers (n = 66, 85%), although several birth fathers (n = 3, 4%) and foster mothers (n = 9, 12%) were primary caregivers. In several cases (n = 19, 24%), additional caregivers participated in ABC sessions; however, in this study, only the primary caregiver’s behavior change data were used. Referral reasons were assessed with an open-ended question answered by clinicians, and responses were coded to characterize the sample. Many families (n = 29, 37%) were referred due to involvement with Child Protective Services or Child Welfare Services. A number of additional families (n = 21, 27%) were referred due to observed harsh or neglectful parenting. Other reasons for referral (n = 29, 36%) included: referrals by other service providers (e.g., home visitors); past separations between the parent and child; observed difficulties with ABC-targeted behavior; and self-referral.

Parents ranged in age from 17 to 46 (M = 28.6, SD = 6.8) years old. Half (n= 39, 50%) were of mixed racial background, and 10 (13%) were white/non-Hispanic, 4 (5%) white/Hispanic, 1 (1%) African American, 7 (9%) Asian American, 8 (10%) Native Hawaiian, and 5 (6%) other Pacific Islander; data were missing for 4 (5%) caregivers. Primary caregivers with mixed racial background frequently had ancestry that included Native Hawaiian (92%), white/non-Hispanic (76%), Asian American (46%), and other Pacific Islander (30%) descent.

Children ranged from 2 to 27 months of age (M =12.2, SD = 7.4) when they began ABC. Most children (n = 57, 73%) had mixed racial backgrounds, and 5 (6%) were white/non-Hispanic, 2 (3%) white/Hispanic, 1 (1%) African American, 1 (1%) Asian American, 7 (9%) Native Hawaiian, and 2 (3%) other Pacific Islander; data were missing for 3 (4%) children. Children with mixed racial background frequently had ancestry that included Native Hawaiian (83%), white/non-Hispanic (77%), Asian American (56%), and other Pacific Islander (47%) heritage, and 12% of children with mixed racial background were identified as Hispanic.

Procedure

Before session 1, clinicians filmed a semi-structured parent-child play assessment. The child was placed in a child seat, and the clinician gave several toys to the child. The parent was instructed to interact with the child and he or she normally would, and parent-child interactions were recorded for 9 minutes.

After the play assessment was conducted, clinicians implemented the ABC intervention with the family. Clinicians conducted 10 sessions of ABC, usually on a weekly basis. Sessions were videotaped for the purposes of video feedback with the family, as well as clinician supervision. After the tenth session, the play assessment was re-administered, using the same toys and protocol, typically on the same day as the final intervention session.

Clinicians sent play assessment and intervention session videos to University of Delaware. There, one team of undergraduate coders coded the play assessments and a separate team of undergraduates coded the intervention sessions. Coders were unaware of hypotheses, and the parent behavior coders were naive to the frequency/quality of in the moment feedback received by the family. At the end of the training period, clinicians were asked to report demographic information about themselves and their cases. They received a $50 gift card for completing these demographic questionnaires. Families’ consent for participation in ABC and video release to University of Delaware was managed by clinicians and agencies, and the team at University of Delaware did not have access to copies of consent forms or participants’ names. Videos were coded to support clinical supervision and program evaluation, and once coded, all archived clinical data lacked any identifying information. As such, the University’s IRB considered the research exempt and approved archival data analysis.

Measures

Parent behavior

Parent behavior was coded from the semi-structured play assessments, using scales adapted from the NICHD Observational Record of the Caregiving Environment (ORCE; NICHD ECCRN, 1996). Although the ORCE scales are often collapsed to create a composite of parental sensitivity, we retained separate scales in order to assess distinct targets of ABC. The scales coded were sensitivity/responsiveness to non-distress (i.e., following the child’s lead), positive regard (i.e., delight in the child), and intrusiveness. To allow greater variability in ratings, the original 4-point ORCE scales were expanded to include 5 points, a decision with precedent in multiple studies (Barnett, Gustafsson, Deng, Mills-Koonce & Cox, 2012; Mills-Koonce et al., 2007; Propper, Willoughby, Halpern, Carbone & Cox, 2007). Four coders completed coding of parent behavior, with pre- and post-treatment play assessments coded by separate coders.

Fifteen percent of play assessment videos were double-coded, and reliability ranged from fair to excellent, with one-way, random effects Intraclass Correlation Coefficients (ICCs) of .53 for positive regard, .65 for sensitivity, and .85 for intrusiveness. One-way, random effects ICCs are appropriate when a variety of coder pairs rate a subset of the sample (Shrout & Fleiss, 1979), and ICCs in the range of .40–.59 are considered fair, .60–.74 considered good, and above .75 considered excellent (Cicchetti & Sparrow, 1981).

In the moment feedback

Clinicians’ in the moment feedback was assessed through the in the moment comment coding system. This coding system includes two parts: first, intervention-targeted parent behaviors are coded, and second, clinician responses to each of these opportunities to comment are coded. When a clinician fails to respond to a targeted behavior, the clinician is coded as missing an opportunity to comment. When a clinician does comment on a targeted behavior, certain characteristics of comment quality are coded: (a) on/off-target, and (b) information components.

On/off-target

The target of clinician comments is the intervention-targeted behavior to which the comment refers. Comments that respond relevantly and appropriately to parent behaviors are “on-target.” In contrast, “off-target” comments may inappropriately reinforce undesired behaviors (e.g., commenting that the parent was following the child’s lead when the parent was actually correcting the child’s play), respond to a relevant behavior with a comment highlighting another behavior target (e.g., referring to a nurturing behavior as following the lead), or address topics unrelated to the ABC intervention (e.g., nutrition, sleep).

Information Components

The number of information components of comments is coded from 0 to 3. Comments can include three components: 1) specifically describing the parent-child interaction (“She said “gaga,” and you said “gaga” back to her”), 2) connecting the parent behavior to the relevant intervention target (“That's following her lead”), and 3) indicating long-term outcomes of the behavior (“That’s helping her to develop the ability to regulate herself”). Comments that include no full components (scored as 0) may describe the parent’s behavior in non-specific terms (“Great responding to that signal”) or give praise without additional information (e.g., “Perfect, that’s exactly what we’re talking about”).

Coding is completed on an Excel spread sheet that automatically calculates summary statistics about the clinician’s commenting in the session. The summary variables include two measures of comment frequency, on-target comment frequency (i.e., number of on-target comments in the 5-minute clip), and percentage of missed opportunities (i.e., percentage of parent behaviors that were not commented on), a measure of frequency that accounts for the number of commenting opportunities available in a 5-minute clip and does not penalize off-target comments. The summary variables also include two measures of comment quality, average number of information components provided in on-target comments, and percentage of comments that were on-target (i.e., number of on-target comments divided by total number of comments).

For this study, 5-minute clips of all sessions available for the 78 families were coded. ABC video clips of 5 minutes in length have measured in the moment feedback effectively in prior work (Meade & Dozier, 2012), and video segments of 5 to 10 minutes have been used in Barnett et al.’s (2014, 2015) work on in vivo feedback in PCIT. In the current study, in which the clips were assigned to both independent coders and clinicians for use in commenting-focused supervision, this length of measurement was also considered feasible by both parties. Though process variables coded from short segments may not accurately represent individual sessions, aggregating short segments results in a reasonably accurate estimate of overall process (Friedlander et al., 1988). As such, session 1–5 data were aggregated at the case level, as described below. Video clips were randomly selected in 5-minute increments from 6 possible segments between minute 10 and minute 40 of session videos (i.e., randomly selected times could be: 10–15, 15–20, 20–25, 25–30, 30–35, or 35–40) by rolling a 6-sided die. If the randomly selected video clip had significant problems affecting coding (e.g., parent off-screen for long segments), the coding times were adjusted forward or backward to eliminate the problem while maintaining as much of the randomly selected segment as possible.

Of the possible 780 videos (78 families×10 sessions per family), video clips were coded for 640 sessions. Twenty-one families dropped out of ABC prior to session 10, and 134 of the expected sessions were never completed. Video files were missing/unavailable for an additional 6 sessions. Coding was completed by 15 undergraduate coders. Coders did not code same-case sessions within 2 sessions of another session they had coded, such that no more than 4 sessions per case (of 10 total sessions) were coded by any individual coder. Interrater reliability for in the moment feedback was assessed from 40 double-coded videos of ABC sessions, with reliability across session-level in the moment commenting summary variables good to excellent (ICCs ranged from .69 to .94). For the purposes of this study, summary variables were averaged across sessions 1–5 to create composite variables for each case. These composite variables were intended to increase reliability and accuracy of in the moment feedback measurement by averaging across multiple observations (cf. Friedlander et al., 1988). The decision to average variables across sessions 1–5 was guided by Kazdin and Nock’s (2003) discussion of establishing a timeline in mediation/mechanism research. Measuring a potential mediator of change (i.e., in the moment feedback) early in treatment reduces the likelihood that the outcome (i.e., parent behavior) is changing before, and potentially evoking observed variability in, the hypothesized mediator.

Analyses

We conducted multivariate models in hierarchical linear modeling (HLM), with cases (at level 1) nested within clinicians (at level 2) to account for the non-independence of data. This approach also allowed us to parse the contributions of predictors at level 1 (i.e., variables at the level of the case, such as frequency of comments directed toward a particular parent/family) and predictors at level 2 (i.e., variables at the level of the clinician, such as average frequency of commenting across cases). Due to the lack of a generally agreed upon method of computing effect sizes in HLM (cf. Niehaus, Campbell, & Inkelas, 2013), we do not present effect sizes.

Retention

After exploring differences in average characteristics of commenting received by cases that dropped out of versus completed intervention, we tested 3 multi-level models to examine how specific aspects of commenting, specified at both the case level (i.e., level 1) and the clinician level (i.e., level 2), predicted treatment retention. Because retention had a binomial distribution, Bernoulli HLM models were used. Each multi-level model followed this general form:

  • Level 1: Prob(Retainedij=1|βj) = ϕij

    log[ϕij/(1 - ϕij)] = ηij

    ηij = β0j + β1j*(Case-Commentingij)

  • Level 2: β0j = γ00 + γ01*(Clinician-Commentingj) + u0j

    β1j = γ10 + u1j

with Clinician-Commenting grand-centered and Case-Commenting group-centered. In these equations, the variables of interest wereγ01, representing the average estimated effect of clinician commenting across cases on retention, and γ10, representing the average estimated effect of case-specific clinician commenting on retention.

Parent Behavior Change

After exploring bivariate correlations between behavior change variables and commenting characteristics, we tested 8 multi-level models to examine how 4 aspects of commenting predicted change in sensitivity and intrusiveness. Level 1 (i.e., case-level) predictors included the case’s session 1–5 average of a commenting variable, and the case’s pre-intervention parent behavior score, as parents who scored low on sensitivity and high on intrusiveness at pre-intervention tended to show the most change (r (56) = −.66, p< .001 for sensitivity and r (56) = −.72, p< .001 for intrusiveness). We also included the commenting variables as predictors at level 2, the clinician level. Each multi-level model followed this general form:

  • Level 1: Behavior Differenceij= β0j + β1j*Pre-Behaviorij + β2j*Case-Commentingij + rij

  • Level 2: β0j = γ00 + γ01*Clinician-Commentingj + u0j

    β1j = γ10 + u1j

    β2j = γ20 +u2j

with Clinician-Commenting grand-centered and Pre-Behavior and Case-Commenting group-centered. In these equations, the predictors of interest wereγ01, representing the average estimated effect of clinician commenting across cases on parent behavior change, and γ10, representing the average estimated effect of case-specific clinician commenting on parent behavior change.

Results

Predictors of Retention

Twenty-one families (27% of the families enrolled) dropped out of ABC prior to session 10; 73% of families were retained through treatment completion. Families who dropped out of treatment were not statistically different from those who completed treatment on a variety of demographic characteristics; a table comparing demographic characteristics of dropouts and completers is available from the authors upon request. To examine potential characteristics of in vivo feedback as predictors of retention in treatment, we first conducted t-tests, comparing the on-target comment frequency, percentage of missed opportunities, percentage of on-target comments, and average number of information components in comments, across sessions 1–5, between families who completed ABC and those who dropped out. As shown in Table 1, families who dropped out of ABC tended to receive comments less frequently, and comments that were less often on-target, than families who completed ABC. Families who dropped out of ABC also tended to engage in more behaviors that were not commented on by clinicians than families who completed treatment, as reflected by higher percentages of missed opportunities. Based on these findings, we specified three multi-level models with on-target comment frequency, percentage of missed opportunities, and percentage of on-target comments as predictors of retention.

Table 1.

Univariate T-tests Comparing Comments Received by ABC Completers and Dropouts

Commenting Variable ABC
Completers		 ABC
Dropouts		 t-ratio p-
value
Session 1–5 On-Target Comment Frequency (# Comments/5 Minute Clip) 5.23 (2.78) 3.17 (1.77) 3.16 <.01
Session 1–5 % Missed Opportunities 59.91 (14.43) 69.39(17.83) 2.41 .02
Session 1–5 % On-Target Comments 83.06 (16.06) 69.76(27.19) 2.11 <.05
Session 1–5 Average # Components 1.05 (.36) .95 (.33) 1.05 .30
Open in a new tab

As shown in Table 2, when clinicians commented more frequently when implementing ABC, the families seen by these clinicians were more likely to be retained throughout treatment. An odds ratio of 2.86 suggests that a family seen by a clinician whose average rate of commenting was 1 comment (per 5-minute clip) above the group mean was 186% more likely to complete treatment than a family seen by a clinician whose commenting was at the average rate. Additionally, when clinicians missed more opportunities to comment in ABC sessions, the families seen by these clinicians were less likely to complete treatment, compared to families seen by clinicians who missed fewer commenting opportunities. An odds ratio of 0.86 suggests that each percentage point of missed commenting opportunities above the group mean of clinicians was associated with a 16% (1/0.86) decrease in likelihood of a family completing treatment.

Table 2.

Hierarchical Linear Models Testing Commenting Variables as Predictors of Retention

Effect Coefficient SE t-ratio OR (CI) p-value
Comment Frequency Predicting Retention
Intercept, γ00 1.30 .62 2.09 3.68 (.85, 16.01) .08
Clinician-level comment frequency, γ01 1.05 .42 2.47 2.86 (1.05, 7.79) .04
  Case-level comment frequency, γ10 .20 .18 1.10 1.22 (.80, 1.87) .30
Percentage of Missed Opportunities Predicting Retention
Intercept, γ00 1.00 .58 1.73 2.71 (.69, 10.62) .13
Clinician-level missed opportunities, γ01 −.16 .06 −2.61 .86 (.74, 0.99) .04
  Case-level missed opportunities, γ10 .00 .02 .03 1.00 (.95, 1.05) .98
Percent On-Target Comments Predicting Retention
Intercept, γ00 1.18 .82 1.44 3.25 (.47, 22.54) .19
Clinician-level percent on-target, γ01 .04 .05 .81 1.05 (.92, 1.19) .44
  Case-level percent on-target, γ10 .04 .02 2.24 1.04 (.999, 1.09) .06
Open in a new tab

Note. SE = Standard Error; OR = Odds Ratio; CI = Confidence Interval.

Predictors of Parent Change

To examine potential predictors of parent change, we first explored bivariate correlations among the commenting variables and parent behavior change scores. As shown in Table 3, there were no significant associations between positive regard changes and any commenting variables. Based on these findings, we specified multi-level models using all four commenting variables as predictors, and change in parent sensitivity and intrusiveness as outcomes.

Table 3.

Means, Standard Deviations, and Bivariate Correlations of Predictor and Outcome Variables

Predictor Mean (SD) 1 2 3 4 5 6 7 8 9
1. Pre-Intervention Sensitivity 2.84 (1.18) --
2. Pre-Intervention Positive Regard 3.65 (1.10) .51*** --
3. Pre-Intervention Intrusiveness 3.01 (1.21) −.72*** −.25* --
4. Session 1–5 On-Target Comment Frequency (# Per 5 Min.) 4.68 (2.70) .13 .13 .07 --
5. Session 1–5 % Missed Opportunities 62.47 (15.88) −.10 −.14 −.05 −.70*** --
6. Session 1–5 % On-Target Comments 79.48 (20.37) .02 .13 .11 .52*** −.41*** --
7. Session 1–5 Average # Components 1.02 (.35) −.01 .04 .15 .48*** −.29* .29* --
8. Sensitivity Change Score 1.04 (1.54) −.66*** −.24 .44** .18 −.15 .33* .22 --
9. Positive Regard Change Score .41 (1.28) −.31* −.71*** .26 .10 .03 .14 .13 .50*** --
10. Intrusiveness Change Score -1.32 (1.54) .47*** .18 −.74*** −.37** .32* −.42** −.19 −.56*** −.23
Open in a new tab

Note.

*

p< .05,

**

p< .01,

***

p< .001.

As shown in Table 4, across several models, the clinician’s average on-target comment frequency, percentage of missed opportunities, percentage of on-target comments, and number of comment components predicted parent behavior change. Both measures of comment frequency, on-target comment frequency and percentage of missed commenting opportunities, were associated with greater reductions in parental intrusiveness. With regard to comment quality, clinicians’ higher percentage of on-target comments and inclusion of more components in comments were associated with larger increases in parental sensitivity and reductions in intrusiveness.

Table 4.

Hierarchical Linear Models Testing Commenting Variables as Predictors of Change

Effect Coefficient SE t-ratio p-value
Comment Frequency Predicting Change in Sensitivity
Intercept, γ00 .71 .23 3.03 .02
  Pre-intervention sensitivity, γ10 −.92 .14 −6.45 <.01
Clinician-level comment frequency, γ01 .28 .12 2.38 .06
Case-level comment frequency, γ20 .08 .07 1.18 .28
Percentage of Missed Opportunities Predicting Change in Sensitivity
  Intercept, γ00 .65 .26 2.51 .05
  Pre-intervention sensitivity, γ10 −.88 .15 −5.90 <.01
Clinician-level missed opportunities, γ01 −.06 .03 −2.03 .09
Case-level missed opportunities, γ20 .01 .01 0.39 .71
Percentage of On-Target Comments Predicting Change in Sensitivity
  Intercept, γ00 .56 .19 2.95 .03
  Pre-intervention sensitivity, γ10 −.90 .14 −6.59 <.01
Clinician-level percent on-target, γ01 .07 .02 3.91 <.01
Case-level percent on-target, γ20 .02 .01 1.71 .13
Number of Comment Components Predicting Change in Sensitivity
  Intercept, γ00 .71 .18 4.02 <.01
  Pre-intervention sensitivity, γ10 −.87 .14 −6.20 <.01
Clinician-level components, γ01 2.19 .67 3.27 .02
Case-level components, γ20 1.05 .80 1.31 .23
Comment Frequency Predicting Change in Intrusiveness
  Intercept, γ00 −.47 .40 −1.18 .28
  Pre-intervention intrusiveness, γ10 −.80 .21 −3.74 .01
Clinician-level comment frequency, γ01 −.61 .19 −3.13 .02
Case-level comment frequency, γ20 −.15 .06 −2.68 .03
Percentage of Missed Opportunities Predicting Change in Intrusiveness
  Intercept, γ00 −.40 .35 −1.15 .29
  Pre-intervention intrusiveness, γ10 −.66 .24 −2.80 .03
Clinician-level missed opportunities, γ01 .13 .03 4.48 <.01
Case-level missed opportunities, γ20 −.00 .01 −.10 .92
Percentage of On-Target Comments Predicting Change in Intrusiveness
  Intercept, γ00 −.66 .26 −2.53 .05
  Pre-intervention intrusiveness, γ10 −.76 .22 −3.49 .01
Clinician-level percent on-target, γ01 −.08 .02 −4.12 <.01
Case-level percent on-target, γ20 −.03 .01 −2.47 .04
Number of Comment Components Predicting Change in Intrusiveness
  Intercept, γ00 −.30 .51 −.60 .57
  Pre-intervention intrusiveness, γ10 −.87 .24 −3.62 .01
Clinician-level components, γ01 −4.57 1.82 −2.52 <.05
Case-level components, γ20 −.67 .59 −1.14 .29
Open in a new tab

Note. The estimates in the coefficient column of represent the expected change in sensitivity or intrusiveness associated with a 1-unit change in each predictor variable.

Additionally, for intrusiveness, case-level on-target comment frequency and percentage of on-target comments were predictors of change, indicating that variations in ABC implementation within a clinician’s caseload also predicted outcome, above and beyond the effects of the clinician’s average frequency/quality of commenting. Within a clinician’s caseload, parents that received more frequent comments and a higher percentage of on-target comments showed the greatest reductions in intrusiveness.

Discussion

This study demonstrated that four characteristics of in the moment feedback predicted treatment outcome in the ABC intervention. Specifically, the frequency of on-target comments, the percentage of missed commenting opportunities, the percentage of comments that were on-target, and the number of components included in comments predicted increases in sensitivity and decreases in intrusiveness, when pre-intervention levels of parent behavior were controlled. Frequency of commenting and percentage of missed opportunities were also found to predict retention in treatment. Findings were robust across four predictors of change and both behavioral and treatment retention outcomes.

We found no association between in the moment feedback and change in parental positive regard. There are two factors that made this association less likely to emerge as significant than other predicted associations. First, interrater reliability for this parenting behavior was the lowest of the three parent behavior scales (ICC = .53). Second, the effect size (and absolute value) of change in positive regard was smaller than the other two parent behaviors, suggesting that change in positive regard was less robust as an outcome, and therefore less useful as an outcome that could be predicted, than change in the other two scales. These factors lead us to interpret the null results for in the moment feedback-positive regard associations with caution. However, it is possible that positive regard is more difficult to change, through parenting intervention in general and the process of in vivo feedback in particular, than other ABC intervention targets.

Though associations between in the moment feedback and treatment outcomes were more consistent at the clinician level, associations were also found at the individual case level. Associations at the clinician level mean that some clinicians tend to generate better outcomes in families than other clinicians, and these clinicians are also more likely to implement ABC with strong adherence and competence (i.e., make more frequent, informative, and on-target comments). Thus, one interpretation of these findings is that some clinicians are stronger than others, both at implementing the intervention as intended, and at retaining parents in treatment and helping them to change, which may not seem particularly ground breaking. Perhaps more compelling, then, are the associations at the individual case level, which show that within each clinician’s caseload, the families that demonstrated the greatest reductions in intrusiveness were those that received more frequent and higher quality comments. By going beyond demonstrating therapist effects, effects at the individual case level provide compelling evidence that the in the moment feedback received by a family predicts their treatment outcome, above and beyond what could be expected from knowing the clinician’s average implementation of comments. We found these case level effects with our most reliable scale of parent behavior, intrusiveness (ICC = .85), suggesting that the lower measurement precision of other parent behavior scales may have interfered with our ability to observe consistent case-level effects.

These results provide evidence that in the moment feedback is an active ingredient of ABC, and support a focus on in the moment comments in training, supervising, and measuring progress of new clinicians. Because frequency and quality of in the moment comments predict parent behavior change and retention in treatment, helping clinicians to refine their use of in the moment feedback is likely to enhance treatment outcome. These results support use of supervision processes that focus attention on making frequent, high quality in the moment comments, including having clinicians code their own comments from video clips of their sessions, and providing commenting-focused supervision sessions led by expert coders.

These results also expand the evidence base for the broader construct of in vivo feedback as an active ingredient in parenting interventions. Prior studies have linked in vivo feedback to increased positive parenting behavior; in addition to replicating this finding, this study demonstrates that in vivo feedback can also predict decreases in negative parenting behavior. In most work demonstrating associations between in vivo feedback and parenting behavior (Barnett et al., 2014; Meade & Dozier, 2012; Shanley & Niec, 2010), parents have been relatively homogenous and primarily European American. Barnett et al. (2015) expanded these findings to a predominantly Hispanic sample, and the present sample, which included many families with Asian American, Native Hawaiian and other Pacific Islander descent, further expands the populations with whom in vivo feedback has been found to predict change. Further, hierarchical linear modeling was used to demonstrate that these predictive associations were present at both the case level and the clinician level, providing robust evidence supporting the role of in vivo feedback as a predictor of parent behavior change. Critically, we found these results in a sample of clinicians working in community agencies, suggesting that in vivo feedback is an active ingredient of intervention across settings, not just in the lab. In sum, this study joins a handful of others to go beyond results suggesting that treatment adherence predicts parent behavior change and demonstrate evidence for a specific active ingredient of parenting interventions.

Thus, the results of this study suggest that in vivo feedback may be a potential mechanism of change in ABC and perhaps other parenting interventions. Kazdin and Nock (2003) defined mechanisms of change as processes that cause therapeutic change, and asserted that studying mechanisms is likely “the best short-term and long-term investment for improving clinical practice and patient care” (p. 1117). This study provided some evidence for several of Kazdin and Nock’s criteria for establishing mechanisms. We showed a strong association between frequency/quality of in the moment feedback and parent behavior change, and provided some evidence for the temporal relation, with parent behavior measured before and after treatment, and characteristics of in the moment feedback averaged across the first half of treatment.

We also found evidence that frequency of in the moment feedback, as assessed in two ways (on-target comment frequency and percentage of missed commenting opportunities), predicted treatment retention, consistent with the findings of Barnett et al. (2015). Although we did not specifically examine the frequency of positive comments as compared to negative or scaffolding comments, the vast majority of in vivo feedback in ABC is positive. Specifically, ABC clinicians are advised not to make any negative or scaffolding comments before session 3 in order to build rapport without threatening the parent, and then are advised that about 80% of comments should be positive in tone. Thus, we attribute the link between comment frequency and treatment retention to more frequent positive comments helping parents to feel viewed positively by the clinician, which may build therapeutic alliance, comfort participating in intervention sessions, and enjoyment and valuing of intervention participation. Parents involved with Child Protective Services have often been told how they should be parenting their children, and may be sensitized to interpret a therapist’s recommendations as criticism of their current parenting. A strengths based approach to parenting programs, in which therapists identify and praise things that parents are already doing well, is thought to promote treatment participation and retention (Katz, Lederman, & Osofsky, 2011).

As our study is limited by its correlational nature, and does not allow for firm conclusions of directionality, additional studies should work to build the case for in vivo feedback as a mechanism of change in parenting interventions. We have interpreted the data in a unidirectional manner (i.e., clinician affects parent); however, therapeutic relationships are transactional. As such, a third variable, such as therapeutic alliance, may influence both the clinician’s comfort making in the moment comments and the family’s engagement and buy-in, leading to the observed associations between commenting frequency/quality and parent behavior change and retention. Inclusion of other potential mechanisms, such as therapeutic alliance, in future research would allow tests of whether in vivo feedback predicts outcome over and above other potential mechanisms, and would further the case for in vivo feedback as a mechanism of change (Kazdin and Nock, 2003). However, past work on alliance in ABC yielded ceiling effects with little variability (Allen & Dozier, 2012), leading us to believe that in the moment feedback would likely predict outcome even when controlling for alliance.

We were unable to demonstrate mediation of treatment effects by in the moment comments because there was no control group in this study. Thus, a critical next step for establishing in vivo feedback as a mechanism of change is demonstration of mediation of parenting interventions’ effects on parent behavior by in vivo feedback. Establishing the consistency of associations between in vivo feedback and parent behavior change across treatments would also provide critical support for in vivo feedback as a mechanism (Kazdin and Nock, 2003). Finally, further research should be conducted with diverse families in other places, as our sample’s unique demographic composition and location may limit the study’s generalizability.

In pursuing this line of research, tools to measure in vivo feedback are critical. The in the moment comment coding system has allowed us to demonstrate associations between the frequency and quality of in vivo feedback and parent change in the ABC intervention. Efforts to develop a coding system to quantify therapists’ live coaching in PCIT (Barnett et al., 2014, 2015) have allowed analysis of associations between live coaching statements and PCIT treatment outcome. Because in vivo feedback is so frequent in both ABC and PCIT, it may be possible to measure it relatively well in selected session segments, minimizing the burden of adding a coding-based process measure to a clinical trial. We encourage other research groups to continue developing and refining in vivo feedback coding systems and to utilize them as a measure of treatment adherence/competence in their study of parenting interventions.

Overall, this study demonstrated robust associations between frequency and quality of in the moment feedback and parent behavior change. These associations were found in a diverse sample in a community implementation context, suggesting that in vivo feedback likely acts as an active ingredient of parenting interventions across settings. By extending the findings on in vivo feedback to community settings, the results have high public health significance. The results discourage adaptations to evidence-based parenting interventions that might decrease in vivo feedback, because such adaptations are likely to decrease intervention effectiveness. The results also suggest that training community clinicians to provide frequent and high quality in vivo feedback is a strategy that may promote treatment outcomes when parenting interventions are disseminated. Focusing training on specific active ingredients can direct clinicians’ efforts to the treatment components that are most likely to help families, and may simplify the process of learning to implement an intervention.

This study reinforced the focus of ABC training, process measurement, and supervision on in the moment feedback. In future work, we plan to examine the process of ABC clinician training at dissemination sites, and specifically the effects of commenting-focused supervision on clinicians’ use of in the moment feedback. Given the strong clinician-level effects of commenting identified in the current study, we plan to explore clinician-level predictors of in the moment feedback, particularly those that may be measured prior to training, which could guide clinician selection processes. We also hope to study the continuing performance of community clinicians, examining whether use of in the moment feedback and parent behavioral improvement is maintained when clinicians are no longer receiving weekly supervision. Additionally, we hope to examine potential moderators of change in community sites, to identify characteristics of families that might predict response to in vivo feedback techniques. Finally, we plan to continue research on the role of in the moment comments as an active ingredient and potential mechanism of change in intervention. It will be important to examine whether in the moment feedback predicts not only short-term effects of ABC on parenting, but also long-term effects on children. Future studies could examine the within-session contingencies between feedback and parent behavior, the session-to-session contingencies between feedback and behavior, and the process of change.

Acknowledgments

We would like to thank Caroline Roben for her help with ABC training and dissemination; Patria Weston-Lee and the Consuelo Foundation for supporting the Hawaii implementation effort; and the parent coaches and families who participated in this study.

FUNDING

This research is supported by the Consuelo Foundation, and by National Institutes of Health Grants R01 MH052135, R01 MH074374, and R01 MH084135 to the third author (MD).

References

  1. Allen EA, Dozier M. Assessing intervention experiences among parents of high-risk children. University of Delaware; Newark, DE: 2012. Unpublished manuscript. [Google Scholar]
  2. Antonini TN, Raj SP, Oberjohn KS, Cassedy A, Makoroff KL, Fouladi M, Wade SL. A Pilot Randomized Trial of an Online Parenting Skills Program for Pediatric Traumatic Brain Injury: Improvements in Parenting and Child Behavior. Behavior Therapy. 2014;45:455–468. doi: 10.1016/j.beth.2014.02.003. [DOI] [PubMed] [Google Scholar]
  3. Barnett MA, Gustafsson H, Deng M, Mills-Koonce WR, Cox M. Bidirectional associations among sensitive parenting, language development, and social competence. Infant and Child Development. 2012;21:374–393. doi: 10.1002/icd.1750. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Barnett ML, Niec LN, Peer SO, Jent JF, Weinstein A, Gisbert P, Simpson G. Successful Therapist–Parent Coaching: How In Vivo Feedback Relates to Parent Engagement in Parent–Child Interaction Therapy. Advance online publication. Journal of Clinical Child & Adolescent Psychology. 2015 doi: 10.1080/15374416.2015.1063428. [DOI] [PubMed]
  5. Barnett ML, Niec LN, Acevedo-Polakovich ID. Assessing the Key to Effective Coaching in Parent–Child Interaction Therapy: The Therapist-Parent Interaction Coding System. Journal of Psychopathology and Behavioral Assessment. 2014;36:211–223. doi: 10.1007/s10862-013-9396-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bernard K, Dozier M, Bick J, Lewis-Morrarty E, Lindhiem O, Carlson E. Enhancing attachment organization among maltreated children: Results of a randomized clinical trial. Child Development. 2012;83:623–636. doi: 10.1111/j.1467-8624.2011.01712.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bernard K, Dozier M, Bick J, Gordon MK. Intervening to enhance cortisol regulation among children at risk for neglect: Results of a randomized clinical trial. Development and Psychopathology. 2015;27:829–841. doi: 10.1017/S095457941400073X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bernard K, Hostinar CE, Dozier M. Intervention effects on diurnal cortisol rhythms of Child Protective Services-referred infants in early childhood: Preschool follow-up results of a randomized clinical trial. JAMA Pediatrics. 2015;169:112–119. doi: 10.1001/jamapediatrics.2014.2369. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bernard K, Simons R, Dozier M. Effects of an attachment-based intervention on high-risk mothers' event-related potentials to children's emotions. Child Development. 2015 doi: 10.1111/cdev.12418. Advance online publication. [DOI] [PMC free article] [PubMed]
  10. Borrego J, Urquiza AJ. Importance of therapist use of social reinforcement with parents as a model for parent-child relationships: An example with Parent-Child Interaction Therapy. Child & Family Behavior Therapy. 1998;20:27–54. [Google Scholar]
  11. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. American Journal of Mental Deficiency. 1981:127–137. [PubMed] [Google Scholar]
  12. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005. [Google Scholar]
  13. Friedlander ML, Ellis MV, Siegel SM, Raymond L, Haase RF, Highlen PS. Generalizing from segments to sessions: Should it be done? Journal of Counseling Psychology. 1988;35:243–250. doi: 10.1037/0022-0167.35.3.243. [DOI] [Google Scholar]
  14. Hulleman CS, Cordray DS. Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness. 2009;2:88–110. doi: 10.1080/19345740802539325. [DOI] [Google Scholar]
  15. Kaminski JW, Valle LA, Filene JH, Boyle CL. A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology. 2008;36:567–589. doi: 10.1007/s10802-007-9201-9. [DOI] [PubMed] [Google Scholar]
  16. Katz LF, Lederman JCS, Osofsky JD. Use of Evidence - Based Parenting Programs for Parents of At - Risk Young Children. Juvenile and Family Court Journal. 2011;62:37–56. doi: 10.1111/j.1755-6988.2011.01067.x. [DOI] [Google Scholar]
  17. Kazdin AE, Nock MK. Delineating mechanisms of change in child and adolescent therapy: Methodological issues and research recommendations. Journal of Child Psychology and Psychiatry. 2003;44:1116–1129. doi: 10.1111/1469-7610.00195. [DOI] [PubMed] [Google Scholar]
  18. Lewis-Morrarty E, Dozier M, Bernard K, Terraciano S, Moore S. Cognitive flexibility and theory of mind outcomes among foster children: Preschool follow-up results of a randomized clinical trial. Journal of Adolescent Health. 2012;52:S17–S22. doi: 10.1016/j.jadohealth.2012.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lind T, Bernard K, Ross E, Dozier M. Intervention effects on negative affect of CPS-referred children: Results of a randomized clinical trial. Child Abuse and Neglect. 2014;38:1459–1467. doi: 10.1016/j.chiabu.2014.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. McNeil CB, Hembree-Kigin TL. Parent-child interaction therapy: Second edition. New York: Springer; 2010. [Google Scholar]
  21. Meade E, Dozier M. “In the Moment” commenting: A fidelity measurement and active ingredient in a parent training program. Unpublished manuscript. University of Delaware; Newark, DE: 2012. [Google Scholar]
  22. Mills-Koonce WR, Gariépy JL, Propper C, Sutton K, Calkins S, Moore G, Cox M. Infant and parent factors associated with early maternal sensitivity: A caregiver-attachment systems approach. Infant Behavior and Development. 2007;30:114–126. doi: 10.1016/j.infbeh.2006.11.010. [DOI] [PubMed] [Google Scholar]
  23. National Advisory Mental Health Council. Blueprint for Change: Research on Child and Adolescent Mental Health. A Report by the National Advisory Mental Health Council’s Workgroup on Child and Adolescent Mental Health Intervention Development and Deployment. Bethesda, Maryland: National Institutes of Health, National Institute of Mental Health; 2001. [Google Scholar]
  24. NICHD Early Child Care Research Network. Characteristics of infant child care: Factors contributing to positive caregiving. Early Childhood Research Quarterly. 1996;11:269–306. doi: 10.1016/S0885-2006(96)90009-5. [DOI] [Google Scholar]
  25. Niehaus E, Campbell CM, Inkelas KK. HLM behind the curtain: Unveiling decisions behind the use and interpretation of HLM in higher education research. Research in Higher Education. 2014;55:101–122. doi: 10.1007/s11162-013-9306-7. [DOI] [Google Scholar]
  26. President’s New Freedom Commission on Mental Health. Report of the President’s New Freedom Commission on Mental Health. 2004 Retrieved from http://www.nami.org/Template.cfm?Section=Policy&Template=/ContentManagement/ContentDisplay.cfm&ContentID=16699.
  27. Propper C, Willoughby M, Halpern CT, Carbone MA, Cox M. Parenting quality, DRD4, and the prediction of externalizing and internalizing behaviors in early childhood. Developmental Psychobiology. 2007;49:619–632. doi: 10.1002/dev.20249. [DOI] [PubMed] [Google Scholar]
  28. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage; 2002. [Google Scholar]
  29. Robbins FR, Dunlap G, Plienis AJ. Family characteristics, family training, and the progress of young children with autism. Journal of Early Intervention. 1991;15:173–184. doi: 10.1177/105381519101500206. [DOI] [Google Scholar]
  30. Shanley JR, Niec LN. Coaching parents to change: The impact of in vivo feedback on parents' acquisition of skills. Journal of Clinical Child & Adolescent Psychology. 2010;39:282–287. doi: 10.1080/15374410903532627. [DOI] [PubMed] [Google Scholar]
  31. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin. 1979;86:420–428. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  32. Weersing VR, Weisz JR. Mechanisms of action in youth psychotherapy. Journal of Child Psychology and Psychiatry. 2002;43:3–29. doi: 10.1111/1469-7610.00002. [DOI] [PubMed] [Google Scholar]
  33. Weisz JR, Jensen-Doss A, Hawley KM. Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. American Psychologist. 2006;61:671–689. doi: 10.1037/0003-066X.61.7.671. [DOI] [PubMed] [Google Scholar]
  34. Caron E, Weston-Lee P, Haggerty D, Dozier M. Community implementation outcomes of Attachment and Biobehavioral Catch-up. Child Abuse & Neglect. 2016;53:128–137. doi: 10.1016/j.chiabu.2015.11.010. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES