Using Video Feedback as a Tool in Training Parent Coaches: Promising Results from a Single-Subject Design

Abstract

The Attachment and Biobehavioral Catch-up (ABC) parenting program focuses on three intervention targets: increasing parental nurturance, increasing parental synchrony, and decreasing parental frightening behavior. Parent coaches are expected to comment “in the moment” when behaviors relevant to these three targets are observed in sessions. Making in the moment comments is a challenging aspect of intervention, and parent coaches have struggled with their fidelity to this critical intervention component. Thus, we developed a system for coding the frequency and quality of comments from video-recorded session clips on a statement-by-statement level. To help parent coaches refine and maintain their skills in making such comments, they are taught to code segments of their own video-recorded sessions, with the expectation that gains would be seen in comments after learning to code. In this paper, we describe the fidelity coding system and present initial results from a yearlong, single-subject design examining the effects of video feedback coding for a parent coach who was learning the intervention. We observed an increase in frequency of in the moment comments during the period of video feedback coding, consistent with a training effect.

Attachment and Biobehavioral Catch-up (ABC) is a parenting program designed to enhance parental synchrony and nurturance, and decrease frightening behavior. Parent coaches implement the intervention in families' homes with parents and children together. A key aspect of the intervention is providing “in the moment” feedback to parents that focuses attention on intervention targets. Such feedback helps parents recognize opportunities for synchronous and nurturing responses, and provides support as they practice such behaviors. We first recognized the effectiveness of in the moment commenting in the sessions of parent coaches whose natural style included use of such comments. In the context of group supervision, we observed that when parents were provided with in the moment feedback, they were able to rapidly change their behavior. Because of these observations, we incorporated in the moment commenting as a key component of the ABC intervention, and we began to train all parent coaches to make comments. However, we found that the process of making comments in the moment was very difficult for many parent coaches, such that they made them rarely, or did not make them well. We developed a system for coding parent coach comments on a statement-by-statement basis, allowing us to monitor progress. At the same time, we began training parent coaches to code their own sessions under the assumption that careful attention to the critical parameters would result in enhanced commenting fidelity. In this paper, we report the results of a single case study that tested whether the introduction of coding one's own session videos would be followed by increased frequency of a parent coach's in the moment comments.

Attachment and Biobehavioral Catch-up: An overview

Children who have experienced early adversity, such as maltreatment, caregiver instability, and institutionalization, are at increased risk for problematic outcomes, including behavioral and emotional problems, poor health, and school failure (e.g., Gilbert et al., 2009). Attachment and Biobehavioral Catch-up is a 10-session, home-based intervention for infants who have experienced early adversity. ABC has been found to result in lower rates of disorganized attachment and higher rates of secure attachment relative to a control intervention (Bernard et al., 2012), to normalize diurnal cortisol production in children (Dozier, Bernard, Bick, & Gordon, 2013), and to enhance children's executive function (Lewis-Morrarty, Dozier, Bernard, Terraciano, & Moore, 2012).

We expect that the ABC intervention leads to positive outcomes for children by changing parenting behavior in specific ways. ABC targets three parent behaviors: nurturance, synchrony, and non-frightening behavior. Nurturance is conceptualized as sensitive responding to child distress, such as asking, “Oh honey, are you okay?” in a concerned tone when a child fusses. When children are not distressed, parents are encouraged to interact with their children synchronously, that is, in a contingently responsive manner that follows their children's lead. Examples of synchrony include repeating a child's vocalization or mimicking a child's manner of playing with a toy. Finally, parents are helped to recognize behaviors that could be frightening or overwhelming for their children (e.g., yelling, forceful physical interactions), and are encouraged to inhibit such behaviors and respond differently.

ABC consists of 10 hour-long sessions, conducted in families' homes, with parents, other caregivers (e.g., grandparents), children, and siblings present, as we wish to intervene in the context of the family's usual environment to promote generalization of the behaviors practiced. All sessions are video recorded from start to finish for the purposes of supervision of parent coaches and video feedback to parents.

Parent behaviors are targeted through manual-guided discussion, structured practice activities, video feedback, and in the moment comments. To provide an example of how these different activities are integrated, session 5 is focused on reduction of overstimulating and intrusive behavior. The parent coach introduces the session topic by asking about the parent's experiences of being overstimulated, such as tickled, as a child, and guides the discussion to how children's responses to such behavior (e.g., uncontrollable laughter) may not always represent how they feel. Then, the parent coach introduces a structured practice activity: playing with puppets, plastic spiders, and other toys that often elicit intrusive behavior, and coaching the parent to follow their child's lead and avoid intrusive play. If appropriate, the parent coach may also show the parent a video clip of a past time when he or she engaged in intrusive or overstimulating behavior, contrasting this clip with a positive clip, in which the parent followed the child's lead. Throughout all of these activities, however, the parent coach's main agenda is to notice ongoing parent behaviors that fit with all intervention targets and give in the moment feedback, at a rate of at least one comment per minute.

In The Moment Comments

Why in the moment comments are important

The parent coach's task is to present the manual content while commenting on opportunities for synchrony and nurturance in the session. These comments are important because they draw attention to the specific behaviors of the parent (so the parent understands what behavior is referred to), link behaviors to the intervention targets (so the parent can see how a specific behavior relates to intervention targets), and point out the effects of the parent's behaviors on the child (so the parent can see the importance of the behaviors for long-term outcomes of the child). Additionally, in the moment comments build the therapeutic alliance and help parents feel supported. Parents may begin ABC thinking that they will be told that they are parenting badly; instead, even early in the first session, the parent coach praises the parent for what the parent is already doing well.

Early in the intervention, in the moment feedback is almost exclusively positive, commenting on the parent's responding to the child's overtures, however fleeting. For example, if the child handed the mother a toy phone and she took it (even with a look of disinterest), the parent coach might say, “Great job following his lead! He handed you that phone and you took it right from him.” Parent coaches can also use in the moment comments to support continued positive interaction. For example, if the parent appears to become bored with following the child's lead in a repetitive activity, the parent coach might say in a lighthearted tone, “I know you're probably getting tired of pretending to eat that pizza, but you know why he keeps giving it to you, right? He loves it when you follow his lead – look at his smile.” Over time the parent coach should increasingly be able to make comments that scaffold the parent's behavior (i.e., suggesting a way to nurture or follow the lead), and even comments that challenge the parent's behavior (i.e., suggesting that the mother is not nurturing or not following the child's lead).

Opportunities for nurturance occur much less frequently than opportunities for synchrony because they are limited to times when the child is distressed or seeks proximity. Thus, it is especially important to notice and comment when those opportunities arise. For example, if a child bumped his toe and stumbled, and his mother said, “You okay?” the parent coach could comment, “Wow. He bumped his toe and you asked if he was okay. That's so important for him knowing that you 're there for him.”

The effect of these comments is to focus attention on intervention targets. Rather than merely discussing the importance of nurturance and synchrony, parents have the experience of having their own behaviors pointed out to them repeatedly. Although we have not yet conducted contingency analyses that would provide empirical support, anecdotal evidence suggests that parents often change quickly in response to these in the moment comments, behaving in more synchronous and nurturing ways immediately following parent coach comments. Changed parent behavior creates more opportunities for comments, and further strengthening of the behaviors. Using the coding system described in the Method, we have found that the frequency of parent coaches' in the moment comments was linked to increases in parent synchrony. Specifically, the frequency of on-target comments in intervention session 3 predicted parent synchrony in session 9, even after controlling for the quality of parent synchrony observed in session 3 (Meade & Dozier, 2012). Because in the moment feedback appears so critical to intervention effectiveness, we use it as an index of intervention fidelity

In the moment comments and similar intervention techniques

In the moment comments are a specific type of in vivo feedback. In vivo feedback is a technique used in several parenting interventions; another type of in vivo feedback, for example, is live coaching, which is a key component of Parent-Child Interaction Therapy (PCIT; McNeil & Hembree-Kigin, 2010), an evidence-supported treatment for young children with behavior problems. In PCIT, therapists coach parents to engage in positive parent-child interactions and use consistent and effective discipline, from behind a one-way mirror using a “bug-in-the-ear” device. Although there are differences in the implementation of in the moment commenting and live coaching, both provide feedback to parents about targeted behaviors and support parents in practicing skills. Thus, in vivo feedback may be an active ingredient across parenting interventions, or an “evidence-based kernel” of behavioral influence (Embry & Biglan, 2008).

Preliminary evidence supports in vivo coaching as a key ingredient of parenting interventions. A meta-analysis that examined components of parent training programs found that of 18 components examined, requiring in vivo practice with the parent's own child was one of the strongest and more consistent predictors of improved parenting behavior (Kaminski, Valle, Filene, & Boyle, 2008). Shanley and Niec (2010) experimentally examined the impact of two 15-minute PCIT-like coaching sessions on mothers' positive parenting skills. They found that mothers assigned to the coaching group increased in the frequency of their targeted parenting skills, whereas these skills tended to decline in the non-coached group (Shanley & Niec, 2010). Thus, providing in vivo feedback about parenting behavior is a technique used by ABC as well as other interventions, and there is evidence to suggest it may act as an active ingredient of intervention.

In the moment comments and video feedback

In vivo feedback is similar to video feedback in that it provides parents with supportive or corrective information about interactions with children. Both techniques focus on the actual behavior of a particular parent-child dyad, rather than discussing behavior in hypothetical terms or role-playing parent-child interactions. In addition, both forms of feedback focus on specific parent behaviors, rather than global constructs, and typically take a primarily positive, reinforcing approach (Smith, Dishion, Moore, Shaw, & Wilson, 2013).

However, there are also key differences between in vivo and video feedback approaches. One difference is the parent's perspective when receiving feedback; when parents receive in the moment comments, they are aware of their behavior in a first-person perspective, and when parents receive video feedback, they experience their behavior from a third-person perspective. Taking a third-person perspective may allow parents to more objectively observe their own behavior and their children's responses (Smith et al., 2013). However, there are also benefits to the immediacy and first-person interactive nature of in the moment feedback. Video feedback is usually somewhat retrospective, and there also may be delays between video feedback and the parent's first opportunity to apply this feedback in interaction with the child, placing demand on the parent's ability to remember the information. In contrast, in the moment comments allow parents to adjust or practice a behavior within seconds of the original behavior. The short intervals between rounds of in the moment feedback can also allow parent coaches to shape successive approximations of desired behavior to a greater extent than video feedback. In addition, the frequency of in the moment feedback allows parent coaches to address many specific examples of parent behavior. Through the identification of many behaviors, parents gain more exemplars of broad constructs such as nurturance and following the lead, allowing them to better understand these constructs and generalize the learning to novel situations.

However, video feedback can be particularly helpful when a parent coach misses the opportunity to respond to an especially important and rarely occurring behavior (e.g., frightening, nurturing) in the moment, and to highlight behavioral sequences that may be more difficult to reflect on in the moment (e.g., parent tickles child, child becomes dysregulated and kicks the cat). Thus, we use both video feedback and in the moment feedback in the ABC intervention, because each approach has its strengths.

Why making in the moment comments is difficult

In sessions, parent coaches are asked to keep two agendas in mind; that is, they need to cover the manualized content while paying attention to every interaction between the parent and child. This is challenging, requiring parent coaches to know the manual well enough that they can flexibly move in and out of the content while attending to the parent-child interaction. A number of factors interfere with parent coaches being able to do this. Sometimes parent coaches indicate that they fear that parents will feel patronized if they make comments, which does not seem to be the case if comments are delivered comfortably. Parent coaches are often concerned that the manualized content should take precedence over in the moment comments. We consider covering the manualized content secondary to making in the moment comments. In addition, new parent coaches' in the moment comments often fail to identify the behavior of interest clearly or indicate why the behavior matters, leaving the parent without enough useful information to lead to behavior change.

Using session videos to train parent coaches to make in the moment comments

We have used several strategies for training parent coaches to make in the moment comments. First, we provided traditional feedback to parent coaches. Video segments of parent coach sessions were viewed in supervision, with the supervisor pausing the video and asking parent coaches to generate examples of in the moment comments they could make following specific interactions. Given that some parent coaches failed to make behavioral changes with this method, we developed the coding system described in the Method section to quantify skill and progress. Parent coaches coded their own sessions, and supervisors used coding to provide feedback during group supervision.

The practice of using videos to provide feedback regarding performance has been used to train professionals in various fields, and has been found effective in improving professionals' interactional skills with clients (Fukkink, Trienekens, & Kramer, 2011). Further, meta-analytic works suggests that video feedback has particularly positive outcomes with professionals when a standard observation form is used to structure feedback (Fukkink et al., 2011). In this paper, we used a single-subject design to examine whether use of an observational coding tool enhanced parent coach ability to deliver in the moment comments.

The Present Study

The present study examined the trajectory of in the moment comments by a parent coach over a 12-month period. Specifically, undergraduate coders assessed the parent coach's rate of in the moment commenting before and after the parent coach began coding her own sessions as a form of video feedback.

Method

A single-subject design was conducted in the larger context of two ongoing randomized clinical trials of the ABC intervention for children adopted internationally and toddlers in foster care. In each trial, a manual based on the Attachment and Biobehavioral Catch-up - Infant (ABC-I) intervention was used. As in the ABC-I intervention, the interventions sought to increase parental nurturing and synchronous behaviors, as well as to decrease intrusive or frightening behavior.

To examine the effects of video feedback guided by a coding system, we used an A/A+B design with one of our primary parent coaches. The parent coach was a White/non-Hispanic female who had recently received a post-baccalaureate level degree in the field of clinical work, but was not yet licensed. She was one of two primary ABC interventionists working for our project during the year when the coding system was developed and introduced to parent coaches, and was selected for this study because of the timing of her hiring and training; the other primary parent coach had been working as an ABC parent coach for several years, and thus had had a good deal of time to develop in the moment commenting skill through regular supervision feedback prior to the introduction of coding.

Supervision Type: Independent Variable

In phase A of the A/A+B design, the parent coach received group supervision, or supervision as normal, for 1.5 hours each week. In the second phase, the parent coach continued participating in group supervision (A), but also coded a 5-minute clip of one of her intervention sessions each week (B). Each phase lasted 6 months. The timing of the implementation of the A + B phase was not random; rather, the parent coach was asked to begin coding clips of her own sessions when we created the coding system, described below. The parent coach did not receive formal coding training before beginning to code her cases, but received informal training in the form of feedback when coding was reviewed in supervision. The 5-minute clip was selected randomly from the full session by the first author. The parent coach was asked to email her coding weekly, and coding was reviewed informally with supervisors during group supervision. Completion of coding was not rigorously enforced, and during the A+B phase of the design, the parent coach completed 15 codings in 23 weeks. The parent coach's frequency of completed coding did not change systematically across the 6-month A+B phase; she completed 5 codings in the first two months, 5 in the next two months, and 5 in the final two months. Although missed coding cannot be assumed to be missing completely at random, the pattern of missingness was not predictive of the dependent variable. More specifically, the number of missed codings per month was not correlated with monthly average comment frequency (p > .05).

Participants

Analyses included 176 sessions of 19 intervention cases; 108 sessions were conducted with 11 families with children adopted internationally, 41 sessions were conducted with 5 families with children adopted domestically, and 27 sessions were conducted with 3 families with foster children. In 8 of the 19 cases, two caregivers consistently participated in intervention sessions. In 10 cases, at least one sibling was present for some of the intervention sessions. On average, children were 2.4 (SD = 0.7) years old when they began the intervention. Parents were, on average 37.9 (SD = 4.0) years old.

Fidelity Instrument: Dependent Variable

Six trained undergraduate coders rated 5-minute clips of each of the parent coach's hour-long sessions (n = 176) over a period of 12 months. Clips were selected randomly by the first author, and coding was assigned such that each coder never coded more than 4 sessions of a single case and coders did not code back-to-back sessions. Of the 176 sessions, 66 sessions were conducted in the 6 months before the parent coach began coding clips of her own sessions, and 110 sessions were conducted in the 6 months afterward. Coders were blind to hypotheses and to timing. Five minute segments were chosen because this length of time was effective in assessing inter-coach and inter-case variability in frequency of parent coach comments in our previous work (Meade & Dozier, 2012). For the current study, 10% of videos were double coded. Assessment of inter-rater reliability for the primary measure of fidelity used in this study, rate of on-target comments, found a one-way, random effects Intraclass Correlation Coefficient (ICC) of .95. This measure of reliability is appropriate when a variety of coder pairs rate a subset of the sample (Shrout & Fleiss, 1979), and ICCs above .75 are considered to represent excellent levels of agreement (Cicchetti & Sparrow, 1981). Reliability for additional measures of fidelity ranged from poor to good (Cicchetti & Sparrow, 1981), with ICCs of .25 for proportion of on-target comments; .30 for average comment level; and .69 for proportion of missed opportunities. Because reliability of these measures was low, these measures of fidelity were not analyzed.

The in the moment coding system is a bipartite system in which parent behaviors relevant to the behavioral targets of the intervention are first described and coded, and parent coach responses to each of these behaviors are then transcribed and coded. Each time a synchronous (or non-synchronous) or nurturing (or non-nurturing) behavior is observed, the system is triggered (i.e., parent behavior and parent coach comment are coded). Here, we describe the simplified version of the in the moment coding system used by parent coaches (i.e., the tool used by the parent coach in the A + B manipulation phase of the study). However, the dependent variable used in this study was obtained using a research version of the tool. Differences between the two tools are described at the end of the section. We describe the parent coach version of the coding system for the sake of simplicity; however, the choice of coding system used did not affect the dependent variable of this study (i.e., the frequency of the parent coach's on-target comments).

Coding of parent behavior

Parents are coded as responding in synchronous ways when they respond contingently to the child or follow the child's lead. Non-synchrony is coded when the parent does not respond contingently to the child through a failure to respond, a mismatched response, or initiation of a parent-led interaction. Nurturance is coded when the parent responds sensitively to the child's distress, or responds warmly to a child-initiated overture for physical proximity. Non-nurturance is coded when the parent does not respond sensitively to child distress, either through a failure to respond or a dismissing or rejecting attitude toward the child's distress. Each instance of relevant parent behavior is coded on a separate line of an Excel spreadsheet, which automatically calculates summary scores, including the frequency of each type of parent behavior.

Coding of parent coach response

Parent coach responses to these parental behaviors are then recorded on a behavior-by-behavior basis. Characteristics of the comment that are coded include: (a) behavior target, as described by the parent coach; (b) on/off target: whether the comment is appropriately matched to the parent's behavior and does not drift beyond the scope of the intervention targets, and (c) level: the number of information components provided by the comment. Using these comment characteristics, the Excel coding spreadsheet automatically calculates summary scores for the session (discussed below).

The target of parent coach comments is coded in a manner analogous to the parent behavior. Comments that match parent behaviors are considered “on target.” “Off target” comments, in contrast, inappropriately address undesired behaviors (e.g., commenting that a mother was following the child when she was in fact starting her own game), respond to a positive behavior with a comment addressing a different behavior target (e.g., referring to a synchronous behavior as nurturance), or address topics not covered by the intervention (e.g., setting limits). When a parent coach does not respond to a parent behavior with an in the moment comment, the parent behavior is coded as a missed opportunity.

The level of parent coach comments is coded from 0 to 3, representing the number of information components included in the comment. Comments can include three components: specifically describing the parent's behavior (“He reached up for his ball and you handed it to him”), linking the behavior to the relevant intervention target (“Nice following his lead”), and indicating long-term effects of the behavior (“That's going to help him with self-regulation as he gets older.”). The level of a comment represents the number of components it includes (i.e., 1-3). Level 0 comments include no full components, and may describe the parent's behavior in non-specific terms (“Nice responding to him.”) or give praise without additional information (e.g., “That's perfect!”).

Coding is completed on an Excel spreadsheet that is programmed to automatically calculate summary statistics. For many of the comment summary statistics, being an “on-target” comment is a prerequisite. The summary variables include rate of on-target comments (i.e., number of on-target comments per minute), average level of on-target comments, proportion of missed opportunities (i.e., parent behaviors that were not responded to with an in the moment comment, divided by the total number of parent behaviors), and proportion of comments that were on-target (i.e., number of on-target comments divided by total number of comments).

Differences between research fidelity coding and parent coach fidelity coding

As described previously, the parent coach fidelity coding system is a simplified version of the research fidelity coding system used in the analyses for this project. The research coding system includes subtypes of the behavior targets described above (e.g., passive non-synchrony vs. active non-synchrony) to allow questions about specific parent behaviors to be asked. Additionally, the research coding system includes an expanded coding of comment level with scores ranging from 1 to 5, more clearly delineating among scores on the lower end of the scale. The research version also calculates individual summary statistics regarding parent behavior for up to two caregivers.

Data Analytic Approach

Hierarchical linear modeling (Raudenbush & Bryk, 2002) allowed us to account for the non-independence of multiple observations nested within cases by simultaneously estimating within- and between-subject variation. This approach also allows for variability in the number and spacing of time points, which was especially important in our data structure, in which different families began and completed intervention at different points during the year.

To examine how coding her own sessions affected the parent coach's on-target comment rate(i.e., number of on-target comments per minute),we used piecewise linear growth modeling. Finding pre-post increases in comments could reflect increased commenting across the training year, rather than increased commenting in response to implementation of coding. A linear growth modeling approach allows examination of rate of change over time (i.e., slope), and piecewise linear growth modeling allows for division of growth trajectories into separate linear components (Raudenbush & Bryk, 2002). Thus, to test whether the parent coach's growth in commenting changed after she began coding, we estimated separate slopes for period “A” and period “A + B,” and tested whether these slopes were significantly different from one another.

In these analyses, the dependent variable was the on-target comment rate in each session. The level-2 data unit was the family or case. Given the between- and within-subject variability in timing of sessions, we restructured the data from occasions of measurement (i.e., days) to time classes (i.e., months), following approaches used previously (e.g., King et al., 2006; Sumner, Bernard, & Dozier, 2010). We expected this approach to make estimates of case-level intercepts and slopes more accurate, enhancing model fit. These time classes were then recoded into two separate level-1 predictors to form a two-piece model, as depicted in Table 1. The first time variable (Pre-Coding) represented change in the parent coach's rate of commenting between when she began seeing cases and when she began video feedback. The second time variable (Post-Coding) captured change in the parent coach's rate of commenting after she began video feedback. Data were centered around the month when the parent coach began video feedback coding, by setting this time class to 0. Based on preliminary examination of a scatter plot of commenting rate across time classes, depicted in Figure 1, we specified a quadratic growth model of the following form:

Table 1. Coding Scheme for Two-Piece Linear Model.

Time Class	Variable
	Pre-Coding	Post-Coding
Month 1	−6	0
Month 2	−5	0
Month 3	−4	0
Month 4	−3	0
Month 5	−2	0
Month 6	−1	0
Month 7 (start of coding)	0	0
Month 8	0	1
Month 9	0	2
Month 10	0	3
Month 11	0	4
Month 12	0	5

Figure 1.

Scatterplot of within-session rates of on-target comments, across 12 time classes corresponding to months.

$${\text{Commenting}}_{\mathrm{t}\mathrm{i}}={\mathbf{\pi }}_{0\mathrm{i}}+{\mathbf{\pi }}_{1\mathrm{i}}\ast ({\text{Pre}-\text{Coding}}_{\mathrm{t}\mathrm{i}})+{\mathbf{\pi }}_{2\mathrm{i}}\ast ({\text{Post}-\text{Coding}}_{\mathrm{t}\mathrm{i}})+{\mathbf{\pi }}_{3\mathrm{i}}\ast {({\text{Pre}-\text{Coding}}_{\mathrm{t}\mathrm{i}})}^2+{\mathbf{\pi }}_{4\mathrm{i}}\ast {({\text{Post}-\text{Coding}}_{\mathrm{t}\mathrm{i}})}^2+{\mathrm{e}}_{\mathrm{t}\mathrm{i}}$$

where π_0i represents the parent coach's rate of commenting with family i in the month when she began video feedback coding; Pre-Coding_ti and Post-Coding_ti represent the time class variables for family i at time t; π_1i and π_2i represent the instantaneous Pre- and Post-Coding linear growth rates in commenting with family i, in the month when the parent coach began video feedback coding (time 0); π_3i and π_4i represent the curvature (i.e., quadratic term) in the Pre- and Post- Coding growth trajectories of commenting with family i; and e_ti represents the within-family error in the parent coach's rate of commenting with family i at month t, that cannot be accounted for by the model.

Results

Results of the piecewise linear growth model are presented in Table 2. Prior to beginning video feedback, the rate of change (i.e., slope) in the frequency of on-target comments was not significantly different from 0, suggesting that the parent coach was not showing improvement in her rate of commenting across months (Pre-coding instantaneous linear slope: β₁₀= 0.17, p > .05; Pre-coding quadratic slope: β₃₀= 0.03, p > .05). In contrast, the post-coding period was characterized by a significant increase in the parent coach's commenting rate. Specifically, there was evidence of positive quadratic growth, with the rate of commenting increasing across months (Post-coding instantaneous linear slope: β₂₀= −.38, p < .05; Post-coding quadratic slope: β₄₀= 0.11, p < .01). The parent coach's model-implied rate of commenting over time is graphed in Figure 2. It should be noted that the negative linear slope (β₂₀) indicates that the tangential slope of the quadratic function was negative at month 0, and thus should not be interpreted as a meaningful measure of linear change across the post-coding period. Post-hoc analyses demonstrated that there were no differences in frequency of comments between month 0 and month 1, 2, 3 or 4. By month 5, post-hoc hierarchical linear analysis demonstrated higher frequency of comments than in month 0 (β₁₀ = 1.05, p < .10).

Table 2. Two-Piece Unconditional Quadratic Model for Changes in Commenting.

Effect	Coefficient	SE	t	df	p
Intercept, β00	1.27	0.18	6.97	18	<.001
Pre-Coding linear slope, β10	0.17	0.14	1.19	18	.25
Post-Coding linear slope, β20	−0.38	0.15	−2.57	18	.02
Pre-Coding quadratic slope, β30	0.03	0.03	0.98	18	.34
Post-Coding quadratic slope, β40	0.11	0.03	3.62	18	.002

Figure 2.

Model-implied levels of the parent coach's rate of on-target comments over time.

Note. Video feedback coding began in Month 0. Months −5 and −6 were excluded because the model produced errors when centered at these time points.

Discussion

The results of this single-subject design provide preliminary support for our hypothesis that the introduction of coding one's own video-recorded intervention sessions leads to improvements in the frequency of in the moment comments. Specifically, the frequency of on-target in the moment comments improved at an increasing rate following the introduction of coding, whereas prior to beginning coding of videos, commenting frequency was fairly stable. Post-hoc analyses indicated that the effects of video coding on commenting frequency were seen after several months, with significant effects in month 5. Thus, in this study, it appeared that several months of coding were required before effects emerged.

There are several reasons why coding one's own session videos may increase parent coach use of in the moment commenting. The process of in the moment coding requires examination of both parent behavior and parent coach behavior. Reviewing videos and coding each occurrence of relevant parent behavior likely promotes parent coaches' skill in quickly and confidently identifying intervention-targeted behaviors, and thus may promote the ease and automaticity of making comments on live parent behavior. In addition, coding one's own in the moment comments forces parent coaches to reflect on various aspects of the in the moment comments they did or did not make. The process of coding each missed opportunity may make parent coaches more attuned to commenting on opportunities in the future. Parent coaches must also decide whether each of their comments is “on” or “off-target,” a process that increases parent coaches' awareness of the relevance of their comments to the specific parent behavior antecedents. Finally, coding their videos may also allow parent coaches to observe the in-session effects of their comments (e.g., a parent increasing frequency of a targeted behavior, or sustaining a targeted behavior that is difficult for him or her), thus promoting parent coaches' valuing of in the moment comments and motivation to make comments in sessions.

These results are consistent with Fukkink et al.'s (2011) meta-analytic findings that video feedback is an effective training technique for professionals, particularly when a standard evaluation form is used. Fukkink et al. (2011) proposed that an evaluation form allows supervisors and supervisees to “zoom in and focus” on targeted behaviors (p. 56). Adherence or fidelity coding forms have successfully been used to provide feedback to professionals. For example, when experts provided Multisystemic Therapy clinicians with structured feedback about adherence scores coded from session audiotapes, 3 of 5 therapists showed increased levels of adherence (Schoenwald, Henggeler, Brondino, & Rowland, 2000). Our findings offer unique support for the benefits of having therapists code their own session videos.

A strength of this study was the relatively long period of time prior to, and after, the date that coding-based video feedback was implemented. When the parent coach began coding-based video feedback, it was after 6 months of weekly group supervision led by the developer of the ABC intervention, as well as after conducting 66 ABC sessions. Thus, she had sufficient time to develop competence in the model through the gold standard process of video-based supervision with an expert. It seems relatively unlikely that the parent coach's rate of change in commenting that began several months into coding-based video feedback was related to the process of group supervision or part of a typical developmental trajectory for a parent coach. Rather, the results of the piecewise growth model suggest that improvement in commenting occurred only after the initiation of coding-based video feedback.

In the current example, the parent coach coded only one session per week and did not receive explicit, formalized feedback on the coding. Further, the parent coach coded only 15 video clips in the 6 months of video feedback. Each video clip requires about 30 to 60 minutes to code, which suggests that the parent coach invested only 7 to 15 hours in coding during this period of time. Therefore, although we consider this a weak example of implementing in the moment coding, change is seen, which suggests how powerful coding one's own sessions may be in improving therapist fidelity.

It is important to note, however, that a single-subject, A/A+B design can provide only tentative support for coding videos as a method to improve in the moment commenting. We infer that the initiation of coding session videos caused the parent coach's increasing rate of in the moment comments. However, changes in commenting rate could be linked to other variables (e.g., unmeasured interventions or changes that occurred between Month 0 and Month 4), or could be part of the normal developmental trajectory of parent coaches receiving only group supervision (i.e., attributable to A). Replication with additional parent coaches, using a multiple baseline design, would provide further support for the effects of coding on parent coaches' commenting frequency. Replication, particularly with parent coaches who more consistently complete coding, could also provide further information about whether change in commenting can be seen earlier than 5 months into the coding intervention.

Future Directions

This study helped us develop and refine our technique of training parent coaches in ABC. Because coding her own sessions appeared to improve this parent coach's provision of in the moment feedback, we more closely integrated coding into our training and supervision procedures in the attempt to more quickly and powerfully enhance benefits to in the moment commenting. When we had informally introduced the coding system to parent coaches, we found that some parent coaches were not able to code reliably, despite hours of discussion of behaviors and comments in supervision. However, undergraduates who were trained formally became skilled coders. We simplified the coding system for parent coaches, and developed a model in which parent coaches now receive coding training and supervision from an undergraduate-level expert coder. In addition, we now introduce and practice coding during the initial training for parent coaches. After picking up intervention cases, parent coaches code 5-minute clips of their sessions that are double-coded and reviewed with coding supervisors on a weekly basis. Coding supervisors discuss discrepancies in coding, and offer suggestions for improving the frequency and quality of in-the-moment comments. This system also holds parent coaches accountable for completing their own coding, given that we found that missed coding was common in the current study, and could have influenced the rate of growth. We are using this system of training and supervision in several dissemination sites at this point, and plan to evaluate the effects of coding supervision using a multiple baseline design.

These results also make us hopeful for the wider application of coding-based video feedback in training parent coaches. Coding-based video feedback could be used to develop a system in which ABC parent coaches can monitor themselves and provide feedback to one another. We expect that peer-based and self-supervision will lead to a reduction in the need for expert-led supervision and an enhancement in treatment fidelity. Further, these results may also be relevant for those training clinicians in other parenting interventions that involve in vivo feedback, such as PCIT. Barnett, Niec and Acevedo-Polakovich (2013) have developed a coding system for therapists' coaching of child-directed interaction in PCIT, which could be used in the training and supervision of PCIT therapists. In addition, coding-based video feedback is likely an effective supervision strategy for a variety of interventions, as suggested by Fukkink et al. (2011), and may promote efforts to disseminate evidence-based treatments to community settings.

These promising implications should be considered in the context of several limitations of this study. As a single-subject design examining the trajectory of a single parent coach, our ability to state that changes occurred as a result of coding-based video feedback is tentative. In addition, we were not able to examine potential therapist variability in response to video feedback. Future studies with multiple parent coaches may demonstrate variability in such growth trajectories. We had a somewhat small sample size of cases (N = 19), which limited statistical power and prevented us from examining family-level predictors. In addition, reliability for several measures of in the moment fidelity was too low to examine change over time, and causes some concern about the usefulness of the coding measure to assess these commenting characteristics. A final limitation of the design was the inability to determine the differential effects of group supervision and video feedback, because group supervision was a constant throughout the year. Thus, the results presented here should be taken as suggestive and as an impetus for future research and replication.