Effects of a video feedback parent training program during child welfare visitation

Abstract

Behavioral parent training programs have documented efficacy for improving behaviors among parents and their children and are frequently used by child welfare agencies to prevent removal of a child from the parental home or to facilitate reunification. Although an ideal time for parent training might be during supervised visits where parents may practice skills with their children under the guidance and support of a therapist or caseworker, this is not typically the case. Most often, parents within the child welfare system receive parent training in small groups without their children present, and to date, few studies have examined effects of behavioral parent training interventions during supervised visitation. In this study, concurrent multiple baseline across behaviors design was used to examine effects of a behavioral parent training program, Filming Interactions to Nurture Development (FIND), on parental skill acquisition with four mothers who had lost custody of their children but were being considered for reunification. Children emitted little or no problem behaviors during baseline or intervention, so parenting behavior was the primary dependent variable. Results obtained across participants documented a clear functional relation between implementation of the FIND intervention and increases in developmentally supportive parenting behaviors. Results of social validity and contextual fit measures suggest the intervention was perceived by mothers to be positive, feasible, and appropriate within the child welfare context. Practical and conceptual implications, limitations of this study, and directions for future research are discussed.

1. Introduction

An estimated 3.4 million referrals involving the alleged maltreatment of children were received by child protective service agencies across the United States during the federal fiscal year of 2012 (Child Welfare Information Gateway, 2014). Among the states reporting a breakdown of service type, two-fifths (38.6%) of victims who received services were placed in out-of-home care, including foster and group facilities (Child Welfare Information Gateway, 2014). Foster home placements are intended to be short-term responses to ensure the safety and wellbeing of children, with the primary goal of reunifying foster children with their biological parents (Brook, MacDonald, & Yan, 2012).

Although approximately 50% of children nationally leave the foster care system through reunification with their parents, reunification often is not successful (D'Andrade & Chambers, 2012). Nationally, nearly 30% of reunified children return to foster care within 10 years, with the majority of children reentering within the first year of reunification (Wulczyn, 2004). Over the past three decades, researchers have identified four parenting concerns that often lead to child abuse and neglect, and the subsequent removal of children from parental custody: parental substance abuse (e.g., Barth, 2009; Besinger, Garland, Litrownik, & Landsverk, 1999), parental mental illness (e.g., Glennon, 2003; Kundra & Alexander, 2009), domestic violence (e.g., Barth, 2009; Casaneuva, Martin, Runyan, Barth, & Bradley, 2008), and child conduct problems (e.g., Burns, Phillips, Wagner, Barth, Kolko, Campbell, & Yandsverk, 2004; Fanshel, 1992). Since reunifying children with their birth parents continues to be a goal of the child welfare system, local and national agencies have taken steps to reducing the reoccurrence of these four parenting concerns through parent education and training (Beyer, 2008; Wulczyn, 2004). As noted by Barth et al. (2005), parent training is often the primary intervention that child welfare agencies provide in trying to prevent child removal or reunify families.

Behavioral parent training is an evidence-based approach for the improvement of parenting behaviors and the prevention and treatment of disruptive behaviors exhibited by children (for recent reviews see Chorpita et al., 2011 or Comer, Chow, Cooper-Vince, & Wilson, 2013). There are a number of different behavioral parent training programs but most are based upon the social-interactional model developed by Patterson and colleagues (e.g., Patterson, 1982; Patterson, Reid, & Dishion, 1992). Behavioral parent training programs teach parents specific skills (e.g., prompting strategies, differential reinforcement, extinction) to provide structure, encourage desired behaviors, and decrease problem behaviors.

Although most behavioral parent training involves in-vivo instruction and rehearsal, video feedback has been used in multiple studies (e.g., Fukkink, 2008; Hitchcock, Dowrick, & Prater, 2003). In such programs, parents are filmed interacting with their child, typically practicing newly learned skills. Parents then observe all or part of the recordings with a therapist. This allows therapists to focus parental attention on specific parent behaviors and to see the effects of those behaviors on their child (Fukkink, 2008). Meta-analytic reviews of video feedback document significant positive effects of video feedback on parenting behaviors, attitudes of parents toward parenting, and the development of their children (Fukkink, 2008; Fukkink, Trienekens, & Kramer, 2011; Hitchcock et al., 2003).

Another variation of behavioral parent training involves an explicit focus on parent and/or child strengths (e.g., Buchanan, Chamberlain, Price, & Sprengelmeyer, 2013; Wulczyn, 2004). A strength-based approach to parent training targets behavioral skills currently in a parent's repertoire and seeks to reinforce those skills and provide prompting to encourage generalization of those skills to others in the same response class. Strength-based practice in child welfare work has a strong theoretical foundation as an effective helping strategy for developing prosocial skills and appropriate behaviors in at-risk children and adults (Brun & Rapp, 2001; Buchanan et al., 2013; Fisher, Kim, & Pears, 2009).

Although behavioral parent training programs are frequently used by child welfare agencies when caseworkers are attempting to prevent child removal or reunify families (Barth et al., 2005), the use of parent training programs for families in the child welfare system has received little examination, and few studies have examined the use of such practices during supervised visitation time (Beyer, 2008). In this study, the effects of a strength-based parent training program, Filming Interactions to Nurture Development (FIND; Fisher, 2012), were evaluated with four mothers who had lost custody of their young children due to neglect, substance abuse, or threats of harm to the child. FIND was adapted from an empirically validated parenting program called Marte Meo (Aarts, 2000). In Marte Meo, the therapist videotapes everyday caregiver-child interactions and then edits the videos, pulling out brief snippets illustrating the caregiver engaging in a desired behavior. The therapist describes the skill to the parent and highlights how the child responded (Aarts, 2000). Marte Meo has been shown to be effective in reducing teacher reports of child problem behavior in schools (Axberg, Hansson, Broberg, & Wirtberg, 2006), improving supports for new adoptive parents (Osterman, Moller, & Wirtberg, 2010), and facilitating healthy mother-infant interactions in an effort to combat postnatal depression (Vik & Rohde, 2014); however it has not previously been examined in the context of child welfare visitation.

FIND was adapted from Marte Meo in the following ways. First, FIND targets between one and four skills depending on the needs of a specific parent whereas Marte Meo targets six parenting skills (see Table 1 for skills taught in Marte Meo versus FIND). Second, FIND is time-limited, lasting 10 weeks with sessions running for no more than 45-min in duration (Marte Meo is not time limited). Third, FIND is structured such that skills are taught and practiced by parents in a specific sequence, rather than simply identified and recorded when they happen to occur. The sequence of the four parenting behaviors is based on the concept of “Serve and Return” that was developed at the Center on the Developing Child at Harvard University (2007). Communication and social skills are supported in the developing brain when children's behaviors, whether verbal (cooing, giggling) or nonverbal (gesturing), are responded to in a nurturing manner (smiling, returning the coo, verbally acknowledging). Such back-and-forth interactions between young children and their caregivers shape the architecture of the child's brain (Center on the Developing Child, 2007). If an adult's responses to a child are inconsistent, absent, or punitive, the developing architecture of the child's brain may be interrupted and may potentially lead to childhood impairments (Center on the Developing Child, 2007). Therefore, FIND, just like Marte Meo, focuses on developmentally supportive parenting behaviors that aim to increase the likelihood that caregivers will notice and respond to child behaviors in appropriate and nurturing ways.

Table 1.

Skills taught in Marte Meo versus FIND.

Marte Meo skills	FIND skills
Element 1. The adult seeks to identify the child's  focus of attention.	Parenting behavior 1. Sharing the focus of attention
Element 2. The adult seeks to confirm the child's  focus of attention or initiative and then adds her  own reaction. The confirmation attempt may be  verbal or non-verbal.	Parenting behavior 2.
	Noticing and encouraging
Element 3. The adult actively awaits the child's  response to her own action.	Parenting behavior 3.
	Turn-taking
Element 4. The adult names both ongoing actions,  events, experiences and feelings as well as  anticipated ones.	Parenting behavior 4.
	Beginnings and endings
Element 5. The adult triangulates the child to the  surrounding world by naming what is there  (within the child's possible focus of attention).
Element 6. The caregiver has the responsibility to  achieve a mutually clear and satisfactory ending  to present activities when possible.

The FIND adaptions were made to (1) allow for specific instruction on core parenting behaviors, (2) maximize the efficiency of the intervention by sequencing skills, and (3) limiting the time of the intervention as well as the time of each therapy session. The goal of these modifications was to make the intervention more structured for training, more feasible to implement, and a better fit for the context of child welfare.

The purpose of this study was to conduct a preliminary investigation, using single-subject methodology, of the FIND intervention on developmentally supportive parenting behaviors exhibited by mothers who had lost custody of their children. The goal for using FIND during supervised visitation time was to support participating mothers in building the necessary parenting skills needed for fostering healthy development, communication, and attachment with their children, in an effort to bolster some of the skills necessary for family reunification. Specifically, this study addressed the following research question:

1. Is there a functional relation between implementation of FIND and an increase in developmentally supportive parenting behaviors across 4 at-risk mother-child dyads?

2. Method

2.1. Setting and participants

2.1.1. Setting

This study took place at the Department of Human Services (DHS) in a suburban county in Oregon. Visitation meetings between participating mothers and their children occurred at three DHS facilities in the participating county. These meetings were scheduled and conducted by the DHS; the schedule, duration, frequency, and location of these meetings were not adjusted or altered for this study. Additionally, participation or lack of participation in this study did not affect visitation rights or other matters related to custody or DHS. Visitation meetings were held as per the DHS guidelines, in a private room that was stocked with toys, books, a sofa, a table, and chairs. The rooms were equipped with a two-way viewing mirror for supervision from an assigned caseworker.

2.1.2. Participants

Participants in this study were four mother-child dyads. Participant recruitment and selection occurred in several steps. First, a Child Welfare Program Manager at the DHS who helped to organize implementation of this study sent a recruitment email out to all DHS Caseworkers. To be eligible for this study, child participants had to be between the ages of 1 and 3 years old, and participating mothers were those who did not have custody of their child but were receiving weekly supervised visitation time with their child at the DHS. Since the goal of this study was to provide parent training in the hopes of increasing appropriate parenting behaviors for future reunification, mothers who were not being considered for reunification with their child or those with partial custody of their child were not included in this study. DHS Case-workers nominated potential families that met the inclusion criteria only after receiving approval from the mother. Individual informed consent was obtained from the mothers who agreed to participate and their caseworkers. The first four mother-child dyads who agreed to participate took part in the study, and they were all given unique pseudonyms to protect their confidentiality.

2.1.2.1. Maria and Makela

Maria was a 20-year-old White mother of three children. Her daughter, Makela, was a 27-month-old multiracial (White and Hispanic) child with developmental delays who was removed from Maria's custody, along with her two siblings, at 18 months old due to medical neglect and Maria's illegal substance use. Prior to beginning this study, Maria had completed intensive outpatient drug treatment, and had participated in a parenting group offered by a local organization, which she did not complete.

2.1.2.2. Denise and Donny

Denise was a 24-year-old Native American mother of two children. Her son, Donny, was a 12-month-old multiracial (Native American and White) child with developmental delays who was removed from Denise's custody at 2 days old due to Denise's illegal substance use while pregnant. Prior to beginning this study, Denise completed intensive outpatient drug treatment and a group parent training program. She also began but did not complete an enhanced visitation program through a local religious organization.

2.1.2.3. Lanita and Leo

Lanita was a 26-year-old White mother of one child. Her son, Leo, was a 14-month-old multiracial (White and Native American) child who was removed from Lanita's custody at 6 months old due to threat to harm, as well as Lanita's illegal substance use and mental health concerns. No health concerns or cognitive delays were reported for Leo. Lanita had completed a brief group parent training program prior to beginning this study, and was actively involved in outpatient opioid treatment.

2.1.2.4. Sandra and Sylvia

Sandra was a 36-year-old White mother of two children. Her daughter, Sylvia, was a 26-month-old White child with selective mutism who was removed from Sandra's custody at 6 months old due to child neglect and illegal substance use. Sandra had not completed any treatment programs prior to participation in this study, but she was actively involved in group parent training and intensive outpatient drug treatment, and graduated from drug treatment 4 weeks into this study.

2.2. Data collection, response definitions, and interobserver agreement

The first author, who served as the therapist in this study, was in the room during supervised visits to record videos but stayed at least five feet from family members and did not interact with them. Three, 10-min video observations were obtained per visitation. Videos were recorded during times the parent and child were interacting with one another and no one else (e.g., the caseworker). All video observations took place during the first 45-min of the visitation session, with 5-min breaks provided in between each 10-min video. During video observations, the caseworker for each participating mother remained on the other side of the two-way mirror and did not interfere with the visit unless a negative parenting behavior was observed.

All parent behaviors were scored using real time data collection on a computerized data collection program. The following parent behaviors were coded (the first four labels are derived from the FIND manual; Fisher, 2012): (1) sharing the focus of attention, (2) noticing and encouraging, (3) turn-taking, (4) beginnings and endings (these four responses collectively define “developmentally supportive parenting behaviors”), and (5) negative affective behavior, (6) negative physical behavior, and (7) inattention/neglect. Partial interval recording across consecutive 5-sec intervals was used for all responses except beginnings and endings, which was coded as a frequency measure, because this response occurred infrequently.

Sharing the focus of attention was scored when the mother directed her gaze toward what her child was looking at or interacting with or, if the child was emoting (e.g., laughing, crying), directing her gaze to the child. No ti c in g a n d enc ou ra gi n g could occur only after sharing the focus of attention and included one or more additional response: (1) naming the child's interest/initiative or context (e.g., “You see a ball”), (2) praising, paraphrasing, or positively verbally acknowledging the child's interest/initiative or context, or (3) giving a nonverbal positive physical gesture (e.g., high five, pat on the back, thumbs up.) Tu rn-taking was scored when the mother and child responded to each other in language and/or action in a reciprocal back and forth rhythm. During turn-taking, the mother mirrored or imitated her child's behavior in a positive way (e.g., child throws a ball to parent, parent throws the ball back to child; baby coos, parent coos back). Beginnings and endings involved the parent starting or stopping an activity or interaction by clearly signaling the change in focus with use of a verbal or physical cue. Parents verbally or physically describe an upcoming transition prior to initiation of that transition (e.g., parent finishes changing baby's diaper and says, “All done with the diaper change,” before taking the child off the changing table). Data were also collected on three negative parenting behaviors that were trained and coded: Negative affective behavior (e.g., sighing or eye rolling at child or a child's behavior), negative physical behavior (e.g., not supporting a baby's head, pulling a child up by his/her arms, or rough handling), and inattention/neglect (e.g., ignoring a child's cues, turning your back to the child). These behaviors, however, were not observed during any of the observation sessions and therefore are not presented in this study.

Graduate students served as trained observers for this study. Prior to beginning data collection, observers were trained to an 85% interobserver agreement criterion on each target behavior. Two observers independently scored 40% of observations during baseline, intervention, and maintenance phases for each of the four mother-child dyads. Total agreement, occurrence only agreement, and nonoccurrence only agreement were calculated for each coded behavior. Total agreement was calculated by dividing the number of intervals that both observers agreed a response did or did not occur by the total number of intervals observed, and multiplying that number by 100. Occurrence only agreement was calculated by dividing the total number of intervals both observers agreed a response occurred by the number of intervals either observer scored a response, and multiplying that number by 100. Nonoccurrence only agreement was calculated by dividing the total number of intervals both observers agreed a response did not occur by the total number of intervals either observer did not score a response, and multiplying that number by 100.

For sharing the focus of attention, total agreement averaged 91% (range = 72% to 100%), occurrence only averaged 90% (range = 70% to 100%), and nonoccurrence only averaged 93% (range = 76% to 100%). For noticing and encouraging, total agreement averaged 83% (range = 53% to 96%), occurrence only averaged 83% (range = 59% to 97%), and nonoccurrence only averaged 85% (range = 58% to 97%). For turn-taking, total agreement averaged 85% (range = 62% to 97%), occurrence only averaged 83% (range = 68% to 95%), and nonoccurrence only averaged 85% (range = 65% to 97%). For beginnings and endings, total agreement averaged 94% (range = 85% to 100%), occurrence only averaged 91% (range = 80% to 100%), and nonoccurrence only averaged 93% (range = 80% to 100%). Across all four mothers, the only coded behaviors with averages that fell below 85% total agreement were noticing and encouraging and turn-taking; likely because the definitions for these responses were similar.

2.2.1. Fidelity of implementation

Fidelity of FIND implementation was assessed by trained observers for 100% of videos and coaching meetings. Fidelity of each was assessed using a checklist documenting key features. For each edited video, the presence or absence of the following was scored: (a) initial frame depicted a positive interaction, (b) each of three video clips documenting the occurrence of targeted parenting behavior, (c) an ending frame depicting a positive interaction, and (d) total duration of video clip was 3 min or shorter. For feedback sessions raters scored the extent to which the therapist (a) provided a summary of the coaching process to the mother, (b) made eye contact with the mother and used a friendly tone of voice during each discussion of the video clips, (c) provided positive praise to the mother for every developmentally supportive parenting behavior, (d) identified for the mother how every developmentally supportive parenting behavior is supportive of her child's development and why, (e) solicited the mother's input, questions, and opinions, and reflectively listened while she shared, and (f) provided the mother with a reminder of the developmentally supportive parenting behaviors to work on during her visitation meeting with her child. Across dyads, fidelity of video editing was 100%. For feedback sessions, mean fidelity was 99.92% (range = 99.66% to 100%).

2.2.2. Contextual fit and social validity

Contextual fit was assessed before the first week of implementation of the FIND intervention phase and again at the end of the study. Participating mothers completed a 4-item contextual fit questionnaire that assessed mothers' understanding of the intervention, their perception of whether the FIND program would help them reach their parenting goals, and whether the program was stressful for them. Scores on the questionnaire were recorded on a Likert scale from 1 to 6, with higher scores indicating a more favorable impression of the contextual fit on the participating mothers.

Social validity of the FIND intervention was assessed at the conclusion of the intervention. A 13-item social validity questionnaire was administered to each participating mother. Items on the questionnaire assessed the extent to which the intervention was perceived to improve parenting skills, improve the mother-child bond, was worth the time and effort, was worth recommending to others, and was easy to participate in. Scores on the questionnaire were recorded on a Likert scale from 1 to 6, with higher scores indicating a more favorable perception of the FIND intervention.

2.3. Design and procedures

A concurrent multiple baseline across behaviors design was used to assess functional control. There were three phases; baseline, FIND implementation, and maintenance.

2.3.1. Baseline

In the baseline phase, the four mother-child dyads were observed during unstructured supervised visitation time at the DHS. All observations were video recorded for later scoring, and no teaching procedures to modify any of the four developmentally supportive parenting behaviors were in effect. However, in the instance that a negative parenting behavior was observed, the therapist would leave the room and instruct the caseworker, who was also observing through a two-way mirror, to intervene and provide support to the mother and child. A minimum of five observations (across a minimum of two visitation meetings) were conducted for each mother-child dyad during the baseline phase.

2.3.2. FIND implementation

For each dyad, implementation occurred sequentially across target parent behaviors, beginning with sharing the focus of attention. Subsequent parent training targeting additional behaviors (noticing and encouraging, turn-taking, and beginnings and endings) was initiated every two weeks, as per FIND guidelines. These four parenting behaviors were taught in the same sequence across all participants, per FIND guidelines, as each parenting skill serves as a prerequisite for the following parenting skill (sharing the focus of attention is the prerequisite skill needed for noticing and encouraging, noticing and encouraging is the prerequisite skill needed for turn taking, turn taking is the prerequisite skill needed for beginnings and endings). Therefore, intervention did not necessarily happen with the most stable behavior. It occurred with the parenting skill being taught that week. Implementation of FIND involved the following components: editing videos, program description for parents, coaching with edited videos, and four training periods.

2.3.2.1. Editing videos

Videos were edited using iMovie, a computerized video editing software application available for MAC and iOS products. Videos taken during the previous session with the mother were trimmed down to show examples of the targeted parenting behavior. Edited videos were no longer than 3-min, and consisted of two still pictures and three video clips of the mother and child engaging in the developmentally supportive parenting behavior that was to be taught in that session. For example, before the first coaching meeting the therapist used baseline videos to create a brief video depicting the mother and child sharing the focus of attention (components of videos were described above under fidelity of implementation). This was the only time baseline videos were used. For each subsequent session, the edited videos shown to the mothers were created from videos taken the previous week. The process of editing the videos took the therapist less than 15-min to complete per family.

Across all participating families, the therapist was always able to identify examples of the developmentally supportive parenting behaviors in previous session videos and therefore never needed to splice together clips from older videos. In the instance, however, that the therapist was unable to identify the targeted behaviors in the previous session videos, the protocol of using older video examples would have been used.

2.3.2.2. Coaching with edited videos

Coaching meetings with participating mothers took place at the DHS facility, approximately 30–45 min prior to their visitation with their child. Since the mothers were unable to practice the developmentally supportive parenting behaviors with their child outside of the context of their visitation meetings, coaching sessions were strategically designed to occur before the beginning of their visitation meeting so that the mothers would have the opportunity to learn about those behaviors, receive positive feedback on those behaviors, and then practice those behaviors during the subsequent visitation with their child. Coaching sessions adhered to the guidelines listed on the fidelity of implementation checklist.

At the beginning of each coaching session, the therapist introduced the film to the mother and explained how the process of coaching would go. The therapist said:

Today we are going to watch an edited film of the different moments I recorded between you and your child last week. The film will begin with a still shot of you and your child. Then there will be a series of three short clips. I will tell you what I want you to focus on before each of the clips we watch. We will watch each clip one time through and then we will look at it frame by frame. The film will end with one more still shot of you and your child. Before we begin do you have any questions?

During each coaching session, the therapist spent 5–10 min discussing each still frame and video clip and solicited input from the mothers. The therapist began with the still picture and explained to the mother why the still picture was chosen. The therapist then asked for the mother's reaction and feelings about the picture. The therapist then introduced the first clip, played it through one time, and then went back through it and stopped at points in which a developmentally supportive parenting behavior occurred. The therapist did that with the subsequent two clips, and ended by showing the final still frame and discussed why it was chosen, in a similar manner to the discussion of the first still frame.

After finishing the film review, the therapist discussed with the mother the developmentally supportive parenting behavior trained that day and reviewed why the example behaviors demonstrated in the clips were good for her child's development. The therapist then gave the mother an information sheet that reviewed the core element of developmentally supportive parenting behavior. The therapist then asked the mother if any behaviors in particular stood out to her from the video that she would like to do more of during her visitation time, and the encouraged the mother to go into her visitation meeting with those behaviors in mind. Before leaving the coaching meeting for her visitation session with her child, the mother was encouraged by the therapist to watch for moments during visitation where she could engage in the developmentally supportive parenting behavior. Video observations took place immediately after coaching, during the first 45-min of visitation with the child, while the trained parenting behavior was freshly on the mind. It should be noted that the focus of the FIND intervention is to teach, model, and reinforce only appropriate parenting behaviors. Therefore no teaching or corrections are made around negative parenting behaviors. In the instance that negative parenting behaviors occurred, the caseworker assigned to the mother would provide teaching and feedback.

2.3.3. Maintenance

Maintenance sessions were conducted approximately 1 week following the conclusion of the intervention. During maintenance sessions, no coaching was given to the mothers prior to their visitation time. The therapist videotaped three 10-min interactions at two different visitation meetings.

2.3.4. Data Analyses

2.3.4.1. Visual analysis

Data were analyzed to assess the extent to which the FIND intervention was functionally related to changes in developmentally supportive parenting behaviors. Direct observation data related to parent and child behaviors were analyzed using visual analysis, which was done by examining each phase and assessing the level, trend, variability, and immediacy of effect across baseline and intervention phases (Horner, Carr, Halle, McGee, Odom, & Wolery, 2005). In a within-subjects multiple baseline design, experimental control is documented when the introduction of an independent variable results in a systematic change in level and trend only for the dependent variable with which the independent variable is applied (Baer, Wolf, & Risley, 1968; Christ, 2007). The independent variable is introduced in a systematic manner across each dependent variable (in this case developmentally supportive parenting behaviors) and experimental control is demonstrated if changes in the dependent variable are observed only after introduction of the independent variable (Baer et al., 1968; Christ, 2007). Participant behavior was considered responsive to intervention if observable and sustained increases in developmentally supportive parenting behaviors were recorded during the intervention phases (Horner et al., 2005). Descriptive statistics (i.e., percentage of overall mean rates and ranges of rates of parent behaviors) were used to analyze data. The stability criterion of comparing the last three baseline sessions to the last three intervention sessions was selected, as this criterion allows for a steady state of comparison that provides a more accurate representation of intervention effects (Johnston & Pennypacker, 1993; Sidman, 1960).

2.3.4.2. Effect size

We used Cohen’s (1988) d to estimate the treatment effect size for each behavior for each mother-child dyad (i.e., (Meantreatment – Meanbaseline) / SDpooled). An ES is the standardized mean difference between two groups, where the scale represents the standard deviation (SD). For example, 0 is equivalent to no difference, and 1.00 is a difference of one SD between groups. A widely used interpretation of ES is that 0.20 is considered “small,” 0.50 “medium,” and 0.80 “large” (Cohen, 1988). Using the partial interval count data, we estimated d for the observed treatment-baseline means, and for the OLS estimated slopes for the treatment and baseline trends (R Core Team, 2016).

2.3.4.3. Descriptive analyses

Fidelity data were measure and analyzed descriptively to ensure the FIND intervention was implemented as planned. Contextual fit data were analyzed descriptively using pre- and post-comparisons to assess the appropriateness of implementing the FIND intervention during supervised visitation time. Lastly, social validity data were analyzed descriptively to gain a better understanding of mothers’ perceptions of the FIND intervention.

3. Results

Results are presented in Figs. 1 through 4. Across participants, target behaviors increased systematically after implementation of FIND. Percent change was calculated for each target behavior based on the last half of baseline data and the last half of intervention data (as recommended by Boyer, Miltenberger, Batsche, & Fogel, 2009). Across participants, the smallest change was observed for turn-taking.

Fig. 1.

Percentage of 5-s intervals with sharing the focus of attention, noticing and encouraging, and turn-taking (top three panels) and frequency with beginnings and endings (bottom panel) for Maria.

Fig. 4.

Percentage of 5-s intervals with sharing the focus of attention, noticing and encouraging, and turn-taking (top three panels) and frequency with beginnings and endings (bottom panel) for Sandra.

Results for Maria and Makela are in Fig. 1. For these participants, a functional relation between the intervention and parent behavior was documented as responding increased systematically only when the intervention was introduced. This effect was documented four times (for each response) and at four different points in time (the intervention was introduced at a different time for each response).

When intervention was introduced with sharing the focus of attention, there was a reduction in variability of responding and an increase in both level and trend. The arrow on the graph indicates when Maria began interacting only with Makela during observations, prior to this point she had interacted with all three of her children. At this point, sharing the focus of attention increased markedly. Overall, intervention resulted in a 60% increase in sharing the focus of attention relative to baseline. Responding further increased in maintenance. Cohen's d was 1.5 for the for the observed treatment-baseline means (Mean_tx = 79.1, Mean_bl = 41.0), and d for the OLS estimated slopes was 1.9 (β_tx = 2.6, β_bl = − 3.3); thus, the estimated treatment effects for sharing the focus of attention were considered quite large.

Noticing and encouraging was the second response targeted for intervention. When intervention was implemented, an increase in responding occurred during the 4th intervention observation. Responding in intervention increased 27% relative to baseline and this level and trend continued in maintenance. The estimated treatment effects for noticing and encouraging were considered large (d_Means = 1.5, Mean_tx = 48.2, Mean_bl = 20.7; d_slopes = 0.7, β_tx = 0.9, β_bl = 0.1). Maria almost never engaged in turn-taking in baseline. Introduction of the intervention resulted in a 21% increase in turn-taking and again, this was sustained in maintenance. The estimated treatment effects for turn-taking were considered quite large (d_Means = 4.1, Mean_tx = 38.1, Mean_bl = 5.5; d_slopes = 1.2, β_tx = 0.8, β_bl = − 0.0). Finally, beginnings and endings occurred infrequently in baseline. Introduction of the intervention resulted in an increase of 6 occurrences of this response per observation. Although levels were somewhat lower in maintenance, an increasing trend was noted. The estimated treatment effects for beginnings and endings were considered quite large (d_Means = 2.4, Mean_tx = 7.4, Mean_bl = 2.2; d_slopes = 1.5, β_tx = 0.2, β_bl = − 0.0). Results for Denise and Donny are in Fig. 2. For these participants a functional relation was demonstrated as well but there are only three demonstrations of the effect at three points in time because the intervention did not result in an observable change on the level of turn-taking (variability was decreased due to intervention however).

Fig. 2.

Percentage of 5-s intervals with sharing the focus of attention, noticing and encouraging, and turn-taking (top three panels) and frequency with beginnings and endings (bottom panel) for Denise.

When intervention was introduced for sharing the focus of attention, an immediate increase in level was noted as was a reduction in variability of responding. This continued in maintenance. Intervention resulted in a 36% increase in sharing the focus of attention relative to baseline. The estimated treatment effects for sharing the focus of attention were considered quite large (d_Means = 1.9, Mean_tx = 107.7, Mean_bl = 77.8; d_slopes = 2.1, β_tx = 0.6, β_bl = − 8.3). A similar effect was noted for noticing and encouraging (intervention resulted in a 19% increase in responding). The estimated treatment effects for noticing and encouraging were considered large (d_Means = 2.5, Mean_tx = 58.7, Mean_bl = 16.9; d_slopes = 0.6, β_tx = − 1.4, β_bl = − 1.9). For turn-taking, although variability in responding decreased following intervention, an increase in occurrence was negligible and level of responding returned to at or below baseline levels in maintenance. The estimated treatment effects for turn-taking were considered quite large (d_Means = 1.1, Mean_tx = 33.4, Mean_bl = 17.3; d_slopes = 1.2, β_tx = − 0.4, β_bl = − 1.3). Finally, intervention resulted in an increase of 5 occurrences relative to baseline for beginnings and endings. The treatment effects for beginnings and endings were considered quite large (d_Means = 2.9, Mean_tx = 6.3, Mean_bl = 1.3; d_slopes = 0.3, β_tx = − 0.1, β_bl = − 0.1).

Results for Lanita and Leo are in Fig. 3. For these participants, functional control was documented as an increase in level and trend was documented across three responses, sharing the focus of attention, turn-taking, and beginnings and endings. Although noticing and encouraging increased markedly following intervention, a downward trend in responding was apparent across the intervention phase.

Fig. 3.

Percentage of 5-s intervals with sharing the focus of attention, noticing and encouraging, and turn-taking (top three panels) and frequency with beginnings and endings (bottom panel) for Lanita.

Lanita shared the focus of attention during the majority of intervals during baseline; intervention resulted in a further increase in responding (a 10% increase relative to baseline) and this was sustained in maintenance. The estimated treatment effects for sharing the focus of attention were considered large (d_Means = 2.0, Mean_tx = 103.9, Mean_bl = 85.2; d_slopes = 0.8, β_tx = − 0.4, β_bl = − 1.7). As noted above, although noticing and encouraging increased markedly following intervention, a downward trend in intervals scored with this response was apparent across intervention, however responding did stabilize in maintenance. Thus, the estimated treatment effects for noticing and encouraging were mixed (d_Means = 3.6, Mean_tx = 71.3, Mean_bl = 14.7; d_slopes = − 6.0, β_tx = − 3.1, β_bl = − 0.3). Lanita rarely engaged in turn-taking in baseline, this response occurred in just 6% of intervals. She increased turn-taking during intervention, an overall 12% increase was observed, and this was sustained in maintenance. The estimated treatment effects for turn-taking were quite large (d_Means = 1.5, Mean_tx = 15.0, Mean_bl = 6.7; d_slopes = 1.1, β_tx = − 0.1, β_bl = − 0.5). Finally, intervention resulted in an increase of 5 occurrences of beginnings and endings per observation session, which also sustained in maintenance, and the estimated treatment effects were also quite large (d_Means = 1.0, Mean_tx = 5.3, Mean_bl = 2.1; d_slopes = 1.0, β_tx = 0.2, β_bl = − 0.1).

Results for Sandra and Sylvia are in Fig. 4. Functional control for this dyad was documented as introduction of the intervention resulted in an immediate increase in the level of all four responses, which remained stable in maintenance. In intervention, a 19% (d_Means = 6.0, Mean_tx = 111.2, Mean_bl = 76.4), 15% (d_Means = 1.1, Mean_tx = 55.2, Mean_bl = 37.3), and 13% increase (d_Means = 5.0, Mean_tx = 19.25, Mean_bl = 3.9) was noted in sharing the focus of attention, noticing and encouraging, and turn-taking, respectively. An increase of 5 occurrences in beginnings and endings per session occurred, relative to baseline (d_Means = 1.6, Mean_tx = 6.5, Mean_bl = 2.3). A large treatment effect was estimated for the slope of turn-taking (d_slopes = 1.0, β_tx = 0.2, β_bl = − 0.1). The estimated treatment effects were negative for the slope of sharing the focus of attention (d_slopes = − 1.5, β_tx = − 0.3, β_bl = 1.0), although visual analysis suggests a relative stable (flat) trend for both, and a clear increase in the behavior upon intervention. As was the case with other participants, a decreasing trend in noticing and encouraging was observed (d_slopes = − 1.0, β_tx = − 1.9, β_bl = − 0.7); however, responding stabilized across the final six intervention observations.

Contextual fit was rated highly prior to and following implementation of FIND. Prior to intervention, mothers rated the contextual fit as a mean of 5.3 (range 1–6). After intervention the mean was 4.8 (range 4–6). Social validity results also were positive across participants. All participating mothers rated four of the 13 items with the highest score of 6, and nine of the 13 items with a mean score of 5.7 (range 4–6).

4. Discussion

Across participants, the FIND intervention resulted in systematic increases in the four targeted parenting behaviors, and these increases maintained for most behaviors during maintenance probes which took place one week post-intervention. Estimated treatment effect sizes supported functional relations findings throughout the study. However, the demonstrations of experimental control are not consistently strong across all four dyads. For Maria, stronger effects in sharing the focus of attention and noticing and encouraging were seen after she began conducting observation sessions with just Makela, and not her other two children. This modification adds a confound to her results since she was not given this prompt during baseline sessions. Denise demonstrated increases in all four developmentally supportive parenting behaviors at four different points in time, although we see turn-taking drop down to baseline levels during maintenance sessions. For Lanita, we see immediate increases in all four behaviors following implementation of FIND, however significant overlap in the data between baseline and intervention phases for beginnings and endings make these data less convincing than the others. Lastly, Sandra demonstrated similar effects as Lanita, with immediate increases in all four trained behaviors following FIND implementation but significant overlap between baseline and intervention data for noticing and encouraging.

4.1. Limitations and future research

Across participating mothers, noticing and encouraging decreased when turn-taking was introduced. This may reflect an order effect whereby the sequence in which these behaviors were trained may have had an impact on how the mothers' learned and retained each skill. Alternatively, noticing and encouraging could have been a more difficult skill to implement such that the introduction of turn-taking may have diverted the mothers away from practice of noticing and encouraging. The decreasing trend in noticing and encouraging may also be an effect of time and the duration that each skill was trained. These data suggest the importance of further exploration on how and when these two skills are trained, and the need for a possible refresher on noticing and encouraging after turn-taking has been introduced. For example, future research may examine whether extending training on noticing and encouraging might attenuate this reduction. Additionally, the authors of FIND may consider that the operational definitions for these two behaviors have similarities and are in need of further clarification on their differences.

The behavioral mechanism for the effectiveness of FIND was not established in this study. The intervention consisted of several potentially important components, any or all of which may have resulted in the positive outcomes obtained by the participating mothers. Decades of research on behavioral parent training support the efficacy of explicit instruction and positive feedback to increase parenting behaviors (Comer et al., 2013) and it seems likely that the components of FIND contributed to the results obtained in this study. Future research may examine the impact of cumulative skills coaching, consistent with other behavioral parent training, whereby training includes introducing new skills as well as reviewing previously learned skills. For the population served in this study, cumulative skills coaching may be particularly beneficial since the parents did not have access to their children to practice learned skills between visits. Future research is also needed on the relative contribution of the video segments when added to coaching. Beyond examining whether the videos contributed to skill acquisition, research should examine which components of the therapist's video feedbackfacilitated acquisition and maintenance of learned parenting skills. As noted by previous research (Fukkink, 2008), the video coaching may have enhanced instruction by allowing the therapist to spotlight specific instances of the target behaviors using the mother as the model. Second, by viewing the video, the mother could observe effects of her own behavior on her child's responding. Anecdotally, mothers reported enjoying the video aspect of the intervention and stated that it clarified the developmentally supportive parenting behaviors and the impact such behaviors had on their children, as they could see their behaviors and their child's responding instead of just hearing about them.

Beyond examining mechanisms responsible for intervention effects, future research is needed to examine the maintenance and generalization of FIND skills. Maintenance data were only collected 1-week post intervention completion, and generalization data were not collected for any of the participating families because none of the children transitioned home during the course of this study. It would be beneficial for future studies to collect data on continued maintenance of the trained skills in the visitation setting as well as generalization of the trained skills to the home setting, for mothers who are reunified with their child. These data would help determine if the parenting skills learned during visitation time generalized to the home setting and maintained over a longer period of time. A longitudinal RCT with three conditions (visitation as usual (control), FIND as described, and FIND plus in-the-moment behavior coaching) may also be beneficial for examining if FIND is effective for long-term behavior change.

With regard to the sample itself, this study was conducted with mothers identified by caseworkers as likely to reunify with their children. Thus, they may have been highly motivated to participate in the study; therefore research is needed on the effects of this intervention on other families or caregivers in different stages of reunification, and with a larger sample. The selection of these mothers was based on pretty strict inclusion criteria; however it was left up to the caseworkers' discretion as to whom they nominated. Whether similar outcomes would be obtained with mothers who have already been reunified with their children, mothers less likely to be considered for reunification, or other parents or guardians is not clear.

It should be noted that a limited amount of information was shared regarding the parenting groups the four mothers had previously participated in and why some of them were not completed. Anecdotally, some mothers reported not enjoying previous parenting groups or finding them more punitive in nature, while others shared that they did not have reliable transportation to attend those groups consistently. Therefore, it is difficult to judge how FIND differed from others services the mothers had access to or how FIND expands upon previously taught skills. It is also difficult to assess if FIND taught these parents new skills, or if they utilized these skills prior their substance use and custody suspension. Conclusions cannot be drawn, either, regarding whether or not FIND addressed the root-cause of the child maltreatment at hand. Although that was not the purpose of this study, it should be noted that supports for families within the child welfare system are complex and multifaceted, often requiring a combination of behavioral parent training, substance use treatment, and therapeutic supports for trauma and violence, to name a few.

The present study only collected social validity and contextual fit data from the participating mothers. This was due to an agreement with DHS that no demands would be asked of the supervising case-workers or attending staff during implementation of this intervention. Although all the mothers rated the FIND intervention as a socially valid intervention that fit well within the context of supervised visitation time, future research may examine if these ratings are consistent across the mothers, their caseworkers, and any other DHS staff members involved in their cases. Caseworker perceptions of the contextual fit of FIND are of particular importance when examining the feasibility of implementing the intervention. FIND utilizes technology that may not be accessible, affordable, or feasible for other clinicians, and while the intervention is brief in duration, extensive training and supervision is required before a clinician can embark on editing videos, selecting clips, providing coaching to parents with fidelity, and teaching the four parenting skills. Effective and supportive parental training is critical to the well-being of families who have lost custody of children due to neglect or abuse, therefore it is imperative that future research examine the feasibility of implementing FIND for clinicians with limited training, funding, equipment, or time. Lastly, the greatest measure of social validity would be reunification of children with their mothers following participation in this intervention, and the collection of generalization data outside of the DHS context would help stakeholders better assess the permanency of learned parenting skills and the safety of children in the custody of their birth parents. We cannot say, based on this initial pilot study, that learning and demonstrating four positive parenting behaviors stops child maltreatment that has happened or prevents future child maltreatment. The goal for using FIND during supervised visitation time was to support mothers in building the necessary skills needed for fostering healthy development, communication, and attachment with their children, in an effort to bolster some of the skills necessary for family reunification. Therefore, future research is needed in order to examine whether participation in the FIND intervention reduces the likelihood of initial or repeated maltreatment of children, or if the FIND intervention needs to be tailored to include specific parenting behaviors in addition to the four behaviors.

4.2. Summary

Child welfare agencies are currently in need of effective interventions for parents who are at-risk of losing custody of their children, have lost custody of their children, and who are working to meet the necessary criteria for being reunified and bringing their children home. Whereas many child welfare agencies provide parent training outside of the supervised visitation context, it may be more efficient for these agencies to provide parents with instruction, modeling, and practice opportunities of appropriate parenting skills during times where they can implement these skills with their children under supervision of a therapist or caseworker. Supervised visitation time for mothers who have lost custody of their children is one such example of a time in which parent training may be provided. This study has demonstrated that in this setting, appropriate parenting skills can be increased.