Implementation of Parent Child Interaction Therapy Within Foster Care: An Attempt to Translate an Evidence-Based Program Within a Local Child Welfare Agency

Abstract

This paper describes an innovative adaptation of an evidence-based intervention – Parent Child Interaction Therapy or PCIT – to foster parent training services. The authors faced multiple problems that commonly plague translational child welfare research as they developed, implemented and tested their model. The paper discusses how the authors addressed these problems when: 1) specifying the child welfare context in which the intervention model was implemented and tested, choosing an intervention model that responded to child welfare service needs, and tailoring the model for a child welfare context; 2) securing external funding and initiating sustainability plans for model uptake; and 3) forging a university-community partnership to overcome logistical and ethical obstacles. Concluding with a summary of promising preliminary study results, a description of future plans to replicate and spread the model, and a distillation of project lessons, the paper suggests that child welfare translational research with PCIT is very promising.

Introduction

This paper will describe an innovative adaptation of an evidence-based intervention – Parent Child Interaction Therapy or PCIT – to a child welfare service context. Consistent with the theme of this special issue, the authors recognize that child welfare services generally lack supportive evidence (Horwitz, Chamberlain, Landsverk, & Mullican, 2010). While the situation is improving (Landsverk, Garland, Reutz, & Davis, 2011).), the problem can be distilled into the following themes: 1) service systems are not yielding expected outcomes (Ai, Foster, Pecora, Delaney, & Rodriguez, 2013); 2) workers are not approaching direct service with evidence-informed frameworks (Barth et al., 2012); and 3) service components do not include validated intervention models (Pecora et al., in press). As intervention scientists, the authors’ concern rests with this last observation: limited application of evidence-based interventions.

However, implementing well-validated intervention models within child welfare poses significant challenges for at least three reasons. First, official child maltreatment is well-recognized to be a low-base rate and generally intractable phenomenon. That is, it does not surface often but when it does it represents caregiver dysfunction that is relatively extreme, multifactorial, and difficult to prevent (MacMillan et al., 2009). In turn, its effects on children are often complex, pervasive, and treatment resistant (Twardosz, & Lutzker, 2010). Consequently, although evidence has been mounting in support of a number of innovative models (e.g., Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009), maltreatment prevention or early intervention programs face difficult odds and have been slow to develop an evidence-base (Reynolds, Mathieson, & Topitzes, 2009).

Second, child welfare agencies lack financial resources to implement gold standard, evidence-based intervention models (Holmes et al., 2014). Initial costs associated with program start-up are often high (Higa & Chorpita, 2007). Training direct service staff often requires substantial outlays of funds (Kazak et al., 2010), and significant expense is typically associated with ongoing service delivery because many of these interventions rely on cost-intensive delivery structures comprised of: a) individual or dyadic formats, b) long duration, or c) both. Therefore, in order for cash-strapped public agencies to invest in high quality intervention models, often the models must be tailored and financial support must be forthcoming.

Third, conducting research to demonstrate the efficacy, effectiveness, and cost effectiveness of any one intervention model as applied to real world settings requires the difficult work of forging university-community partnerships (Aarons, Hurlburt, & Horwitz, 2011). Doing so within child welfare settings poses multiple challenges. For instance, a number of nonprofit agencies within large metropolitan areas often receive public child welfare dollars to deliver contracted services (Unruh & Hodgkin, 2004). Partnering with one agency may limit the research in various ways, e.g., sample size and generalizability, while collaborating with multiple sometimes competing agencies may create conflict. Motivating busy administrators to participate in a research project may prove to be a formidable task, and selecting the appropriate providers to deliver the intervention within any one agency may also prove challenging given most child welfare workers do not hold clinical licenses (Clark, Smith, & Uota, 2013). Once agency collaborations are established and intervention providers are identified, the research team and the agency must agree upon a number of procedural guidelines. Planning these steps in the research process requires resolution of myriad logistical and ethical concerns upon which researchers and practitioners commonly differ. There is no question that this collaborative, community-based implementation research is messier than lab- or clinic-based experimental science (Dodge, 2011; Green & Seifert, 2005). Consequently, best practices often do not reach society’s most vulnerable families.

Despite the difficulties inherent in conducting implementation science, scholars are increasingly recognizing its value both inside and outside the child welfare context (Littell & Shlonsky, 2010). In fact, research groups are forging models of rigorous evaluation science to inform a specific brand of this research, sometimes referred to as T2 research (Woolf, 2008). T2 or second phase translational science describes research that evaluates treatment protocols in health care settings, mental health care clinics, or social service agencies. As opposed to T1 science, which entails basic or experimental research with implications for usual care, T2 science directly tests innovations within usual care contexts that were developed at the T1 phase. The purpose of T2 science is to accelerate the dissemination of evidence-based practices, a function for which social work scholars are well-suited given the interdisciplinary and applied nature of their work (Brekke, Ell, & Palinkas, 2007). Inspired by the growing emphasis on T2 research, the authors set-out to develop, implement, and test a modified version of PCIT within a local child welfare setting. Future projects will pursue the goals of replication, re-testing and sustainability.

In the following pages, the authors will describe the ways in which they addressed the previously-mentioned problems that commonly plague initial phases of implementation research projects within child welfare settings. To elaborate, the paper will outline how the authors: 1) specified the child welfare context in which an intervention model would be implemented and tested, chose an intervention model that responded to child welfare service needs, and tailored the model for child welfare; 2) secured external funding and initiated sustainability plans for model uptake; and 3) forged a university-community partnership to overcome logistical and ethical obstacles endogenous to such work. Moreover, the paper will include a brief summary of preliminary intervention study results, a description of future plans to replicate and re-test the model throughout the local area, and a distillation of project lessons relevant for readers of the Journal of Public Child Welfare. The fundamental message that the authors currently take away from their project titled Project Connect is that adaptation of PCIT for foster parent training conceptually addresses limitations of child welfare services and empirically shows promise in enhancing relevant outcomes. Because of the project’s initial success, the authors propose to expand the foster care application of the model within existing infrastructure and also to deliver and test the model in revised form with biological caregivers receiving child welfare services.

Model Selection and Modifications

Foster Care Services

With the intent of identifying an intervention model that would enhance caregiver capacity and ameliorate child maltreatment effects, the authors targeted licensed, primarily non-relative foster caregivers and their foster children with intervention services for several reasons. First, we surmised that foster parents receiving child welfare services would be generally higher functioning compared to biological parents receiving these same services (for non-relative foster parent profiles see U.S. Department of Health and Human Services, 2001). Consequently, these parents would be more capable of selecting into and remaining in an available program.

Second, foster parents themselves can be ill-equipped to address the often extreme behavior problems with which foster children present (for foster child profiles see U.S. Department of Health and Human Services, 2001). These caregivers rarely receive the kinds of skills-focused training in behavior management that have been shown to mitigate severe disruptive behavior (McNeil, Herschell, Gurwitch, & Clemens-Mowrer, 2005). The training foster parents do receive is typically didactic in nature and lacking in empirical support (Dorsey et al., 2008). Thus, despite the need for foster parents to develop effective behavior management skills, there is little evidence that the training they receive actually promotes these competencies.

Third, foster children themselves may face a greater number of development insults relative to children receiving child welfare services who remain in their birth family homes. On top of experiencing abuse and/or neglect, foster children endure separation from biological caregivers which places them at risk for compromised attachments and impaired behavioral regulation (Fish & Chapman, 2004). Behavioral dysregulation in the form of externalizing behaviors reflects a common outcome of child maltreatment and parental separation (Saxe, Ellis, & Kaplow, 2006) and affects a substantial percentage of foster children. Studies estimate that anywhere from 20-78% of children in out-of-home care present with externalizing behaviors (Keil & Price, 2006). Moreover, early onset of behavior problems heightens risk for prolonged, unstable placement histories (Leathers, 2002). Fourth, empirically-validated treatments shown to reduce these common symptoms of child maltreatment and out-of-home placement are seldom available to foster children, particularly young foster children (Fisher, Burraston, & Pears, 2005). These children are consequently vulnerable to long-term behavioral and mental health impairments given the known linkages between untreated externalizing problems, unstable foster home placements, and poor health-related functioning later in life (Kiel & Price, 2006).

Parent-Child Interaction Therapy

The authors, therefore, selected Parent Child Interaction Therapy (PCIT) as a model to develop, implement and test within a local foster care service context. PCIT is a well-validated intervention with a number of features that are closely aligned with the treatment needs of foster children and the training needs of foster parents. For instance, PCIT is an experiential parent training program designed to reduce externalizing behaviors in children ages 2 - 7 (McNeil & Hembree-Kigin, 2010). Like similar programs, PCIT targets parents of young children because caregivers appear to have the greatest influence on children’s behavior early in life, caregiver behavior is theoretically alterable (Knafo & Plomin, 2006), and early childhood is sensitive to corrective experience and fundamental to future growth (McNeil & Hembree-Kigin, 2010).

PCIT belongs to a wide class of parenting interventions that produce differential effects. Studies reveal that child-inclusive models incorporating individualized treatment plans, skill-building activities, and live coaching appear to be most effective (Kaminski, Valle, Filene, & Boyle, 2008). Consistent with these themes, PCIT is distinguished by several key features: (a) inclusion of children and caregivers in treatment, (b) use of assessment to guide treatment, (c) tailoring treatment to the level of the child’s development and parent’s skill mastery, and (d) live parent coaching during parent-child interactions (McNeil & Hembree-Kigin, 2010).

As a mastery-based intervention, PCIT is structured according to a two-stage treatment model (Eyberg & Robinson, 1982). In stage 1, Child-Directed Interaction (CDI), a clinician provides coaching instruction to a parent during live parent-child interactions in order to promote authoritative parenting and positive parent-child bonding. Grounded in play therapy techniques and social learning principles, the parent follows the child’s lead in play activities while offering consistent attention, affection, and guidance. The clinician reinforces coaching sessions with modeling, role play, and didactic instruction (McNeil & Hembree-Kigin, 2010).

When parents master CDI skills, they graduate to the second phase of treatment, Parent-Directed Interaction (PDI). During PDI, clinicians help parents develop effective child management and discipline skills while applying the same clinical techniques used in CDI, e.g., coaching. PDI is also informed by social learning principles, particularly Patterson’s coercion theory, which reveals the process by which parents reinforce children’s disruptive behavior via coercion or capitulation. PDI is designed to minimize “aversive exchanges” between parents and children by establishing consistent contingencies for child behavior (Herschell & McNeil, 2005).

Repeated trials have shown that participation in PCIT is associated with significant reductions in child behavior problems in clinical, home, and school settings (Gallagher, 2003). PCIT has also been linked to improved caregiver outcomes (Gallagher, 2003). A meta-analysis by Thomas and Zimmer-Gembeck (2007) reported that standard PCIT, compared to waitlist controls, is associated with medium-to-large effects on child behavior and to sizeable effects on parent behavior. Longitudinal evaluations have also shown that gains associated with PCIT are maintained 1-6 years post-treatment (Eyberg et al., 2001; Hood & Eyberg, 2003). Although most PCIT research has examined Caucasian samples (Eyberg, 2005), PCIT has been shown to yield similar retention rates and effect sizes for families of color (Capage, Bennett, & McNeil, 2001).

Compared to other parent training interventions guided by social learning principles, PCIT represents a discrete intervention usually requiring a relatively small dosage delivered over a brief duration of time. Typically, PCIT therapists conduct 12-14 weekly, hour-long sessions during which they adhere to a standard yet adjustable phase-based format. PCIT is, therefore, portable and has been tested within child welfare service contexts given overlapping goals between PCIT and child welfare. In fact, studies have shown that standard PCIT is associated with significant improvements in caregiver and child outcomes among families referred for child welfare services. For instance, Timmer, Ware, Urquiza, and Zebell (2010) tested the effects of PCIT with maltreated children receiving child welfare services versus non-maltreated children receiving clinic-based services. Both groups were retained in treatment at equal rates and demonstrated equivalent gains across several child outcomes, e.g., anxiety. As a result of studies such as these, PCIT has qualified as a trauma-informed treatment according to the National Child Traumatic Stress Network (Pearl et al., 2012) and a promising model for child welfare services.

However, integrating standard PCIT into child welfare practice may be impractical. While PCIT appears to be a cost-effective treatment model (Goldfine, Wagner, Branstetter, & McNeil, 2008), the average cost per child ($1,000) prohibits most child welfare agencies from offering PCIT to foster children with externalizing problems (Barth, 2009). Therefore, PCIT model adaptations may be required to motivate dissemination among child welfare agencies, and promising yet limited evidence suggests that non-standard PCIT can be integrated successfully into such contexts. For instance, McNeil et al. (2005) evaluated an adaptation of PCIT in which the standard format was replaced by a two-day group workshop delivered to foster parent-child dyads. Caregivers reported significant improvements in child behavior at one month and five months post-treatment, high levels of satisfaction with the workshop, and frequent use of skills learned. No comparison group was established for this single-group designed study.

PCIT Modifications for Project Connect

Extending McNeil et al.’s work (2005) and addressing gaps in foster child services and foster parent training, the authors designed a modified training version of PCIT for foster care. The novel design retained the active ingredients of PCIT, e.g., coaching, while introducing several innovative modalities, i.e., group training and phone consults. To specify, the authors created a 2 to 3-day PCIT group training series for foster families that included PCIT phone consultation. The model was time-limited as opposed to mastery-based, and the authors developed two treatment models differing only by dosage and duration in order to test differential effects of exposure. One model consisted of two PCIT training workshops and six PCIT phone consults over the course of eight weeks, while the second included a booster session at the 8-week mark along with four additional phone consults extending over the course of six weeks. Training workshops accommodated 6-8 families each and ran 7-8 hours in length. Phone consultations lasted approximately 15 to 20 minutes and occurred weekly or biweekly usually in the evening time when parents were available. Trained PCIT providers delivered the face-to-face and phone services.

The training workshop schedule unfolded as follows. Families arrived at the training site around 8:30 a.m. on a given Saturday. Each family included one eligible target child identified for services and potentially other foster or biological children, all of whom were directed to the onsite childcare program. From 9 a.m. to 10 a.m., parents participated in a group learning session during which providers introduced CDI or PDI parent skills via didactic instruction and modeling role-play. From 10:15 to 11 a.m., providers coached parents as the parents practiced CDI or PDI skills with one another. The parent role-plays functioned as preparation for the live parent-child coaching sessions that took place over the course of three 45-minute periods from 11:15 a.m. to 12 noon, 1 to 1:45 p.m., and 2 to 2:45 p.m.

During these live parent-child coaching sessions, three providers coached parents at three separate sites within the training facility. In one of coaching sites, a two-way mirror separated an observation room from a formal playroom. A parent-child dyad interacted in the playroom while the clinician, situated behind the two-mirror, delivered instructions to the parent through a bug-in-the-ear device. The other two coaching sites required that the clinician interact with the parent directly, sitting physically behind the caregiver and whispering instructions in her or his ear. All parents received at least two live coaching sessions per training workshop, and all parents completed at least one of their coaching sessions in the formal playroom. In addition, all parents watched at least one live coaching session from the observation room. From 3:00 to 3:30 p.m., providers and parents reconvened in a conference room to consolidate gains, discuss questions and homework, and schedule phone appointments.

Phone consultation sessions lasted 10-15 minutes, included structured conversation as opposed to coaching services, and fulfilled multiple purposes. They were intended to augment customary PCIT homework activities through the following objectives: 1) increase the probability that parents completed the customary 5 to 10-minute PCIT daily homework with their foster children; 2) ensure that parents were implementing PCIT skills during homework sessions with fidelity to the PCIT model; 3) motivate parents to continue engaging in PCIT homework activities through discussion of child gains, positive reinforcement of parents’ effort, and reiteration of the importance of homework completion; and 4) detail the homework assignment for the following week or two. Together the phone consults and homework practice reinforced the training workshops through two mechanisms: in vivo practice and overlearning. In vivo practice increases the probability that skills learned in training are generalized to the home. Overlearning, the repetitive practice of developed skills, increases the likelihood that such skills will be effectively deployed in stressful situations (Long, 2000).

The overarching purpose of the phone consultation service and the group-based training format was to enhance exposure and promote program effects. Project Connect contained costs by delivering concentrated PCIT services with only a few providers to multiple parent-child dyads within a predetermined period of time. While this approach helped to limit the allocation of resources, it reduced the dosage of PCIT coaching relative to standard PCIT. To elaborate, clients receiving standard PCIT services complete anywhere from 10 to 12 coaching sessions, but Project Connect participants completed on average two coaching sessions per full-day workshop, for a total four or six coaching sessions. Prior research revealed that as few as four PCIT coaching sessions can yield parent behavior change (Berkovits, O’Brien, Carter, & Eyberg, 2010); nonetheless, the authors introduced the group format and phone consultation sessions to increase the chances of parent skill acquisition while capitalizing on their documented benefits of these PCIT modifications. For instance, clinical phone services represent a promising strategy when coupled with homework (Leach & Christensen, 2006).

Like the clinical phone service, the group modality was meant to amplify program effects through specific means. First, the group training model limited participant burden relative to standard PCIT, requiring substantially fewer trips to a clinic setting. The authors expected lower burden to translate into higher PCIT retention rates as standard PCIT suffers from high dropout rates, i.e., 27 to 47% in efficacy trials alone (Lyon & Budd, 2010). In turn, improved retention rates were expected to contribute to greater overall program effects. Second, group formats reduce stigma associated with participation in psychosocial services (Mittal, Sullivan, Chekuri, Allee, & Corrigan, 2012). Project Connect was characterized as a training program that teaches specialized parenting skills to caregivers of foster children with very difficult-to-manage behaviors. By placing emphasis on the unique needs of the foster children and the uncommon nature of the skills required to parent them, the authors hoped to assuage fears that participation signaled deficits in parenting. Still, seeing competent co-participants learn new skills within the context of group training possibly alleviated residual concerns about participation. While the authors did not test the hypothesis, they conjectured that non-stigmatizing services would not only attract more participants but would also motivate successful program completion.

Third, the group format provided myriad social learning opportunities for participants. From the observation room, parents watched one another practice PCIT skills during live coaching sessions. At this time, observing parents were asked to code parent-child interactions in order to become more familiar with the PCIT “do” and “don’t” skills. This exercise optimized the learning potential of peer modeling that is unique to group formats. Moreover, parents interacted frequently during the training, normalizing problems that caregivers commonly face as foster parents while often generating useful solutions to these shared challenges.

Fourth, the group format potentially enhanced the cultural sensitivity of the training workshops. The authors expected the group setting to be well-received among parents of color who comprised approximately half of the program participants. The social group setting is congruent with the collectivist versus individualistic values of non-dominant or minority cultures (Bhawuk & Brislin, 1992). Additionally, delivering services with multiple providers enabled the authors to match the racial composition of the trainer team to the racial make-up of the participant group – half of the primary trainers (2 of 4) were of minority status. Furthermore, four of the six childcare provider heads were of minority status. Notably, racial matching is considered an important feature of culturally sensitive service delivery (Bernal & Sáez-Santiago, 2006). In sum, while the group format limits individualized attention for participants, it also introduces unique benefits which may help explain why parent trainings have been successfully deployed in group settings (Ruma, Burke, & Thompson, 1996).

Funding and Sustainability

Aside from selecting and modifying the intervention model, early project activity also involved the development of funding plans. Capital was required to cover initial start-up and implementation costs as child welfare agencies could not be expected to fund these activities without assurance that investments would yield benefits. Consequently, the authors sought funding not only to support implementation activities but also to rigorously test the performance of the finalized model. Results of planned analyses would inform model uptake at the agency level. The authors targeted the National Institutes of Health (NIH): National Institute of Child Health and Human Development (NICHD) as a viable source of funding for several reasons.

PCIT directly treats externalizing behaviors in children, a significant predictor of health-related problems later in life, and Project Connect’s focus on young children corresponds to the work of the NICHD, which recognizes the critical importance of early childhood development to lifelong health trajectories (Brooks-Gunn, Han, & Waldfogel, 2002). In addition, the vision for Project Connect also aligned with the criteria upon which a proposal for funding would be judged, e.g., significance and innovation. The proposed plan reflected scientific significance insofar as it contributed to implementation science as applied to child welfare services, to PCIT research, and to the treatment of maltreated children. The project also reflected practical significance insofar as it aimed to improve child welfare services as usual. As for the second criterion, Project Connect promised three distinct innovations. First, it proposed to modify the standard PCIT model in novel ways and to test its effects. Second, it aimed to introduce an evidence-based, experiential parent program to foster parent training services that typically are neither evidence-informed nor experiential. Third, the analytic strategies proposed were meant to illuminate key features of the PCIT model that have not been well-tested, e.g., dosage.

In order to submit a competitive proposal, the authors developed a rigorous research design for Project Connect. The sampling design included random assignment to one of three research conditions (waitlist control, treatment group 1 attending two training workshops, and treatment group 2 attending three training workshops) and incorporated balancing procedures to ensure group equivalence. The assessment design included up to five time points and a battery of tests that consisted of validated parent report and observational measures, and the analytic plan entailed multilevel modeling. As a result of design features, the authors could: a) confidently attribute observed differences between conditions to the intervention; b) compare group trajectories over time; and c) assess effects of differential dosage.

After several years of concept development, partnership building, and grant proposal preparation, the authors’ received funding through an R15 NIH grant mechanism. The R15 is dedicated to relatively small projects affiliated with universities that do not receive major NIH funding. While the external funding launched the project, it also reinforced the need to develop sustainability plans. Regardless of program success, funding would be temporary. Therefore, the authors followed several strategies to increase the probability that PCIT modified for child welfare services would be implemented in the local area beyond the life of Project Connect.

First, the authors intended the tests of differential exposure to inform model uptake in the future. These planned analyses would yield insight into the performance of the two and three-day training sequences, respectively. Understanding if one model rendered better outcomes or if both generated comparable effects accounting for attrition would help guide integration into routine services. Second, the authors sought to train both working professionals and graduate students in the PCIT standard and modified models in order to develop a growing workforce that could deliver PCIT services beyond the life of the project. Third, ongoing talks between university and community stakeholders were meant to specify sustainability plans. These conversations explored extant sources of funding such as state dollars for foster parent training programs.

In-depth project-related discussions between university and community stakeholders also addressed critical topics unrelated to expansion and sustainability, e.g., research procedures and staffing decisions. Successful implementation of Project Connect required strong collaboration between the authors and their community partners before and after the funding announcement. The following section outlines the development of this university-community partnership.

University-Community Partnership

A seminal publication from The National Implementation Research Network outlines effective strategies for researchers to undertake when forging university-community partnerships (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). These include the following: articulate ways in which the study contributes to the organization’s mission and strategic plans, co-develop model adaptations, clarify roles and responsibilities while considering feasibility, identify resources to support organizational efforts, and modify procedures as needed. The authors speculate that application of these or similar themes helped promote positive university-community relations. In fact, positive developments emerged during initial conversations.

At the project’s outset, the first two authors who are based at nearby universities met with administrators from a nonprofit agency in the upper Midwest that is contracted by the state government to serve foster parents in the state’s largest city. After several discussions, both parties agreed that the partner agency would supply Project Connect with service providers, physical space, and participant referrals. In exchange, the project would train providers and outfit space in order to enhance agency capacity for PCIT.

However, other agencies in the area serve foster families and limiting the participant pool to the primary partner’s client families would in turn limit sample size and generalizability. Accordingly, the authors recognized the cross-system nature of local foster care services, and following Maher et al.’s (2009) lead, engaged multiple agency stakeholders during initial project discussions. For instance, the authors met with officials from another nonprofit agency that provides a large percentage of area foster families with an array of services. Meetings resulted in the creation of referral protocols. Moreover, it was established that the agency’s graduate social work student interns would receive training in PCIT and the agency’s administrators would engage in ongoing talks about model integration. Although competitors for the same child welfare service dollars, the two primary community partner agencies collaborated effectively.

The authors also met with officials who oversee the local child welfare bureau. These administrators sanctioned the project, circulating endorsement memos to staff and partners. They also supplied the original sampling frame for the study and personally expressed enthusiasm for the project, acknowledging its innovative nature and potential to improve family outcomes.

In these and subsequent meetings with administrators from each partner agency, the authors formulated shared goals, discussed PCIT and PCIT model adaptations, developed feasible pre-service and ongoing project protocols, and agreed upon grant proposal budgets while also discussing key concepts for the grant proposal text. To elaborate, when the authors were selecting and modifying an existing intervention model, agency partners provided information to help identify service needs, highlight emergent agency benchmarks, and shape the model. When developing procedures for recruiting participants, the authors turned to the agency partners for strategies to identify eligible foster families, accept specific referrals, outreach to foster families and their child welfare case managers, screen prospective participants, and obtain foster parent informed consent to participate in the study. Both parties discussed ways in which agency personnel would directly contribute to these processes. When preparing the grant proposal, the authors negotiated budget line items with the primary partner agency and circulated concept papers to the agencies for feedback. In many instances, these documents reflected the results of collaborative planning sessions between the authors and their collaborators.

During service delivery periods, ongoing communications between the authors and agency representatives helped to hone recruitment processes and generate additional referrals. Moreover, interactions between the authors’ research team and the nonprofit agency’s case managers helped to increase participant retention. It also helped to improve family communication with case managers in some instances, according to case management report.

Beyond the themes identified by Fixson et al. (2005), the authors noticed the following dynamics. Offering real benefits to the agencies in exchange for project participation helped to engage administrators in Project Connect. Drawing on the expertise of these administrators to shape the project from its inception helped also to promote active participation of these busy professionals. Introducing well-vetted yet flexible project proposals for administrators’ consideration facilitated involvement while deferring to the administrators’ judgment on matters falling within their scope of work promoted both positive relations and program success. It also appeared that these agency officials devoted time to the project because they shared the authors’ enthusiasm for an evidence-based, trauma-informed model that could enhance services, improve benchmark outcomes, and contain costs. Discussions about model uptake and sustainability generated great interest, so much so that agency officials initiated the topic.

The partner agencies had been integrating promising intervention models into their menu of services in the recent past in an effort to keep pace with cutting edge child welfare practice. Therefore, not only were administrators open to the authors’ proposal to implement a modified version of PCIT into foster parent training, but they also had integrated master’s level licensed clinicians within their case management units, easing the burden of finding appropriate staff to deliver the intervention. Once selected, providers received extensive PCIT training.

Subsequently, the authors and staff discussed and negotiated procedural guidelines in order to strike a balance between the scientific imperatives of a randomized control trial and the clinical imperatives of child welfare services. For instance, the providers were concerned about randomization resulting in exclusion of motivated families from PCIT services. The authors consequently explained in detail the principles and procedures associated with the waitlist control design. Also, providers lobbied to serve more than one child or one parent within any one family that selected into the study. In response, the authors presented the key rationale behind serving just one parent-child dyad per family, i.e., due to limited dosage, the authors set-out to optimize the chances that one parent would master PCIT skills with one child. Had target parents attempted to share the training opportunity with another parent or generalize PCIT skills to another child may, they may have experienced truncated or delayed PCIT skill mastery.

In addition, to ensure that providers conducted all PCIT sessions with fidelity to the model, the authors introduced well-specified integrity checklists adapted from the PCIT manual (Eyberg & UF, 2010). Staff appreciated the guidance provided by the checklists but voiced concerns about overly-prescribed service delivery and unclear checklist directions. Extensive talks during clinical supervision resulted in articulation of an evidence-based approach to following the checklists (Gamrbrill, 2011) along with revisions to the checklist documents. The nature and goals of Project Connect dictated that the authors enter into sincere dialogue with all community partners, including service providers, when planning and implementing the project. While time consuming, doing so increased the chances of project success.

Preliminary Results and Future Research

At the time the authors conducted preliminary analyses, 92 participants had enrolled in the study, and of those, 68 completed a final assessment. The authors plan to conduct follow-up interviews with stakeholders to explore ways in which study attrition can be prevented; nonetheless, Project Connect’s retention rate of nearly 75% compares favorably to previous studies of PCIT in which retention rates ranged from only 53% to 73% (Lyon & Budd, 2010). Waitlist control group members enrolled in Project Connect were retained at 76%, and participants in both the two-day and three-day training were retained at approximately 73%. The latter finding suggests that the third training or booster session did not compound project attrition.

The results of preliminary outcome analyses, which are forthcoming, indicate that both parents and children benefitted from their participation in Project Connect services. To this point, three outcomes were assessed including parent tolerance of child behavior problems, child externalizing behavior problems, and child internalizing behavior problems. Two well-validated instruments were used, i.e., the Eyberg Child Behavior Inventory (ECBI) and the Child Behavior Checklist (CBCL). The ECBI (Robinson, Eyberg, & Ross, 1980) is a widely-administered 36-item parent report instrument that measures children’s problem behaviors and caregiver distress associated with each potential problem. The ECBI yields a Problem Scale indicating parent tolerance and distress associated with the behaviors along with an Intensity Scale, measuring the frequency of a child’s problem behaviors. The CBCL (Achenbach & Rescorla, 2001) is an oft-used, standardized measure of child emotional and behavioral problems. Foster parents with a child less than age 6 at enrollment completed the CBCL for ages 1.5 to 5, and parents with a child age 6 at enrollment completed the CBCL for ages 6 to 18. The CBCL/1.5-5 consists of 99 items, while the CBCL/6-18 consists of 118 items. Both versions of the CBCL produce two broadband scales—Externalizing Problems and Internalizing Problems.

Analysis with multilevel or mixed models indicated the following. First, the intervention was associated with significant increases in parent tolerance of child problems over time, according to the EBCI Problem Scale. The ECBI Intensity Scale along with the Externalizing Problems scale of the CBCL indicated significant reductions in the externalizing behaviors of children in the intervention groups compared to children in the control group. In addition, the Internalizing Problems scale of the ECBI provided some evidence that foster children’s internalizing behaviors improved as a result of the intervention. Detailed results are forthcoming (see also Mersky, Topitzes, Brondino, & Grant-Savela, 2014). When assessing differential exposure, data showed that the three-day training series yielded significant but modest gains on most measures relative to the two-day series. This finding coupled with the attrition results suggested that the three-day training may be superior to the two-day format as higher dosage and longer duration translated into slightly better outcomes and no greater likelihood of dropout. Given our small sample size, these findings warrant further attention.

Based on these results and the strong collaboration between the authors and the community partners, the collaborative team plans to replicate, retest and integrate the current modified model as well as to adapt and test another version of PCIT. First, project partners are developing plans to implement the Project Connect PCIT model into the foster parent training system in the southeastern part of the state. Slight adjustments to the model are required to fit it within the state approved foster parent training structure. If planned evaluations of this full scale implementation project produce promising results, the pre-existing delivery system and stable funding source will likely ensure model sustainability. Second, the authors and their partners have tailored a hybrid PCIT model for biological parents whose children have been removed from the home due to safety concerns. These families are preparing for reunification through supervised visitation and other child welfare services. Project partners propose to deliver to these families a brief course of outpatient PCIT coupled with a longer course of in-home PCIT. Innovative telemental health technology will facilitate the in-home component, and all sessions will take place during supervised visits. Rigorous evaluation of the model will inform future integration. The long-term vision is to coordinate PCIT services for the foster and biological parents of the same child in order to synergize program effects.

Conclusion: Lesson Learned

As the authors near completion of the first phase of their overarching plan to integrate modified versions of PCIT into local child welfare contexts, several lessons emerge that can help inform their future implementation efforts. First, implementation research requires strong collaboration between investigators and their community partners. Dialogue must unfold from the inception of the project, and researchers must be willing to relinquish full project control in the service of external validity and model uptake. Second, conducting such work is not for the faint of heart. Protocols must be created, implemented, assessed, and revised. Myriad problems must be solved and competing interests must be balanced. Not everyone on the original research or clinical team demonstrated the requisite motivation and/or perseverance to meet the demands of this work. A significant number of clinical and research staff either resigned from their post prematurely or were released from the project due to poor performance. Third, funding is scarce, and great attention should be devoted to proposal development, cost containment and funding sustainability. Last, implementing modified versions of evidence-based programs in complex real world settings with fidelity to the key elements of the model requires: a) attention to model detail, b) strong supervision, and c) ongoing dialogue with providers. While challenging, the work promises to improve the lives of vulnerable families and to advance applied science.