From Theory to Practice: One Agency’s Experience with Implementing an Evidence-Based Model

Abstract

As evidence-based practice is becoming integrated into children’s mental health services as a means of improving outcomes for children and youth with severe behavioral and emotional problems, therapeutic foster care (TFC) which is a specialized treatment program for such youth, is one of few community-based programs considered to be evidence-based. “Together Facing the Challenge” (TFTC) which was developed as a component of a randomized trial of TFC has been identified as an evidence-based model. We describe the experiences reported by one of the agencies that participated in our study and how they have incorporated TFTC into their on-going practice. They highlight key implementation strategies, challenges faced, and lessons learned as they moved forward towards full implementation of TFTC throughout their agency.

Introduction

There has been tremendous emphasis on dissemination and implementation of evidence-based treatments in children’s mental health over the last decade (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). Much of this work has focused on the challenges and factors associated with increased likelihood of successful implementation of empirically supported approaches (Green & Aarons, 2011; Hoagwood, Murray, & Jensen, 2006). Such efforts suggest that successful implementation is difficult but possible, and that organizational factors (e.g., readiness, leadership, receptivity to evidence-based practice) play an important part in changing practice and implementing empirically supported treatment (Harwood, et al., 2006; Johnson & Austin, 2005,). However, there is relatively little attention to just how much work is required and what types of issues are encountered from both the treatment developer/disseminator’s perspective and from the perspective of the implementing agency.

This paper provides a case study of such an effort to implement and create the infrastructure to maintain long-term fidelity for implementing Treatment Foster Care (TFC); TFC is foster care offered by providers (typically mental health agencies) in which the foster parents have a high level of training and receive clinical support and supervision. This differentiates it from “standard” foster care (usually provided by departments of social service) which is designed more for dependent children. “Together Facing the Challenge” (TFTC) is a newly-developed empirically-supported approach to TFC (Farmer, Burns, Wagner, Murray & Southerland, 2010; Murray, Dorsey, Farmer, Potter, & Burns, 2007; Murray, Southerland, Farmer, & Ballentine, 2010). In this paper, we provide an overview of the process and highlight lessons learned for other academic-practitioner partnerships and community-based providers who may be considering such initiatives to improve practice.

Background

TFTC is an approach to training TFC supervisors and treatment parents that includes an implementation resource toolkit developed via a set of studies funded by the National Institute of Mental Health (NIMH). An initial study (1998-2002) provided descriptive data from 46 agencies in a statewide sample on practices and outcomes in usual care TFC. These results suggested that usual care TFC varied widely in standard practice settings, showed only moderate conformity to then-current national standards of care (FFTA, 1995), and was a very “watered down” version of the only existing evidence-based version, Multidimensional Treatment Foster Care (MTFC; Chamberlain, 1994, 2002; Farmer, Burns, Dubs, & Thompson, 2002; Farmer, Burns, & Murray, 2009). However, analyses revealed that key elements of both national standards of care and MTFC, when present, were associated with improved outcomes for youth (Farmer, et al., 2009). Hence, TFTC was built on these findings to strengthen what already existed and encourage better implementation of key underutilized elements (e.g., proactive behavior management, improved supervision/support of treatment parents, better adult/child relationships within homes). To do this, TFTC utilizes a training/consultation approach to working with agencies.

The intervention model includes a cognitive-behavioral parent-training curriculum developed to be used with families working within TFC settings (described in Murray, et al., 2010). In the completed randomized trial, university-based trainers conducted a six-week training with agency staff (supervisors and treatment parents) on core components of the intervention through a combination of didactic presentation, video segments, role play, and other hands-on activities to ensure participant understanding of the materials presented and to transfer this knowledge into on-going practice. In addition to traditional behavior management techniques, the curriculum focuses on the importance of positive relationships (both between youth and treatment parents and between treatment parents and agency supervisors), incorporating a future-orientation into the treatment parents’ work with youth, and the importance, for treatment parents, of taking care of themselves (Murray, et al., 2007).

In addition to the parent-training manual, the toolkit also includes a manual for supervisors to use for on-going supervision with staff working directly with the treatment parents. Material from this manual assists the supervisor while working with their staff in their role as teacher and coach while offering their staff the support needed to effectively intervene with the youth and families in TFC. In addition to the training materials and discussion of the supervisor’s role, the toolkit provides helpful take-along “cheat sheets” to assist staff in providing supervision and consultation with their treatment foster families. For example, one of the handouts is a Strategic In-Home Treatment form, which is used to keep both staff and parents focused on the targeted goals of the youth’s treatment plan, provides a place to document specific interventions used to address behavioral problems, and to reinforce pro-social behaviors while tracking progress made along the way.

During the course of this work on practice improvement, the research team has been involved with a variety of agencies to develop the broader agency-level supports necessary to implement and sustain this model. It has become apparent that training and consultation is only one slice of the process that makes it possible to implement a new approach and integrate it into the everyday practice of an agency (e.g., Fixsen, et al., 2005, McHugh & Barlow, 2010). The following provides an overview of one agency’s experience in implementing TFTC and the iterative and ongoing process of integrating it into their improved usual practice.

Recently published work has discussed the development, implementation, and outcomes of TFTC (Farmer, et al., 2009; Farmer, et al., 2010; Murray, et al., 2010). Findings from an NIMH-funded randomized trial of TFTC illustrate both the challenges faced in conducting this type of intervention, as well as the potential to influence and change practice and outcomes. Improving practice in the wide range of existing agencies that provide TFC is a critical part of improving the overall quality of care for children (Farmer, et al., 2010).

Gaps in the Intervention

In the initial randomized trial, TFTC training consisted of training for agency supervisory staff and treatment parents (conducted by university-employed staff). In line with the train-the-trainer philosophy, agency supervisory staff were trained prior to the 6-week training with treatment parents, so that they could serve as co-facilitators during training sessions with treatment parents, help with break-out exercises and groups, and work with their assigned treatment parents as they implemented the intervention. Training also included follow-up consultation and support with the TFC supervisors and other agency staff involved with implementation of the model. This set of activities was based on the available literature on effective training approaches for changing practice among existing staff (see Murray, et al., 2010 for more detail on this approach and logic). A train-the-trainer approach to dissemination potentially has several advantages, including building in a structure for sustainability, adaptability, and relationship building (Hill, Palmer, Hein, Howell, & Pelletier, 2010).

When the research team developed TFTC, the variability in skill level and need among supervisory staff had not been fully anticipated or incorporated. The team began with the assumption that the staff supervising the TFC parents had the skill set that would enable them to incorporate the tools from the training into practice with modest levels of support and consultation. The team learned through this process and from ongoing collaboration that in order to fully implement TFTC into ‘practice as usual’, additional steps were needed.

The current paper illustrates the real world experiences reported by one of the participating agencies from the randomized trial (2003-2008) and how they have incorporated TFTC into usual practice. From initial training and consultation provided by the intervention staff on the research team, the following describes this agency’s process, at all levels of their organizational infrastructure, to facilitate implementing and infusing the model into their everyday practice. The discussion focuses on challenges and lessons learned from this experience. The goal of this detailed case study is to provide guidance to other researchers, program developers, and agencies that are considering (or currently beginning) an effort to improve practice in existing settings. To capture the real world experience of this agency, while providing information that is potentially beneficial to others, this description and discussion is organized around themes that emerged during the training and implementation period. From our perspective and experience with a wide variety of agencies across the country, these factors appear to be relevant for many organizations undertaking a major change initiative.

Brief Description of the Focal Agency

The focal organization, KidsPeace North Carolina, is part of a national not-for-profit agency, KidsPeace National Centers, Inc., providing child and family mental health services in 10 states and the District of Columbia. Much of the administrative support (human resources, contract development, billing, accounting, etc.) for North Carolina’s services comes from the corporate office in Pennsylvania.

KidsPeace North Carolina operates TFC services from four offices in central North Carolina, providing care for approximately 100 youth at any point in time. These services are accredited, licensed, and provided in accordance with state and federal regulations for child mental health Medicaid services in North Carolina.

Even prior to involvement with TFTC, the organization utilized an intervention framework that was based on a model of strengths, empowerment, and resiliency (Fraser, 1997; Luther & Zigler, 1991; Masten & Coatsworth, 1998). When KidsPeace administrators were approached about participating in a randomized trial of TFTC, they felt that the interventions contained in the TFTC curriculum supported the mission of KidsPeace National Centers. Under their organizational structure and policies, the NC branch of the agency had the autonomy to make decisions about treatment models and, therefore, to implement the TFTC curriculum.

In working with agencies across the state and country, it is clear that TFC programs use a variety of terms for key positions, and organizational structures vary widely across agencies. To clarify communication, the following terms are defined as they are used by KidsPeace. Family resource specialists (FRS) are staff members who recruit, train, and license therapeutic foster parents. Family consultants are staff members who provide supervision and support to therapeutic families. Family consultant supervisors are the immediate supervisors, and thus provide supervision and support, to family consultants. Program managers provide oversight to a regional office. The clinical supervisor provides clinical oversight while the state manager oversees administrative operations for all offices in the state and is the manager of the entire agency.

KidsPeace’s adoption and desire to fully integrate TFTC into their usual practice was initiated by the agency and supported via ongoing collaboration with the researchers/developers following the completion of the randomized trial. As this suggests, KidsPeace was very forward-thinking and attuned to the needs to improve practice and incorporate empirically-supported treatments into their way of doing TFC. Baseline data from the randomized trial suggest that the main office of KidsPeace NC was slightly higher on receptivity to evidence-based treatments than other participating study sites. Based on staff responses to the Evidence-based Practice Attitude Scale (EBPAS; Aarons, 2004), supervisory and administrative staff in KidsPeace’s primary office had a mean of 3.5 (compared to 3.2 in the other KidsPeace offices and in the overall sample of sites that participated in the randomized trial). Overall, these means suggest quite high receptivity at all offices, as means are approximately a standard deviation higher than national norms reported for the EBPAS (Aarons, Glisson, Hoagwood, Landsverk, Cafri, 2010).

KidsPeace participated for approximately two years in a randomized trial of TFTC as a part of a research study, before implementing this agency-wide change on their own. During that time, their staff received training on TFTC and they began to utilize the intervention. Later, they began the process of more fully integrating the model into usual care, creating documentation and forms that would assist in this process, etc. However, during the initial research study (since it was a multi-site randomized trial), they were working essentially on their own on these activities (because research staff were interacting with all participating intervention sites using an established protocol). Hence, the iterative and interactive work between the agency and the researchers/developers did not begin formally until the randomized trial concluded. Figure 1 provides a graphic representation of the sequence of events for the set of practice-changing activities described here.

Figure 1.

Together Facing the Challenge (TFTC) Sequence of Implementation

Key Elements for Implementing New Approaches: The Agency’s Perspective

Infrastructure and Orientation

Before embarking on a major change in training and practice, KidsPeace created infrastructure to guide the process. Implementation of TFTC in KidsPeace NC was facilitated through the development of a program improvement committee. This committee included the state manager, program managers, family consultant supervisors, and clinical supervisor. The overall goal of the committee was to improve clarity and consistency of roles and efforts.

For several years prior to KidsPeace’s involvement with the randomized trial of the TFTC program, the leadership staff had been involved in a self-initiated set of activities to improve practice. Their concerns were that their model of care, training, and supervision, did not provide specific direction to staff and families to realize the agency’s goals. They recognized the limitations of pre-service training to prepare treatment parents to effectively work with youth who have complex needs, and the limitations of family consultants (often less experienced and without much residential experience) to provide the necessary support to assist treatment parents with the real challenges they face. They understood the obstacles confronted in implementing programmatic consistency with clients in care due to the limited monitoring and feedback that could be provided to treatment parents residing in homes spread across several counties.

Previous efforts to address these issues focused on increasing family consultants’ and treatment parents’ understanding of resiliency theory, i.e., the need for their program to increase protective factors in the lives of the children served. They determined those factors over which they had control (e.g., caring relationships, opportunities for spiritual development, extracurricular activities, improving reading deficits, increasing structure) and those with which they had no control (e.g., temperament, cognitive level, early childhood experiences). Work in this area focused on assessing risk and protective factors in the lives of their clients, determining specific protective factors on which to focus, and providing staff and treatment parents with the direction and resources to assist the youth in acquiring protective factors.

In spite of these efforts it became clear that treatment foster families needed increased skill in managing and responding to client behavior and that the staff charged to support the families needed additional tools and clarity in providing that support. Alone, increased clarity around enhancing protective factors did not increase the skill level of staff and families in intervening in problem behavior. KidsPeace maintained that problem behavior would decrease with the acquisition of increased protective factors, but struggled to effectively intervene and manage problem behaviors in the interim. Thus, when approached by the study team about being involved in research on improving practice and outcomes in usual care TFC, they were very enthusiastic. It fit with their desire for a more specific and coherent approach to practice improvement.

Fit of Model with Agency Philosophy

In the TFTC team’s work with various agencies, this recognition of “fit” appears to be an important initial step in the implementation process. The commitment by KidsPeace to implement the TFTC curriculum came through a recognition of the need for a better behavior management program, with a realization that TFTC supported increasing resiliency. The curriculum’s focus on relationships, structure and predictability, problem solving skills, and social competence, was consistent with and supported their resiliency-theory-based approach.

KidsPeace understood that effective implementation of TFTC would require staff and treatment parents to have a clear understanding of the why behind the TFTC interventions, and that it is important that the intervention approaches be used in the manner in which they were designed. This increased understanding occurred as the agency leadership conveyed a coherent and consistent rationale for implementing this change. Key concepts conveyed to staff and foster families from the beginning of TFTC implementation included an understanding of (a) the effects of trauma, (b) the role of protective factors in enhancing resiliency, (c) how TFTC interventions increase protective factors, and (d) that the key components, when used with fidelity, have been shown through research to be effective.

Implementing the Model

It is important to recognize that the initial training in TFTC occurred as part of a randomized trial. The study provided training to the supervisory/administrative staff and treatment parents employed by the agency when the initial training occurred. Initial training presents its own set of challenges (buy-in, logistics, motivation to change, etc.), which have been described previously (Murray, et al., 2010). Moving beyond initial training to integrate implementation of the new practices into ongoing work requires a higher level of commitment. This process is described from the agency’s perspective in the following section.

Needs Assessment for Sustaining TFTC

Planning for sustainability of TFTC within KidsPeace began with an assessment of foster parents and staff; specifically, which foster parents and staff members were currently trained and how training could be provided to those remaining staff and treatment parents. KidsPeace was fortunate to have a number of treatment families and supervisory staff which had completed TFTC training during the initial study, but turnover in both staff members and treatment parents, and agency growth, left the agency with a number of staff untrained in the model. The experience of KidsPeace in the initial study was that there were two important skill sets - clinical skill and training skill. Agencies can count on (and hire for) clinical expertise. However, not all clinicians are comfortable in the role of trainer (a role more similar to teacher or adult educator). In the opinion of the agency’s state manager, the development of training skill, as well as the time and effort to prepare supervisors to champion the use of TFTC interventions in their program were both underestimated. In order to increase the skill of trainers, these staff members had opportunities to practice inservice delivery, and were given instruction in principles of adult education.

Training – Existing Staff

The approach to training existing employees in the use of TFTC was to show the strong commitment the agency had to implementation. The plan included a TFTC kick-off training event, where, over the course of two consecutive days, all existing supervisory foster care staff were trained by a family consultant supervisor who had already been trained. It is important to note that the formal position of the trainer is less important than the skills and motivation that the trainer possesses. KidsPeace chose an individual who was enthusiastic about the curriculum and could therefore present the information in a way that spread that enthusiasm to others. Given the agency’s time constraints, devoting two full days to supervisor training showed the importance that the leadership staff was placing on this training. For those employees who had been involved in the initial training on the curriculum, the information was familiar. However, this format proved a valuable refresher, and more importantly served as a team-building experience that signified that the group was moving forward with implementation together. Following this training event, ongoing refresher trainings were conducted at least once per quarter for all family consultants, led by the family consultant supervisor.

Training – New Staff

As new staff were hired, they were informed early in the interview process that the TFTC curriculum was a key element of the KidsPeace program. On each new hire’s first day of employment, he/she is now presented with the TFTC video as an introduction to the program. In addition, he/she receives the power point training slides, handouts utilized in the TFTC training, and the “Info-to-Go” manual. Because the TFTC progress note that is used as a framework for visits/consultation with treatment families requires knowledge of TFTC interventions, new hires receive training on this as quickly as possible. Family consultant supervisors ensure that newly hired family consultants understand the material, can select appropriate interventions specific to the needs of children they serve, and have the ability to teach and review the material with treatment parents. In addition to didactic instruction regarding these interventions, new hires accompany experienced family consultants on home visits so that they can observe the practical application of these strategies. Supervisors also observe new hires as they conduct home visits or trainings to ensure proper implementation. Furthermore, the TFTC toolkit includes four sessions of training for new family consultants. These sessions are conducted one-on-one between the family consultant supervisor and the family consultant. Each session includes an evaluation form to be completed by the supervisor that ensures that relevant skills and abilities are present.

The development of a new KidsPeace office provided an opportunity to implement TFTC with staff for whom the curriculum did not represent a change in existing practice, but instead was the only curriculum they knew - thus eliminating the factor of change. Post-training evaluations showed that the new hires in this setting indicated that TFTC was easy to learn and to utilize effectively. Experience thus suggests that implementing an intervention in a new setting is much easier than changing practice in an existing setting.

Curriculum Implementation and the Role of Supervision

The role of the family consultant and the family consultant supervisor are key factors in assuring implementation. These roles provide supervision and coaching to the treatment families in their effective use of the intervention. Family consultants must have an adequate understanding of the curriculum, and must be capable of coaching and teaching families in the use of the intervention during their consultation sessions. Family consultant supervisors were charged with providing training to staff, providing an overview of the TFTC tool kit, and providing regular refreshers. A strong working relationship between these professionals is crucial in the implementation of this program. KidsPeace had a well-developed structure for this set of relationships.

Consistent implementation of this curriculum was challenging due to geographically dispersed offices and the various skill levels of the family consultant supervisors. KidsPeace learned that although TFTC is easily understandable, there is much more work and skill needed from the supervisory role to properly implement the curriculum throughout the TFC homes. This again speaks to the difference in clinical skill and skill as an educator or coach.

Attempts to insure supervisor readiness to support TFTC included providing an overview of the curriculum by a skilled trainer, and requiring the family consultant supervisor to be familiar enough with the curriculum to train their staff in its use. While thorough knowledge of the curriculum improves implementation, implementing a new curriculum requires much more than a thorough understanding of the material. Effective implementation also requires agency leaders to “sell” the curriculum to their staff and motivate their employees to use the model. In order to do this and overcome resistance to change they must share the “big picture” view of the agency leadership.

Infusing Training into On-going Practice: Training Treatment Parents

Once the majority of staff were trained, the program improvement committee focused on the need for the curriculum to be presented to all newly licensed treatment foster families going forward as part of the agency’s pre-service training.

Family resource specialists (the persons responsible for recruiting, pre and in service training, and licensing foster families) must be comfortable and effective in presenting the TFTC material. KidsPeace leadership stressed to these staff the need to present this training to treatment parents clearly and to stress to families that the use of the intervention will be required of them in their role as treatment parents for KidsPeace.

TFTC then became an integral component of the KidsPeace pre-service training and was added to the required trainings already offered by the agency and already approved by state licensing agencies. While the conceptual framework was consistent, it took time to combine the two sets of training curricula. Revising the pre-service training took about 6 months to complete. Once revised, it was presented to the state licensing agency for its review, and after some minor adjustments to meet the 40 hours of required training, the curriculum was returned and approved for use.

Gaining Expertise as a Trainer

Upon completion of the new pre-service curriculum, family resource specialists learned how to train other staff on the materials. All staff came together and were asked to evaluate the family resource specialists on their presentation of the content. While presenting the material to their peers was uncomfortable for some staff, it was important to conduct this activity for several reasons. It allowed agency leadership to convey a renewed and strong commitment to full implementation of TFTC. It conveyed a message to staff concerning the importance of being well-versed on the content of the curriculum. And it conveyed that not only was clinical skill important, but so was skill in training and an understanding of principles of adult education. These three things combined provided an opportunity to discuss with family consultants the importance of their role in “keeping the learning going” through their role as coaches with treatment parents.

Response from Treatment Parents

Many people who come to the work of being treatment foster parents have successfully raised their own children and truly enjoy the parenting role. It is quite tempting for them to believe that the same techniques used with their own biological children will be effective with treatment foster children. That can often lead to comments and beliefs like: “clients just need to be loved” or to a stern, no nonsense, “old school” style of parenting. Helping these foster parents understand that the techniques used with their own children may not be effective with children who come into care with backgrounds of trauma, mental illness, or multiple placement disruptions is an important part of the work of TFTC.

Overall, treatment parents were receptive to TFTC. Most treatment parents participated in the TFTC training without complaint or opposition. The TFTC interventions were easy to understand and treatment parents verbally committed to using the TFTC approach. However, TFTC is deceptively simple. It is one thing to understand an intervention; it is another to possess the skill to implement the intervention in vivo. So, while treatment parents sometimes thought they were doing TFTC, this was not always the case (based on observation and assessment of their implementation of the intervention).

It also became apparent that it was actually quite difficult to quantify when supervisors and treatment parents were implementing an intervention correctly. As a response to this, KidsPeace (in conjunction with TFTC researchers) developed and implemented a progress note that provided a consistent format and structure for documenting development and implementation of behavioral intervention. The treatment parent response to this note was mixed. Some liked the note because it kept them focused on TFTC intervention. Many were neutral about the note. Some protested that it took too much time to complete. One of the agency’s best treatment parents complained that the note “forced me to think every day about what I am doing with the kid” (which is exactly what the note was intended to do!). Some anecdotal comments from foster parents led us to believe that they found the form to be unwieldy. It was an original part of the randomized trial, but later changes and adjustments made it more user friendly during the sustainability phase. The note remains in use, and it is hoped that it will motivate the treatment parents and family consultants to incorporate TFTC intervention into their on-going practice.

An unintended benefit of TFTC is that it gave everyone a common language. For instance, there are multiple ways to talk about power struggles or family meetings. When everyone had been trained in TFTC, everyone knew exactly what was meant when references were made to power struggles or family meetings. This enabled all agents of KidsPeace to communicate about components of the intervention with more accuracy.

Other Tools Developed to Highlight Components of the Intervention

Agencies incorporating the Together Facing the Challenge curriculum will require the use of tools to assist in assuring proper implementation of the intervention components. These will change from one agency to the next based on a variety of factors (state requirements, breadth of agency services, etc.). KidsPeace utilized some of the tools available with the TFTC curriculum as written, and revised others in order for them to fit their model of care. By using agency-created tools, rather than outside or commercially available tools, a greater sense of ownership was developed on the part of KidsPeace staff and foster parents. Some of the tools KidsPeace incorporated to assist in implementation included a tool used during in-home support visits to assist in focusing on behavioral goals; a tool to monitor client response; and a manual that provided staff with easy access to materials.

The implementation of these tools to assist in incorporating TFTC into usual care practice was a longer process than was anticipated as the use of a new tool also meant additional work for family consultants, treatment parents, and others. Therefore when developing new tools designed to increase effective implementation of the TFTC curriculum, KidsPeace aimed to replace a document or process that was currently in place, rather than adding a new one. Rolling out a new requirement was met with much less resistance when prefaced with “we will discontinue the use of this old form/process.”

Summary

Lessons Learned: What Other Agencies Should Know as They Embark on Change

The process of implementing a new intervention may take much longer than anticipated. To fully implement a new practice requires a change in the behavior of staff and treatment foster families. The parts that build on what an agency already does are much easier to implement than those that require a new approach to practice. For example, KidsPeace had a long history of providing relevant in-service trainings to their TFC families and staff, but it did not have a history or structure to provide the follow-up coaching and consultation to assist staff and treatment foster families in using the material presented. These changes from the “old normal” seem to be particularly difficult for long-established staff and families. New staff and treatment families who entered the agency during this transition period did not have a history of “how it’s always been done” and, hence, tended to be more receptive to the new practice.

Change requires commitment across time and across levels. Discussions and training with staff and foster parents need to be presented differently from the beginning and at each interaction along the way. This starts with leadership having a strong commitment to the change that is being implemented, a solid understanding of the material and the goal of full implementation, as well as strong leadership skills necessary to fully implement the changes over time. Buy-in from staff and treatment families is essential to the full-scale implementation of a new approach. While it requires patience to go through the process of getting input from all relevant stakeholders in the organization as changes are being developed, it appears to create a sense of ownership and understanding throughout the organization that facilitates implementation.

Organizations need to be continually self-reflective about the change process. Having a committee that met monthly to continuously ask “is the TFTC intervention being used effectively in all foster homes in all offices?” was critical. This committee needs to be composed of representatives from all relevant levels of the organization and must have authority to provide direction and instigate modifications as deemed necessary. Such meetings also serve as a team building experience that contributes to more consistent understanding of the goals for implementation. They also provide a forum for very practical modifications and strategies, such as the development of specific protocols and tools to assist with implementation.

Lessons Learned: What Researchers Need to Consider as They Embark on Change

TFTC was developed and tested through a series of research studies in an attempt to improve practice in existing TFC agencies. The implementation work reported here occurred because an agency that participated in the initial randomized trial wanted to incorporate the newly developed approach into usual practice. Hence, the above description is an account of an agency-initiated implementation of a new approach. It is not a description of a researcher-designed dissemination strategy or study. This, in itself, is a good lesson for others – once an organization has been exposed to an intervention, the organization’s adoption, adaption, and incorporation of the approach becomes their own. Researchers can be partners in this process, but if it is going to be integrated into daily practice, the shift of momentum and ownership moves from the researchers to the practitioners. Dissemination studies can manipulate and test specific elements or strategies, but it is critical to remember how much effort is required of the agency to make the changes, and how much buy-in, leadership, and commitment are critical to sustaining the long-term commitment to successfully utilizing the new approach.

It is critical for researchers/developers to have a good understanding of the agencies’ starting point, current practices, management structure/approach, and philosophy. The focal organization for this case study, KidsPeace, took on the challenge and made a conscious decision to move away from practice as usual by incorporating an enhanced model of care into their service delivery. Their high level of motivation to change existing practice was observed in their willingness to push the boundaries forward and persevere. As noted above, this focal organization was already actively engaged in a self-initiated attempt to improve practice before the researchers entered the picture.

From field-based experience and researcher documentation involved in dissemination and implementation processes, KidsPeace’s experiences are both illustrative and instructive. It is important to note that they occurred within the context of an agency that was already very quality focused and self-reflective. The organization also had strong local and national administrators who wanted to improve their model of care. Such agencies are often viewed as places where change is most easily implemented, i.e., the “early adopters” (Rogers, 1983). The current work suggests that even in agencies that are eager for change, the process is challenging.

Some of the challenges faced by KidsPeace included customizing tools and other internal resources to make the changes fit within existing operating procedures, developing structures that ensured fidelity, and getting everyone to buy into the changes that were made in direct practice. This experience emphasized the importance of incorporating the model at all levels of the organizational infrastructure to facilitate implementation and infusion of the model into everyday practice. From a research perspective, each of these issues was a reality check. The need for buy-in is well known among researchers, and up-front activities to assure buy-in are common in field-based research. However, the extent to which buy-in is needed to be recreated and reinforced on an ongoing basis was clearly more labor intensive and more collaborative than initially expected.

Although no specific cost data were collected during the randomized trial and the expenses for the initial training were covered by the grant, some costs after the study ended (additional staff time for ongoing training, coaching, new tool development, providing meals, snacks, etc.) were covered by the agency. According to the agency director, these additional costs were balanced by the benefits received. Some of the costs involved staff time and new learning by foster parents, but this was a “one time” cost, as once these processes were implemented they then became standard practice.

Creation of tools and instruments to guide practice was an ongoing learning process. It seems incredibly obvious in retrospect, but the various cheat sheets that the research team created in conjunction with the agency to guide daily practice appear to be essential elements for guiding interactions and changing long-established practice patterns.

Overall, researchers need to be humble and well-aware of their expertise and limitations. The process and results of this effort suggest substantial potential to improve practice in sites that had not previously been implementing evidence-based interventions. It also shows the effort, resources, and commitment required by an agency’s staff to do so. It should be noted that this was all done within the context of ongoing regular contact between the treatment developers/researchers and the agency personnel. The field of dissemination/implementation research is expanding rapidly at the moment. This case study highlights challenges and strategies that require additional systematic study to assess common elements, potentially beneficial strategies, and factors that influence both the process and outcomes of attempts to change practice.