Partnering to Translate Evidence-Based Programs to Community Settings: Bridging the Gap Between Research and Practice

Abstract

Implementing evidence-based programming in diverse community settings is an essential translational research step to make effective programs widely accepted and accessible, and thereby improve public health (National Institutes of Health, 2010). This process is challenging and complex, yet we have few examples to guide our efforts. We present our experience as an example of using a university-community partnership approach to aid in translating an evidence-based program (EBP) into a small community setting as a resource for researchers and community partners wishing to implement evidence-based programming in community settings. We review the steps of systematic planning and client needs assessment to decide on an EBP; adapting the EBP to appeal to the community while maintaining program fidelity; building staff and organizational capacity; implementation and family engagement; and program evaluation. We focus on research-to-practice links, and highlight each partner's role and activities in facilitating successful translation of an EBP to this community setting. We also present lessons learned and recommendations. Using partnerships to prepare community-based organizations to implement EBPs is a vital mechanism for bridging the discovery-delivery gap and moving toward real-world applications of research discoveries.

Implementing evidence-based programs (EBPs) in community settings is challenging, yet vital for improving public health (Burgio 2010, Kerner, Rimer & Emmons, 2005). The NIH Roadmap urges evaluating basic research applications in real world settings as the goal of Phase 4 translational research (National Institutes of Health, 2010). Addressing this challenge requires a complex balancing act of matching EBPs to communities (Brownson, Gurney, & Land, 1999), and readying communities to implement EBPs across settings, including community-based organizations (CBOs) (Simpson & Flynn 2007). We used a community-university partnership to facilitate this translational process. We implemented an evidence-based behavioral intervention, the Strengthening Families Program (Kumpfer, DeMarsh, & Child, 1989), in a grassroots CBO serving children of incarcerated parent(s) and their families in Flint, Michigan. Drawing on organizational readiness to change, interactive systems, and university-community partnerships frameworks (see Simpson & Flynn, 2007; Spoth, Greenberg, Bierman & Redmond 2004; Wandersman et al., 2008 for theoretical discussions of these), we present steps taken by university and CBO partners to prepare this small organization to implement an EBP: client needs assessment, program selection/adaptation, capacity building/sustainability, delivery, and evaluation. We describe experiences and lessons learned to illustrate how our university-community partnership helped bridge the discovery-delivery gap.

This paper is a joint product of a university partner and Motherly Intercession (MIC), a small CBO (~6 staff, ~20 volunteers). MIC's mission is to create a community support system for children with incarcerated parents. It historically focused on addressing children's academic needs and jail visitation, but sought additional programming for families. The Director contacted university partners at a community forum on children's health. We jointly obtained NIH funding to implement an EBP to address family needs. Goals were to draw on local expertise to tailor the program and evaluate our efforts. We seek to inform research-to-practice links for researchers looking to translate EBPs to community settings, and CBOs considering implementing an EBP. We use MIC as a case example to illustrate our steps, focusing on each partner's role and joint activities (see Table 1).

Table 1.

Partnership and Translation Activities in Preparing a Community-Based Organization (CBO) to Implement an Evidence-Based Program (EBP)

Phase	Approximate Timeline	CBO Partner Activities	Joint Partnership Activities	University Partner Activities
Client Needs Assessment	~ 5 months to gather and synthesize data	○ Gather information about client needs one-on-one, in context of existing relationships ○ Share data on services provided	○ Interpret and synthesize results of client needs assessment data-gathering efforts ○ Prioritize client needs in context of CBO mission	○ Conduct focus groups with clients, as an independent observer ○Analyze focus group, service use data to identify needs
Program Adoption and Adaptation	~ 8 months from initial EBP review to training; EBP adaptations are ongoing	○ Judge whether EBPs are consistent with CBO mission ○ Align existing staff skills with EBP requirements ○ Draw on local expertise for EBP adaptations ○ Document adaptations	○ Identify criteria for EBP selection ○ Create system for documenting EBP adaptations (content, logistics)	○ Review existing programs that address identified client needs ○ Summarize pros/cons ○ Manualize EBP adaptations, verify in keeping with original structure and principles
Capacity Building and Sustainability	~ 6 months to create job descriptions, hire and train interns; ~ 2 months to hold specialized trainings; ~ 8 months to develop database; Grant seeking is ongoing	○ Outline current staff roles and duties ○ Evaluate how EBP implementation will affect current/future resources (e.g., budget, infrastructure, staff) ○ Seek funding to continue program	○ Use flow charts, logic models to detail staff activities, CBO needs ○ Recruit interns from local institutions ○ Complete specialized trainings (e.g., with EBP developers; research process; working with jailed clients)	○ Develop database infrastructure to serve multiple organizational needs ○ Create system to collect and manage process and outcome evaluation data ○ Provide training in research (e.g., informed consent, research ethics, program evaluation goal)
EBP Implementation and Family Engagement	~ 2 months to recruit families; ~ 4 months to implement program (per cohort)	○ Provide ongoing staff support ○ Recruit families through local connections and events	○ Plan and hold celebrations to acknowledge and honor staff and family achievements	○ Suggest strategies for retention ○ Develop systems to track family contact information, attendance
Program Evaluation	~ 6 months to plan outcomes, measures ~ 4 months to develop protocols Data collection and analyses ongoing	○ Define outcomes of interest ○ Facilitate data collection on-site	○ Develop data collection protocols ○ Review possible measures, assessments ○ Adapt current intake forms ○ Plan how to use data to inform practice, find additional funding	○ Plan evaluation design ○ Develop/adapt measures ○ Provide analytic support ○ Generate program evaluation outcome reports
Partnership Development	~ 2 years (unfunded) grantwriting ~ 2 years project activities	Ongoing communication with partner regarding goals, priorities, challenges

Background: Partnering to Bring EBPs to Communities

EBPs are theory-driven, empirically-based programs with demonstrated intervention trial efficacy. EBPs are often manualized, aiding fidelity of replication. It remains challenging for some children and families to overcome participation barriers and benefit from EBPs (Spoth, Redmond, Hockaday, & Shin, 1996). Thus, implementing EBPs in small CBOs that serve families, like MIC, is an essential translational research step to make EPB's more widely accepted and accessible (Kerner et al., 2005; NIH, 2010). A CBO can become the central hub of a community in need, but often does not have the expertise or resources to implement an EBP (Saul et al., 2008; Spoth et al., 2004). CBOs may be interested in EBPs for many reasons, including a desire for programs with proven track records; securing future funding by showing program evaluation results; and augmenting staff skills through training. Implementation is a significant undertaking for a small CBO, however, and may require organizational change and support to succeed (Simpson & Flynn 2007; Sobeck & Agius 2007; Wandersman et al., 2008). Partnering with universities to provide resources can be useful (Spoth, Clair, Greenberg, Redmond & Shin 2007). We focus on the role of partnership when implementing an EBP in a small CBO setting.

Successful partnerships require deep understanding and respect of each partner's goals and skills, resource-sharing, and commitment to the project on behalf of each partner (Green, Daniel & Novick 2001; Seifer & Vaughn 2004; Spoth et al. 2004). Working together on a focused goal, while each contributing unique expertise, community and university partners can develop efficient ways to deliver services and evaluate their effects (Green et al. 2001). A goal of our partnership was to capitalize on each partner's strengths to prepare MIC for implementing this program. To facilitate this, we communicated explicitly about expectations for each partner.

Table 1 presents examples of CBO-led, university-led, and joint partner activities, and time involved at each phase. Our project budget was approximately equally divided between university (57%) and community partners (43%), an important aspect of resource-sharing that enabled equal partner participation. Partner roles were determined by expertise. MIC's extensive experience working with children of incarcerated parents, for example, was essential in engaging families and making program adaptations. University partners developed systems to track research data and address other CBO needs (e.g., reporting number of clients served). Thus, community and university partners each brought skills to the partnership that enabled effective translation of the EBP to this setting. Developing respect for both partners’ priorities and skill sets was important; CBO partners learned to appreciate research issues (e.g., control groups; IRB requirements), and university partners learned to value CBO concerns (e.g., limited staffing, space, funding). Understanding each others’ priorities and challenges was helpful when planning implementation and evaluation activities, because each partner could grasp why issues that may not affect their primary agenda were significant for the partner (e.g., needing flexible timelines due to IRB constraints; limited staff). We describe each partner's role at each phase of readying MIC to implement an EBP, to inform future partnership for translation efforts.

Methods

Phase 1: Client Needs Assessment

CBOs have a general sense of the needs of the communities and clients that they serve. MIC defined their community of clients as children of incarcerated parents, and their caregivers, in the county. MIC sought an EBP that would address client needs, be a good use of resources, and be consistent with their mission. Thus, our first step was to assess client needs. University and community partners worked together to assess needs using complementary approaches. MIC staff built on ongoing relationships with families and one-on-one conversations to assess caregiver and child needs. Importantly, prior to this work, MIC had not provided programming specifically for caregivers. However, caregivers informally expressed needs for services To gather more explicit information, the university partner facilitated focus groups with caregivers to discuss concerns and desired services. Participants spoke favorably of existing MIC programs for children (e.g., tutoring), and noted that they trusted MIC as a safe place where they were not judged based on their circumstance. They also mentioned unmet needs, including stress management, little peer support, strained family relationships (e.g., with incarcerated parents), and worry about child behaviors (Brown, Ramsay, Cochran & Miller, 2010). MIC staff observations and focus group responses together revealed needs for family-level programming to provide support for parenting (caregiving) and child social-emotional functioning. Providing caregiver-focused programming, which had not occurred before at MIC, would enable MIC to meet these needs and reach more people.

Phase 2: EBP Adoption and Adaptation

CBOs must be able to evaluate EBPs that address identified client needs. Selecting an EBP to adopt is a complex task, requiring review of relevant programs (Brownson et al., 1999; Wandersman et al., 2008) and appraisal of CBO capacity to implement programming. Although data are available online, it takes time and expertise to synthesize information and match programs to client and agency needs (Brownson et al., 1999; Saul et al., 2008). University partners can be helpful in evaluating options and implementation strategies (Campbell & Zimmerman, 2009; PRC of Michigan 2000-2009). MIC's Director worked closely with the university PI to define criteria for an EBP to meet CBO and client needs. Elements considered in our selection process included: family-focused; developmentally-appropriate; evidence-based; addressed alcohol/drug use; provided support for training, implementation, and evaluation; reasonable cost for training, materials, and implementation.

We also sought a program that we could adapt to local needs. Many programs addressed parenting, but MIC's mission required adapting materials for children's caregivers while the parent was incarcerated, and addressing transitions if the parent was released. Balancing adaptation of a program with fidelity can stretch limited resources, but if original materials do not appeal to the focal population, an EBP may not be effective in promoting behavior change in the new context (Bernal, Jimenez-Chafey & Domenech Rodriguez, 2009). In contrast, tailoring materials may promote family and CBO engagement (Aktan 1999). When making adaptations, it is important that the original program structure, components and timings remain intact (August, Gewirtz, & Realmuto, 2009; Kumpfer, Alvarado, Smith & Bellamy, 2002). Researchers can work with CBOs on modifying EBPs to maintain integrity, while adding local relevance.

We selected the Strengthening Families Program (SFP; Kumpfer et al. 1989) because it met all of the above criteria. SFP is group-based EBP shown to enhance family-level communication, parenting, and child problem solving skills among high-risk families (e.g., Aktan, Kumpfer, & Turner, 1996; Kumpfer & Alvarado, 2002). SFP has been adapted to suit local needs (e.g., Kumpfer et al. 2002), and program developers encourage this. SFP also includes training resources and comprehensive manuals supporting implementation, fidelity assessment and evaluation. We thus had models and materials (e.g., images of different ethnicities) for adaptation while maintaining basic principles of the intervention in order to deliver our adapted version with fidelity. During the grantwriting and planning phases, the university PI and MIC Director reviewed adaptation ideas with SFP developers (e.g., involving incarcerated parents), to ensure that our strategies would not violate the program structure. In our adaptation process, which extended through our first implementation round, MIC staff kept notes on topics, activities, and materials that worked (or not) during each session. University partners created color-coded manuals summarizing the changes and containing materials. These manuals have become part of MIC's library and “required reading” for all SFP staff.

Phase 3: Capacity Building and Sustainability

Agency capacity and readiness for change can drive EBP implementation success and sustainability (Simpson & Flynn 2007; Spoth et al., 2004). Devoting resources to supporting staff during implementation and long-term commitment to a program can increase the likelihood of successful implementation and sustainability (Klein & Knight 2005). Although we did not explicitly assess organizational readiness, MIC's Director identified several factors that could affect current and future implementation of the new EBP (e.g., cost of hiring/training staff). We thus directed resources toward building capacity in multiple ways, including finding additional staff; developing a database to prepare reports and provide data for future funding proposals; articulating the functions and roles of existing staff members; and developing EBP training and implementation protocols.

We sought to involve existing staff without additional burden. Challenges included staff anxiety about increased workload, and delineating new roles and responsibilities. We developed flow charts and logic models diagramming program needs and activities to streamline existing workloads (e.g., using database to generate reports automatically). We also created procedure manuals and cross-trained staff members to perform critical functions. Partners agreed that additional staffing was necessary for leading SFP sessions. SFP developers advised hiring interns as group leaders. University partners helped create internships for health education and social work students. We developed job descriptions ensuring opportunities met requirements for different training experiences, and recruited interns from local universities and community colleges. Interns implemented child or caregiver SFP groups and co-facilitated family sessions. Hiring interns increased MIC's capacity, and likelihood of SFP sustainability, by strengthening connections with local university partners. Interns have also spread the word about their positive training experiences with MIC.

We also sought to enhance both intervention-specific and general organizational capacity by developing personnel skills through training. Initial intervention training was a two-day workshop for all MIC and university partners, facilitated by SFP program trainers. During this time group leaders worked with SFP professionals to learn methods and discuss adaptations for MIC's service population. All staff also completed human subjects training, which provided opportunities to educate about research ethics and program evaluation. Discussions covered maintaining confidentiality in a small CBO setting, and adapting recruitment and consent processes (e.g., oral vs. written informed consent) (May, Craig & Spellecy 2007). By involving everyone from the project in these trainings, we sought to foster connections among MIC staff, interns, and university partners; facilitate program buy-in; and promote sustainability. Our SFP resource manuals address training and implementing SFP at MIC, and provide the groundwork to sustain the program through a training framework for new staff.

Phase 4: Implementation and Family Engagement

During implementation, we provided ongoing supervision for staff/interns working with families to maintain enthusiasm and avoid burnout (Aarons, Fettes, Flores & Sommerfeld 2009). We partnered with a social worker who met weekly with group leaders to consult and debrief about the process of each session, including crisis management. All staff members were trained in working with high-risk children and families, including when and how to report clinical concerns (e.g., child abuse). To sustain enthusiasm, we held staff retreats and appreciation days to honor their hard work and celebrate their vital role in implementation.

Successful intervention implementation also hinges on engaging participants (Spoth et al., 1996). MIC had a dedicated recruiter who visited the jail every week to promote the SFP with inmates and caregivers who brought children to see their parents. The recruiter's knowledge of local gathering places and traditions was helpful in enrolling and maintaining contact with these hard-to-reach families (see Spoth et al., 2007). Although it was ideal to have one person in this role from the participant's perspective, it presented a challenge when she became ill. The fact that we had developed systems to track families, and cross-trained staff members in SFP activities was helpful because it was possible for others to step in to share her duties.

Encouraging continued family involvement after recruitment can be difficult given barriers to participation, including scheduling, lack of transportation, and child care needs (Spoth et al. 1996). Addressing these issues was critical for the overburdened families with whom we worked. We held meetings in the evening and provided transportation, culturally appropriate meals, and care for children too young to participate. We also asked local businesses to donate small gifts for weekly door prizes. Every session, families arrived to a hot meal and had time to socialize with others caring for children of incarcerated parents. Participants told us they found this time to connect with each other to be extremely valuable and rewarding because they shared a common experience of being stigmatized and socially isolated (Hagen & Myers 2003; Severance 2004). These relationships with other families may have helped sustain participation.

CBOs may engage hard-to-reach families because they may be perceived as more personal and less intimidating than hospitals, guidance clinics, or schools. CBOs typically arise from a need or interest within the community. Thus, they are likely to be responsive to such needs, and may have reputations and relationships developed over time that are more trusted than larger institutions which typically deliver EBPs (Kramer 1999). Participants also tend to establish trust at interpersonal rather than organizational levels (White-Cooper, Dawkins, Kamin, & Anderson, 2009), so small CBOs like MIC may be more likely to establish personal connections compared to larger institutions. MIC's intimate setting allowed us to host family engagement activities (e.g., ice-breaker games, taking family photos to give as gifts in frames decorated by the children) and celebrations (e.g., awards ceremonies). Participants offered testimonials reflecting how appreciative and grateful they were for the new program. Families who attended SFP have since enrolled their children in additional MIC programs, and shared their enthusiasm about the program with friends. Thus, MIC expanded their community outreach through a network of personal connections.

Phase 5: Program Evaluation

Finally, evaluation is necessary to sustain an intervention and improve practice. Extensive outcome evaluation may be beyond a CBO's capacity, but efforts to document activities, collect participant feedback, and obtain information for program improvement and future funding are vital organizational management issues (Stevenson, Florin, Mills, & Andrade, 2002). Process evaluation can help a CBO manage resources wisely and obtain support for high-quality programs that address community needs. We documented program adaptation and implementation through group leader notes from weekly sessions and post-session debriefings, using the notes to revise manuals for the next round, and developing a template and examples to structure future notes and fidelity observations.

We also gathered participant outcome and satisfaction surveys, conducted fidelity observations, and recorded attendance. Outcome surveys were developed with the goal of creating an internal evaluation resource that MIC could use to assess caregiver and child functioning at intake. Surveys included standard SFP questionnaires (see Kumpfer, 1989) covering caregiving attitudes, family relationship dynamics, and child behaviors. We also assessed social support, mental health, and caregiving stress because these were identified in the client needs assessment as salient issues for caregivers. MIC's Director and the university PI reviewed surveys to ensure all items would be appropriate and understood by participants. We administered surveys using a group format to maximize efficiency and participation, reading items aloud to reduce literacy concerns. Although it is beyond the scope of this paper to report on outcomes (currently being collected), we have thus far observed high implementation fidelity and received high client satisfaction ratings (e.g., 90% of families said they would recommend SFP). Data have also been used to improve practice by providing information for client referrals (e.g., depression symptoms).

University and community partners also worked together to develop a system to manage process and outcome evaluation data. We assessed MIC's need for different program reports, what data to collect, and how best to organize data for reports. Matching CBO reporting needs to the format of the data collected was a significant undertaking, but resulted in a flexible and comprehensive database that was manualized so that CBO and university partners could use it to extract information. Resulting data could be used to secure funds for sustainability, future programming, and ongoing capacity-building. Such implementation and evaluation procedures enabled MIC to show that they could implement an EBP with fidelity, and evaluate outcomes.

Discussion: Partnering for Translation

Table 2 illustrates some concerns and lessons learned regarding partnership and translation. Partnership lessons included building trust, participation, role definition, and support. University and community partners met frequently to discuss project goals, clarify the role of each partner organization, and facilitate MIC's readiness for EBP implementation. Specialized experiences that MIC and university partners each provided were also important both to increase staff members’ skill sets and build respect for each others’ priorities. Specifying unique contributions of each partner and assigning responsibilities (e.g. interpreting client needs assessment results) appeared to enhance each partner's investment in the project and build trust. To support staff as they took on new responsibilities, the University PI and MIC Director met with them to describe project goals and rationale, and define roles for each staff member. We also held celebrations for staff in order to acknowledge their contributions.

Table 2.

Partnership (P) and Translation (T) Lessons Learned in Preparing a CBO to Implement an EBP

Topic	Concern	Lesson Learned	Actions
Trust and Respect for Partner Priorities (P)	○ Partners may not understand or respect each other's agenda	○ Emphasize equal partnership ○ Develop respectful relationship ○ Learn to “speak each other's language”	○ Work together on a common idea ○ Identify mutual, overall project goals ○ Specialized trainings (e.g., research ethics; jail visiting)
Partner Participation and Role Definition (P)	○ Participation may not be equal	○ Important tasks for everyone in partner organizations ○ Define unique partner roles	○ Partners jointly identify client needs ○ Select and adapt EBP together ○ Jointly decide on evaluation procedures and measures
Staff Support (P)	○ Low morale	○ Acknowledge achievements ○ Create positive climate ○ Regular communication with all staff	○ Celebrate small and big successes ○ Engage staff with interesting tasks and meaningful responsibility
Logic Model (T, P)	○ Not knowing who does what and why	○ Communicate reasoning behind project duties and activities ○ Clearly define roles on project	○ Staff training and cross-training ○ Regularly checking in with staff on details of project progress
Organizational Strengths and Needs (T)	○ Immediate needs may be too overwhelming to add something new	○ Focus on building long-term sustainability of EBP ○ Develop specific strategies to keep staff, obtain funding	○ Enhance existing activities to serve multiple functions ○ Identify and build on staff skills ○ Develop manuals for training new staff
Evaluation (T)	○ Evaluation will be burdensome	○ Close project monitoring and management ○ Obtain client feedback on what to assess	○ Build data infrastructure ○ Create systems for data collection and documentation ○ Use data to enhance practice

Translation lessons concerned organizational capacity for implementation and evaluation. Agency directors are influential in setting the stage for EPB translation (Proctor et al., 2007); MIC's small staff was dedicated to the Director and the agency and anecdotally, enthusiasm for this program was high. MIC's Director also acknowledged feeling anxious about being able to deliver on research goals (e.g., recruiting participants) and whether EBP implementation efforts, particularly staffing, would overwhelm her need to provide other services. To address these concerns, we tried to enhance functions of MIC that would aid in EBP implementation and also other programming (e.g., training staff; establishing internships; developing database, adapting intake questionnaires). Connecting with other local University partners to find interns was particularly helpful. Similarly, university partners had concerns about data collection procedures being burdensome. By communicating directly and frequently about such issues throughout the implementation process, we proactively addressed many concerns (e.g., using group questionnaire administration). MIC's Director and the University PI maintained frequent phone contact and sent group email updates to keep everyone connected. In these ways, a strong partnership helped us address concerns about translation.

Finally, certain elements of MIC may have uniquely contributed to our ability to translate the EBP to this setting. MIC had a strong Director who appreciated the need for evaluation data to secure funding, and devoted resources to this by getting client files into the database and involving student interns to help with grantwriting. NIH funding for the two years of this project provided more resources than were typically available for MIC, but our investment in capacity-building and focus on sustainability helped secure continuation of the program once NIH funding ended. Finally, the Director and University PI worked together on proposals prior to and after this project was funded. This helped them appreciate each other's strengths, goals, and interests, which facilitated program implementation.

Conclusion and Recommendations

In sum, we learned several lessons for partnering to translate EBPs to a community setting. Understanding each other's concerns and priorities, supporting staff, and defining unique roles for each partner built trust between partners and a focus on achieving mutual goals. Considering organizational capacity helped define project scope and feasibility; enhancing capacity was a goal of our work. Developing protocols for EBP implementation and evaluation that built on existing organizational strengths were also important for sustainability. We recommend that researchers and CBOs who are preparing to partner to translate EBPs to community settings attend to these issues. Effective partnerships can foster research-to-practice links, help bridge the discovery-delivery gap, and ultimately improve health outcomes for under-resourced children and families.