Development and Evaluation of a Toolkit to Assess Partnership Readiness for Community-Based Participatory Research

Abstract

An earlier investigation by academic and community co-investigators led to the development of the Partnership Readiness for Community-Based Participatory Research (CBPR) Model, which defined major dimensions and key indicators of partnership readiness. As a next step in this process, we used qualitative methods, cognitive pretesting, and expert reviews to develop a working guide, or toolkit, based on the model for academic and community partners to assess and leverage their readiness for CBPR. The 75-page toolkit is designed as a qualitative assessment promoting equal voice and transparent, bi-directional discussions among all the partners. The toolkit is formatted to direct individual partner assessments, followed by team assessments, discussions, and action plans to optimize their goodness of fit, capacity, and operations to conduct CBPR. The toolkit has been piloted with two cohorts in the Medical University of South Carolina’s (MUSC) Community Engaged Scholars (CES) Program with promising results from process and outcome evaluation data.

Faculty from the MUSC Center for Community Health Partnerships (CCHP) and their community partners began a conversation several years ago regarding reasons why some academic–community partnerships are successful in meeting their goals, whereas others are not. The group reached consensus that some partnerships and partners are more “ready” than others to engage and work together effectively, and that readiness is the key to success. However, “readiness” remained an ambiguous term with varied definitions among group members. Continued discussions about the concept of “readiness” led to the development of a National Institutes of Health (NIH) grant application and subsequent funding to explore dimensions and key indicators of “partnership readiness” for CBPR. As a result of this work, the study team, composed of both community and academic partners, developed a model for CBPR partnership readiness.¹ Further reflections on the emerging major dimensions and indicators of readiness identified the need for a means to guide partners in assessing their readiness, and generating partnership dialogue and action plans to optimize readiness to conduct CBPR. This paper provides a brief overview of the CBPR Partnership Readiness Model, the development of the CBPR Partnership Readiness Toolkit, and a description and initial evaluation of the toolkit.

OVERVIEW OF THE CBPR PARTNERSHIP READINESS MODEL

The study investigators, a nurse researcher (academic partner) and a director of community nonprofit agency (community partner) used semistructured key informant interviews and focus groups (n = 36) to explore the key dimensions and indicators of partnership readiness for CBPR. The analyses of these data informed the development of the “CBPR Partnership Readiness Model.” A complete description of the methods and analyses for the model development is available elsewhere.¹

CBPR partnership readiness is defined as the degree to which academic/community partners “fit” and have the “capacity” and “operations” necessary to plan, implement, evaluate, and disseminate CBPR projects that will facilitate mutual growth of the partnership and influence positively targeted social and health needs in the community. The antecedents of CBPR partnership readiness are a catalyst and/or mutual interest. The preferred outcomes are sustainable partnership and products, mutual growth, policy, and positive social and health impact in the community. The dimensions (and indicators) of CBPR partnership readiness are (1) goodness of fit (shared values, compatible climate, mutual benefit, and commitment), (2) capacity (effective leadership, inclusive membership, complementary competencies, and adequate resources), and (3) operations (congruent goals, transparent communication, conflict resolution, and equal power). The model is depicted in Figure 1.

Figure 1. CBPR Partnership Readiness Model.

Readiness is an iterative and dynamic process. The assessment of readiness begins with “goodness of fit.” If there is not a good fit between partners, then the other dimensions of readiness and the desired outcomes are threatened. If the partnership is deemed to be a good fit, the next steps are to assess the capacity and operations of the partnership, which can be assessed simultaneously. Each dimension is interdependent on the other, and a change in one dimension could impact the other dimensions and indicators.

The assumptions of the CBPR Partnership Readiness Model are: The dimensions (goodness of fit, capacity, operations) need to be viewed within the community, organizational, partnership, and partner contexts; readiness is issue specific; readiness is partnership specific; varying levels of readi ness exist and are influenced by a range of factors; readiness is amenable to change; and readiness is essential for long-term sustainability of the partnership and community outcomes.¹

DEVELOPMENT OF THE CBPR PARTNERSHIP READINESS TOOLKIT

Concomitant with the development of the CBPR Partnership Readiness Model, review and feedback from the eight-member study advisory board (academic researchers and community partners) identified a need for a practical tool to guide a systematic assessment of readiness through a dyadic lens. To accommodate this request, the academic investigators initially began developing a quantitative instrument using the major dimensions and indicators of the CBPR Partnership Readiness Model to guide the generation of items. The original instrument (approximately 100 items) was drafted with the goal of producing scores for each subscale (goodness of fit, capacity, operations) and a total “readiness” score. The intent was to develop an algorithm with recommendations for the partnership based on their scores. This “traditional academic approach” and instrument draft was presented to the study advisory board and two of the original key informant interviewees. After several meetings with the board, consensus was reached that a tool promoting equal voice, transparent bidirectional dialogue, and specific actions by the partnership members would be more useful than a tool that yielded quantitative “scores” only. These discussions led to the concept of a “toolkit” in a workbook format with the intent that each partner would receive a personal copy and complete individual assessments based on his or her own perceptions. After individual assessments, the partners would come together for a team dialogue and form consensus or further action plans based on their discussions.

Subsequently, the investigators generated a 16-page toolkit to guide individual assessments of each dimension of readiness with “yes/no” responses, confidence responses with a 5-point Likert scale, and additional qualitative exercises for the team to complete together. Two focus groups (n = 20 academic and community members) were used to receive both written and verbal feedback on the toolkit and further validate the findings for the key dimensions and indicators of the CBPR Partnership Readiness Model. Each participant was provided a copy of the toolkit and asked to respond verbally to semistructured questions regarding the toolkit, and to make written notes and suggestions on the toolkit itself. The participant toolkits were turned in to the investigators at the end of the each focus group. The focus groups were recorded and transcribed verbatim. The investigators, two members of the study advisory board, and an expert consultant listened to the audio recordings and reviewed the transcripts. The team applied a “framework analysis approach” to analyze these, as well as the written feedback data.^2,3 Validity of the findings was enhanced with the triangulation of multiple sources of data (focus groups, observation, transcripts, written responses) and multiple investigators analyzing the data with final consensus and convergence of findings.⁴ Two focus group participants reviewed the analyses (i.e., member checks) for validation of the findings.

Data revealed a saturation of preferences for individuals to assess confidence of each indicator (versus “yes or no” items indicating presence or lack of presence of the particular indicator) and to use a broader range of responses (i.e., 1–10 Likert response versus 1–5 scale) as a starting point for the team discussion. Participant preferences indicated that if an assessment item was scored less than 6 by an individual, then this would prompt further action by the partners to address the particular readiness indicator. Other general preferences identified for the toolkit included additional definitions of the indicators to promote mutual understanding of the terminology, to include exemplars, and to provide additional reference information in the toolkit. Feedback and recommendations were provided for the order of the content, specific items, and language used for the individual assessments, team discussions, and action plans. Reviewers highlighted the need for additional materials to prompt an adequate assessment, such as tables to document information on specific indicators based on the partners, organization, and community of interest.

After these analyses, the investigators revised the toolkit that is based on the current published version. The investi-gators conducted a third focus group (n = 10 academic and community partners) to further validate the toolkit content, format, and overall approach. After the focus groups, the academic investigator conducted cognitive pretesting with four academic and four community members using techniques of “think-aloud” and verbal probing.^5-7 This process was used to gain in-depth insight into individuals’ interpretation of specific toolkit content, subsequently resulting in revisions and clarification of items assessing individual’s level of confidence with their ability to meet the three major dimensions of readiness. Content validity was established through review by the study advisory board and two scientific experts.

OVERVIEW OF THE CBPR PARTNERSHIP READINESS TOOLKIT

The CBPR Readiness Toolkit is approximately 75 pages and has 6 chapters. It is available, free of charge online: http://academicdepartments.musc.edu/nursing/cchp/cchp_products

Chapter 1: toolkit Overview (highlights)

• Background information

• Introduction to CBPR principles

• Description of the Partnership Readiness Model

• Instructions for toolkit use

Chapter 2: Basic tenants of the Partnership (highlights)

• Description of the priority health issue, the community of interest, and evidence that the issue is a significant problem in the community

• Individual assessments and team discussion

• Description a vision statement

• Short- and long-term goals for the partnership

Chapter 3: Goodness of fit (highlights)

• Definitions and key indicators of “goodness of fit”

• Definitions, individual assessments, team discussion, and action notes for each indicator: Shared values, compatible climate, benefits, and commitment

• Team exercises to delineate and discuss past history (positive histories and challenging histories between partners, their respective organizations, and the community) and credibility issues (both positive and challenging), which may facilitate or impede the partnership and intended CBPR project

• Team exercise to discuss the organizational and community climate (people, processes, and structures) within the academic organization, community organization, and community

• Team exercise to discuss commitment (partners time and partner/organization resource commitment), with resources on memorandum of understanding and formalized contracts

• Chapter summary with individual assessments, team dis cussion, and action plan summary for “goodness of fit”

Chapter 4: Capacity of Partnership/Project (highlights)

• Definitions and key indicators of capacity

• Definitions, individual assessments, team discussion, and action notes on each indicator: Leadership, inclusive membership, competencies, and resources

• Checklist of possible competencies needed (skills/abilities, project evaluation, personal behaviors, and knowledge)

• Team exercise to delineate current resources and needed resources

• Chapter summary with individual assessments, team discussion and action plan summary for “capacity”

Chapter 5: Partnership Operations (highlights)

• Definitions and key indicators of operations

• Definitions, individual assessments, team discussion, and action notes on each indicator: Communication, conflict resolution, and equal power

• Team exercise to delineate preferred communication methods and how communication exchange will occur

• Team exercise to describe the structure for the partnership (meetings, agendas, decision making, task groups or subcommittees, and advisory boards)

• Team exercise to describe expectations of workload allocation

• Team exercise to plan for dissemination of products

• Chapter summary with individual assessments, team discussion, and action plan for “operations”

Chapter 6: Are we ready? (highlights)

• Team reflection and summary discussion of each readiness dimension (goodness of fit, capacity, and operations)

• Revisiting and revisions (if needed) of partnership goals from Chapter 2

• Compilation of action notes to develop comprehensive action plan to move forward with the partnership, including timeline and evaluation components

• Description and resources for logic model

HOW TO USE THE TOOLKIT

We recommend that the toolkit be used with a trained facilitator to guide the process of systematically assessing the readiness of a partnership. Specific suggestions include:

1. Schedule a retreat (1–2 days), or five or six 2-hour sessions, or three or four 3-hour sessions in which all partners can attend.

2. Provide a toolkit for each partner for his/her personal use.

3. Before the retreat or scheduled sessions, make assignments for the chapters to be discussed at the session, with each partner reviewing and completing the reading materials and individual exercises before the planned session.

4. During the partnership sessions, individual responses are shared, discussed as a team, and tentative action plans are made. A trained facilitator is recommended to guide this process. At the end of all of the sessions, the team derives a final, comprehensive action plan to guide next steps.

EVALUATION OF THE TOOLKIT

The investigators have piloted the toolkit in training sessions for the past 1.5 years with the MUSC CES program, representing 35 academic and community partners (11 partnership teams) who receive 12 months of didactic training, mentorship, and CBPR pilot funds. The toolkit is used in months 2 and 3 of the program, totaling 15 hours of assessment including facilitated discussions, individual assessments, and internal discussions among team partners.

Participants in the CES program evaluated the toolkit as being especially helpful for both new and established academic–community partners to establish a foundation for long-term sustainability of the partnership. New partners indicated that the toolkit prompted assessments and dialogue of factors they had not yet considered, but deemed important and vital for the success of a partnership. The participants agreed that the toolkit provided a solid foundation to ensure compatibility, and provoked a comprehensive assessment of capacity needs and operational issues necessary for successful CBPR partnerships.

Process evaluation related to implementation of the toolkit in the CES program revealed several key outcomes. After assessment of readiness, one partnership team terminated a relationship with a partner who was not determined a good fit (lack of commitment) and solicited a new partner. With the addition of the new partner, who brought additional competencies and resources (capacity), this team has not only completed a pilot CBPR project, but also has submitted an NIH grant to further test their intervention. Additionally, training has been sought by partners based on deficiencies noted in competencies, additional members have been added to several of the partnership teams, and several teams have added advisory boards to guide their projects. We have noted that the readiness tool facilitates transparent and open discussion among partners, which seems to promote additional trust, understanding, and engagement with the conduct of their CBPR project. Both academic and community partners stated the need for such assessment to position their teams for optimal success.

Outcome evaluation for the six teams participating in the first cohort resulted in the conduct of six pilot CBPR studies, nine presentations by academic and community partners, and three grant submissions (foundation and NIH) with community partners as co-investigators. In addition, multiple teams have been highlighted by local and national organizations as exemplars for community-based research.

The investigators presented the toolkit to approximately 60 participants (academic and community) at the 2010 Community–Campus Partnerships for Health National Conference. The presentation included breakout sessions that provided the participants with the opportunity to work through small sections of the toolkit. Responses to the toolkit were overwhelmingly positive and participants agreed the toolkit would not only be useful for partners to assess their readiness, but to provide a foundation for both the partnership and CBPR project. This group validated the utility of the toolkit for academic–community partners regardless of their time and history together, with the rationale that partnerships are dynamic, change over time, and almost always have opportunities for further growth.

LIMITATIONS OF THE TOOLKIT

This toolkit was developed using a purposive sample of academic and community partners in one geographical region. Second, the toolkit has not yet been adequately tested with other partnerships outside our region, and has not been tested with a comparison condition. Additional validity testing of both the CBPR Partnership Readiness Model and the toolkit is warranted. The toolkit is lengthy, requires considerable time to complete, and is best utilized with a trained facilitator. Further research is needed that may delineate stages of partnership readiness, as well as strategies that support or hinder readiness for each stage. Well-designed studies that may test the effectiveness of the toolkit to facilitate readiness of CBPR partnerships and partnership outcomes are indicated.

CONCLUSIONS

The CBPR Partnership Readiness Toolkit was developed in response to academic–community partners’ interest in exploring ways to improve partnership outcomes. The toolkit provides an opportunity for each partner to have a voice in the assessment of the partners’ readiness and to facilitate honest and transparent discussions about their own strengths and challenges. The toolkit is comprehensive and requires a substantial time commitment to complete. However, the intent is to promote an investment of the partners to lay a firm foundation for their relationship and operations to not only achieve desirable outcomes, but to sustain the partnership and its products over time.