Community Capacity Building: A Collaborative Approach to Designing a Training and Education Model

Abstract

Background

Successful community–academic research partnerships require building the capacity of both community-based organizations (CBOs) and academics to conduct collaborative research of mutual interest and benefit. Yet, information about the needs and goals of research-interested CBOs is lacking. Our partnership aimed to conduct a community research needs assessment and to use results to develop future capacity-building programs for CBOs.

Methods

Based on our review of the literature, informal interviews with research-interested CBOs and community-engaged research groups locally and nationally, we developed a needs assessment survey. Key domains of this survey included history and experience with research collaboration, interest in specific research topics, and preference for learning format and structure. We trained community health workers (CHWs) to recruit senior leaders from CBOs in New York City (NYC) and encourage them to complete an on-line survey.

Results

Fully 54% (33/61) of CBOs completed the needs assessment. Most (69%) reported involvement with research or evaluation in the last 2 years and 33% had some funding for research. Although 75% had collaborated with academic institutions in the past, 58% did not rate this experience well. The four areas respondents prioritized for skills building were program evaluation, developing needs assessments, building surveys, and understanding statistical analyses. They were less interested in learning to build collaborations with academics.

Conclusions

A formal needs assessment of research training and educational needs of CBOs revealed that most had experience, albeit negative, with academic collaborations. CBO leaders wanted to build skills to conduct and analyze assessments and program evaluations. Our community-academic partnership is using these findings to develop a research capacity-building course. Other partnerships should consider conducting such assessments to transform the capacity of CBOs to be active research partners and leaders.

When representatives of CBOs and community leaders work in partnership with academics to conduct research, this community engagement can inform the research question, process, and methods, by yielding critical insights and ideas, introducing innovative questions, raising community and ethics concerns, and offering knowledge of diverse cultures within its community.

There are many factors that play a role in community-academic partnerships that prove to be ineffective and do not meet the expectations of its partners. An important factor is the current disparity in research capacity and experience between academic and community partners, a critical dynamic that can exacerbate inequities and tensions within the collaboration.

To date, there has been inadequate attention to and resources focused on preparing CBO staff in research fundamentals so that their engagement in the research process is as an informed participant and collaborator. Building the capacity of CBOs to engage effectively in research enhances their ability to develop and conduct their own research and program evaluation.

There are few published articles that discuss building the capacity of CBOs to design, develop, and implement research. The Clinical and Translational Science Awards’ (CTSA) Task Force on the Principles of Community Engagement prioritizes “research collaboration ethics” as the first step in community capacity building and active engagement in research collaboration. Some efforts to date have focused on ethics training as a means of building research capacity.¹ These articles describe the University of Pennsylvania’s ethics training course that includes examples of unethical research conducted in communities^2,3 and the University of Colorado Clinical Translational Institute’s monthly seminars for faculty, staff, and community members that focuses on community engagement, funding, health disparities, and partnership building.⁴ Many of the recent efforts by academic institutions to engage communities were propelled by the creation of the National Institutes of Health’s CTSA and its required community engagement cores.^5–7 Yet, from a review of the literature and program websites, institutions that are creating a research training series seem to base their curriculum on perceived areas of interest of community members rather than through a formal needs assessment of the identified community. Such efforts, although important, may reflect the challenges that catalyzed the field of CBPR: That research initiatives are enriched when there is bidirectional learning and dialogue on community needs and methods to achieve identified outcomes. This paper describes an essential first step, often overlooked, in community research capacity building—conducting a needs assessment to develop a research capacity-building training model.

In 2009, Mount Sinai established a community engagement core, The Centers for Community and Academic Research Partnerships (CCARP), as part of Conduits, their CTSA.⁸ Nurturing and developing community-engaged research affecting the well-being of diverse communities and population groups in NYC has been CCARP’s underlying and driving purpose.⁹ To achieve these goals, CCARP, led by a partnership board, shapes the community-engaged research portfolio for conduits, builds researcher and community skills in conducting community-engaged research, and sparks and supports community-academic research partnerships. Led by community and academic co-chairs, the board is composed of leaders of 22 research-interested CBOs and 12 community-interested academics from Mount Sinai and affiliated institutions in NYC. The community members reflect NYC’s diverse population and lead key research-interested organizations and constituencies including faith-based, social service, advocacy, and healthcare organizations. The board supports the continuum of community-engaged research, ranging from early introductions of community stakeholders to researchers through implementation of CBPR projects. The board meetings provide an opportunity to exchange ideas and participate in research-related activities initiated by community leaders and academics who are board members or who are part of our larger, city-wide community and academic networks.

Through retreats and bimonthly meetings, the board chose key areas of focus and developed subcommittees for each. Community partners immediately identified the need to build research capacity of CBOs and, with board approval, formed a research capacity building subcommittee made up of two researchers and four CBO leaders, including co-authors Peggy Shepard and Veronica Piedra.¹⁰ The subcommittee reviewed the community capacity-building work of community engagement cores of other CTSAs including Northwestern’s Alliance for Research in Chicago Communities. The alliance provides seed grants to community–academic partners for capacity building and the CTSA at New York University provides community capacity building through 1- and 2-day conferences.^11,12 The subcommittee then conducted a literature review, informal interviews with research-interested CBOs and community–engaged research groups locally and nationally, and a search of key websites such as CORUS¹³ and Community-Campus Partnerships for Health.¹⁴

Through this work, the subcommittee concluded that (1) most CBOs have diverse skills and capacity that will facilitate research involvement, (2) cuts in traditional sources of funding for many CBOs might be an incentive for them to expand their skills to new areas, such as research to develop diversified funding streams, and (3) the work and credibility of CBOs could be enhanced by incorporating appropriate research and program evaluation methods into their projects. The team also recognized that it was crucial to hear from front-line CBOs in NYC to ensure that the board’s work reflected the true needs and priorities of the organizations it aims to serve. Thus, to develop a capacity-building strategy to support local CBOs, the subcommittee decided to survey local CBOs to better understand their interest in research, the skills and knowledge they would like to gain, and their preferred learning formats.

METHODS

The subcommittee developed a CBO research needs assessment survey (Appendix). First, we identified several key domains using the sources outlined above, including (1) prior history with research participation and academic collaboration, (2) level of satisfaction with collaboration, (3) perceived research and program evaluation needs and interests (i.e., survey development, organizational assessment, data analysis, literature review, ethics, negotiating academic collaborations, and funding), and (4) preferred learning format (i.e., individual, group, in person, on line, length, and frequency of training). We developed questions because we could not find validated scales to measure these domains. We piloted the survey with board members’ organizations and revised the survey based on their feedback.

To collect survey data, we employed CHW students from a local community college who conducted this work to complete a 70-hour internship requirement. All CHWs received training in research ethics and the project was approved by the Program for the Protection of Human Subjects Office at Mount Sinai.

To identify organizations to be surveyed, we generated a preliminary list of 80 NYC organizations through networks of board members and through a listing in the Foundation Center’s Directory of organizations that had received grants in 2009–2010 in health services or research. 15 We contacted these organizations, and after discussions with their senior executives eliminated 19 organizations no longer in operation or disinterested in research. The CHWs then sent an email invitation with a Survey Monkey link to the remaining 61 organizations, following up by mail, email, and telephone calls over a 90-day period to ensure that any questions or concerns by organizations were addressed appropriately and that the survey was submitted to a person senior enough in the organization to ensure complete knowledge of the research activities and interests of the organization. To maximize results, we sent the survey link to non-responders on three occasions, phoned leaders 10 times, and offered to administer the survey by phone or in person if the contacts preferred. The subcommittee worked with CCARP’s statistical support team to analyze the data, using simple frequencies for most results and analysis of variance between groups to compare the mean response between the groups and Pearson correlations to compare within-group responses.

RESULTS

Of the 61 organizations contacted, 33 responded (54%). There were no differences between respondents and non-respondents by organization, location type, or population served. Their median size was 42 full-time staffers with 5 volunteers. As summarized in Table 1, most were in Manhattan, provided direct social services, served diverse vulnerable populations of all ages, and provided services related to many common health conditions.

Table 1.

Description of Organizations in Needs Assessment (N = 33)

Organization	%
Location
Manhattan	85.0
Bronx	12.0
Queens	3.0
Type of Organization
Direct social service	48.5
Health service	21.2
Religious	6.1
Academic/research	6.1
Government	15.2
Other	21.2
Population Served*
Adults	72.7
Children/adolescents	87.9
Families	60.6
Immigrants	87.9
LGBTQ	75.9
Underrepresented	39.4
Minorities	78.8
Service Direction*
Access to care	51.5
Asthma	60.6
Arthritis	24.2
Cancer	36.4
Diabetes	60.6
Cardiovascular disease	33.3
Infectious disease	29.4
Violence prevention	27.3
Maternal child health	39.4
Mental health	45.5
Obesity	57.6
Occupational health	18.2
Reproductive health	42.4
Substance abuse	45.5

LGBTQ, lesbian, gay, bisexual, transgendered, queer.

^*Responders could choose multiple options.

Research Experience and Collaboration

As shown in Table 2, although less than one half of CBOs have a research or evaluation unit and only one in three have ever had funding for research or evaluation, most have conducted some research or program evaluation in the last 2 years. Although three in four have collaborated with academics and are interested in future academic collaborations, fewer than one half rated their history of academic collaboration as very good or excellent.

Table 2.

Research Experience and Collaboration of Organizations (N = 33)

Research Experience	%
Active research or evaluation unit	42
Conducted research/evaluation in the last 2 years	69
Ever funded for evaluation/research	33
History of academic collaboration	75
Rating of academic collaboration:
Poor/unsatisfactory	25
Good	33
Very good/excellent	42
Very/extremely interested in future academic collaborations	75

Training and Education Interests

The four areas of greatest interest reported were program evaluation, developing needs assessments, statistical analysis, and survey development. There was less interest in how to establish research collaboration with academics. With regard to what method of training or education would work best for the responding organizations, the response was mixed: 33% preferred a hybrid, in person/online program; 30% preferred a 1-day conference; and 27% preferred a multi-week, formal course. Respondents interested in research collaboration were more likely also to be interested in program evaluation, organizational evaluation, human subjects protection, or general research design (Table 3). There were no differences in interests or experiences between the two largest types of organizations: Direct social service and health service (data available on request). Organizations that consider themselves to be providing non-direct health services had the highest levels of interest in program evaluation and needs assessment development. Programs that provide direct health services had higher levels of interest in statistical analysis (data available on request).

Table 3.

Areas of Expressed Research Interest (N = 33)

Areas of Interest	Mean	Standard Deviation	Pearson Correlation If Interest in Collaboration Expressed	p value
Interest in collaboration	6.03	2.157
Program evaluation	6.42	2.292	.481**	.007
Organizational assessment	5.61	2.404	.369*	.045
Needs assessments	6.35	1.992	.221	.240
Negotiating to establish a partnership/ collaboration	5.55	2.278	.426*	.019
Survey development	6.00	1.673	.213	.259
Human subjects protection	5.35	2.550	.551*	.002
General research design	5.48	2.541	.551*	.002
Statistical analysis	6.10	1.777	.316	.089
Conducting a literature review	5.32	2.023	.231	.220

Each respondent organization provided feedback on areas of expressed research interest based on their experience and/or current needs as shown above in Table 3. The domains of research interest were: interest in collaboration, program evaluation, survey development, organizational assessment, needs assessment, negotiation to establish a partnership, human subjects protection, general research design, statistical analysis, and conducting a literature review.

Use of the Survey to Develop a Capacity-Building Training Program

The subcommittee presented their findings and recommendations to the board to develop an action plan. The board decided to develop a 12-session capacity-building training program with monthly rotating in-person and on-line workshops. To take next steps, the subcommittee met to develop a curriculum.¹⁶ In addition to topics of interest identified by survey respondents (program evaluation, survey development, and data analysis), we chose to supplement those with basic research topics that people often “don’t know that they don’t know or don’t know they need to know,” as expressed by one CBO member. These include setting realistic goals, defining a research question, selecting the best study or program evaluation design and methods to answer the question, and principles and strategies to engage research participants. The subcommittee planned to offer these basic research topics to participants briefly, and expand on them if participants expressed interest in learning more about them. Participants will have access to the university’s computing system for literature review, and we will have computer terminals available for hands-on class practice on data analysis.

We contacted board members and survey respondents to identify potential participants. An on-line application process requires applicants to have a leadership role in a research-interested CBO, a written letter of support from its executive director to ensure they will have protected time to take the course and complete coursework, and a research or program evaluation question in mind. To facilitate experiential learning, the participants will work to develop and implement a research project. Through group learning and individualized technical assistance, course teachers (both community and academic experts) will help participants to shape their research question, apply new knowledge of methods and research design, practice conducting and analyzing data, and interpret results. The subcommittee has developed an evaluation plan that includes administering pre- and post-evaluation questions to participants so that we can refine recruitment and course content, and targeted follow-up with all CBOs who participated in the research needs assessment.

DISCUSSION

To build community organizations’ capacity to conduct research and program evaluation, our community–academic partnership developed, implemented, and analyzed a survey to inform development of a participatory, goal-directed research training program. Several important lessons emerge from this work. First, through an egalitarian partnership, we were able to identify a community priority: To increase research and evaluation capacity of CBOs. Through the work of the Partnership Board, we developed a needs assessment and used the results to address research capacity, and we were able to incorporate the board members’ wide range of expertise in needs assessment, education and training, research capacity building, and CBO-skills, operations, and challenges. As a result, we have built a strong team to address these goals and a training program with great educational and collaborative potential.¹⁷ Just as it is important to use community-engaged approaches to conduct high-impact research that benefits local communities, we have found it critical to move beyond researchers’ assumptions about what CBO leaders need to know about research, and engage communities to develop research capacity building programs that meet their needs.

Second, similar to other community-engaged projects, this kind of community research capacity-building project requires consistent staffing and ongoing support from academic institutions and CBOs. It is important to explore and have a clear understanding of what resources are available from all partners, articulate how these will be deployed and shared, and ensure they will be sufficient to take such a task through to its completion.¹⁸ In this case, the academic institution had resources to provide staffing and analytic support and to help community representatives on the board to collect, analyze, and interpret data. Community partners had expertise and networks that were invaluable. They identified critical survey domains and led the planning of a training and education model reflective of CBO interests, needs, and preferences.

Third, it may take this type of longitudinal approach to develop true community–academic partnerships with the opportunity to foster trust and mutual exchange of ideas and experiences. The time and resource investments made by CBOs and academic institutions in supporting CBPR are a crucial component in designing education and training programs responsive to community needs, but they are also an example of a collaborative intention to level the playing field. Short-term outcomes will likely include stronger, more informed community–academic partnerships, and better informed CBO focus on program evaluation. Academic and CBO returns on investment may include novel ideas, increased funding, enhanced capacity building, and movement toward new and important organizational directions.¹⁹ Both stakeholders—researchers and communities—may also begin to see each other as strong partners and collaborate to advanced commonly held goals to improve community health.²⁰

Last, this goal-oriented board partnership was working through its own partnership process while it was developing the survey and course. Each step was iterative, and we will use lessons learned to implement and evaluate the training program, and assess additional needs of CBOs as the workshop proceeds infuses by the experience of board members and the communities they represent. Through this process, we hope to create a sustainable model for community research capacity building.²¹

We recognize that the study had limitations. The sample size was small, which limits generalizability and precludes more robust analysis of subgroups within those who responded. Despite use of extensive follow-up procedures, we had a 54% response rate. Nonresponders may have had different priorities and interests, although their basic characteristics did not differ from those of respondents. Despite these shortcomings, we were able to build a partnership with the participating CBOs, identify CBO research priorities, and use information gained to offer an interactive, learner-centered program to increase their capacity to be involved substantively in research, and ultimately to direct their own research agendas.

Appendix A. Community Needs Assessment Survey