Building Community Research Capacity: Process Evaluation of Community Training and Education in a Community-Based Participatory Research Program Serving a Predominately Puerto Rican Community

Abstract

Background

Education and training build community research capacity and have impact on improvements of health outcomes.

Objectives

This manuscript describes the training and educational approaches to building research capacity that were utilized in a community-based participatory research program serving a Puerto Rican population and identifies barriers and strategies for overcoming them.

Methods

A process evaluation identified a multitiered approach to training and education that was critical to reaching the broad community.

Results

This approach included four major categories providing a continuum of education and training opportunities: networking, methods training, on-the-job experience, and community education. Participation in these opportunities supported the development of a registry, the implementation of a survey, and two published manuscripts. Barriers included the lack of a formal evaluation of the education and training components, language challenges that limited involvement of ethnic groups other than Puerto Ricans, and potential biases associated with the familiarity of the data collector and the participant. The CBPR process facilitated relationship development between the university and the community and incorporated the richness of the community experience into research design. Strategies for improvement include incorporating evaluation into every training and educational opportunity and developing measures to quantify research capacity at the individual and community levels.

Conclusions

Evaluating training and education in the community allows researchers to quantify the impact of CBPR on building community research capacity.

Like many disadvantaged communities, the Lower West Side neighborhood of Buffalo, New York, suffers from high rates of asthma. The housing stock is old, and many residents are exposed to triggers (specifically cockroaches, mold, and tobacco smoke) that are known to increase asthma morbidity.¹ Moreover, the community is adjacent to a major highway and it is the U.S. terminus of a high-volume international border crossing where trucks often experience lengthy delays, resulting in elevated levels of diesel emissions and other exhaust.

The Lower West Side has approximately 25,000 residents representing more than 20 racial/ethnic groups, 30% of which are Hispanic, most of whom are of Puerto Rican descent. Fifty-four percent of the community is African-American. The median household income is $15,688; 17.5% of households receive public assistance. Thirty percent of the population is under age 18.²

Although the entire neighborhood is presumably exposed to environmental factors that could exacerbate asthma, rates are especially high among Puerto Rican residents. A community health assessment in the 1990s found that 14% of Puerto Ricans 12 years and older and 21% of Puerto Ricans under age 12 had asthma.³ Puerto Ricans have been found to have higher rates of asthma than other Hispanic subgroups.⁴

In response to the high asthma burden, particularly among Puerto Ricans, and growing concern about the impact of air pollution in the neighborhood, a partnership of university researchers and community activists formed and secured funding from the National Institute of Environmental Health Sciences (NIEHS) to create the West Side Community Asthma Project (WSCAP) in 2001. Although the WSCAP was accessible to the entire Lower West Side of Buffalo, its target population was the Puerto Rican residents living within this community. WSCAP’s overall goal was to enhance the community’s ability to participate in asthma research and prevention/management activities. This paper describes the training and educational approaches to building research capacity that were used in this community and identifies barriers and strategies for overcoming them.

CONCEPTUAL FRAMEWORK

Asthma is a chronic disease with important environmental and lifestyle components. Successful management depends on engagement of patients and their families/households. As suggested by the higher asthma prevalence among Puerto Ricans in Buffalo’s Lower West Side, it is likely that cultural factors also play a role. The organizers of the WSCAP decided, therefore, that engaging the community in the design and implementation of asthma interventions could improve the probability of success. Specifically, they adopted an approach based on the NIEHS model, which states that CBPR “(1) promotes active collaboration and participation of every stage of research, (2) fosters co-learning, (3) ensures projects are community-driven, (4) disseminates results in useful terms, (5) ensures research and intervention strategies that are culturally appropriate, and (6) defines community as a unit instead of identity.”⁵

Community research capacity represents the skills and knowledge individuals within the community have that enable them to participate in research activities. Building community research capacity through training and education ensures that co-learning is occurring. Understanding and adapting research methodologies to meet community needs develops trust between researchers and the community to facilitate research by translating the research paradigm into language that is meaningful to the community. For example, learning about misperceptions of disease, alternative methods for treating disease, and appropriate dissemination of information are important in the translation of research. Zayas and colleagues highlighted that lay definitions of disease do not always match the biomedical model that drives treatment.⁶ Understanding these differences may lead to more meaningful interventions that improve health outcomes.

Examples of building research capacity are documented in the literature. In Roxbury, Massachusetts, youth were trained to educate the community on the relationship between air pollution and health, empowering them as leaders.⁷ The Community Action Against Asthma program trained outreach workers as “Community Environmental Specialists” to conduct household assessments and personal monitoring of exposure.⁸ In Brooklyn, New York, community health educators were trained to conduct interviews and facilitate focus groups.⁹ Adams and colleagues described the importance of co-learning through building research capacity to ultimately empower the community to initiate and participate in future research agendas.¹⁰

In the WSCAP, the framework for action was similar to the “Pathways to Health Model,” recently described by Wallerstein,¹¹ in which community-based participatory research acts through training and education to improve employment opportunities, income, and the potential for advocacy or policy changes. These empowerment and community capacity outcomes in turn drive positive health outcomes.¹¹ Specifically, Wallerstein’s model states that “empowering strategies” create “empowerment and capacity outcomes.” These outcomes, in turn, have a direct impact on “health outcomes” and “developed effectiveness.” All of this occurs within the context of ‘global, national, and local contexts” in addition to “political, human rights, economic, socio-cultural, racial, and environmental contexts.” Figure 1 shows how this model was adapted to address the WSCAP. The training and educational opportunities were empowerment strategies that led to empowerment and capacity outcomes such as skill development and increased confidence. The community member became a university partner and was recognized as the local expert. These outcomes lead to improved health outcomes through community participation in the registry, survey, and focus groups. Personal wellness can also be expected from improved capacity.

Figure 1.

Pathways to health model (Wallerstein) adapted for the West Side Community Asthma Project.

METHODS

Needs, Objectives, and Strategies

At the program’s inception, WSCAP prioritized community education as an integral part of the CBPR process considered by Wallerstein’s model as a capacity outcome. An informal plan to educate the community involved attendance at national meetings, human subjects tutorial training, and training on research methods and research ethics.

Training and education were needed on several levels in the community. Basic education was needed to improve understanding of what asthma is and how it is managed. As part of the participatory process, community members were engaged in the development and implementation of research design. Educational sessions and training helped to dispel the mistrust that existed between the community and the university through an understanding and appreciation of the processes. For example, methods training received by community members allowed the processes to be described in a more understandable way. Also, sessions led by nurses to teach how to use medications properly was essential because the community felt that the university was giving something back to them. In addition, training allowed the community to give insightful comments to benefit the research.

Educational opportunities were presented in schools, nursing homes, and other local venues to provide information on what asthma was, how to use inhalers properly, how to understand the medication that was given, and how to develop strategies to best manage asthma. Stakeholder meetings incorporated general education and training elements, including how to interpret the local data that were available. Newsletters, available in English and Spanish, were distributed throughout the community. These newsletters included information on asthma-related events occurring in the community, asthma resources, and updates on legislation related to asthma (e.g., inhaler availability in schools). Educational opportunities were targeted to diverse groups such as seniors, grade school children, and teens, as well as to the general public. A translator was available at all events.

More focused training and education was provided to address the project-specific research needs. These training sessions were adapted to the level of learner and were specific to a given project. Both didactic and experiential learning strategies were incorporated to provide the most beneficial training to the learners.

Implementation

A bilingual and bicultural community member was hired as project coordinator to facilitate the participatory process. In many cases, the coordinator provided the education or arranged for experts such as nurses to speak with various groups.

Community members were recruited to participate in the data collection process. Experience with asthma, rather than preexisting skills, drove participation. The background and experience of the participants were diverse. For example, the coordinator had a Master’s in Social Work; she also had asthma herself and had children with asthma. Another interviewer worked in a social service occupation within the community and also had grandchildren with asthma, one of whom died as a result of an exacerbation. Similarly, one of the interviewers ran an in-home day care and was also a foster parent. Many of the children she cared for had asthma.

Evaluation

Process evaluation was used to categorize training activities based on audience, formality of training, individual offering the training, and activities involved. Categories represented the various approaches that were used to address training needs within the community. Logistic and planning limitations, and barriers to current approaches were identified and strategies were offered to improve the training and education component of this community-based research program.

RESULTS

Education and Training Outcomes

The process evaluation identified four categories of training activities: networking, methods training, on-the job training, and community education. Networking was defined as scheduled local and national opportunities that the community members attended to share their work and learn from others. Methods training was training received in a standardized format led by an expert team member and adapted to meet the specific needs of the team. These opportunities were presented according to need. Human subjects training was also included. On-the job training was defined as the opportunity to apply classroom learning in the field. These are actual hands-on experiences to practice the skills that were taught in a more didactic manner. Community education was defined as opportunities open to the community that were educational in nature and built a knowledge base but did not necessarily result in skill building. This was often general knowledge that was presented to large groups. Opportunities presented here reflect a continuum of training offered to the community. Table 1 summarizes the number of community participants in various educational and training opportunities. Table 2 high-lights additional outcomes related to the educational and training opportunities and documents the number of community residents that participated in research opportunities.

Table 1.

Continuum of Educational/Training Opportunities to Build Research Capacity Among Residents of the Lower West Side of Buffalo, New York

	Training Opportunities	Community Participants (n)
Networking	Attendance at national conferences (n= 6)	13
	Attendance at local conferences (n = 1)	1
Methods training	Human subjects education	11
	Qualitative research	2
	Focus group facilitator training	1
	Interviewing techniques	6
	Train the trainer initiatives	1
	Exposure to various skills and expertise	11
On-the-job experience	IRB requirements	11
	Developing, administering surveys	6/6 (100%)
	Conducting, analyzing focus groups	2/2 (100%)
	Formative evaluation	6/6 (100%)
	Preparations of grants, manuscripts	1
	Networking	1
Community education	Speak out	90
	Educational sessions	161
	Committee participation	1

Table 2.

Additional Outcomes Related to Educational/Training Opportunities Provided to Community Residents and Research Participation in the Lower West Side of Buffalo, New York

Outcome	n
Number of students that provided training	9
Number of manuscripts published that involved community residents	2
Number of formal presentations given by community residents	2
Number of committees involving community residents	6
Number of community residents enrolled in an asthma registry	719
Number of community residents that participated in three focus groups	36/719 (5%)
Number of community residents that participated in an asthma survey	204 (28.4%)

Networking

Community members participated in formal training opportunities, including workshops and conferences with specified training objectives. Specifically, the coordinator attended the National Asthma Conference and presented a poster at the U.S. EPA 2005 Community Involvement Conference and Training. The coordinator and one community member attended the WE ACT Northeast Environmental Justice Network meeting. Four community members attended the Environmental Leadership Summit and three attended Dialogues for Improving Research Ethics in Environmental/Public Health. Program findings were presented at the annual NIEHS Awardees meeting, and the local Environmental and Society Colloquium. Selection for participation in networking opportunities was purposeful depending on content of the event.

Methods Training

Community members were trained on project-specific research methods by experts. Opportunities to participate in the collection of data were open to the entire community. Flyers, word of mouth, and advertisements in community newspapers were used to recruit individuals. Interested individuals participated in the training.

A bilingual-bicultural sociocultural anthropologist offered training to facilitate and analyze focus groups. Standardized didactic training was used that was adapted to address the level of learner and the sociocultural nuances. An exercise in analysis was conducted. The trainer participated as an observer during the focus group facilitation and in the analysis to provide feedback to the learners and keep the process on track. A researcher and the coordinator led three focus groups of 36 people on perceptions of pollution and asthma. Two community members participated in the analysis of previously conducted focus groups, contributing to two published manuscripts.^12,13

Six community members received training on survey methodology and interviewing techniques from an epidemiologist. A standardized training manual, adapted to address the cultural nuances of the community as well as the individual project, was used. Role playing was used to teach survey administration, including the tracking of paper-work. A table of contents of the training is included in Appendix A.

Sampling strategies and study protocols were high-lighted. This training prepared community members to enroll 719 families in a community asthma registry and conduct surveys on 204 families. Nine public health and epidemiology students shared expertise on database development and website design with the project coordinator; the community members participating in the data collection activities. A listserv was created for easier communication. Exposure to computer software and hardware was also provided.

WSCAP examined previously collected data (The Lower West Side Health Needs Assessment data³ and the Buffalo Asthma Study^1,6) within the community to raise awareness of the work that was already done and to identify gaps. For example, the community identified a need for school-based programs to address the asthma problem among youth.

Eleven individuals from the community completed the National Institutes of Health’s training for human subjects and have received certification.

On-the-Job Training

Data collectors (n = 6), hired from the community, contributed to the design and implementation of an asthma survey. They critically evaluated proposed protocols and suggested ways to better sample, raise response rates, and make projects more acceptable to the community. This research was conducted with scientific integrity while developing translational strategies. For example, there was high turnover in address and phone numbers in the registry. The interviewers organized families by street, starting with the families they knew. This strategy allowed for effective follow-up and assisted in finding other families. As new families were enrolled in the registry, they were immediately invited to participate in the survey. One hundred percent of the individuals who received methods training also received on-the-job experience.

The coordinator was involved in the preparation of manuscripts, grants, and presentations. One manuscript was published¹³ and one is currently in press.¹² She also played an active role in the preparation of this manuscript and is included as a co-author. Two community members participated in analysis of focus group transcripts that served as the basis for the manuscript in press.¹² Several grants were submitted and progress reports prepared for currently funded initiatives. Posters were prepared and presented at local and national forums.

Involvement in various organizations increased awareness of community activities and approaches being used. Specifically, the coordinator was a board member of the Citizens Environmental Coalition and a general member of the Coalition of Impacted Neighborhoods. These organizations are leaders in advocacy and legislative negotiation. In addition, the coordinator participated on the leadership boards of four community organizations.

Community Education

The most far-reaching efforts to build research capacity were through community education targeted to organizations serving Puerto Rican residents, such as presentations to school staff, teenagers, and nursing home residents. Community education sessions conveyed asthma education and the research process. All sessions were open to all community members in the targeted areas. A teen education night involved 10 teens and 11 parents. One hundred forty staff from Buffalo public schools located in minority communities participated in a school education session.

The coordinator and other community members attended community health fairs and gave presentations at community organizations on behalf of the WSCAP. The coordinator participated in the organization of community “Speak Outs” to gather the community’s perspective of areas of health need, gaps in service, and incorporation of initiatives into community activities. Ninety community residents attended. The “Speak Outs” included a small grant program to address these areas of need. Community members were offered grant writing training and were paired with experts to prepare their proposals.

CONCLUSIONS

Barriers

As has been the case with other community-based participatory research projects, the WSCAP experienced the challenge of balancing desires for activities and immediate results against the longer term and less tangible nature of the research process itself.^7,14,15 This tension was most readily apparent in the project’s evaluation component, where pragmatic choices early on resulted in utilization of a descriptive process evaluation rather than a more quantitative analysis of outcome measures. Being a young program, our focus was on building relationships with the community and conducting research. In essence, the measurement phase of the capacity building component of this work was neglected.

Language proved to be another barrier. Many training opportunities were limited to those who were bilingual because many of the trainers only spoke English. Even though there are more than 20 different languages spoken in the target community, only Spanish and English speakers were able to participate.

In addition, interview activities may have been subject to methodological bias, particularly in the case of sensitive information, owing to the familiar relationships many of the interviewers had with participants. Although interviewers were instructed to follow the script exactly, there was concern about the conversational nature that ensues with ones we know well. Record keeping needed more attention and follow-up. In addition, there was a challenge with confidentiality within the community. As researchers, we view each participant in a confidential nature; however, as a community member, the “everyone knows everyone” phenomenon makes maintenance of confidentiality challenging.

Strengths

The multitiered approach to integrating research training into this Puerto Rican community reached many different factions of the community. Focusing formal training in a small group provided skills that were taken back into the community. This added to the relationship building that continued. Interviewers were trusted and had success in getting people to participate. Training bilingual individuals will allow more training of predominately Spanish-speaking residents. Extending research training to students living in the community may increase the number of minority researchers.

As community residents became involved in the process and were trained in various methodologies, they were better able to inform the research process to become integrated and accepted in the community. This built confidence in themselves and in the research process. The research team also began to better appreciate the value added by community input.

Strategies for Improvement

There is a need to develop measures to evaluate education and training within the community to substantiate the impact community-based research programs have on building research capacity in the community. Process, formative, impact, and outcome evaluation will be incorporated into all training and educational sessions as part of program planning. Community members that participate in providing education or research activities will receive more comprehensive evaluation training and be provided with the tools to conduct a strategic evaluation plan.

Specifically, in future training and education efforts we will develop an evaluation tool specific to each training/educational event that is directly linked with the learning objectives for the event. In addition, baseline information will be collected to assess empowerment and health management skills. A one-page evaluation tool will be developed that can be used across events to determine the impact of the training or educational event on building community research capacity. Follow-up will be conducted among those who participated in data collection to identify whether they continue to be actively using the skills they received. Changes in empowerment will be assessed. In accordance with Wallerstein’s model,¹¹ changes in health management skills will also be assessed to identify the impact that education and training have on health outcomes.

SUMMARY

The WSCAP was able to provide research training and education to the community, meeting our initial expectations. Based on our application of elements of Wallerstein’s model,¹¹ we would expect this to translate into skill building that would, in turn, have an impact on health outcomes, advocacy, and policy. Because of the lack of a complete outcomes evaluation, however, it cannot be formally demonstrated that this was the case. Anecdotally, we have seen the impact on the community. For example, one community member checks in regularly to see if other research opportunities are available. She is now confident of being involved in other initiatives. However, we do not have data to elaborate on such observations.

Evaluation of all components of the community-based process is critical to defining the impact that CBPR has on the community and the co-learning that is part of the process. Developing evaluation protocols and measures to quantify research capacity, educational impacts, and satisfaction with the CBPR process is necessary to moving the methodology forward.

Appendix

Appendix A.

Interviewing Training Manual

Session 1	Brief introduction into the project
	Roles and characteristics of the interviewer
	Break
	Introduction to survey methodology
Session 2	Interviewer techniques and guidelines
	Videos on techniques
Session 3	Detailed description of the project
	Detailed role of the interviewer in the project
	System to be followed by interviewer
	Introduction to the survey instrument
Session 4	Review of the instrument
	Role play
Session 5	Role play of revised instrument
	Last chance for any questions
	Distribution of interviewer materials
	Kick-off of project