Evaluation of a Partnership Approach to Translating Research on Breast Cancer and the Environment

Abstract

Background

The growing literature on community-based participatory research (CBPR) suggests that a participatory approach benefits science in important ways. However there have been few formal evaluations of a CBPR approach itself, and few standards developed to assist in such efforts.

Objectives

This evaluation used CBPR guidelines developed by Green and colleagues to evaluate the participatory approach of the Community Outreach and Translation Core (COTC) of the Bay Area Breast Cancer and the Environment Research Center (BABCERC) in translating scientific findings from two key projects to the public.

Method

To assess key stakeholders’ perceptions of alignment between the projects and the guidelines, four COTC members, four researchers, and four community members rated the projects on each of the 26 guidelines. These data were triangulated with transcripts from interviews with the same participants and a focus group with a subset of the participants.

Results

The participatory approach by the COTC resulted in many important benefits including improved relationships among diverse stakeholders, knowledge generation, increased sensitivity and propriety of the research, and increased community support of research. However, several atypical features of this collaboration—for example, the basic and etiological nature of the science being undertaken, and the multiple communities (lay and activist/advocate) involved—resulted in different levels and qualities of participation among stakeholders.

Conclusions

Further research should focus on the adaptation of participatory research principles for different kinds of community partners and on the development and refinement of standards and tools to assist in evaluating the process and outcome of participatory research.

The COTC of the BABCERC¹ used principles of CBPR to influence two research projects designed to improve understanding of the role of environmental factors in pubertal development, as a window on breast cancer etiology. Project I was a basic research study to explore mammary gland development in animals to determine vulnerability to environmental agents in the prepubertal period that may influence breast cancer development in adulthood. Project II was a longitudinal, epidemiological study to assess the influence of the environment on the onset of puberty in young girls.

The literature suggests that a CBPR approach benefits the translation of such research by fostering co-learning, ensuring projects are community driven, increasing trust between the community and researchers, and increasing the local relevance of the findings.^2–6 Nevertheless, the literature provides few standards for assessing a participatory approach,⁷ few formal evaluations of a CBPR approach itself,^8–10 and few data about the adoption and effectiveness of CBPR within the context of a multisite, multidisciplinary center. Our objective was to evaluate the extent to which the approach used by the COTC was participatory, and to ascertain the benefits and challenges of the participatory aspects of the project as perceived variously by community, advocacy, and scientific partners. This evaluation used CBPR guidelines developed by Green and colleagues^11–14 to assess key stakeholders’ perceptions of alignment between a planned CBPR effort and the guidelines. Questions guiding this evaluation were as follows.

1. To what extent was the translation process used by the COTC consistent with the participatory research guidelines?

2. What facilitated the participatory research process that should be retained for future endeavors?

3. What hindered the participatory research process that should be improved upon or eliminated from future endeavors?

METHODS

Creation of the BABCERC

In 2002, the director of the NIEHS heard community concerns about possible environmental causes of high breast cancer rates at a town hall meeting held by Zero Breast Cancer, a community-based organization in Marin County California.¹ At this meeting, he announced a Request for Applications (RFA) for breast cancer and the environment research centers to foster interdisciplinary research on this topic; the RFA required two types of projects, one using animal models to characterize pathways related to breast and endocrine system development over the life course and a second epidemiological study of the determinants of puberty in girls. Each center was required to create a COTC to ensure community input into the research. Local researchers from the University of California—San Francisco Comprehensive Cancer Center, Lawrence Berkeley National Laboratory, and Kaiser Permanente partnered with community members from Zero Breast Cancer to respond to the RFA with a proposal to create the BABCERC with a COTC component. They were awarded this grant in 2003, establishing the BABCERC.

Description of the BABCERC Research Projects and Selection

Project I employed mouse models to study the impact of environmental stressors on breast cancer and elucidate the effects of timing of these exposures during critical windows of vulnerability in breast gland development. Project II was a research study following 400 seven-year-old girls for at least 5 years as they transitioned through puberty to better understand the ways young girls mature and how that is affected by environmental, nutrition, activity, and developmental factors. Decisions as to the exact nature and design of the scientific projects were made by the scientists based on their expertise and experience and what they thought would be most competitive.

The COTC of the BABCERC and Its Target Communities

The purpose of the COTC was to develop, implement, and evaluate strategies to ensure ongoing community input into the center’s research, the translation of the center’s scientific findings for the public and policy makers, and the development and communication of “key messages” based on research findings and precautionary principles. Translation focuses on increasing public understanding of the science and on translating the research findings into prevention strategies, public health practices, and public policy.¹⁵ To fulfill its translation-related responsibilities, the COTC adopted a CBPR approach and a COTC member was assigned a co-investigator role on each of the two scientific projects (I and II). Other translation strategies included the development of educational materials, publications, and presentations at scientific and public meetings, resource/information tables at local events, and multiple town hall meetings, discussions, and/or focus groups where researchers and the general public have come together.

The membership of the COTC consisted of three representatives from each of three county departments of public health (Alameda, Marin, and San Francisco), three representatives from community-based organizations, and three breast cancer survivors. Members of the COTC were selected by Zero Breast Cancer. The COTC served as the designated liaison between the community and researchers. Defined broadly, the target community of the BABCERC included individuals who shared concern that the high incidence of breast cancer in the San Francisco Bay Area was due to environmental factors. Diverse community members lived in Marin, Alameda, and San Francisco counties, with a focus on Bayview Hunters Point, a predominantly African-American, low-income community in San Francisco. This area was targeted because of high rates of industrial pollution and rates of breast cancer almost double those in other parts of the Bay Area.^16,17

Overview of the Evaluation of the Participatory Approach of the COTC

Institutional review board approval for this evaluation was obtained from a private board not affiliated with any stake-holders. This was an independent evaluation conducted by the first three authors of this paper. The evaluation included three components: (1) a quantitative rating of CBPR experiences, (2) interviews, and (3) a focus group. In consultation with the BABCERC Executive Committee, composed of the BABCERC Principal Investigator and the Chair of the COTC, the evaluation team, twelve people representing three key stakeholder groups (community members, COTC, and researchers) were selected to participate in the first two components of the evaluation. A subset of respondents, including people who had and had not been part of the ratings and interviews, was recruited to participate in the focus group.

METHODS

We developed a quantitative rating form (Appendix 1) that drew on a set of previously validated CBPR criteria, ratings and guidelines.^11–13 The guidelines were adapted in consultation with the BABCERC Executive Committee to provide a better fit with the purpose and circumstances of the COTC. A list of twenty-four criteria-based questions covered four areas of participatory research, namely, participants’ involvement, shaping the purpose and scope of the BABCERC Projects, research implementation and context, and interpreting the research outcomes.

Each respondent could rate each of the twenty-four guidelines as high, medium, or low. A “high” rating equated with a positive perception of the participatory process, whereas a “medium” rating reflected a slightly less positive perception of the participatory process, and a “low” rating reflected the least positive perception of the participatory process.¹⁴ For each question, a participant could write in “not sure” and provide an explanation.

The evaluation team sent the rating form to the twelve interview participants a few weeks in advance of the interview and asked them to complete the rating form and bring it to the interview. Our interview guide was developed based on guiding principles of CBPR and to further explore the four areas of participatory research covered by the rating form.

The evaluation team conducted twelve interviews and asked respondents to share any concerns or questions raised by the rating form as appropriate during the interview. For example, several respondents brought up concerns about the question of agreements, which were then further explored during the interview. The in-depth interview sought to assess ways in which the approach used by the COTC was consistent with the participatory research guidelines, identify the facilitators of and barriers to the participatory research process, and explain and explore the quantitative findings about guideline alignment.

The evaluation team conducted a focus group with six respondents (two from each of three stakeholder groups) to validate key findings from the interviews and permit new perspectives to emerge. For example, the focus group further examined interview findings that revealed divergent views among stakeholders of the benefits of science and the participatory process; it also sought to explore new ideas for solving some of the challenges of the participatory process. To achieve both objectives, those invited to the focus group included both people who had and had not been part of the ratings and interviews. In advance of the focus group, participants were asked to review preliminary evaluation findings; during the focus group, they were asked to share their reactions to and interpretations of the findings. The BABCERC Executive Committee met regularly with the evaluation team to provide input on the evaluation, including the design of the instruments, recruitment of respondents, and interpretation of major findings.

Analysis

The evaluation team reviewed the quantitative results to determine frequencies of responses by type of stakeholder and the extent to which respondents rated the project as more or less participatory. Interviews were analyzed according to a realist approach¹⁸ and guiding principles of CBPR.¹⁹ This included assigning codes to meaningful segments of transcript text and recording memos to help make sense of the data and facilitate more abstract development of theories about the data. Themes that repeatedly emerged in interviews and those emphasized by the respondent were identified as “important.” Resulting themes cut across evaluation criteria. Data quality was ensured through a continual process of member checking and triangulation.^20,21 Member checking occurred at each phase of the evaluation, with the evaluation team presenting initial findings to the BABCERC Executive Committee for comment, clarification, and further questions. Findings were validated through triangulation from multiple sources, including the quantitative rating form, the interviews, interview notes/summaries, the focus group, and the COTC, community members, and researchers.

RESULTS

Quantitative Findings

Alignment of the Project With Principles of Participatory Research

Nine of the twelve interview respondents returned the quantitative rating form. The majority of participatory research criteria and guidelines received a high rating by respondents, reflecting a net positive perception of the participatory processes among respondents. Of 216 possible responses, 40% (87/216) indicated high alignment with the principles of CBPR; 25% (53/216) of responses indicated moderate or medium alignment, 16% (34/216) of responses indicated low alignment with the principles of CBPR; and 19% (40/216) of responses indicated “not sure” or “no answer” (Table 1). A fraction of the responses had an explanation; several were uncertain about whether agreements existed and some said they did not understand the question. Among the four areas, participants showed the most positive perceptions on questions related to the areas of “Participants and the Nature of their Involvement” and “Shaping the Purpose and Scope of the BABCERC.” For example, most participants felt that the Project I and II research questions were developed through a collaborative process. Greater variance showed in responses in the areas “Research Implementation and Context” and “Nature of the Research Outcomes.”

Table 1.

Quantitative Results from Adapted Participatory Research Guidelines

	Alignment With Tenets of Participatory Research
Question Reflecting Participatory Research Guidelines	Low	Moderate	High	No Answer/ NA
Participants and the Nature of Their Involvement
Are BABCERC’s intended users of research described in a way sufficient to assess their   representation in the project?		2	5	2
Is the mix of respondents included in research process sufficient to address needs of the   project’s intended users?	1	1	5	2
Is effort made to address barriers that might limit participation of underrepresented   intended users in the research process?	1	3	4	1
Has provision been made to build trust between researchers and other intended users?		4	5
Do the researchers and other intended users participating in the research process have   an explicit agreement (verbal or written) regarding project management?	1		4	4
Shaping the Purpose and Scope of the BABCERC
Were the research question(s) developed through a collaborative process between   researchers and other intended users?	2		6	1
Has BABCERC applied the knowledge and experience of intended users in   conceptualizing and/or designing the research?	1	1	6	1
Does BABCERC provide for mutual learning among intended users and researchers?		4	4	1
Has BABCERC planned to build the capacity of intended users to address multiple   factors that contribute to breast cancer risk?	2	2	3	2
Research Implementation and Context
Does BABCERC apply the knowledge and experience of intended users in   implementing project activities?		3	5	1
Does BABCERC provide intended users participating in the research process with   opportunity to learn about research methods?	1	2	5	1
Does BABCERC provide researchers with opportunity to learn about COTC and   community members’ perspectives on the issue being studied?	1	4	3	1
Do the researchers and other intended users participating in the research process have   an explicit agreement regarding mutual decision making about potential changes in   research methods or focus?	4		2	3
Does BABCERC provide intended users with opportunity to participate in planning and   executing the data collection?		6	2	1
Does BABCERC provide intended users to participate in planning or executing the   analysis?	3	2	3	1
Does BABCERC provide intended users with opportunity to participate in the   interpretation of the research findings?	1	2	3	3
Nature of the Research Outcomes
Does BABCERC reflect sufficient commitment by researchers and other intended users   participating in the research process to action?	2	4	1	2
Do the researchers and other intended users engaged in the research process have   an explicit agreement for acknowledging and resolving in a fair and open way any   differences in the interpretation of research results?	5		2	2
Do the researchers and other intended users engaged in the research process have an   explicit agreement regarding ownership and sharing of the research data?	1	2	3	3
Do the researchers and other intended users engaged in the research process have an   explicit agreement regarding feedback of research results to intended users?	1	1	4	3
Do the researchers and other intended users engaged in the research process have an   explicit agreement regarding the dissemination of research findings?	1	4	2	2
Does BABCERC provide intended users with opportunity to participate in   dissemination of project findings to other intended users and researchers?	1	1	5	1
Is the provision for assistance to intended users sufficient to indicate a high probability   of research results being applied?	2	2	3	1
Does BABCERC plan for sustainability in relation to the purpose of the research?	3	3	2	1

Qualitative Findings: Facilitators of and Barriers to Community Participation

Project Structure as Facilitator and Barrier

Respondents cited project structure as both a facilitator of and a barrier to community participation. Several discussed the active participation of (and funding for) COTC members on each of the research project teams as evidence of COTC (hence community) integration into the research and oversight of the project. The COTC representative brought concerns of diverse constituencies to researchers, participated in the research process, and shared scientific updates with community members.

The … structure is, up front, explicitly designed to involve COTC members in an ongoing way. So in all the working conference calls and meetings and town halls, we hear from … members of the COTC on a regular basis, and they hear about efforts to pull off the science and see it in all its glory and ugliness, you know, and difficulties.

Integration of a COTC member into the study teams allowed for the community to provide input into the research. Several researchers and COTC members shared concrete examples of how the COTC and other community members had been involved in shaping the research, such as helping to identify environmental exposures to be included in Project II plans for analysis of the biospecimens. For example, a COTC member said:

So that’s where that information from the different communities came into play, where I was able to say [to the project team], “Here are the environmental exposures that are of the top concerns, the top priorities for these communities. … So I was really active in making sure that the top concerns of the communities were actually part of the discussion when planning was taking place for the analysis of the specimens.

Many respondents spoke about the important role of annual town hall meetings in accomplishing the goals of translation and dissemination in this project. These meetings brought community members and researchers together, raised community members’ awareness about the scientific objectives and progress of the BABCERC’s research, and provided opportunities for community members to share their concerns. Town hall meetings provided opportunities for researchers—with assistance from the COTC—to communicate findings in ways the lay public could understand. For example, there was a joint effort between the COTC and Project I scientists to make a video (“Of Mice and Women”) to improve the public’s understanding about how and why mouse models are needed to study breast cancer. This video was mentioned by several respondents as a key accomplishment toward more effective translation of the research. The process of making this video helped the COTC to understand the science while teaching the scientists how to communicate more effectively with the public.

Some aspects of the project structure, such as budget and timeline, limited full engagement of the community in the research. For example, although there were opportunities for the COTC to attend researcher meetings, there were no similar opportunities for researchers to attend COTC meetings. One participant alluded to the complexities of data collection, particularly when time is short:

There’s also time constraints as there are always with data. So … personally I feel like there’s less opportunity to add things to the survey … although I think that the feedback [from community] is always sought out, whether it will happen or not is questionable.

Lack of Explicit Agreements Regarding Stakeholder Roles

Few, if any, written agreements clarified stakeholder roles in research design, implementation, interpretation, translation, and dissemination. Several respondents indicated that the lack of a clear agreement about how dissemination of project findings would happen creates uncertainty about who will be involved, how they will be involved, and what findings will be shared. One respondent, however, expressed concern that Projects I and II were not intended to adhere to prescribed CBPR standards; therefore, such agreements were neither expected nor necessary and an inappropriate criterion for this evaluation.

Several respondents mentioned one written agreement among partners—a protocol for the selection of topics for analysis, publication and authorship. One respondent described this agreement:

The Publications Committee was formed pretty much close to the beginning of the center’s project in an attempt to provide a frame-work and a set of protocols and policies for how things will get published, and to provide a framework for the process flow. And to set some standards for authorship.

This document was developed over a year of discussion with multiple drafts, but not everyone shared the same perception of either the document or its benefits. For example, some respondents felt that comparable, existing agreements had not been implemented to meet expectations nor had they created a clear mechanism for how things would happen (e.g., how people were invited to participate and how authorship was shared).

Nature and Stage of Research Influence Community Involvement

Differences in the nature of the two scientific projects also contributed to variable community involvement. Project I, a basic science project, was funded to answer specific questions and to conform to a required research design, so opportunities for meaningful community input into project design were not really possible. Nevertheless, at least one non-research respondent expected to have more meaningful input into Project I than merely participating as a curious audience. From the perspective of one researcher, however, allowing someone with limited scientific expertise to influence basic research might threaten the integrity of the science.

I think participatory science is really important for the community members to identify areas that they’re concerned about and they want scientific data on. But they have to realize the scientific process is restricted for good reason in order to get … the best possible data you can.

Opportunities to influence the research (e.g., questionnaire design) were available in Project II. The need to collect the same data at subsequent time points, however, meant that the core baseline questionnaire in Project II could not be modified substantially in annual follow-up surveys.

Stakeholder Skills, Priorities, and Needs Tied to Level of Participation

Tied to the stage of research are skills need to participate in it. In the above quote, the researcher alludes to the need for advanced training so that participants can understand and contribute, particularly to basic science. Lack of such training led to the challenge of developing equitable partnerships when community members or their representatives do not have the skills or knowledge needed to participate meaningfully in the research. For example, a COTC member said,

if the COTC came up with a question that they wanted to look at the data … I feel that we would not have difficulty doing that. The difficulty with the COTC is that we don’t know how to do that.

Other respondents felt that not having research capacity might limit the ability of community members to act as full partners, for example, participating in authorship, translation, and dissemination.

Respondents discussed the challenge of having partners with different priorities and needs. For example, although researchers have publication demands, activists/advocates are issue focused and want to take action. Research does not always satisfy this need. A COTC member said,

The challenge is all research takes a long time, and advocates are really impatient. … And I think another challenge is not everybody grasps this concept the same way. It’s just going to be different levels of willingness on the part of scientists … to truly accept or embrace the philosophy of working with the advocates and working in a sort of a collective way. It’s not easy to do that, I’m sure. And the challenges are that people’s feelings sometimes get hurt, and it can get ugly, I guess because people can say things. But I do think the benefits outweigh the challenges.

Some respondents noted difficulty involving community members in research that may be perceived as not truly benefiting the whole community. For example, one community member shared the perception that researchers need to be more understanding of and responsive to the needs of low-income people of color and the uninsured. Respondents also poignantly illustrated the challenge inherent in being involved in research when basic needs, such as health care, were not being met. In addition, community members might not always make the distinction between a researcher and a health care provider, particularly when the researchers come from an institution, such as the University of California at San Francisco, also best known in the community for providing quality medical care.

Communication Challenges the Participatory Process

Communicating across stakeholder groups challenges CBPR, particularly when motivations for being involved vary. For example, community members were generally interested in the practical (How can I protect myself and my family? How can I prevent breast cancer?) and did not always see the value of basic research, particularly research that represents an early step toward advancing knowledge about breast cancer. Researchers want to present their research and get directly relevant feedback. One respondent summed it up in this way:

I think people get into their little silos and get resentful that people are asking them questions that aren’t related to whatever they’re presenting, whether it’s biology or whether it’s activism. It happens on both sides of the coin. You hear the researchers grumbling about, “Why are we talking about this activist issue again?” And you hear the folks on the other side grumbling about, “Why can’t I understand anything that’s coming out of this person’s mouth?” You know?

Communication from within these different perspectives could result in resentments, frustrations, and communication challenges across these perspectives, which in this study sometimes played itself out at town hall meetings. The dispersion of stakeholders across a wide geographic area, such as the three counties in this study, also made communication across boundaries more difficult.

Lack of Trust Hinders Participation

The challenge of overcoming community members’ (particularly those in the African-American community) mistrust of research emerged as a theme. One community member said,

People don’t really understand the true needs of low-income people of color and uninsured. … People need to be involved and really and truly understand the needs. Because the thing is if you’re talking about breast cancer and then you have a person of color, uninsured, and, low-income who has no resources and then they go to try to access some help and then they get turned away, then it is very difficult for them to say. I’ll be a part of something because you know … when they needed you, you weren’t there.

Several respondents discussed the legacy of having been repeatedly used by researchers for their own professional gain without ensuring that community members benefit from the research.

Community-Level Benefits of Participatory Research

Several important perceived benefits of community participatory research emerged from the analysis (Table 2).

Table 2.

Benefits of Participatory Research

Benefits of Participatory Research	Illustrative Quotes
Improved Communication and Sharing of Knowledge Between Stakeholders	I think that one thing I’ve learned from the community members is, no matter what I say, somebody’s going to ask me about their breast cancer. . . . And so I think I’ve learned how to make a better—a broader talk to try to encompass the kind of things people are interested in hearing and to keep them engaged. So I think that’s been very useful. It helps me to try to understand how to convey science approaches. And I think the COTC has worked really hard to make that useful to a broad group of people, and I think that’s been very useful, learning about how you convey science, what to convey and how to speak to community groups.
Enriches Data and Contributes to Knowledge	In our particular site, the COTC was particularly interested in potentially doing geo coding and looking at other environmental factors as a result, and so we incorporated residential and school and daycare history in our [epidemiological] project, which was subsequently adopted by the other two centers. And we probably would not have incorporated that to study, initially, if there hadn’t been a strong feeling about that. . . . That’s a direct benefit from that involvement that we really would have missed.
Strengthens Relationships	The researchers trust community members to be there to support them. I think they have a better understanding of the fact that we [members of the COTC] can understand the work they’re doing.
Increased Sensitivity and Propriety of Research	I think the community members and the COTC members, when they’re hearing about what we’re planning, have always raised questions. You know, “What’s Mother going to think?” Or, “Is this appropriate for a 7-year-old girl?” It’s the same kind of questions a human subjects committee would ask or IRB would ask but, in this case, it’s coming from the community itself, so they’re very legitimate questions and we need to be able to answer them.
Community Becomes More Supportive of Research	I think research, just like doing clinical trials, if you get people to buy into it, you can probably have more people participating and try to work on us finding a cure. . . . The thing is that if you ever get someone to buy into something, then they feel a part of it. It’s a win–win situation.

Improved Communication and Sharing of Knowledge Among Stakeholders

Nearly all respondents spoke about improved communication and mutual learning among the stakeholders. Researchers gained a greater understanding of community concerns and ways to be more responsive to community needs. In this study, researchers gained an appreciation for the knowledge of the literature, expertise, and connection to the community that these stakeholders share. Advocates and activists learned how to creatively communicate the research to the public. One respondent described the good faith effort by some of the scientists to make their work more understand-able to the lay person by using less scientific jargon, analogies, and audiovisual aides.

Enriches Data and Contributes to Knowledge

Participatory research purportedly enriched the questions asked and interpretations of data, and made the research more responsive to community needs. Examples from this evaluation include the research questions, instruments, plans for analysis of biospecimens, and sharing of findings with the participating community. For example, in Project II, the biospecimen analysis included environmental exposures that were of interest and concern to the community, but also reasonable to measure, given the objectives of the science. In addition, the participatory approach itself contributes to identifying beneficial ways to engage community members in the conduct of research.

Strengthens Relationships

The participatory approach strengthened relationships among stakeholders and created trust, communication, and understanding of stakeholder stance. Broadening the perspective of different stakeholders was cited as an important benefit of participatory research.

Increased Sensitivity and Propriety of Research

Supporting the claims of the CBPR literature, the evaluation found benefits of community involvement that increased sensitivity and propriety of the research effort. Examples included how community members raised awareness among researchers about potential ethical concerns at various organized forums, such as town hall meetings.

Community Becomes More Supportive of Research

Involving community members in research, according to many respondents, helped to overcome community mistrust of research and increased community members’ willingness to participate in research, particularly when they felt that academic partners were genuinely interested in giving back to the community.

DISCUSSION

This evaluation identified important ways the BABCERC and its partners benefited from and grappled with a collaborative approach. Integrating the COTC into the research resulted in benefits for Project II, the epidemiological study, but challenges for Project I, the basic science project. COTC Participation in Project I was primarily restricted to dissemination and translation, via the creation of educational materials and lay abstracts of publications, because providing meaningful input into the science itself would require years of advanced training. However, findings suggest that understanding what “participation” means in the context of this type of project may have differed among stakeholders. In contrast, Project II—an epidemiological investigation—allowed for levels of participation more generally regarded as appropriate in participatory research. Community members provided input into the design of the questionnaire and helped to identify compounds that should be included in the analysis of biospecimens. Some project participants clearly felt that the participatory process was hampered by the lack of clear agreements about how participation and collaboration in different areas (including publication and dissemination) would unfold. Although there were communication challenges related to different motivations and values of the stakeholders involved, there were some notable successes in bridging gaps in knowledge and communication among scientists, advocates, and lay community members. Researchers’ understanding of community perspectives and concerns improved and the public gained a more sophisticated appreciation for and understanding of the scientific process and its benefits.

A number of benefits of implementing a CBPR approach have been reported in the literature, including increased community capacity, increased relevance of data, increased trust, translation of research into policy, and emergence of new research questions.^2,5,22–32 This study reinforced some of these benefits, particularly the increased relevance of data in the epidemiological study, strengthened relationships among stakeholders, and improved translation and dissemination to the public.

Some challenges in this evaluation revolved around the atypical nature of the CBPR collaboration. First, levels of participation in basic science and etiological (“discovery”) research may differ from participation in most intervention research projects employing a participatory approach. Others involved in a CBPR clinical outcomes project found that, although the CBPR approach initially benefited both teams, divergent principles and methods eventually threatened the integrity of the research conducted.³³ This evaluation suggests that stakeholders should openly communicate their expectations about how a project will unfold and what levels of participation, and by whom, are appropriate before embarking on a project. As was indicated in this evaluation, clear agreements about roles and responsibilities may help stakeholders to negotiate participation. However, other successful CBPR projects have developed without such written agreements and their organizers argue that they can cause the group to get “bogged down” in legalistic language that may prohibit open dialog involving all partners and the development of broader agreed upon principles and procedures.³⁴

Second, atypical also was the involvement of two “communities”: the “lay” community and the breast cancer advocate and activist community (COTC members). Whereas the former had a varied level of understanding and interest in the issues, the latter had a more sophisticated understanding of research and was highly experienced in translating science into effective advocacy. The different interests and roles of these communities in this project and the different levels of understanding of research and the scientific process might also call for further adaptation of participatory research principles for different kinds of community partners, as well as for different kinds of research. As with all partnerships where stakeholders have varied levels of expertise, interests, and different agendas regarding the topic of interest, the involvement of all stakeholders in planning how they will work together and negotiate differences in their principles and methods is critically important to the success of the partnership.

This evaluation had several limitations. First, the quantitative instrument administered to participants was originally designed to evaluate the extent to which participatory research proposals and funded projects aligned with the tenets of participatory research. The instrument used in this study may not have been sufficiently adapted to a project intending to use a participatory approach in the translation of research findings. Second, the study was cross-sectional and thus only provided an understanding of how stakeholders viewed the participatory process up to the point of the evaluation. Third, the sample of participants was small, although they were representative of stakeholders involved. The low response of community members to the quantitative survey may have led to some underrepresentation and biased reflection of the community perspective. Fourth, the findings reflect the perceptions of participants in the evaluation, and there was no attempt to assess the relative veracity of participants’ perceptions. Finally, because of the unique context and structure of this project, findings may not be generalizable to other participatory research projects.

Despite these limitations, it is clear that the participatory approach applied to this project facilitated the translation efforts of the COTC. CBPR has flourished over the past 15 years, but the benefits for health above and beyond traditional research remain understudied. More work is needed in the development and adaptation of tools for evaluating the CBPR approach in “discovery” research, which may present unique challenges as well as potential for significant benefits to the scientific process and its outcomes. Findings from this evaluation will assist stakeholders in the partnership negotiate roles and responsibilities for the translation and dissemination of the science to the public. Upon completion of the project, stakeholders hope to conduct another evaluation to determine whether the CBPR approach enhanced research translation and improved research outcomes. Future research should focus on how partnership approaches can benefit basic science and epidemiological projects, and on negotiating appropriate roles and expectations for different stakeholders.

CONCLUSION

The participation of community advocates in the research resulted in improved communication between the different stakeholder groups, as well as increased sensitivity to each others’ perspectives. The science benefited from community input via community advocates who brought key community concerns to the attention of researchers. Researchers learned how to communicate more effectively their science to the public, thus increasing the likelihood that their findings will improve community health.

Appendix. Quantitative Rating Form: Adapted Community-Based Participatory Research Guidelines

Evaluation of the Participatory Research Activities of the BABCERC

General Instructions

Below are questions to help assess how the BABCERC project meets the expectations for participatory research. Community-based participatory research (CBPR) is a “collaborative approach to research that equitably involves all partners in the research process and recognizes the unique strengths that each brings.” Please draw from your own experience with the BABCERC project to evaluate the extent to which you believe this project has met the expectations of a CBPR project. For each question, check only one box. If you feel you have insufficient information to know how to answer a specific question, please write in “not sure”and a brief explanation. You may use the additional space provided after each question to further explain your responses. There are no right or wrong answers; we are interested in your honest reflections based on your experience with this project. Please complete this questionnaire and bring it with you to your interview. On behalf of the BABCERC Executive Committee, thank you very much for your time and participation in the evaluation of this project.

1. Participants and the Nature of Their Involvement

   • 1aAre the BABCERC’s intended users (i.e., COTC members, researchers and community members)^* of the research described in a way sufficient to assess their representation in the project?

     • no description or description provides minimal help in assessing representation [or there are no plans to include intended users in the research process]

     • description provides partial but not sufficient help in assessing representation

     • description is sufficient for assessing representation

   • 1bIs the mix of participants included in the research process sufficient to address the needs of the project’s intended users?

     • the mix suggests that the research will not address or will minimally address the needs of the intended users[or there are no plans to include intended users in the research process]

     • the mix suggests that the research will partially but not sufficiently address the needs of the intended users

     • the mix suggests that the research will sufficiently address the needs of the intended users

   • 1cIs effort made to address barriers that might limit participation of under-represented intended users in the research process?

     • no or minimal effort to address barriers that might limit their participation [or there are no plans to include intended users in the research process]

     • partial but not sufficient effort to address barriers that might limit their participation

     • sufficient effort to address barriers that might limit their participation

   • 1dHas provision been made to build trust among between researchers and other intended users participating in the research process?

     • no or minimal provision has been made [or there are no plans to include intended users in the research process]

     • moderate provision has been made

     • substantial provision has been made

   • 1eDo the researchers and other intended users participating in the research process have an explicit agreement (verbal or written) regarding management of the project?

     • there is no mention of an explicit agreement or of plans to develop an explicit agreement [or there are no plans to include intended users in the research process]

     • there are plans to develop an explicit agreement

     • an explicit agreement has been developed

2. Shaping the Purpose and Scope of the BABCERC

   • 2aWere the research question(s) developed through a collaborative process between researchers and other intended users?

     • research question(s) was (were) developed mostly or entirely by the researchers with no or minimal contributions from the intended users [or development of the research question(s) is not discussed]

     • research question(s) was (were) developed mostly or entirely by the intended users with no or minimal contributions from the researchers

     • both researchers and other intended users made relatively substantial contributions to development of the research question(s)

   • 2bHas BABCERC applied the knowledge and experience of intended users in conceptualizing and/or designing the research?

     • knowledge and experience of intended users has not been applied or has been minimally applied

     • knowledge and experience of intended users has been partially but not sufficiently applied

     • knowledge and experience of intended users has been sufficiently applied

   • 2cDoes BABCERC provide for mutual learning among intended users and researchers?

     • no or minimal provision for mutual learning

     • moderate provision for mutual learning

     • substantial provision for mutual learning

   • 2dHas BABCERC planned to build the capacity of intended users to address multiple factors (including things such as genetics, lifestyle and environmental exposure) that contribute to breast cancer risk?

     • no or minimal plans to build capacity

     • moderate plans to build capacity

     • substantial plans to build capacity

3. Research Implementation and Context

   • 3aDoes BABCERC apply the knowledge and experience of intended users in implementing different project activities?

     • no or minimal application of knowledge and experience of intended users

     • moderate application of knowledge and experience of intended users

     • substantial application of knowledge and experience of intended users

   • 3bDoes BABCERC provide intended users participating in the research process with opportunity to learn about research methods (whether or not the intended users choose to take that opportunity)?

     • no or minimal opportunity to learn about research methods [or there are no plans to include intended users in the research process]

     • moderate opportunity to learn about research methods

     • substantial opportunity to learn about research methods

   • 3cDoes BABCERC provide researchers with opportunity to learn about COTC and community members’ perspectives on the issue(s) being studied?

     • no or minimal opportunity to learn about user perspectives

     • moderate opportunity to learn about user perspectives

     • substantial opportunity to learn about user perspectives

   • 3dDo the researchers and other intended users participating in the research process have an explicit agreement (verbal or written) regarding mutual decision-making about potential changes in research methods or focus?

     • there is no mention of an explicit agreement or of plans to develop an explicit agreement [or there are no plans to include intended users in the research process]

     • there are plans to develop an explicit agreement

     • an explicit agreement has been developed

   • 3eDoes BABCERC provide intended users with opportunity to participate in planning and executing the data collection (whether or not the intended users choose to take that opportunity)?

     • no or minimal opportunity to participate

     • moderate opportunity to participate

     • substantial opportunity to participate

   • 3fDoes BABCERC provide intended users with opportunity to participate in planning and/or executing the analysis (whether or not the intended users choose to take that opportunity)?

     • no or minimal opportunity to participate

     • moderate opportunity to participate

     • substantial opportunity to participate

   • 3gDoes BABCERC provide intended users with opportunity to participate in the interpretation of the research findings (whether or not the intended users choose to take that opportunity)?

     • no or minimal opportunity to participate

     • moderate opportunity to participate

     • substantial opportunity to participate

4. Nature of the Research Outcomes

   • 4aDoes BABCERC reflect sufficient commitment by researchers and other intended users participating in the research process to action?

     • no or minimal commitment to action by both researchers and intended users; or partial commitment by one and no or minimal commitment by the other [or there are no plans to include intended users in the research process]

     • partial but not sufficient commitment to action by both researchers and intended users; or sufficient commitment to action by one and partial, minimal, or no commitment by the other

     • sufficient commitment to action by both researchers and intended users

   • 4bDo the researchers and other intended users engaged in the research process have an explicit agreement (verbal or written) for acknowledging and resolving in a fair and open way any differences in the interpretation of research results?

     • there is no mention of an explicit agreement or of plans to develop an explicit agreement [or there are no plans to include intended users in the research process]

     • there are plans to develop an explicit agreement

     • an explicit agreement has been developed

   • 4cDo the researchers and other intended users engaged in the research process have an explicit agreement (verbal or written) regarding ownership and sharing of the research data?

     • there is no mention of an explicit agreement or of plans to develop an explicit agreement [or there are no plans to include intended users in the research process]

     • there are plans to develop an explicit agreement for ownership and sharing of data, or an explicit agreement has been developed regarding one of ownership or sharing of data but not both

     • an explicit agreement has been developed regarding both ownership and sharing of data

   • 4dDo the researchers and other intended users engaged in the research process have an explicit agreement (verbal or written) regarding feedback of research results to intended users?

     • there is no mention of an explicit agreement or of plans to develop an explicit agreement [or there are no plans to include intended users in the research process]

     • there are plans to develop an explicit agreement

     • an explicit agreement has been developed

   • 4eDo the researchers and other intended users engaged in the research process have an explicit agreement (verbal or written) regarding the dissemination (and/or translation or transfer) of research findings?

     • there is no mention of an explicit agreement or of plans to develop an explicit agreement [or there are no plans to include intended users in the research process]

     • there are plans to develop an explicit agreement

     • an explicit agreement has been developed

   • 4fDoes BABCERC provide intended users with opportunity to participate in dissemination of project findings to other intended users and researchers (whether or not the intended users choose to take that opportunity)?

     • no or minimal opportunity to participate in dissemination to other intended users and researchers; or moderate opportunity to participate in dissemination to either intended users or researchers and no or minimal opportunity to participate in dissemination to the other

     • moderate opportunity to participate in dissemination to other intended users and researchers; or substantial opportunity to participate in dissemination to either intended users or researchers and moderate, minimal, or no opportunity to participate in dissemination to the other

     • substantial opportunity to participate in dissemination to other intended users and researchers

   • 4gIs the provision for assistance (for example, in communicating with policy makers) to intended users sufficient to indicate a high probability of research results being applied?

     • no or minimal provision for assistance has been made

     • partial but not sufficient provision for assistance has been made

     • sufficient provision for assistance has been made

   • 4hDoes BABCERC plan for sustainability in relation to the purpose of the research (e.g., by fostering collaboration between intended users and resource providers, funding sources, policy makers, holders of community assets, etc.)?

     • no or minimal plans for sustainability

     • moderate plans for sustainability

     • substantial plans for sustainability