Piloting interventions within a community-based participatory research framework: Lessons learned from the Healthy Environments Partnership

Abstract

Background

Community-based participatory research partnerships implementing pilot interventions experience unique opportunities and dilemmas.

Objectives

We describe challenges and opportunities associated with conducting a pilot intervention within a longstanding CBPR partnership, lessons learned for use of a participatory process to pilot community interventions, and recommendations to funders on mechanisms for funding pilot interventions to help address these challenges.

Methods

We conducted key informant interviews and convened a group discussion with host organization leaders and project personnel.

Lessons Learned

Findings highlight: opportunities and challenges related to needs and desires of community constituents and the ability of pilot interventions to meet those needs, and the importance of ongoing communication to address anticipated and unanticipated challenges that arise in the context of short-term pilot interventions in community settings.

Conclusions

We suggest consideration of several funding mechanisms for supporting the implementation of larger scale interventions following promising pilot efforts in community settings.

Introduction

Increasingly, researchers, policymakers, community members, and funding organizations have recognized the contributions of participatory approaches to research efforts that aim to investigate and ultimately eliminate inequities in health associated with social, environmental, and structural factors (1–5). In particular, community-based participatory research (CBPR) is a partnership approach to research that equitably involves all partners throughout the research process and in which all partners contribute expertise and share decision making and ownership (6). Over the last decade, the available funding aimed specifically at facilitating CBPR has grown considerably and includes such initiatives as planning grants, partnership infrastructure support, and resources directed at community-based organizations (5, 7, 8). These funding opportunities support and sustain partnerships’ collaborative efforts, and may also influence challenges faced by partnerships engaged in CBPR in ways that have not been examined thoroughly in the literature.

In this article, we describe the lessons learned from one component of a formative evaluation of a community-academic partnership that implemented a pilot neighborhood walking program in Detroit, Michigan as part of a larger CBPR planning grant. Funding for pilot interventions, in general, can afford research partnerships various opportunities by, for example, providing time and resources to evaluate a new intervention strategy, collect preliminary data, or test new methods. At the same time, pilot interventions often do not have a guarantee of long-term funding or commitment. Thus, while pilot interventions can be invaluable to understanding the methods and approaches that are likely to be most effective, they may be at odds with the needs of community members and community-based organizations. Drawing on our experience piloting a community intervention, we describe: challenges and opportunities associated with conducting a pilot intervention within a longstanding CBPR partnership, lessons learned for use of a participatory process to pilot community interventions, and recommendations to funders on mechanisms for funding pilot interventions that may help address some of these challenges.

Background

Healthy Environments Partnership

The Healthy Environments Partnership (HEP) is a community-based participatory research partnership affiliated with the Detroit Community-Academic Urban Research Center (9). HEP was established in 2000 to investigate and develop interventions to address social, physical, and environmental factors associated with risk of cardiovascular disease (CVD) in three Detroit neighborhoods (eastside, northwest, southwest) (10). HEP’s research efforts are guided by a Steering Committee (SC), which meets monthly and is comprised of representatives from the community, community-based organizations, health service providers, the city health department, and academic researchers from the University of Michigan School of Public Health. Upon its inception, the partnership adopted a set of “Community-based Research Principles” that emphasize: involving all partners in all aspects of the research process; strengthening collaboration among partners; conducting research that is beneficial to the community; enhancing the capacity of partners; and disseminating findings to the community in ways that are understandable and useful (6). In addition, the partnership has developed dissemination guidelines that outline processes for disseminating HEP results through community and academic channels, including a commitment to co-presentation and co-authorship with SC partners. HEP conducts evaluations of the partnership process annually, and uses the findings from these evaluations to inform discussions regarding ways to strengthen the partnership.

The neighborhoods in which HEP collaborates, two of which are predominantly African American and one of which is largely Hispanic, all face adverse health, environmental, and economic conditions. Age-adjusted mortality rates due to heart disease are considerably higher in these neighborhoods than national and state estimates (10–12). Residents of these neighborhoods have an elevated prevalence of cardiovascular risk factors compared with national rates (13), and the neighborhoods are characterized by limited access to grocery stores, poor quality produce, and limited opportunities for physical activity (14). In addition, the median household income in Detroit was estimated to be $29,109 in 2007, compared with $50,007 for the U.S., and approximately 27 percent of families in Detroit live below the federal poverty level, compared with 10 percent of families nationally (15). At the same time, there exist within these communities a sense of shared identity, a prior history of positive working relationships, skills and resources, and organizations and residents with a strong commitment to the community and a demonstrated interest in promoting heart health.

Community Assessment and Intervention Planning Process

In 2005, HEP was awarded funding from the National Center on Minority Health and Health Disparities (R24 MD 001619) to conduct a community-based participatory assessment and intervention planning process toward the development of a multilevel intervention to improve heart health in Detroit. This three-year planning grant was part of a larger CBPR funding initiative that included subsequent funding opportunities for project implementation and dissemination (8, 16). During the planning phase, the HEP partnership, in a project referred to as Community Approaches to Cardiovascular Health (CATCH), engaged community members through focus groups and youth photovoice processes, in discussions of barriers and facilitating factors for physical activity and healthy eating in their neighborhoods. The Healthy Environments Partnership also convened community forums to discuss results from the community assessment and generate ideas for interventions. A series of intervention planning meetings with community leaders helped to further refine and prepare recommendations for intervention approaches. Members of the HEP SC were extensively involved in this process, including: the SC planned the forums, the community-based organization members of the SC in each of the three HEP neighborhoods hosted and co-facilitated the community forums, and the SC planned and participated in intervention planning team meetings. Following the planning process, the SC convened a day-long retreat in which the group synthesized the recommendations and developed a broad framework for a multilevel intervention to strengthen individual, organizational, and community capacity to promote heart health and active living in Detroit (17). This framework provided the basis for a grant proposal submitted for the implementation phase of HEP-CATCH, which the SC members were involved in designing.

Walk Your Heart to Health Pilot Intervention

The above planning process culminated with the implementation and evaluation of a pilot intervention, which was one component of the multilevel intervention framework developed by the SC. The pilot intervention, Walk Your Heart to Health (WYHH), was a six-week neighborhood walking program hosted by two community-based organizations and one faith-based organization in three Detroit communities. Representatives from the two community-based organizations were members of the HEP Steering Committee, and leadership at the faith-based organization was involved in the CATCH intervention planning process. Along with other members of the HEP SC, these organizational leaders contributed to the intervention design, helped to identify study personnel, including walking group facilitators, and distributed promotional materials to aid the recruitment of participants. The walking groups met for two hours, three times per week, and were designed to promote physically active lifestyles through information, skill building, social support, and group walks using an empowerment approach to behavior change (18). Trained community residents and the HEP Health Educator worked in teams of two to co-facilitate the program at each of the three sites. Memorandums of Understanding were collaboratively developed and outlined the roles and responsibilities of the lead agency and each host organization and specified funds provided to the host organizations for space rental, staff time related to coordination of host site facilities, and provision of child care for children of pilot participants, as needed.

Methods

The planning grant represented the first stage of funding within the larger NCMHD initiative that also included funding opportunities for a second stage to carry out the intervention conceptualized during the planning process, and a third stage to disseminate research findings. The purpose of the WYHH pilot was to assess the feasibility of the intervention strategy and the study design, including the methods used to evaluate the walking program, to inform the study design, and the grant proposal, for the multilevel intervention. Specifically, the evaluation of the pilot intervention assessed the: 1) feasibility of enrolling community residents in the walking groups; 2) feasibility of collecting baseline, follow up, and pedometer data; 3) challenges and opportunities of hosting the walking groups at community and faith-based organizations; and 4) impact of the intervention on physical activity and clinical and anthropometric indicators of cardiovascular risk. Concerns that emerged in response to the third objective, challenges and opportunities experienced by sites hosting the walking groups, provided the motivation for this manuscript.

For this piece of the evaluation, key informant interviews were conducted by the lead author with organizational staff and leaders at each host site to inquire about challenges and facilitating factors associated with the implementation of the pilot intervention at the host organizations. These open-ended interviews were conducted about one month following the end of the intervention and lasted approximately 20 minutes for organizational staff and 30 minutes to one hour for organizational leaders. Interview questions inquired about the benefits and costs to the organization as a result of hosting the WYHH program, the extent to which organizational responsibilities were consistent with respondents’ expectations, factors that aided the implementation as well as challenges or unforeseen events, and suggestions for improving the program. The lead author prepared verbatim field notes for each interview, reviewed and summarized these notes for each organization according to the topics listed above and compared and contrasted the themes identified across the three sites. The lead author then met with other project personnel to review and further refine the themes. Examples of themes included factors that facilitated the implementation of the program at the host organizations (e.g. the focus of the pilot on physical activity and healthy lifestyles corresponded well with each organization's mission) as well as challenges (e.g. communication challenges between host organizations and project personnel), among others.

We then convened a group discussion with the host organization leaders and key project personnel to explore similarities and differences across sites that emerged from the key informant interviews. The overarching goal of this discussion was to draw on our experience piloting a community intervention to identify lessons learned from this process that may be informative to community-based participatory research partnerships engaged in similar efforts and to potential funders of community pilot programs. Specifically, using a summary of the key points from the interviews as a guide, we discussed in greater depth the factors that influenced the implementation of the program, and the challenges that arose, including those pertaining to program implementation as well as to participant, organizational, and community responses to the program. As part of this discussion, we considered how the structure of pilot interventions contributed to our experience, with a particular emphasis on delineating opportunities and challenges presented by pilot efforts. The host organization leaders, two of whom were HEP SC members and the other participated in the intervention planning process, are all coauthors on this manuscript. The lead author summarized the information generated and developed a draft of the salient points from the discussion. Coauthors reviewed this information, provided feedback, and contributed to the preparation of this manuscript. The research presented here was exempt from review from the Institutional Review Board at the University of Michigan.

Results and Lessons Learned

Results: Opportunities and Challenges of Pilot Interventions for CBPR Partnerships

The pilot intervention was conducted within the context of a planning grant, which afforded the partnership valuable time and resources to engage the community in an extensive assessment and intervention planning process that resulted in a community-driven intervention strategy. Conducting the pilot intervention provided the partnership with an opportunity to assess the feasibility of the intervention design through the collection of process and impact evaluation data. As a result, the partnership was able to examine the process and identify strategies that worked well and those that might be modified, findings that contributed greatly to the study design proposed for the full intervention. Finally, having the time and resources to conduct a thorough process evaluation allowed for in-depth investigation of participants’ experiences, and of facilitating factors and challenges related to implementation of the pilot at the host organizations. This process led to thoughtful discussions about how the intervention could be improved to more effectively address participants’ needs and better fit the structure and organization of the host sites.

We also identified several challenges presented by the pilot, particularly in relation to differences between the needs and desires of the community and the ability of the pilot program to meet those needs. The first challenge encountered by the partnership and the host organizations involved the disappointment expressed by participants as the program ended who wanted to see it expand and continue. In response to participants’ requests, HEP worked with each host organization to determine the feasibility of continuing the walking groups on a limited basis by, for example, hosting the groups once rather than three times per week and providing space for participants to meet but without providing a group facilitator. The ability to continue the walking groups depended on the organizations’ availability of meeting space, competing priorities for limited resources, and the determination of the walking group participants to assume greater leadership roles in organizing and coordinating the group activities. The walking group at one organization continued to meet for several months following the completion of the pilot, while the groups at the other two sites did not for a variety of reasons related to host site and group capacity.

A second challenge for the partnership and the host organizations concerned managing participant and community expectations for the timely implementation of the full intervention. For some, the concept behind a pilot raised expectations that if the program was successful, it would be implemented on a larger scale in the near future. For research studies, the process through which research funds are obtained and the dependence on the timing of funding cycles often result in delays between completion of a successful pilot and initiation of broader implementation. In addition, although the proposal for the multilevel intervention was submitted shortly after the walking program ended, decisions about subsequent funding were contingent upon the review process. The inability to provide a definitive timeline for if and when an expanded program would begin contributed to disappointment among pilot participants and other community members interested in the walking groups

Third, the lag time between the pilot and the full intervention risked a loss of interest and momentum in the community for the particular intervention as well as other work carried out by the partnership and the host organizations. During group discussions with program participants, participants commented that just as momentum for the program, and for walking groups in the community more broadly, was growing, the six-week pilot period was over. There was concern that this loss of momentum may make it more difficult to cultivate that energy in the future.

Each of these challenges has the potential to jeopardize relationships with the community and threaten the credibility of both the partnership and the host organizations. Trust with community members cannot be taken for granted, and research partnerships and community and faith-based organizations must continually foster and build that trust (19–21). In general, because two of the host organizations were also members of the HEP SC, it is difficult to disentangle the challenges experienced by the host organizations versus those experienced by the partnership. Walking group participants interacted directly with staff, and on occasion leadership, at the host organizations. In addition, the HEP partnership has a longstanding history of positive working relationships in the involved communities, and many community and civic groups participated in the community assessment and intervention planning process that led to the development of the pilot intervention. Thus, tensions that arose as a result of the finite nature of the pilot or the uncertainty regarding the implementation of an expanded program were experienced by both the partnership and the host organizations.

Recommendations and Lessons Learned for Conducting Pilot Programs within CBPR Partnerships

It is important to consider the above challenges within the context of the purpose of pilot interventions. The purposes of pilot interventions include assessing feasibility, identifying ways to enhance effectiveness, and informing future program design and implementation. In the case of the WYHH pilot, the goal of the pilot was to test the feasibility of the walking group concept as one component of a planned larger multilevel intervention. In the course of conducting the pilot, challenges pertaining to program sustainability and organizational capacity were illuminated that were relevant to the broader aims of the longer term intervention, which involve building community and organizational capacity to support active communities. This finding reinforced the need for community-based participatory research partnerships and funders to consider issues of sustainability and community capacity when funding and developing opportunities for pilot interventions.

Through discussions of our experience carrying out the walking groups and the opportunities and challenges discussed above, we identified the following lessons learned for partnerships engaged in piloting community interventions.

1. Ensure transparency regarding the research process and the purpose of the pilot intervention.

   Throughout the project, beginning with recruitment and continuing in the group sessions, participants were informed that the walking program was being piloted as part of a research study. Project staff explained that the purpose of the pilot program was to evaluate the feasibility of this strategy for promoting physical activity in the community and to identify ways in which the program could be improved. Participants were encouraged to share their feedback through informal and formal mechanisms, and embraced their role in helping to improve and shape future phases of the program. Understanding the role of the pilot helped participants view the program within the context of the larger research and public health objectives.

2. Build realistic expectations regarding the limitations of pilot interventions but recognize the potential for disappointment and frustration in the community.

   Although participants and host site leaders were aware that the program was designed to last for six weeks, feelings of disappointment upon the program’s completion occurred nevertheless. Thus, while it is important to be up front regarding the goals and limitations of pilot interventions (e.g. the finite nature of the program), it is equally important for the partnership to anticipate and be prepared for the disappointment among participants related to differences between the objectives of pilot projects and the desires of community residents for longer term health promotion efforts. .

3. Recognize the tensions that may arise between the desire of community members and the purpose of a pilot intervention, and discuss in advance how the partnership will address these tensions. .

   The motivation behind conducting a pilot is to evaluate the approaches employed and to identify ways to improve upon these strategies, for the purpose of informing the design of a longer term intervention, as appropriate. Hence, the conduct of a pilot is inherently a short-term endeavor. Within the context of piloting a community program, partnerships, and particularly host organizations, may grapple with potentially competing expectations whereby the research approach may not be in accordance with community desires. In this study, tensions arose from the desire expressed by community members for program continuation and the short-term nature of the pilot intervention. In this situation, it is important for partnerships to discuss, at the beginning of the research process, under what conditions efforts might be made to continue the program beyond the formal pilot phase, including discussion of questions such as: 1) is there evidence that the pilot program benefits the participants?; 2) is there evidence of benefit to the host organization?; 3) are there resources available to continue to support it?; and 4) is there a “champion” in the group or organization who is willing to facilitate continuation? Consideration of these issues, and some agreement among the host sites and the partnership concerning the conditions under which continuation might be considered, would help assess the appropriateness and potential for continuing the program beyond the formal pilot period.

4. Integrate the pilot program within the structure of the host organization to the extent possible.

   Integrating programs within the structure of a host organization promotes organizational ownership of the program and makes use of existing communication systems. Although a long-term objective of the multilevel intervention is to facilitate such integration, it was not a specific short-term objective of the pilot. However, the evaluation of the pilot suggested that the involvement of organizational leadership and volunteers in the program at one host site created a strong sense of ownership and group cohesiveness that subsequently facilitated both the organization’s and participants’ commitment to continuing the program beyond the pilot phase.

5. Maintain reciprocal communication with program participants, the host organizations, and the community during and after the pilot intervention phase.

   Facilitating opportunities for communication with program participants, host organizations, and the broader community is critical to sustaining community engagement, particularly after the pilot has ended, and provides opportunities for continual feedback. HEP has shared results of the pilot program with community leaders and policymakers involved in the intervention planning process and with all HEP research participants, community partners, and others in the community who have been involved. In addition, the comments and feedback provided by the host organizations and program participants contributed greatly to the design of the full intervention and helped identify ways to address concerns and capitalize on the program’s successes.

Recommendations to Funders

Although the above lessons learned may mitigate some of the challenges that can arise in the context of piloting interventions in community settings, our experience underscores the value of promoting pilot programs that are linked to longer-term funding opportunities as well as the need to develop funding mechanisms that maintain community engagement and enthusiasm in the process. We provide three recommendations, some of which have been proposed by others (22, 23), for how such funding initiatives might be structured. First, funding agencies could build in time and resources for a pilot intervention to be conducted during an initial phase of a longer-term funded CBPR project. This would ensure that the partnership has the resources to transition into a full intervention following a successful pilot and also reduces the lag time between the pilot and the full program. Second, funding opportunities for CBPR interventions that involve multiple phases of research could be designed such that partnerships that meet funders’ expectations of an initial pilot grant and subsequent proposal submission, would be guaranteed to receive further project funding, providing peer-review standards are met. Nonetheless, if partnerships know up front that fulfillment of grant requirements will lead to further funding, that could help to foster continued collaborative work and avoid jeopardizing community trust, relationships, and momentum. Third, funding agencies could provide bridging funds to promote continued community engagement and capacity building during the lag time between funding for pilot and full interventions.

Concluding Remarks

Our experience evaluating the implications of conducting a pilot within a longstanding CBPR partnership illustrates the iterative nature of CBPR and the ways in which partnerships can develop strategies to address challenges that arise in the context of carrying out a pilot intervention. Examining in detail the opportunities and challenges presented by the pilot helped to crystallize the distinction between interventions focused on providing a service versus those that maintain a research orientation. Based on our experience, we suggest that community pilot interventions can provide information that is invaluable to the research design. In the absence of longer term funding to foster continued community collaboration, however, the structure of pilot programs may be at odds with community needs and desires. The lessons learned underscore the importance of ongoing communication throughout the research process among all partners involved to address both anticipated and unanticipated challenges. Funding mechanisms that provide opportunities for longer-term support or bridging funds focused on building community capacity greatly expand partnerships’ ability to address these challenges. Importantly, while this discussion focuses specifically on pilot interventions, many of these points apply to community interventions in general and emphasize the need to ensure that all interventions balance research design with value to the communities involved. Public health efforts to promote meaningful change in communities require long-term investment of time and resources to develop trusting relationships, and support capacity for sustainability. Funding organizations that share this long-term vision are a critical component of efforts to strengthen the quality, relevance, and subsequent benefits of community-based participatory research to communities.