Abstract
Improving community health “from the ground up” entails a comprehensive ecological approach, deep involvement of community-based entities, and addressing social determinants of population health status. Although the Centers for Disease Control and Prevention, the Office of the Surgeon General, and other authorities have called for public health to be an “inter-sector” enterprise, few models have surfaced that feature local health departments as a key part of the collaborative model for effecting community-level change. This paper presents evaluation findings and lessons learned from the Partnership for the Public’s Health (PPH), a comprehensive community initiative that featured a central role for local health departments with their community partners. Funded by The California Endowment, PPH provided technical and financial resources to 39 community partnerships in 14 local health department jurisdictions in California to promote community and health department capacity building and community-level policy and systems change designed to produce long-term improvements in population health. The evaluation used multiple data sources to create progress ratings for each partnership in five goal areas related to capacity building, community health improvement programs, and policy and systems change. Overall results were generally positive; in particular, of the 37 partnerships funded continuously throughout the 5 years of the initiative, between 25% and 40% were able to make a high level of progress in each of the Initiative’s five goal areas. Factors associated with partnership success were also identified by local evaluators. These results showed that health departments able to work effectively with community groups had strong, committed leaders who used creative financing mechanisms, inclusive planning processes, organizational changes, and open communication to promote collaboration with the communities they served.
Keywords: Community health partnerships, Local public health departments, Community-based health promotion, Resident involvement, Collaboration, Social determinants of health
Introduction
A number of comprehensive, community-based health initiatives have been implemented that take a broad-based approach to improving community health, including Community Care Networks,1 Health Improvement Initiative,2 Work in Health,3 and Partnerships for Health.4 This approach is best represented in the “Healthy Cities and Communities” movement,5–7 in which “social determinants of health,” such as economic development, housing, and education are seen as important factors influencing community health.8 These initiatives typically emphasize community collaboration and partnerships in shaping intervention approaches and building community capacity.9–12
Although a number of these initiatives have involved local public health departments as partners, few have explicitly included health departments as a key part of the collaborative model for effecting community-level change. Turning Point13 created coalitions that included community representation but focused more on systems change in state and local public health. There are a number of reasons why local public health departments can be effective partners for communities as they strive to improve the overall health and well-being of their citizens, including a mission focused on community-level health improvement, the presence of developed infrastructure and programs, staff that often are trained and interested in population health, and an understanding of governmental processes for making policy and systems changes.
This paper presents evaluation findings and lessons learned from the Partnership for the Public’s Health (PPH), a comprehensive community initiative that featured a central role for local health departments. Funded by The California Endowment (TCE), PPH provided technical and financial resources to 39 community partnerships in 14 local health department jurisdictions to promote community and health department capacity building and community-level policy and systems change designed to produce long-term improvements in population health. We report on findings regarding the progress made towards meeting the initiative’s goals, factors associated with high and low levels of progress, and partnership sustainability.
Methods
Initiative Description
The PPH was a $40 million, 5-year Initiative funded by TCE to develop partnerships between California communities and local health departments. Fourteen county and city health departments were funded under the PPH Initiative along with 39 local community groups. The Initiative goals were to: (1) strengthen the capacity of communities to engage residents to act on their own and in partnership with health departments and other institutions to protect and improve the community’s health and well-being; (2) enhance the capacity of health departments to respond to community-based and community-driven priorities; (3) create sustainable partnerships between communities and health departments that promote and define mutual responsibility for improving community health; and (4) develop state and local policies that support and sustain local capacity to improve community health. Although there was a statewide policy effort that was part of PPH, most of the Initiative resources went to building and supporting the work of local partnerships. Each local partnership was funded for a total of 4 years. Funding was allocated to both the community group and the health department. Community groups received approximately $80,000/year and health departments (HD) received between $150,000 and $180,000/year depending on the number of community groups with which they had partnerships. Each health department partnered with anywhere from two to five separate community groups within their jurisdiction; none of the partnerships involved multiple community groups.
The four broad PPH goals noted above were revised over time and made more concrete to help the partnerships formulate their community action plans and to guide the evaluation. In the final iteration, the partnerships between HDs and community groups were expected to create an action plan that included activities in each of five major PPH goal areas: (1) community group capacity building, (2) health department capacity building, (3) partnership capacity building, (4) community health improvement, and (5) policy and systems change. Community group capacity building focused on organizational development and skill building for the community-based organizations and residents involved in the partnership. Health department capacity building involved modifying internal structures and processes to allow the health department to work more effectively with community partners. Partnership capacity building worked on the interface between the community groups and health departments, striving to break down traditional barriers to collaboration. Community health improvement involved programmatic activities ranging from educational classes to improving the delivery of social and health services. Policy and systems change focused on more long-term structural approaches to community-level health promotion. More examples of activities in each of the five goal areas are given below in “Results.”
We identified four broad categories of community groups within PPH based on their structure and whether they involved primarily non-professional community residents, local nonprofit agencies, or a combination of the two: (1) neighborhood/grassroots—primarily made up of community residents, with resident-driven decision making; (2) agency/resident collaborative—combination of nonprofit agencies and residents, with power and decision making shared more or less equally between the two groups; (3) agency collaborative—organizationally based, a coalition of service agencies/nonprofits; (4) program/service agency—dominated by a single nonprofit agency or program. The number of community groups that fell into each of these categories is presented in “Results” below.
The PPH Office was established as a grant-making office to develop, implement, test, and disseminate model community-based public health approaches in California. The PPH Office was established before community grants were awarded. The responsibilities of the PPH Office included management of the community-level grants, oversight of evaluation efforts, development of technical assistance resources, support of contractors, and development, and advocacy for statewide policy efforts.
Evaluation Design
There were two broad goals of the PPH evaluation: (1) document the impact of the Initiative and provide formative feedback and (2) disseminate lessons from PPH to public health practitioners, community organizers, and funders interested in promoting community-based public health. The primary questions guiding the PPH evaluation focused on community impact and how the combination of PPH strategies and local partnership actions contributed to the goals of the initiative. Two additional evaluation questions addressed the role of the PPH office in bringing about health-related policy and systems changes at the state and local-level in California and assisting the partnerships in achieving their goals.
The PPH evaluation team included PPH staff, an external Initiative-level evaluator—the Center for Community Health and Evaluation (CCHE)—and local evaluators based in each of the 14 health jurisdictions. CCHE created the overall evaluation design, gathered selected initiative-wide data, and was responsible for overall coordination of the evaluation, data analysis, and report writing. The local evaluators were responsible for implementing data collection tools designed by CCHE, creating local evaluation plans and logic models, documenting accomplishments and challenges, providing timely formative feedback to grantees, and supporting the initiative-level evaluation through feedback and insights drawn from working with local partnerships.
Data Collection
Multiple data sources were used in the PPH evaluation including open-ended key informant interviews, closed-ended surveys of partnership members, participant observation, and document review (including partnership progress reports):
Key Informant Interviews. Structured open-ended key informant interviews were conducted with a sample of community informants at the midpoint of the initiative. Interviews were conducted with 183 local partnership members. Local evaluators working with the partnerships recruited participants who were actively involved in their local partnership. Respondents included health department staff (n = 44), community group staff (n = 35), and community participants (n = 104). When possible, community participants were volunteer residents, but when volunteer residents were not available, agency and community group staff were included. Interview questions addressed community assets and barriers to success, opportunities for capacity building, and perceived changes in state/local policy.
Partnership Surveys. Closed-ended surveys of partnership members were conducted in 2002 and 2003. Questionnaires were distributed at local partnership meetings, and individuals in attendance completed them at that time. Instruments were mailed to people fitting the sample criteria but not attending on the days questionnaires were distributed. In some cases, local evaluators used follow-up procedures (phone calls, self addressed mailers, etc.) to increase response rates. A total of 313 and 371 surveys were completed across the 39 partnerships in 2002 and 2003, respectively. The closed-ended questions covered a number of areas, including partnership capacity, organization, decision making, communication, leadership, and benefits of participation.
Participant Observation. Local evaluators attended partnership meetings, including both overall and work group meetings. Although formal observation protocols were not used, the evaluators were able to get a much better sense of partnership dynamics, including leadership, conflict, and decision making. In addition, local evaluators actively worked with the partnerships to build evaluation capacity, develop an annual Local Evaluation Plan, and implement other local-level evaluation activities. During the course of this work, they were able to gather a more in-depth understanding of the partnerships.
Document Review. The primary documents were annual progress reports to the PPH filled out by the grantees. The progress reports were organized by the five PPH goal areas and included information about challenges and lessons learned for activities in each goal area.
Analysis
The analysis of the data for this paper consisted primarily of (1) designing and implementing a process to rate the progress of each partnership in each of the five goal areas and (2) identifying factors associated with more and less successful partnerships.
To capture progress at the local partnership level, a brief case study (Partnership Summary) was developed that summarized individual partnership accomplishments in each of the five goal areas. Data sources used in creating the Partnership Summaries were described in the previous section, including interviews, surveys, participant observation, and document review (progress reports). Local evaluators were trained to prepare the final Partnership Summaries using a participatory process involving local partnership members in drafting and reviewing the summaries. Each Partnership Summary used the same template with sections for community characteristics, partnership history, accomplishments to date, factors associated with success (or lack thereof) in achieving partnership goals, resident involvement, sustainability, lessons learned, and recommendations for program improvement. The Partnership Summaries were then used as the primary source of data in a process to rate progress in each of the five PPH goal areas. Progress was rated independently by four separate groups: the partnerships themselves, CCHE, local evaluators, and PPH staff. Representatives from PPH staff, CCHE, and the local evaluators met to resolve differences between the four sets of raters. This consensus rating was labeled the “Initiative-level” rating. If the initiative-level score differed from the partnership’s rating of their own progress, the partnership was given an opportunity to provide a rationale for their score. In some cases, this process resulted in a revision of the initiative-level rating.
The progress ratings were based on a four-point scale: high, high moderate, moderate, and low. The criteria used in assigning the ratings varied slightly by goal area, but in general, the definitions were the following: high progress—implementation of activities that either substantially strengthened the organization/entity or were likely to significantly improve long-term community health; high moderate progress—implementation of activities with the potential to move into the “high” category with a modest level of additional effort/funding; moderate progress—a limited number of activities and/or activities of modest scope; and low progress—little or no activity in the indicated area.
To systematically identify factors associated with partnership success, local evaluators rated the extent to which a list of 17 positive and 17 negative factors were associated with partnership success (or lack of success) in each of the goal areas. The list of factors was compiled from the previous years’ Partnership Summaries and cross-referenced with the literature on success of community partnerships. For example, one positive factor was “having strong leadership;” one negative factor was “inability to develop and/or maintain clear purpose and vision.” Some factors were associated with a specific goal area; others applied to more than one goal area. Local evaluators rated the factor as a “major factor,” a “minor factor,” or “not a factor” in the partnership’s ability to make progress (or not make progress in the case of the negative factors). The number of times a positive factor was mentioned for high or high moderate performing partnerships was then tabulated (e.g., “for 70% of high performing partnerships strong leadership was a major factor in their success”). Similarly, the number of times a negative factor was mentioned for low or moderate performing partnerships was tabulated (e.g., “for 50% of low performing partnerships lack of a clear vision/mission was a major barrier to their success”).
Results
Results are presented in this section describing the PPH partnerships and communities, summarizing partnership progress in each of the five goal areas, listing factors associated with successful (and less successful) partnerships, and describing progress toward sustainability.
Community/Partnership Characteristics
Table 1 shows selected characteristics of the 39 PPH partnerships selected for funding (note: two of the partnerships were de-funded during the Initiative and are not included in the progress ratings shown in Tables 3 and 4 below). The table is organized by the size of the health department jurisdiction, divided into three groups: small (<300,000 population), large (>300,000), and Los Angeles (LA) county as a special case (with almost 10 million total population, but divided into smaller administratively independent Service Planning Areas [“Spas” each with a population of approximately 1–2 million]). Five of the eight LA SPAs were included in PPH, and four of those were funded continuously throughout the Initiative.
TABLE 1.
Selected PPH partnership characteristics
| Size of health department jurisdictiona | Overall | |||
|---|---|---|---|---|
| <300K | >300K | Los Angeles | ||
| N, Health departments | 5 | 8 | 1 (5 SPA’s | 18 |
| N, Communities | 14 | 20 | 5 | 39 |
| Population served | ||||
| <10K | 10 (71%) | 1 (5%) | 1 (20%) | 12 (30%) |
| 10–30K | 3 (21%) | 9 (45%) | 1 (20%) | 13 (34%) |
| 30–70K | 8 (40%) | 8 (21%) | ||
| >100K | 1 (7%) | 2 (10%) | 3 (60%) | 6 (15%) |
| Ethnicity | ||||
| Majority Caucasian | 8 (57%) | 2 (10%) | 1 (20%) | 11 (28%) |
| Majority Latino | 3 (21%) | 2 (10%) | 3 (60%) | 8 (21%) |
| Mix of ethnic groups | 3 (21%) | 16 (80%) | 1 (20%) | 20 (51%) |
| Community groupb | ||||
| Neighborhood/Grassroots | 3 (21%) | 2 (10%) | 1 (20%) | 6 (15%) |
| Agency/resident collaborative | 2 (14%) | 4 (20%) | 3 (60%) | 9 (23%) |
| Agency collaborative | 5 (36%) | 9 (45%) | 1 (20%) | 15 (38%) |
| Program/service agency | 4 (29%) | 5 (25%) | 9 (23%) | |
aSize of HD is the population in the jurisdiction (greater or less than 300,000). Los Angeles is divided into eight Service Planning Areas (SPAs) five of these participated in PPH, four of which were continuously funded throughout the Initiative.
bComposition of community group/decision-making process: neighborhood/grassroots—primarily made up of community residents, with resident-driven decision making; agency/resident collaborative—combination of nonprofit agencies and residents, with power and decision making shared more or less equally between the two groups; agency collaborative–organizationally based, a coalition of service agencies/nonprofits; program/service agency—dominated by a single nonprofit agency or program.
TABLE 3.
PPH partnership progress ratings, by size of health department: capacity building goal areas
| Size of health department jurisdictiona | Overall | |||
|---|---|---|---|---|
| <300K | >300K | Los Angeles | ||
| N, Health departments | 5 | 8 | 1 (4 service areas) | 17 |
| N, Communities | 13 | 20 | 4 | 37 |
| Capacity building: Community group | ||||
| Highb | 7 (54%) | 6 (30%) | 2 (50%) | 15 (41%) |
| High/moderate | 4 (31%) | 5 (25%) | 1 (25%) | 10 (27%) |
| Moderate | 0 (0%) | 6 (30%) | 1 (25%) | 7 (19%) |
| Low | 2 (15%) | 3 (15%) | 0 (0%) | 5 (13%) |
| Capacity building: Health Department | ||||
| High | 2 (40%) | 2 (25%) | 0 (0%) | 4 (24%) |
| High/Moderate | 1 (20%) | 2 (25%) | 0 (0%) | 3 (18%) |
| Moderate | 2 (40%) | 3 (38%) | 3 (75%) | 8 (47%) |
| Low | 0 (0%) | 1 (12%) | 1 (25%) | 2 (11%) |
| Partnership development | ||||
| High | 5 (38%) | 6 (30%) | 0 (0%) | 11 (30%) |
| High/moderate | 5 (38%) | 6 (30%) | 3 (75%) | 14 (38%) |
| Moderate | 1 (8%) | 6 (30%) | 0 (0%) | 7 (19%) |
| Low | 2 (16%) | 2 (10%) | 1 (25%) | 5 (13%) |
aSize of HD is the population in the jurisdiction (greater or less than 300,000). Los Angeles is divided into service eight Service Planning Areas (SPAs) five of these participated in PPH, four of which were continuously funded throughout the Initiative.
bProgress rating scale criteria: High progress—implementation of activities that either substantially strengthened the organization/entity or were likely to significantly improve long-term community health; high moderate progress—implementation of activities with the potential to move into the “high” category with a modest level of additional effort; moderate progress—a limited number of activities and/or activities of modest scope; low progress—little or no activity in the indicated area.
TABLE 4.
PPH partnership progress ratings, by size of health department: community change goal areas
| Size of health department jurisdictiona | Overall | |||
|---|---|---|---|---|
| <300K | >300K | Los Angeles | ||
| N, Health departments | 5 | 8 | 1 (4 service areas) | 17 |
| N, Communities | 13 | 20 | 4 | 37 |
| Community health improvement | ||||
| Highb | 4 (31%) | 4 (20%) | 1 (25%) | 9 (24%) |
| High/moderate | 6 (46%) | 6 (30%) | 3 (75%) | 15 (41%) |
| Moderate | 3 (23%) | 8 (40%) | 0 (0%) | 11 (30%) |
| Low | 0 (0%) | 2 (10%) | 0 (0%) | 2 (5%) |
| Policy/systems change | ||||
| High | 4 (31%) | 7 (35%) | 1 (25%) | 12 (33%) |
| High/Moderate | 4 (31%) | 4 (20%) | 2 (50%) | 10 (27%) |
| Moderate | 4 (31%) | 5 (25%) | 0 (0%) | 9 (24%) |
| Low | 1 (7%) | 4 (20%) | 1 (25%) | 6 (16%) |
aSize of HD is the population in the jurisdiction (greater or less than 300,000). Los Angeles is divided into service eight Service Planning Areas (SPAs) five of these participated in PPH, four of which were continuously funded throughout the Initiative.
bProgress rating scale criteria: High progress—implementation of activities that either substantially strengthened the organization/entity or were likely to significantly improve long-term community health; high moderate progress—implementation of activities with the potential to move into the “high” category with a modest level of additional effort; moderate progress—a limited number of activities and/or activities of modest scope; low progress—little or no activity in the indicated area.
Each partnership specified a geographic target area that defined their community of focus. These ranged from areas with small populations [e.g., 12 partnerships (30%) focused on areas with less than 10,000 population] to relatively large cities or small counties [e.g., six partnerships (15%) had target areas of 100,000 people or more]. Smaller HDs tended to have smaller target communities: 71% of HD partnerships in small jurisdictions had target areas with fewer than 10,000 people vs. 5% of larger HDs, and 20% (one out of five) of the LA SPAs.
Many of the PPH partnerships were in areas with underserved, low-income, and ethnically diverse populations. As indicated in Table 1, only 28% of the target communities were majority Caucasian; the remainder were either majority Latino (21%) or a roughly equal mix of ethnic groups (51%). Smaller HDs were less likely to be working in ethnic minority communities: 57% of the communities were majority Caucasian for the small HDs vs. 10% and 20%, respectively, for larger HDs and LA. Larger HDs were working mostly in communities (80%) with mixed ethnicity and LA communities were predominately Latino (60%).
Regarding the type of community organization participating in the partnership, the largest number were agency collaboratives (n = 15 or 38%), as opposed to more resident-driven groups or single agencies. There were six community groups (15%) that could be classified as purely “grassroots” or resident-driven; and nine (23%) where there was a mix of resident and agency involvement. The nine remaining community groups (23%) were dominated by a single agency or program.
Partnership Progress Results
As described above, the progress rating process involved an independent assessment by the partnerships, evaluators, and PPH staff, which resulted in an Initiative-level rating that attempted to reconcile the views of all raters. The progress ratings reported in this study are based on the Initiative-level ratings, which differed somewhat from the self-ratings by the partnerships themselves. A comparison of the partnership self assessment ratings to the Initiative-level ratings showed that the partnerships were more positive in their assessments. However, partnership- and initiative-level ratings were identical about half the time (48%), and 77% of the disagreements were by only one assessment category (e.g., high vs. high moderate, or moderate vs. low).
Before presenting the results for partnership progress, some examples of high performing partnerships may be helpful in illustrating the activities that the partnerships were carrying out in each of the five goal areas. Table 2 provides brief descriptions of the activities of one high performing partnership in each goal area. One LA community group capacity-building effort involved increasing networking and communication among partner agencies, creating a “Neighborhood College” for resident leadership training, and implementing a more effective group governance structure. The Contra Costa health department created new capacity in several areas, including a computerized referral system, a new unit to consolidate the expertise of HD staff working in community projects, and new personnel procedures that reflected an increased commitment to community engagement. One of the San Joaquin County partnerships increased access to and use of each partner’s resources (staff and materials), developed clear governance structures, increased trust and conflict resolution skills, engaged in joint programmatic activities, and developed a sustainability plan.
TABLE 2.
Examples of high performing PPH partnerships
| Goal area | Example |
|---|---|
| Capacity building: Community group | Inglewood/Lennox/Hawthorne Community Health Council (ILH-CHC) in Los Angeles County increased networking and communication opportunities among the partner organizations, improved communication mechanisms with parent agencies, created a Neighborhood College curriculum and timeline for integrating five of the graduates into ILH-CHC‘s Steering Committee, and designed and implemented a group governance structure |
| Capacity building: Health Department | Public Health Division of Contra Costa Health Services Department developed a computerized referral system that describes health department and other county services, organized a new unit to consolidate the expertise of the PPH and Health Neighborhood Project staff (a pre-existing community-health department collaborative project with goals similar to the PPH and a very strong track record of community building and tangible accomplishments), trained staff (in collaboration with community partners and Alameda County Health Department) on effectively engaging residents, and changed personnel procedures to demonstrate an increased commitment to community engagement |
| Partnership development | The Asian Pacific Self-Development and Residential Association/Calaveras River Central Community Collaborative—San Joaquin County Public Health Services increased access and use of each partner’s resources (staff and materials), developed clear governance structures and communication processes, involved high-level decision makers in partnership governance, increased trust and conflict resolution skills, engaged in joint activities including a community health assessment and a program to promote dental health, built community-based research and evaluation skills, and jointly developed a sustainability plan |
| Community health improvement | Shingletown Activities Council—Shasta Public Health Partnership increased the availability of food for seniors by securing Meals on Wheels frozen meals, built relationships with the distributors of Brown Bag and commodity packages that resulted in an increase in the number of people receiving these services, and created an emergency food closet. This partnership also enhanced preventive services for seniors; decreased traffic-related fatalities; and increased awareness of traffic safety through data collection, public education, and advocacy. Other accomplishments were the creation of a network of partnerships and the organization of a youth council |
| Policy and systems change | South Bay Partnership—San Diego County Health and Human Services through their resident-based Health Leadership Teams (HLT), protested applications for liquor licenses in local cities, worked with local water park that received a license to serve alcohol to limit alcohol distribution, and lobbied the high school’s district to change its position regarding vending machines on campuses. The partnership also participated in two national campaigns: (1) Mothers Against Drunk Driving, and (2) child car seat use in Latino communities |
Examples are also presented in Table 2 for the two community-change goals: community health improvement and policy/systems change. The Shingletown partnership in Shasta County carried out community improvement activities, including increasing the availability of food for seniors and creating an emergency food closet. This partnership also enhanced preventive services for seniors and increased awareness of traffic safety through data collection, public education, and advocacy. The South Bay partnership in San Diego County worked to promote policy change around alcohol use, including successfully protesting new liquor licenses, limiting alcohol distribution at a local water park, and participating in a national anti drunk-driving campaign.
Table 3 shows the progress ratings for the capacity building activities of the PPH partnerships. Considering the results overall, the area of greatest improvement was in community group capacity building where 15 partnerships were rated “high” in terms of progress (41% of the 37 partnerships rated). Only 4 (24%) of the 17 health jurisdictions (counting each of the four continuously funded LA SPAs as a separate jurisdiction) made high progress in health department capacity building; 11 (30%) were rated high in partnership capacity building.
Although the sample size was too small to make meaningful statistical comparisons, there were generally more successes in community group capacity building in the smaller health department jurisdictions: 86% made high or high/moderate progress vs. 55% in the larger health departments. Three of the four continuously funded SPAs in LA made high or high/moderate progress in community group capacity building. There were no major differences in health department capacity building between large and small jurisdictions. The four LA service planning areas had difficulty making progress—all were rated moderate or low in HD capacity building. The smaller HD’s made more progress in overall partnership capacity building—76% rated high or high/moderate vs. 60% of the larger ones.
Table 4 presents results for the community change outcomes: community health improvement and policy and systems change. Overall, 9 (24%) partnerships made high progress in community health improvement and 12 (32%) made high progress in policy and systems change. More partnerships struggled with policy and systems change; 15 (41%) were rated as having low or moderate progress in the area. There were no clear patterns by size of HD; although smaller HDs were more likely to make high or high/moderate progress in community health improvement (76% vs. 50% of larger HDs).
Factors Associated with Progress
Table 5 lists the factors in each goal area that contributed to or were barriers to progress, as identified by the local evaluators. The first column shows the most frequently mentioned factors for partnerships making high or high moderate progress; the second shows factors most frequently mentioned among the less successful partnerships (moderate or low progress). Frequently cited factors that cut across multiple goal areas included working in a small and/or well-defined community, open communication and sharing of information, and strong leadership skills and commitment to working with community. Communication challenges, lack of resident engagement, and other challenges of working with community appear in all five goal areas as a factor related to low progress. Leadership issues and the ability to develop and maintain a clear vision are also mentioned in multiple goal areas. Inadequate funding and resources is mentioned but at a lower frequency.
TABLE 5.
Factors associated with high or low performance, by PPH goal area
| Goal area | High performance factors | Low performance factors |
|---|---|---|
| Capacity building: Community group | Having strong leadership (i.e. facilitation skills, shared leadership, etc.)—80% | Staffing issues related to recruitment and turnover—50% |
| Possessing dedicated and talented staff—76% | Communication challenges—50% | |
| Working in a small and/or well-defined community—84% | Other challenges working in the community/trust building—50% | |
| Establishing relationships with residents and/or skilled outreach—84% | Leadership problems—42% | |
| Lack of resident engagement—42% | ||
| Other organizational problems—33% | ||
| Capacity building: Health Department | Having strong leadership committed to working with community—80% | Communication challenges—35% |
| Working with community on data collection and distribution—75% | Lack of leadership/organizational support for community-based approaches to public health—29% | |
| Bureaucratic nature of the health department impedes work with community—29% | ||
| Lack of resident engagement—29% | ||
| Other challenges working with community—29% | ||
| Partnership development | Mutual understanding and trust between the health department and community group—80% | Communication challenges—both internal and external—50% |
| Establishing relationships with residents and/or skilled outreach—76% | Lack of resident engagement—50% | |
| Working in a small and/or well defined community—72% | Other challenges of working in the community—42% | |
| Having a stable, knowledgeable group of core members—60% | Inability to develop and/or maintain a clear purpose of vision—33% | |
| Open communication and sharing of information—60% | ||
| Community health improvement | Being opportunistic and flexible—83% | Communication challenges—both internal and external—46% |
| Working in a small and/or well defined community—58% | Lack of resident engagement—46% | |
| Establishing relationships with residents and/or skilled outreach—75% | Other challenges of working in the community—46% | |
| Communicating openly and sharing of information—67% | Inability to develop and/or maintain a clear purpose of vision—38% | |
| Inadequate funding or resources—23% | ||
| Policy/systems change | Ability to network effectively—82% | Communication challenges—both internal and external—60% |
| Having access to community leaders—72% | Lack of time and skills required to plan and implement policy change activities—46% | |
| Working in a small and/or well defined community—72% | Other challenges of working in the community—40% | |
| Establishing relationships with residents and/or skilled outreach—77% | Lack of resident engagement—33% | |
| Open communication and sharing of information—68% | Inability to develop and/or maintain a clear purpose or vision—33% | |
| Inadequate funding or resources—26% |
Factors identified by local evaluator (see text). Percentages show the number of high (or low) performing partnerships with the indicated positive (or negative) characteristics.
Sustainability of Local Partnership Efforts
Improvement in long-term health outcomes requires that the activities and relationships created or improved during PPH are sustained beyond the period of grant funding. Initial results were encouraging. At the end of PPH funding (August 2004), almost 70% (25 out of 37) of health department and community groups had specific plans to continue working together, including six local partnerships that had signed Memoranda of Understanding specifically outlining key elements of their continuing relationship. Thirteen local partnerships had specific jointly implemented projects they were continuing. In most cases, because of the lack of resources to sustain regular partnership meetings, it did not appear that the specific partnership structures developed under PPH would continue. However, several partnerships indicated that they would rely on the same committee structure to carry out their activities.
All but three of the partnerships had plans to continue at least one of the major activities begun under PPH. Twenty-six local partnerships (70%) had PPH-related activities for which they already had secured continuation funding. By the final year of the Initiative, nearly 80% (29 out of 37) of partnerships had applied for funding to sustain the work that was started under PPH. At the end of the PPH Initiative, partnerships had applied for over 18 million dollars in funding. Over 4.5 million dollars had been received and an additional 4.7 million was pending.
Discussion
The PPH was a large, complex Initiative with over 50 separate grantee organizations and ambitious goals for community change promoting long-term improvements in population health. In general, the PPH Initiative was able to manage the complexity effectively and produce a number of substantial achievements. In particular, of the 37 partnerships funded continuously throughout the initiative, in each of the five goal areas, between 25% and 40% were able to make a high level of progress.
Other efforts have been made to promote collaboration between health departments and community groups, including Mobilizing for Action through Planning and Partnerships (MAPP)14 and Turning Point. MAPP is a community assessment process developed collaboratively by the National Association of City and County Health Officials and the Centers for Disease Control and Prevention. It was designed using principles similar to PPH but is strictly an assessment process with no funding attached for the implementation of joint HD/community projects. PPH funded the use of MAPP in PPH sites as one tool of many that health departments could use to get the community and other stakeholders in the broader health system engaged. As noted in the introduction, Turning Point13 created coalitions that included community representation but focused more on systems change in state and local public health, vs. PPH which focused on more community improvement that was also programmatic.
As with many multi-site community initiatives,2,15,16 some partnerships were more successful than others. Understanding the factors associated with partnership success may help in designing future initiatives, and for this reason, a systematic attempt was made to identify success factors in each goal area. The following expands briefly on the results presented earlier for two of the five PPH goal areas: partnership development and HD capacity building.
Partnership development was a key goal area because a strong, long-term relationship between HDs and community groups was viewed by PPH as the primary pathway to sustainable community-level programs and policy and systems changes. Many of the important factors identified in this study have been found elsewhere in the extensive literature on partnership development and sustainability.17–24 Trust among members and effective methods of communication have been shown to be key elements in partnership success factors in a number of studies.24 Devising effective means of gathering input from community members is another key element in successful coalitions.17,22 The other factors we identified—working in a small or well-defined community and having a stable group of core members—have not been identified explicitly in the literature. However, many of the structural and process factors that have been identified as important for success, e.g., setting up workable decision-making processes and finding a match between community priorities and partnership activities,20,21 are made easier in smaller communities with a stable partnership membership.
A second goal area was building HD capacity to work more effectively with communities. Only two factors were identified consistently by the local evaluators as associated with HD capacity building: strong leadership committed to working with the community and working collaboratively with the community on data collection and dissemination. A number of factors worked against HD capacity building, including the bureaucratic nature of HDs, lack of leadership, and communication challenges, related to differences in institutional culture between HDs and community groups.
We conducted a more in-depth investigation using the Partnership Summaries and other data in an attempt to understand the factors that led some HDs to be more successful in working with communities. The most critical ingredient was leadership—health department leaders with a strong commitment to a community-based approach to public health (CBPH). Strong leadership was needed to overcome the bureaucratic and HD cultural factors that are barriers to working flexibly and creatively with community groups. All the health departments we examined that were effectively implementing the CBPH had dynamic executive leadership (i.e., health director and/or health officer) that was strongly committed to changing the way public health approached its mission. These leaders took risks to work with community (e.g., agreeing to approach other agencies on the community’s behalf) and were flexible in the consideration and approval of the types and/or scope of projects the health department worked on with the community.
Successful HD leaders used financing of CBPH, planning, organizational change, and communication to promote effective collaboration with community groups. Regarding financing, “model” health departments (i.e., those making high progress in their building capacity to do CBPH) were able to find innovative ways around categorical funding constraints to support CBPH efforts. Model HDs built the capacity of community partners to apply for grants, designated a portion of categorical funding toward work in CBPH, and used their limited flexible funds to support their work with communities. Model HDs demonstrated a strong commitment to including community members in their planning processes, including planning related to revising mission statements, making major organizational changes, and promoting changes in background and deployment of their workforce. Organizational changes made by model HDs included increasing organizational flexibility, increasing the workforce resources dedicated to working with community, creating units or offices designed to work with community partners, and developing mechanisms for community input into health department planning and practice. Finally, model health departments communicated with the community rather than to the community. Communication strategies for these health departments were consistently designed to build capacity or support community needs for policy and systems change.
A significant limitation in our evaluation was our inability to track long-term changes in population-level health outcomes. This was a deliberate decision based on the relatively short 5-year time frame of the initiative, and it resulted in ratings of significance of the community change outcomes that were largely subjective. Other limitations of the evaluation included difficulties in gathering comprehensive data given the large number of partnerships, the open-ended nature of partnership activities, and the limited data tailored specifically to the geographic areas designated as PPH communities.
A significant strength of the evaluation design was the role of the local evaluators. The local evaluators in many cases became integral partnership members, attending meetings regularly and participating in discussions and decision making. This enabled them to gather better data on partnership structure, processes, activities, and outcomes, and also helped them feed back the data they were collecting more effectively to the partnerships. The relationships they developed with the partnerships assisted in the progress rating process, which required trust that being candid about shortcomings would not affect future partnership funding.
In conclusion, PPH showed that, given the right circumstances and support, health departments can be effective partners with community groups in broader health improvement efforts. These HD–community partnerships led to substantial programmatic, policy and systems changes that, if sustained, can be expected to lead to long-term improvements in community health outcomes.
Footnotes
Cheadle is with the Department of Health Services, University of Washington, Seattle, WA, USA; Hsu, Pearson, and Beery are with the Center for Community Health and Evaluation, Seattle, WA, USA; Schwartz is with the Kaiser Foundation Health Plan and Hospitals, Oakland, CA, USA; Greenwald is with the School of Policy, Planning, and Development, University of Southern California, Los Angeles, CA, USA; Flores is with the The California Endowment, San Francisco, CA, USA; Casey is with the Partnership for the Public’s Health, Oakland, CA, USA.
References
- 1.The National Community Care Network Demonstration Program, (CCN). Hospital Research and Educational Trust; April 1995.
- 2.Cheadle A, Senter S, Procello A, et al. The California wellness foundation’s health improvement initiative: evaluation findings and lessons learned. Am J Health Prom. 2005;19(4):286–296. [DOI] [PubMed]
- 3.California Endowment. Work in Health Initiative. January 2000.
- 4.The Colorado Trust. Partnerships for Health Initiative; 2005–2013.
- 5.Wallerstein N. Power between evaluator and community research relationships within New Mexico’s healthier communities. Soc Sci Med. 1999;00:1–15. [DOI] [PubMed]
- 6.Enhancing Community Capacity. California Healthy Cities and Communities Annual Conference, Center for Civic Partnership, San Diego, California; April 2000.
- 7.El-Askari G, Freestone J, Irizarry C, et al. The healthy neighborhoods project: a local health department’s role in catalyzing community development. Health Educ Behav. 1998;25(2):146–159. [DOI] [PubMed]
- 8.Anderson LM, Scrimshaw SC, Fullilove MT, Fielding JE, and the Task Force on Community Preventative Services. The community guide’s model for linking the social environment to health. Am J Prev Med. 2003;24(3S):12–20. [DOI] [PubMed]
- 9.Alter C, Hage J. Organizations Working Together. Newbury Park, CA: Sage; 1993.
- 10.Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion. Health Educ Res. 1993;8:315–330. [DOI] [PubMed]
- 11.Chavis DM. The paradoxes and promise of community coalitions. Am J Comm Psych. 2001;29(2):309–320. [DOI] [PubMed]
- 12.Lasker RD, Committee on Medicine and Public Health. Medicine and Public Health: The Power of Collaboration. Chicago: Health Administration Press; 1997.
- 13.The National Association of County and City Health Officials. The Community Public Health System Improvement Plan: A Blueprint for Transformation. The Turning Point Initiative, sponsored by the W.K. Kellogg and Robert Wood Johnson Foundations.
- 14.The National Association of County and City Health Officials. A Strategic Approach to Community Health Improvement: MAPP Field Guide. National Association of City and County Health Officials; 2007.
- 15.COMMIT Research Group. Community intervention trial for smoking cessation (COMMIT): I. Cohort results from a four-year community intervention. Am J Pub Health. 1995;85(2):183–192. [DOI] [PMC free article] [PubMed]
- 16.Wagner EH, Wickizer TM, Cheadle A, et al. The community health promotion grants program in the West II: Changes in attitudes, environments and behaviors. Health Serv Res. 2000;35(3):561–589. [PMC free article] [PubMed]
- 17.Shortell SM, Zukoski AP, Alexander JA, et al. Evaluating partnerships for community health improvement: tracking the footprints. J Health Polit Policy Law. 2002;27(1):49–91. [DOI] [PubMed]
- 18.Mitchell SM, Shortell SM. The governance and management of effective community health partnerships: a typology for research, policy, and practice. Milbank Quarterly. 2000;78(1):241–290. [DOI] [PMC free article] [PubMed]
- 19.Roussos TS, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Ann Rev Pub Health. 2000;21:369–402. [DOI] [PubMed]
- 20.Metzler MM, Higgins DL, Beeker CG, et al. Addressing urban health in Detroit, New York City, and Seattle through community-based participatory research partnerships. Am J Pub Health. 2003;93:803–811. [DOI] [PMC free article] [PubMed]
- 21.Green L, Daniel M, Novick L. Partnerships and coalitions for community-based research. Pub Health Rep. 2001;116(supplement):20–31. [DOI] [PMC free article] [PubMed]
- 22.Galea S, Factor SH, Foley M, et al. Collaboration among community members, local health service providers, and researchers in an urban research center in Harlem, New York. Public Health Reports. 2001;116:530–539. [DOI] [PMC free article] [PubMed]
- 23.Lasker R, Weiss ES. Broadening participation in community problem solving: a multidisciplinary model to support collaborative practice and research. J Urban Health. 2003;80(1):14–60. [DOI] [PMC free article] [PubMed]
- 24.Schulz AJ, Israel BA, Lantz P. Instrument for evaluating dimensions of group dynamics within community-based participatory research partnerships. Eval Program Plann. 2003;26:249–262. [DOI]