Strengthening the Capacity of Local Health Agencies Through Community-Based Assessment and Planning

SYNOPSIS

Objectives.

We evaluated the effectiveness of the Protocol for Assessing Community Excellence in Environmental Health (PACE EH) in building competency in essential environmental health services and renewing efforts to engage the community in problem solving. Competency and community engagement have been identified by environmental health practitioners as important to meet new threats to public health.

Methods.

We conducted a national survey and 24 case studies of public health agencies. We invited 917 organizations to participate in the national survey because they had requested a copy of the protocol.

Results.

We received 656 total responses: 354 had not considered implementation, 302 had considered implementation, and 66 had implemented PACE EH. For the 24 case studies, we interviewed 206 individuals in communities implementing PACE EH. We found that PACE EH has had a positive effect on building community and professional networks, enhancing leadership, developing workforce competence, and expanding definitions of environmental health practice.

Conclusions.

With appropriate investments, PACE EH can be an effective tool to meet the environmental health challenges identified by local environmental health practitioners and state, tribal, and federal agencies.

Environmental public health activities related to sanitation, food safety, and water quality played a large role in the remarkable increases in life expectancy in the United States in the 20th century.¹ However, new threats to public health—air and noise pollution, solid and hazardous wastes, ionizing radiation, terrorism inside our national borders, and emerging infectious diseases—have catalyzed a recent reassessment of the roles and competencies of environmental health professionals.^2–7

At the local level, environmental health practitioners began to voice a concern that environmental health agencies were in a highly reactive mode, addressing the “topic of the week” (Personal communication, Heidi Klein, Lead Staff Member, National Association of County and City Health Officials [NACCHO], Protocol for Assessing Community Excellence in Environmental Health [PACE EH] Development Workgroup, December 2006). They lamented the silos in which they worked, in particular the lack of effective linkages among environmental health, environmental protection, and public health.⁸ They also lamented their separation from the community.⁹

A renewed focus on the community is echoed nationally in the Healthy People 2010 vision of “healthy people in healthy communities.”¹⁰ Assuring healthy communities is featured in the Institute of Medicine report, The Future of Public Health,^11,12 and appears as one of the 10 essential public health services: mobilize community partnerships and action to identify and solve health problems.¹³ The benefits of partnering with communities are also reflected in the Centers for Disease Control and Prevention (CDC) health protection goal of “Healthy People in Healthy Places.”¹⁴ Community collaboration was one of the main components that shaped the development of PACE EH.¹⁵

Developed in the mid-1990s in a joint effort between CDC’s National Center for Environmental Health (NCEH) and NACCHO, PACE EH is one of a number of recent tools designed to engage the community in health planning and assessment activities. Other tools include Planned Approach to Community Health, Assessment Protocol for Excellence in Public Health (APEX PH), and Mobilizing for Action through Planning and Partnerships.^16–18 The joint effort to develop PACE EH started when state and local health departments reported that APEX PH did not suitably guide the development of environmental health indicators. PACE EH, unique in its exclusive focus on environmental health, was first implemented in 10 pilot sites across the country between 1996 and 1999.¹⁹

PACE EH was designed to strengthen local capacity to anticipate and respond to a wider variety of environmental public health concerns. The protocol (Figure) guides a local partnership, usually under the leadership of a local health department, through a series of tasks that are aligned with the core functions of environmental health. Through this process, the agency assesses the community’s environmental health needs and capacities (assessment), and mobilizes a partnership to establish priorities and develop action plans (policy development). This sets the stage for implementing action plans and evaluating results (assurance). The PACE EH steering committee believed that this process would help generate the necessary dialogue between communities and local health agencies to respond proactively to environmental health concerns and revitalize environmental public health practice.

Figure.

Protocol for Assessing Community Excellence in Environmental Health tasks

This article presents the results of a national evaluation of PACE EH. Based on the early experiences of pilot sites, anticipated near-term impacts included new relationships among local public health agencies, communities, and other professionals (building networks); improved leadership (enhancing leadership); new work skills (developing workforce competence); and broader and more flexible working definitions of environmental health practice (broadening environmental health activities).^8,9,19,20,^8,9,19,20 Given the protocol’s recent implementation history, the evaluation did not address resulting long-term changes within agencies, communities, or the environment.

METHODS

Interviews with key stakeholders involved in the development and early marketing of PACE EH informed the evaluation. We used two primary data collection methods: (1) a national online survey of all local agencies that expressed interest in implementing PACE EH and (2) qualitative case studies with agencies implementing PACE EH.

The Battelle Institutional Review Board approved the data collection protocols.

National survey

We administered the survey via the Internet. The sampling frame consisted of qualified organizations that requested the PACE EH Guidebook from CDC or -NACCHO or that had participated in PACE EH leadership training (e.g., PACE National Summits) from its initial release in the mid-1990s through April 2006. Qualified organizations had to be within the U.S., be nonfederal, and have considered implementing PACE EH. We did not include organizations that never expressed interest in PACE EH. We had a response rate of 72%, but more than half of the responses were deemed ineligible based on initial screening questions. A total of 302 eligible organizations completed the survey (245 completed; 57 partial), 11 actively refused to participate, and two were lost to contact. We collected survey data using SPSS® mrInterview²¹ and transformed the data into SAS® software²² for analysis.

Case studies

We chose a complementary case study design because it is well suited to examining activities—in this instance, implementation of PACE EH—that are not easily understood outside of the context in which they occur.²³ Context is important because successful strategies both reflect and depend on the organizational and cultural setting. The case study unit was a local implementation partnership that had progressed at least through Step 4 (define goals) of the PACE EH protocol.

We selected case study interviewees based on their relationship to PACE EH (e.g., leader, team member, and other community leader) and the type of organization they represented. We collected data during site visits in 2004 (eight sites) and 2006 (16 sites). In total, we interviewed 206 people individually or in small groups. We recorded interviews with participant consent, imported electronic interview notes to a qualitative data analysis software program (QSR N6®),²⁴ and coded them using a systematic coding scheme.

RESULTS

Sixty-six of the 302 survey respondents indicated that they had at least partially implemented PACE EH. As shown in Table 1, most respondents were local health departments. The remaining implementers consisted of either state or tribal health departments, or of other or unknown agency type.

Table 1.

Distribution of survey respondents by Protocol for Assessing Community Excellence in Environmental Health implementation status

NA = not available

Several organizational factors appeared significant in the decision to adopt PACE EH. These factors related to organizational capacity, the range of public health services offered, existing community partnerships, and the presence of an in-house champion. As shown in Table 1, the majority of implementers were from large county health departments with a relatively large number of staff, and a larger staff focused specifically on environmental health issues. They typically provided a larger set of services, including occupational and health safety, radiation control, outdoor air quality control, drinking water safety, and lead abatement. We did not collect information on the cost to implement, but implementers reported a lower percentage of their operating budget from local general funds or fee-for-service arrangements, and more from state or federal funds.

Implementing agencies were also more likely to have a champion for their PACE EH process than were non-implementers and those who had not yet decided to implement. Only 6% of implementers indicated there was no champion compared with 57% of non-implementers and 45% of undecided agencies. The environmental health director, agency director, or other staff internal to the agency was most likely to serve as the champion across all three groups.

We asked survey respondents about the intermediate impacts they attributed to PACE EH. Table 2 shows the average response by implementation status. Specifically, we compared those that had completed Task 12 (33 had developed an action plan; 30 answered) to those that had not (33 had no plan; 23 answered). Most sites with a plan had begun to implement at least one action (n=27). Sites that did not implement at all were not asked about impacts they attributed to PACE EH.

Table 2.

Mean impact rating^a attributed to implementing Protocol for Assessing Community Excellence in Environmental Health by implementation progress

^a1 = strongly disagree; 2 = somewhat disagree; 3 = neither/not sure; 4 = somewhat agree; 5 = strongly agree

^bStatistically significant difference between the two groups of implementers at the p<0.1 level

EH = environmental health

Case study interviews included 89 public health staff members and 117 community members or representatives of community organizations. Interviewees shared their opinions regarding the impact of PACE EH on the selection of priorities, building capacity in the 10 essential public health services, and other impacts on agencies, partnership members, and the community at large. The following presentation of findings draws from both the survey and case studies.

Building networks

Survey respondents reported impacts in building networks, promoting collaboration, and increasing trust among key players in environmental health. The impacts appeared stronger among those sites that had progressed to the development of action plans, although the difference between the two groups was not statistically significant. Nevertheless, it is clear that relationship building began early in the process.

Case study interview data provided strong support for PACE EH as a vehicle for building stronger relationships. These strengthened relationships were evident between different jurisdictions (e.g., city and county); between departments within local government (e.g., health and environment); and among the government, the public, and the Board of Health. For example, one public health practitioner noted, “It got people out of their comfort zones and opened doors to new relationships across agencies; it built trust.” Of particular note was a reconnection of public health and environmental protection. Another public health practitioner noted:

We just weren’t connected and haven’t been since the [Environmental Protection Agency (EPA)] split off from public health in the early 1970s. This gave us a way to get EPA people to the table with us to talk about environmental health.

These stronger networks can lead to joint planning and information sharing. Interviewees claimed the benefit of having more individuals they could call for information, to collect data, or just to discuss a new idea, as illustrated by one public health practitioner: “I have contacts now that I would never have made except through PACE EH. I know who to call now.”

Enhancing leadership

The survey findings supported the notion that PACE EH can strengthen the perception of the health agency as a leader in environmental health. The health agency appears to gain additional status as a leader when the process is completed through the development of an action plan, although the difference between completers and those earlier in the process was not statistically significant (p=0.13). Leadership was interpreted in diverse fashions within the case study sites. Some viewed PACE EH as an opportunity to create leadership within the lead agency by being proactive instead of reactive to environmental health issues. Other sites talked more about the need to develop community leadership around these issues. The relative weight of these two approaches varied across sites, although both were common. Within case study sites, community-based interviewees spoke about their greater awareness of what environmental health professionals do, while public health staff spoke about their increased visibility and credibility. Public health managers mentioned the growth in leadership skills evident in their staff. As one practitioner noted, “…both the collaboration and the leadership abilities of our staff [increased] through this process.”

Developing workforce competence

Assuring a competent workforce is one of the 10 essential public health services. As shown in Table 2, implementing sites credit PACE EH with improving the knowledge and skills of participants in both environmental health issues and collaborative processes. Interviewees credited PACE EH with enhancing the ability of staff to interact with the public. They cited the need for staff to be not only technically competent, but also to use “the tools of current public health practice,” which include mobilizing communities, facilitating meetings, and presenting issues to the community, “which we are now a lot better at than we were then,” according to one public health practitioner. Some evidence was also provided that managers may consider these additional competencies when hiring staff. “It has broadened the way I think about hiring employees,” explained a practitioner.

Interviewees also mentioned cultural competency of staff as an outcome through the broad exposure that PACE EH provides to community members and to new types of staff often hired to lead the effort. One interviewee noted, “We have had people work for the county [PACE EH outreach staff] who would not normally have wanted to work for the county and they have taught us [staff] a lot about cultural competency along the way.”

Broadening environmental health activities

Survey responses (Table 2) indicated only modest support for the idea that PACE EH led to new programs, changes in priorities within the health agency, or political support for implementing the plan. Respondents disagreed that PACE EH led to increased funding for environmental health or changed how health agencies defined at-risk populations. Respondents did agree, however, that it strengthened agency focus on environmental justice—one of the underlying philosophical principles of PACE EH.

We also asked survey respondents to list the priorities that emerged and to indicate if these were new or preexisting. A combined total of 170 environmental health priority topics were listed, half of which were existing priorities within the respective agencies. Many topics could be broadly grouped into air quality, water quality, and waste management concerns. Others were more specific such as injury prevention, lead contamination, food safety, and physical activity. The finding that half of the priorities were not preexisting gives stronger weight to the idea that new environmental health priorities emerged from the PACE EH process than was indicated from the survey.

Many interviewees, both community members and public health professionals, stated that a major impact of PACE EH was the expansion of their conception of environmental health. For example, in multiple communities, participants discussed and prioritized public safety, walking, and the creation of pedestrian-friendly places—issues that they noted were not typically viewed as part of environmental health. Their growing awareness of what can be considered environmental health is exemplified in the following comments:

[We discussed] stop signs and other pedestrian-friendly measures that you don’t normally think of as within the environmental health arena.. .. What the PACE process showed, however, is that environmental health can also include the whole planning and zoning process, and trees, and walking paths.

Finding these nontraditional environmental health issues like street lighting or abandoned houses—learning how those are linked to public health—was very eye-opening to me.

Land use, another domain that is typically not central to health departments, was frequently raised as an issue within PACE EH, as illustrated by the following comment:

Land use probably would not have been identified as a priority without the PACE EH process. Not everyone views these [land use] issues as environmental health. Part of our work has been to become educated and to educate others about the linkage between land use and environmental health.

Other nontraditional environmental health priorities that rose to the surface included stray animals, violence prevention, and all-terrain vehicle (ATV) safety.

One of the biggest impacts of PACE EH in many sites came in the form of a reevaluation of environmental health professionals’ roles. Staff described their primary mindset prior to PACE EH as regulatory. PACE EH has helped expand their perception of their role to one that is much broader, and more in line with the essential public health services. This expanded role includes assessing the community, providing community outreach, being proactive about environmental health issues, and reviewing competencies in light of the 10 essential public health services. The following quotes provide some sense of how profound these changes could be in the eyes of environmental health practitioners and community members:

I think that Environment Health thought of themselves as mainly a regulatory agency and didn’t really see that this assessment of the community, taking a look at what’s going on, and working with the community was part of their role, too. (public health practitioner)

The department is typically more reactive, and this knowledge base allows them to be more proactive. (community member)

The whole regulation and enforcement role that we are accustomed to has changed. We now have the element in this department of community outreach. There’s a lot of collaboration now. (public health practitioner)

More of our [environmental health] programs now have some mechanism for community input with regulatory decisions. (public health practitioner)

Changing perspectives have led to changes in practices, such as new monitoring approaches (e.g., developing environmental report cards), more opportunities to engage the public, and better communication and education practices, such as improved websites. These changes, however, are not universal, as evidenced by the following comment by a public health practitioner: “When I look at these things [the 10 essential public health services], I don’t really think about [them] as part of my job.” The key difference may be in how wholeheartedly the department embraced PACE EH. The interviewee who cited a lack of impact went on to say, “We are just a little project. It may have had that impact if more people were involved.” This statement is consistent with respondent and interviewee data about the importance of champions and the dedication of resources both for the adoption of PACE EH and its continued support.

DISCUSSION

This evaluation provides support for PACE EH as a useful tool for effecting change and revitalizing the role of environmental health professionals. And it appears that the further a partnership moves toward developing and implementing a plan, the stronger the effects. The results are based on self-report by implementing sites without the benefit of a control group; instead, we asked respondents to reflect on the extent to which they believed that selected impacts could be attributed to PACE EH.

Our evaluation provides evidence that PACE EH is having an impact on building networks. Evaluation participants reported stronger and broader networks outside their agencies and saw PACE EH as a tool that could help bring disparate groups together. Several participants especially noted the renewed connection with environmental regulatory agencies, confirming earlier reports.⁸

We also found support for the idea that PACE EH promotes leadership. PACE EH implementers believe that the process strengthens the public perception of the health agency as an environmental health leader. Many also believe that it has strengthened community leadership, which is one of the primary goals of the EPA-CDC/Agency for Toxic Substances and Disease Registry collaboration to support community-based environmental health activities.²⁵ Specifically, PACE EH was described as an outreach and empowerment vehicle. One community member noted, “It is a vehicle to create action and inspire leadership; it is a way to teach people to fish rather than a way to catch a fish.”

As a process for building workforce competence, the evaluation provided evidence that PACE EH builds skills in many of the nontechnical competencies that were identified in 2000 as necessary for environmental health practitioners to successfully address environmental health challenges.⁴ These competencies, such as information gathering, collaboration, and communication, are the skills that are needed to effectively perform the 10 essential public health services.

The survey revealed mixed results regarding the effects of PACE EH on the breadth of priorities and programs. Few respondents felt that participation had changed the health agency’s priorities or that new programs were being implemented, yet nearly half of the priorities that resulted were new. One interpretation is that even when new environmental health issues are identified and prioritized through PACE EH, respondents do not think that this has translated or will translate into a change in health agency priorities.

There are several possible explanations. One explanation is that the impact of PACE EH has been primarily a shift in emphasis rather than in priorities. It is also the case that many sites had not yet finished PACE EH and thus may not yet have known the full extent of its potential impact on environmental health programs. Another explanation may be that PACE EH succeeded in bringing issues to the forefront that were not within most health agencies’ jurisdiction. Examples include air pollution, which is often handled at the state rather than the local level, and land use planning, which is typically the jurisdiction of planning departments. In many sites, it was not clear what role the health agency might or even could play in implementing these action plans, given the lack of programmatic flexibility. Yet, some clearly felt that PACE EH could facilitate a different approach to these programs. As one public health practitioner noted, “We can’t change in the short run the way our programs are established; they are driven by rules and regulations. But now we look for opportunities for the community to be involved in programs.”

Overall, most PACE EH implementers we surveyed or interviewed were strong supporters, using words such as “fabulous,” “gratifying,” and “tremendous” to describe the process. As with all methods, however, PACE EH had its detractors. Some felt that the time spent was not worthwhile because they never succeeded in getting to action. One practitioner said, “We accomplished only the relationships and my knowledge [growth]. We didn’t really get even one thing accomplished; we needed to have implemented something.”

The impacts were clearly uneven. This suggests that how the protocol is implemented can make a difference. For some health agencies and communities, implementing PACE EH has had a profound impact on the way the agency and its staff carry out their environmental health responsibilities. PACE EH has led them on a journey in which they have moved beyond their regulatory responsibilities to embrace a larger role in which they actively engage with the community in addressing environmental health issues of importance. In so doing, they have broadened their conception of environmental health and have redefined their own role to be more in line with the 10 essential public health services. In a very few cases, this transformation has reached into the community and begun to change residents’ perceptions of their environment, the agencies that serve them, and their ability to effect change. These are the sites that offer proof that the transformative vision of PACE EH can be realized.

For other health agencies and communities, however, the journey has been much less profound. A dialogue has begun and participants in the process have grown from the experience, both personally and professionally, but this has not resulted in broader changes within the agency or community. These are the communities in which PACE EH is likely to be viewed as a project. The resulting action plans now sit on a shelf and no one takes responsibility for implementation.

CONCLUSIONS

The factors that determine the outcome are not clear, but this study suggests that both resources and the presence of a champion with political clout within the agency are key factors. The sites in which this transformation was most pronounced had staff members who were passionate about the value of community involvement and about redefining the ways in which the agency works with the community. These sites also had dedicated implementation resources. Agencies with few resources are likely to be able to make only small steps in the process. These agencies hope to have a blueprint for action—PACE EH action plans—if resources become available.

The current emphasis nationally to promote the use of the 10 essential public health services and to build leadership capacity at the local level will serve to promote the types of changes that PACE EH is designed to facilitate.^5–7 As more environmental health practitioners embrace a new role for environmental health, and as workforce competencies increase, the conditions for successfully using a community-based environmental health assessment tool to empower local agencies in partnership with the community to achieve positive environmental health outcomes will be increasingly present. When a local agency is ready to embrace community-based planning, PACE EH will be an important tool for local communities to consider.