The Oregon Public Health Policy Institute: Building Competencies for Public Health Practice

Abstract

The Oregon Public Health Policy Institute (PHPI) was designed to enhance public health policy competencies among state and local health department staff. The Oregon Health Authority funded the College of Public Health and Human Sciences at Oregon State University to develop the PHPI curriculum in 2012 and offer it to participants from 4 state public health programs and 5 local health departments in 2013.

The curriculum interspersed short instructional sessions on policy development, implementation, and evaluation with longer hands-on team exercises in which participants applied these skills to policy topics their teams had selected. Panel discussions provided insights from legislators and senior Oregon health experts.

Participants reported statistically significant increases in public health policy competencies and high satisfaction with PHPI overall.

Innovative policy solutions to address public health problems are becoming increasingly important, particularly because chronic diseases constitute a growing share of the disease burden in the United States. Public health policies increasingly aim to shape an environment that encourages healthy behaviors, such as physical activity or healthy eating.^1,2

Although traditional public health programs often target smaller groups, such as those infected with or at elevated risk of particular infectious diseases, public health policies can influence the behavior or environment of large populations. Such policies, which include laws, regulations, rules, or operational decisions intended to improve population health, can help jurisdictions meet population health goals because they work “upstream” of heath care services and even many traditional health promotion programs.

The Public Health Division (PHD) of the Oregon Health Authority (OHA) is working to enhance the competencies of the state and local health agencies in Oregon to develop, implement, and evaluate public health policies; these competencies receive limited attention in existing degree programs or in-service training. The PHD funded Oregon State University (OSU) to develop a Public Health Policy Institute (PHPI) tailored to the state’s institutional and political environment, and designed to train public health professionals to address public health problems with upstream policy solutions. We describe the existing public health policy background and training resources nationwide and in Oregon, and outline the structure and content of the PHPI curriculum. We present the results from the evaluation of the first offering of PHPI and reflections on lessons for other states.

BACKGROUND

Many prominent authors and organizations have recently highlighted the importance of policies in meeting public health goals. The director of the Centers for Disease Control and Prevention (CDC), Thomas Frieden, MD, MPH, a leader in pursuing policy approaches and the former director of public health in New York City, identified policy approaches as one of the foundational layers of a pyramid of public health interventions.³ Public policy is a key to changing the environment to make the default option the healthy option; historic examples include placing iodine in salt and fluoride in water. Some modern policies pioneered in New York City, such as restrictions on sugar-sweetened beverage consumption, have been debated in medical journals⁴ and in the courts.⁵ By contrast, policies to control tobacco use have become increasingly common at the state and local level, with 49% of the US population protected by a state or local law prohibiting smoking in workplaces, restaurants, and bars.⁶

The Institute of Medicine issued several reports in which public health policy was a central theme, by laying out the role of policy in combatting chronic disease,⁷ highlighting the role of public health law in improving population health,⁸ and identifying the importance of policy in reducing obesity.⁹ Other authors described political challenges that must be addressed when developing and implementing public health policies, including the need for public health agencies to work effectively with elected officials and community organizations.¹⁰ The evaluation and performance measurement aspect of public health policy were also clarified by authors affiliated with the CDC¹¹ and by the Public Health Foundation.¹²

Public health policy requirements also come from Public Health Accreditation Board standards for health department accreditation.¹³ In particular, the standards call for health departments to “serve as a primary and expert resource for establishing and maintaining public health policies, practices, and capacity” (Domain 5, Standard 5.1). Other required health department capabilities that support policy development and implementation are found under Assessment (Domain 1), Community Engagement (Domain 4), and Public Health Law (Domain 6).

Policy development, implementation, and evaluation have been highlighted as important aspects of public health workforce development. Core competencies promulgated by the Council on Linkages Between Academia and Public Health Practice¹⁴ include a domain devoted to public health policy (Domain 2). However, technical assistance provided by the American Public Health Association Center for Public Health Policy¹⁵ found that relatively few state or local health departments focused on proactive policy activities, such as implementing policies, disseminating information, and policy impact evaluation. Health departments varied widely from a few that offered regular training to many that offered no training. In 2005, the Washington State Department of Health offered a 2-day policy training session to more than 200 participants from health departments, as well as contractors and other partner organizations in the community.¹⁶ Participants reported improved self-efficacy in policy work, and most of them subsequently included policy change priorities in their official work plans.

Existing training programs cover some, but not all, important segments of the continuum of public health policy development, implementation, and evaluation competencies. The Shaping Policy for Health curriculum, which was sponsored by the Directors of Health Promotion and Education,¹⁷ was concurrently offered to staff from community-based organizations and Tribal or county health departments in Oregon. However, it emphasized advocacy, in which state and local health department staff were constrained by being public employees, and gave limited weight to the quantitative analyses that underlay policy development and evaluation. The Evidence-Based Public Health framework was grounded in rigorous quantitative methodologies,^18–20 but did not have an exclusive focus on policy, and it emphasized analysis more strongly than communication and implementation planning.

OREGON PUBLIC HEALTH POLICY INSTITUTE CURRICULUM

In 2012, OHA funded the College of Public Health and Human Sciences of OSU to develop a curriculum for PHPI. Frameworks were synthesized from the literature, and existing curricula were reviewed. The result was a series of public health policy “steps” that could form the basis of a curriculum and equip PHPI participants with specific competencies that they could then apply in their current positions. These competencies were grounded in the disciplines of epidemiology, economics, and public policy, and informed by the practical lessons of public health practice (Figure 1).

FIGURE 1—

Steps in public health policy development, implementation, and evaluation: Oregon Public Health Policy Institute, 2012–2013.

Steps 1 through 6 focus on policy development: identifying and quantifying the problem to be addressed by public health policy, synthesizing available evidence about effective policy options, assessing current local policies and stakeholder viewpoints, and drafting descriptions of the proposed policy. Steps 7 through 9 focus on policy implementation and communication with stakeholders. Step 10 addresses the evaluation of the policy’s implementation and impact. Although the steps are presented in a linear fashion, in practice they are closely interwoven and may not advance in a neat sequence. Some steps may occur in parallel, and there could be loops in which the results of a step lead to reconsideration or refinement of results from previous steps.

The curriculum was designed to be problem-oriented and team-based, because most work in public health agencies is collaborative. Teams of 2 or 3 participants each were recruited from local health departments or state public health programs. Before attending PHPI, teams were asked to complete step 1 of the process by identifying an important local public health problem or need that aligned with 1 or more of the 5-year goals of the OHA Public Health Division Strategic Plan.²¹ This ensured that each team’s needs and context were considered, and that the Institute supported work they were planning or were already doing. Faculty used this information to ensure resources were available to support each team’s selected topic.

The majority of curriculum time within each step was devoted to hands-on exercises that would be engaging for adult learners.²² Short (usually 15-minute) lectures provided participants with an introduction to each step. In-depth exercises (30–90 minutes) followed each lecture and allowed personalized support and technical assistance to teams. Panels of outside speakers were included to share real-world experiences in development and implementation of public health policy in Oregon.

Templates or detailed instructions were developed for each exercise. The quantitative analysis exercise of the local health problem and need took participants through the steps of enumerating the population to be targeted by policy and concisely presenting the underlying problem in terms of incidence, prevalence, and if possible, economic impact. In the stakeholder analysis exercise, participants categorized the particular interests of key stakeholder groups with the aim of maximizing support for the policy and identifying potential partners or opponents in the community. The policy selection exercise required teams to summarize their proposed policies in writing and then explain them succinctly to other teams. The economic analysis exercise, based on a published example, asked teams to compare the costs and benefits of 2 alternative designs for an immunization program. In the communications exercise, participants took turns playing a role—such as their supervisor, a member of the community, or an elected official—and were able to practice communicating about their policy in a low-risk environment.

Figure 2 summarizes the PHPI curriculum as it was offered; however, because of scheduling reasons and to intersperse quantitative and qualitative exercises, the sequencing of lectures and exercises deviated slightly from the order of steps in Figure 1. The 2-and-a-half day length was chosen so that participants could arrive the evening before the PHPI started, return home on the last day, and still have 1 or 2 available workdays that week.

FIGURE 2—

Public health policy institute summary agenda: Oregon Public Health Policy Institute, 2013.

First Offering of Oregon Public Health Policy Institute

OHA Public Health Division was proactive in developing its public health policy capacity. It hired a full-time policy director in 2013, and later that year elected to use CDC infrastructure funds to fund a pilot session of PHPI. OSU worked in collaboration with OHA to solicit team applications from state public health programs, and with the Conference of Local Health Officials to solicit applications from local health departments throughout Oregon. Team members were state and local public health professionals whose participation in PHPI was approved by their supervisors as a professional training activity. Twenty participants represented 4 state public health programs and 5 local health departments from northern, western, and central Oregon. Policy topics chosen by the teams included alcohol abuse prevention, immunizations, reproductive health, and sugar-sweetened beverages.

The PHPI curriculum was offered in August 2013 at the Corvallis campus of OSU. Core OSU faculty were supplemented by the OHA Public Health Division policy director and a consultant with expertise in public health policy communication and enactment. Public health policy doctoral student associates with experience in public health settings assisted faculty in facilitating exercises and providing technical assistance to teams. Panel members included sitting and former state senators, the founding president of a large Oregon health foundation, and the chief operating officer of 1 of Oregon’s integrated Medicaid care delivery organizations.

Evaluation Methods

To assess whether PHPI met its educational goals, a voluntary questionnaire asked participants to describe (on a 5-point Likert scale) their knowledge of several specific public health policy topics before the first session and again after all sessions had been completed. A random code allowed pre- and post-PHPI questionnaires to be paired anonymously for analysis. Eighteen complete pairs of questionnaires were received, for a response rate of 90%. The paired t-test (performed in Stata, version 12; StataCorp, College Station, TX) was used to evaluate the statistical significance of differences in pre- and post-PHPI responses.

At the end of each day, participants were asked to evaluate (on a 1–5 scale) each of the day’s sessions, and on the last day, they were asked to evaluate PHPI overall. These questionnaires also asked for open text comments about what participants found most valuable and what should be added, changed, or dropped. Response rates on these anonymous questionnaires were 70%, 100%, and 90% for days 1 through 3, respectively. Quantitative ratings were calculated for each session and for PHPI overall. We transcribed the open text responses, assigned them to 1 of several analytical domains (satisfaction, lecture or exercise content, pace and scheduling, panel discussions, and logistics), and then summarized responses by domain.

Evaluation Results

Participants reported that their overall knowledge of public health policy development and implementation improved significantly after completing PHPI, as did their levels of 6 specific skills, including identifying evidence-based policy solutions, designing or drafting policies, working with elected officials, and communicating with external stakeholders (Figure 3). Overall evaluation ratings (Table 1) showed that participants found PHPI useful and were highly likely to recommend it to colleagues. They also expressed confidence in their ability to lead or participate in public health policy work.

FIGURE 3—

Participants’ pre- and postassessment of public health policy skills: Oregon Public Health Policy Institute, 2013.

Note. All pre- and post-differences were significant at P < .001.

^aLevels of self-reported knowledge or skill: 1 = very limited; 2 = limited; 3 = moderate; 4 = strong; and 5 = very strong.

TABLE 1—

Participants’ Overall Ratings of Oregon Public Health Policy Institute (PHPI): 2013

Score	Overall, How Useful Was the PHPI (All 3 Days?), No. (%)	How Confident Do You Feel About Your Ability to Lead or Participate in Public Health Policy Work in Your Program or Health Department?, No. (%)	Would You Recommend the Oregon PHPI to Your Colleagues?, No. (%)
5 (high)	13 (72.2)	4 (22.2)	15 (83.3)
4	5 (27.7)	9 (50.0)	3 (16.6)
3	0 (0)	5 (27.7)	0 (0)
2	0 (0)	0 (0)	0 (0)
1 (low)	0 (0)	0 (0)	0 (0)

Open text comments frequently described the team exercises as very valuable. The stakeholder analysis exercise energized teams, with participants reporting they were given a new perspective on the strengths of different stakeholders and the need to identify where those strengths are best used. The policy selection exercise was noted as being quite helpful, because it pushed teams to make their proposed policies explicit. The communication exercise was also very popular; participants reported that it allowed them to anticipate the concerns of different audiences and highlighted the need to frame policy within the perspective of the target audience.

The panel discussions were generally well received, with very positive reactions to the expert panel that addressed the future role of public health and its integration with health care services. Participants appreciated hearing directly from legislators, but noted that, as public employees, they felt constrained in their ability to speak directly to legislators about public health policy issues.

Some sessions were identified as offering opportunities for improvement. Some participants felt that insufficient time was available for quantifying their problem or searching for evidence, especially if regional or local data, or examples of similar policies were not readily available. It was suggested that this work could be done before attending the Institute. Participants also suggested that the economic analysis exercise be simplified or have more time devoted to it, and that the evaluation material could be focused more explicitly on specific aspects of evaluating policies rather than programs.

Participants found the sequential policy development–implementation–evaluation framework quite useful, but also requested that a larger number of specific public health policy examples be incorporated. They found virtually all the sessions valuable, although several noted that the Institute covered a great deal of material given the available time.

SUMMARY AND FUTURE IMPLICATIONS

State and local health departments are playing a greater role in policy, ranging from specific policies to reduce tobacco use, increase physical activity, or promote healthy diets, to health impact analyses of policies proposed by other departments. However, few public health professionals have received training in policy development and implementation, either in their degree programs or subsequent in-service training.

A framework that captures key policy development, implementation, and evaluation competencies for public health professionals was developed and used to create a hands-on curriculum offered to state and local public health teams from Oregon. Participants’ ratings of this initial PHPI were quite positive, and the pre- and post-assessments showed measureable improvement in their self-reported policy skills. Several refinements for future offerings were also identified. The public health policy framework was based on an extensive review of the relevant literature, recommended competencies for public health professionals, and existing health department accreditation standards. The framework was explicitly sequential, to guide structuring of a curriculum, but it had flexibility to accommodate the inherently iterative nature of policy development and implementation. PHPI participants found this framework useful, but there were challenges to covering a diverse range of skills—including evidence synthesis, policy formulation, communication, implementation planning, and evaluation—in less than 3 days.

The curriculum structure successfully built on a team approach and hands-on exercises. This structure appealed to adult learners and matched how policy work is actually done in health departments. Participant engagement was further enhanced by having teams pick real policy problems before attending PHPI. The curriculum exercises then provided them a dedicated time for group work that would be difficult to carve out in their normal work settings. Future PHPI offerings will include faculty follow-up with participants to promote ongoing collaboration by their teams.

Participant learning appeared to be effective, as reflected by self-reported improvements in several specific policy skills. Global ratings and open text comments were also positive. Faculty observed participants’ increased confidence in addressing policy issues, for example, in teams’ draft policy statements and their ability to communicate their policy goals.

The most highly rated exercises were the opening presentation on the evolution of tobacco control policies, the stakeholder analysis exercise, the policy expert panel discussion, and the role-playing communications practice. In addition to team exercises, participants enjoyed networking together in person, especially as mixed groups of state and local public health professionals.

A consistent theme that emerged from discussions and evaluations was the political challenge of dealing with skeptical elected officials locally and busy legislators at the state level. Future PHPI offerings will emphasize that state and local health department staff are most effective when they communicate with elected officials and legislators concisely and on the basis of evidence. They should also not advocate for a specific policy unless elected officials have given it their support.

Other exercises will be targets for future improvements. Quantifying the problem or need to be addressed by policy requires substantial in-depth analysis. Although participants received a compendium of online public health policy resources, synthesizing evidence about effective policies is time-consuming. Economic analysis is challenging to address in a single session. It can also be challenging to identify specific evaluation measures at the early stages of policy development.

The development, implementation, and evaluation of public health policy is complex and evolves over time. In future PHPI offerings, strategies to engage in more pre-PHPI work with teams (in problem or need quantification and evidence synthesis) and follow-up afterward (regarding implementation plans and evaluation) are planned. Online modules, such as for economic analysis or evaluation, are being considered so that in-person time can be devoted to the team exercises and faculty facilitation.

Our report had limitations that should be noted. It covered a single state and the pilot offering of this curriculum; the limited resources did not allow for long-term follow-up. The curriculum is being planned to be offered at PHPI again and with a more extensive follow-up period.

Policy development, implementation, and evaluation are increasingly important competencies for the public health workforce. In Oregon, state health department policy experts and a college of public health collaborated to develop and deliver a PHPI curriculum focused on key policy competencies. PHPI also provided a leadership training opportunity for a new generation of public health professionals, and might serve as an example for other states.