20 Years of Public Health Economics and Decision Sciences at the US Centers for Disease Control and Prevention: The CDC Steven M. Teutsch Prevention Effectiveness Fellowship, 1995-2015

Abstract

The CDC Steven M. Teutsch Prevention Effectiveness Fellowship was started in 1995 to provide postdoctoral training in public health economics. This article describes the origins and state of the fellowship and the practice of prevention effectiveness research at the Centers for Disease Control and Prevention. The fellowship can be seen as one successful example of a demand-driven public health innovation to develop crucial capacity for the contemporary health system. Nearly 150 individuals have been trained through the program since its inception.

Because resources for public health are limited, research is needed to identify effective and efficient investments of these public health funds.¹ The Institute of Medicine has outlined challenges associated with efficiently allocating health resources.² It has emphasized the importance of all communities enhancing their understanding of what they receive in return for their public health investments. The Institute of Medicine has advocated use of economic studies to determine the cost, efficiency, cost-effectiveness, and economic influence of strategies and interventions to prevent and control disease. The Centers for Disease Control and Prevention’s (CDC’s) Director Dr Thomas Frieden commented that “the dissemination of accurate information on costs and benefits may be the best way to reduce opposition and implement effective public health actions.”^3(p1859) However, a trained workforce is required to conduct studies that will provide such information. CDC’s Prevention Effectiveness (PE) Fellowship was started in 1995 to provide the agency with the capacity to conduct applied public health economics studies. We describe the history and state of the PE fellowship (renamed the CDC Steven M. Teutsch Prevention Effectiveness Fellowship in 1997 to acknowledge Dr Steven M. Teutsch’s multiple contributions to the field and practice of health economics at CDC), as well as illustrate influences that fellows and alumni have had regarding public health policy making. Such information will aid other public health agencies and academic units to design their own applied public health training programs.

Introducing PE at CDC

During the late 1980s, researchers at CDC began to focus on integrating economic evaluation and decision sciences into the work of CDC. Before that, episodic efforts were limited to specific studies related to cost-effectiveness of vaccination programs and human immunodeficiency virus prevention activities.^4–7 When Dr William Roper began his term as CDC Director in 1990, his priorities included demonstrating the value of prevention by applying tools such as decision analysis, economic evaluation, applied mathematics, and modeling to public health challenges.

Drs Roper and Stephen B. Thacker first described applied public health economics at CDC in 1990, introducing the term “prevention effectiveness” to the public health literature.⁸ They identified the following 5 strategies for making the integration of health economics into CDC’s policy planning and program evaluation a practical reality: (1) apply systematic strategies for evaluating the effectiveness of prevention; (2) conduct PE assessment by using consistent methods for determining outcomes and economic effects; (3) set research priorities with prevention in mind; (4) use PE study results to design prevention programs; and (5) use effectiveness data more often during policy making. Published articles subsequently defined PE tools, including cost analysis, economic evaluations, decision and disease transmission modeling, regulatory analysis, and budget impact analysis.^9,10

Dr Roper assigned a team to refine the PE concept and its practice. This team, composed of epidemiologists, medical doctors, policy analysts, and contracted economists, developed trainings and delivered them to CDC program staff and Epidemic Intelligence Service officers, hired staff (including CDC’s first full-time economist), performed economic and decision science technical assistance for CDC programs, formed the PE Technical Working Group, performed an evaluation of how to organize PE activities, and authored the first textbook regarding PE methods.¹¹ This team identified PE research at CDC to include the use of econometric, decision, simulation, operations analytic, and mathematical modeling to describe determinants of health, morbidity, mortality, health inequalities, and health care use and expenditure.¹¹ Finally, the team suggested the development of a training program in PE that ultimately became the PE fellowship.

CDC Steven M. Teutsch Prevention Effectiveness Fellowship

The PE fellowship emerged from the developmental stage in the early 1990s as a 2-year, postdoctoral, applied research and training program in the field of public health economics, quantitative policy analysis, and integrative health services research. The PE fellowship mission is “to establish a cadre of quantitative policy analysts at CDC whose work provides information for health policy decision makers in CDC, Congress, and nongovernmental entities regarding allocation and use of resources to maximize the impact of public health programs.” The PE fellowship is unique in that it is an applied research fellowship focusing on the application of decision sciences and economics to public health, it is hosted within the federal government, it provides fellows with exposure to a wide range of disease and programmatic areas, it encourages fellows to collaborate with epidemiologists, it provides fellows with opportunities to contribute to emerging public health challenges (bioterrorism, pandemic influenza, and Ebola and Zika virus disease), and it has a substantial competency-based training component. The PE fellowship is responsible for the development of a majority of the existing economics and decision science capacity at CDC (Figure 1). For example, PE fellowship alumni make up more than 70% of the full-time, PhD-level economists at CDC. CDC has more economists than any other operating division of the Department of Health and Human Services (https://infogr.am/federal-health-economist?src=web).

FIGURE 1.

Cumulative CDC Prevention Effectiveness Capacity by Year Attributable to the Prevention Effectiveness Fellowship

The PE fellowship program welcomed its initial class of 5 fellows in 1995. During 1995–2015, the PE fellowship trained and graduated 133 fellows in 19 classes and there are 21 fellows in the 2015 and 2016 classes (Table 1). The fellowship has engaged approximately 10 fellows per class over the last decade, making the fellowship the largest of its kind in the United States. The number of fellows engaged each year is demand-driven and dependent on the availability of host organization funds to pay the fellows’ salary, benefits, and other expenses. To be eligible for the program, candidates must have a doctoral degree in economics, health services research, engineering, public policy, policy analysis, or a related quantitatively oriented field. The majority of PE fellows (66%) have been graduates of economics programs. Fellows are competitively selected and matched to CDC programs that have submitted requests and committed funding. The fellowship routinely receives more than 10 eligible, competitive applications for each fellowship position. The fellowship advertises through various job Web sites and accepts applications at www.cdc.gov/pef from September through January.

TABLE 1.

Numbers of Steven M. Teutsch Prevention Effectiveness Fellows by Year of Entry Into the Program, Sex, Educational Background, and Postfellowship Employment at CDC

Class	Total	% Male	Educational Background					% Employed at CDC Postfellowshipa
			Economics	HSR	Policy	IE or OR	Other
1995	5	80	3		1	1		100
1996	11	82	9		1		1	45
1997	8	50	8					63
1998	4	25	3			1		25
1999	3	67	3					33
2000	9	33	8			1		56
2001	16	38	15	1				75
2002	5	80	5					80
2003	5	40	3	1		1		40
2004	6	100	6					67
2005	4	25	3	1				50
2006	3	67	2			1		100
2007	4	50	3		1			100
2008	4	50	2	2				100
2009		60	2		3			60
2010	8	63	3	1		2	2	50
2011	7	57	2	3			2	43
2012	13	46	7	2		4		62
2013	13	31	5	3	2	2	1	85
2014	10	10	3	2	1	1	3	60a
2015	12	25	5	1	2	1	3	…a
2016	10	30	6	1	3	0	0	…b
	165	53	106	18	14	15	12	64

Abbreviations: CDC, Centers for Disease Control and Prevention; HSR, health services research; IE, industrial engineering; OR, operations research.

^aCurrently with CDC as of May 2016.

^bStarting in August 2016.

Early operational challenges included finding suitable mentors, developing a sustainable funding stream, and developing initial administrative policies and procedures. The fellowship program has had to respond to the needs of CDC programs to gain stability and growth. From 2008 on, the fellowship has engaged an increasing number of graduates from noneconomics programs (eg, engineering, operations research, health services research, and health policy), evidence of an ever-growing disciplinary diversity of fellows (Figure 2). This wider range of disciplines reflects the complex needs of CDC programs. Since the program’s inception, fellowship assignments have been evenly divided between infectious disease and chronic disease areas at CDC. The majority of fellows (63%) find employment with CDC at the conclusion of their fellowships. Fellows who leave CDC at the conclusion of their fellowship find positions in academia, state/local health departments, consulting firms, pharmaceutical firms, or other public service organizations. As the fellowship has matured, a standard set of policies and procedures has emerged; fellows now occupy Title 42 full-time equivalent positions at CDC (paid at the GS grade 12, step 1 level), they receive full federal benefits, they are all paired with experienced mentors, and non-US citizens are brought on board on a range of visa options.

FIGURE 2.

Educational Background (Economics vs Noneconomics) for Fellows of the CDC Prevention Effectiveness Fellowship, 1995–2015, by Year of Entry

Abbreviation: CDC, Centers for Disease Control and Prevention.

Fellowship Didactic and Experiential Curriculum

The fellowship training supports a set of competencies that were identified in 2011 and includes didactic and experiential components. Five PE competency domains were identified: public health science and practice; quantitative analysis; policy analysis and communication; interpersonal and professional communications; and foundations of leadership. All fellows participate in a program of didactic trainings (Table 2). The fellowship’s core didactic curriculum consists of courses taught by CDC scientists and external subject matter experts and take up approximately 7% of a fellow’s time. The curriculum is continually evaluated and updated to meet the needs of fellows. For example, recent additions to the curriculum include Budget Impact Analysis, Regulatory Impact Analysis, Introduction to Population Health Science and Practice, and Developing a Policy Brief.

TABLE 2.

CDC Steven M. Teutsch Prevention Effectiveness Fellowship Didactic Courses by Competency Domain^a

Competency Domain	Courses
Public health science and practice	Introduction to the CDC and History of the CDC
	Introduction to Population Health Science and Practice
Quantitative analysis	Introduction to Prevention Effectiveness Research
	Prevention Effectiveness Research Methods
	How to Read and Evaluate Prevention Effectiveness Research
	Epidemiology Methods
	Introduction to Disease Transmission Modeling
	Introduction to Burden and Cost Analysis
	Introduction to Economic Evaluation
	Introduction to Regulatory Impact Analysis
	Introduction to Budget Impact Analysis
	Introduction to Health Impact Analysis
	Advanced Topics in Decision Analytic Modeling
	Cost-effectiveness Analysis the “CDC Way”
	Methods for Systematic Reviews of Economics Literature
	Introduction to MS Excel for Public Health Modeling
	Visual Basic Applications for Excel
	Introduction to TreeAge Software
	MEPS Data
	MedStat/MarketScan Data
	Prevention Effectiveness Project Management
Policy analysis and	Economics and Policy at CDC
	Writing a Policy Brief
Interpersonal and professional	Scientific Writing
	Developing and Delivering Effective Presentations
	Networking for Success
	Getting a Job at CDC
Foundations of leadership	Action Learning for Professional Development
	Fellowship Survival Skills and Navigating the CDC Culture
	Fellowship Goal-Setting and Performance-Planning

Abbreviations: CDC, Centers for Disease Control and Prevention; MEPS, Medical Expenditure Panel Survey.

^aCourses vary from year to year on the basis of need and resource availability.

PE fellows are assigned to different CDC units where they undertake their experiential training—taking a lead role in PE studies, including the assessment of policies, programs, and practices by determining their cost, effectiveness, and overall public health influence. Fellows are expected to complete a set of performance requirements that demonstrates achievement of the competencies. These include authoring 2 scientific papers suitable for publication in a peer-reviewed journal, delivering 2 scientific presentations, developing a policy brief, and delivering 2 methods-based education sessions. Progress toward completion of performance requirements is evaluated by the PE fellowship program, the fellows’ supervisor, and the fellows’ mentor on an ongoing basis. Each fellow is assigned a nonsupervisory mentor who is an alumnus of the program to provide technical guidance as well as support navigating the culture of CDC. In addition to their work with the program to which they are assigned, fellows can be deployed to the “field” to participate in outbreak responses or other projects.

Influence of the Fellowship

All public health professionals need a basic understanding of economics (particularly cost analysis), decision science, and health impact assessment, but the need for a cadre of highly skilled professionals to do the rigorous technical work, such as advanced cost-effectiveness analyses, modeling, and decision analysis, has been noted.^12,13 CDC’s PE fellowship program fills a critical public health need by identifying the cost and cost-effectiveness of public health interventions and engaging practice and policy leaders in the implementation of these interventions. The PE fellowship has been the core resource for building CDC’s capacity to assess the effectiveness and value of public health interventions and facilitating evidence-based decision making. As mentioned earlier, the vast majority of economists and decision scientists at CDC are PE alumni. These scientists have changed the culture at CDC and have led the development of the science around public health economics at CDC. PE fellowship alumni have authored or coauthored approximately 3700 publications since the fellowship started in 1995 (CDC unpublished data, 2016). Programs with teams of economists trained through the PE fellowship have been successful in developing research, policies, and practices that have reflected reasonable decisions, respectful of limited health resources. The following research by PE fellows and alumni exemplifies the influence of PE on population health.

• Demonstrated potential reduction in cost as a result of prevention and health promotion. A 2010 Health Affairs publication presented an analysis of the estimated cost of adopting a package of 20 proven preventive services (eg, tobacco cessation screening, alcohol abuse screening, and daily aspirin use) against the estimated savings that could be generated.¹⁴ The authors reported that greater use of proven clinical preventive services in the United States could avert the loss of more than 2 million life-years annually, and increasing the use of these services from current levels to 90% could result in total savings of $3.7 billion, or 0.2% of US personal health care spending. The authors concluded that “effective clinical preventive services can achieve the dual goals of improving the health of all Americans and making prudent use of scarce resources.”

• Demonstrated influence of CDC programs. Another Health Affairs article from 2014 presented a historical economic model to measure the net economic benefits of preventing central line–associated bloodstream infections in Medicare and Medicaid patients in hospital critical care units during 1990–2008, a time when reductions attributable to federal investment resulted primarily from CDC efforts.¹⁵ The authors concluded that CDC central line–associated bloodstream infection prevention efforts prevented approximately 40 000 to 75 000 infection cases and produced net benefits of at least $640 million during 1990–2008.

• Demonstrated potential lives saved by public health interventions. A recent study published in Pediatrics concluded the routine vaccination of children born in 2009 will prevent 42 000 early deaths and 20 million cases of disease, with net savings of $13.5 billion in direct costs and $68.8 billion in total societal costs.¹⁶

• Demonstrated enhancements in population health. A 2013 collaboration among CDC, the American Cancer Society, and the National Cancer Institute published in Preventive Medicine reported implementing a provision of the Affordable Care Act—the elimination of cost sharing for recommended preventive services—resulted in a significantly increased use of these services, particularly for blood pressure checks, cholesterol checks, and influenza vaccinations.¹⁷

• Demonstrated effectiveness of public health preparedness. CDC developed an influential mathematical model to estimate the potential magnitude of the Ebola epidemic in West Africa. The model indicated that the possibility existed to reverse the epidemic if a sufficient number of Ebola-infected patients were effectively isolated, either in treatment units or in other settings.¹⁸

The PE fellowship has also provided CDC with the capacity to provide public health officials and decision makers with estimates of potential effects of influenza pandemics and bioterrorist events, as well as the potential influence of interventions. These analyses have enhanced the planning and preparation for such events and influenced the spending of billions of dollars of federal preparedness funds. The first such work coauthored by a PE fellow provided an economic analysis of the value of preparing for a bioterrorist event with anthrax, tularemia, or brucellosis, which illustrated that substantial societal benefits resulted from investing in systems and technologies that allowed rapid responses to such events.¹⁹ PE fellows also led the first analysis of the potential effect of a 1968-type influenza pandemic (ie, what would happen if the 1968 influenza pandemic were to occur now), as well as estimating benefits of an influenza vaccination program.²⁰ Perhaps, most important has been the translation of various models into basic influenza pandemic planning software, called FluAid, FluSurge, FluWorkLoss, FluLabSurge, and CommunityFlu.^21,22 Other such planning tools produced by PE fellows include those for smallpox or influenza pandemic mass vaccination clinics, determining optimal size of the national pediatric vaccine stockpile, and planning for medical care needs after a disaster (eg, earthquake or severe hurricane).^23–25 These tools are online (http://www.cdc.gov/flu/pandemic-resources/tools/index.htm) and designed to help state and local public health officials plan and prepare for the next influenza pandemic. The influence of such tools can be judged, in part, by the fact that they have been downloaded approximately 100 000 times (CDC, unpublished data, 2015).

PE fellows have also provided key support to the 2014–2016 Ebola outbreaks in West Africa. Two second-year PE fellows published a study in The Lancet, titled “Preventive Malaria Treatment for Contacts of Patients With Ebola Virus Disease in the Context of the West Africa 2014–15 Ebola Virus Disease Response: An Economic Analysis.”²⁶ This study reported that administration of malaria treatment to contacts of patients with Ebola virus disease was cost saving for contacts of all ages in Liberia, Sierra Leone, and Guinea, during both seasons, from a health care provider perspective and should be considered by public health officials when addressing Ebola virus disease outbreaks in countries and seasons where malaria reaches substantial transmission levels.

Conclusion

The influence of public health economics, decision sciences, and modeling at CDC is in substantial part attributable to the capacity built through the PE fellowship. In a recent supplement to the American Journal of Preventive Medicine, titled “The Use of Economics in Informing U.S. Public Health Policy,”²⁷ 5 of the 10 articles are coauthored by graduates of the PE fellowship program and Dr Steven M. Teutsch coauthored 3 others. This is an exceptional addition to the national dialogue around economics and public health policy and displays the impact of the PE fellowship. Since its inception in the early 1990s and formal beginning in 1995, the PE fellowship has trained fellows, built capacity in various centers across CDC, continually updated its public health economics competencies, raised the visibility of public health economics, communicated the value of economics and decision sciences, and, ultimately, created the capacity in public health economics and decision sciences that sets CDC apart from other federal agencies. According to a recent poll of the PE fellowship graduates at CDC, 98% agreed or strongly agreed with the statement “I would not be here now if it weren’t for the PE fellowship” (CDC unpublished data, 2016). Key opportunities for growth involve expanding core economics and decision sciences capacity in CDC centers and divisions that do not currently have such capacity; expanding public health economics and decision sciences capacity to state and local health departments; continuing to engage the health in all policies agenda^28,29; and participating in population health initiatives sponsored by CDC and its partners that bring CDC economics and decision science expertise to health care providers, community organizations, health payers, and the business community. Despite the substantial contributions of the PE fellowship over the past 20 years, there still exists a considerable gap between existing and needed competency in economic evaluation among public health practitioners.³⁰

In his 2014 American Journal of Public Health article, titled “Six Components Necessary for Effective Public Health Program Implementation,” CDC Director Frieden underscores the importance of PE: “To establish an effective intervention package, it is critical to understand the full range of available evidence-based strategies, the size and characteristics of the population to be reached, the projected impact of each intervention, and the estimated cost.”^31(p19) The PE fellowship has been and will continue to be instrumental in developing the analytic capacity to provide such assessments and thus improve the science of public health.

Implications for Policy & Practice.

• Over 20 years, the post-doctoral CDC Steven M. Teutsch Prevention Effectiveness Fellowship has been extraordinarily successful at integrating new disciplines (health economics and disciplines with similar analytic skills) into the already multi-disciplinary field of public health.

• Public health organizations that wish to replicate this success to enhance their analytic and mathematical modeling capabilities can use the CDC PE Fellowship as a model of how to recruit, suitably train, and successfully integrate recent graduates from academia into applied public health settings.

• Key elements to ensure the success of such a program include:

  1. a formal, focused, competency-based training program;
  2. pre-set fellowship requirements that match closely with the needs of the organization (eg, produce manuscripts of immediate use to practicing public health officials);
  3. identification of supervisors who have both the experience and appropriate temperament to supervise fellows;
  4. assigning (and training in some cases) mentors that can provide the fellows and supervisors with relevant perspective and advice; and, finally;
  5. the establishment of an office or program whose main task is to assure the success of the fellowship.