Skip to main content
PMC home page
Preventive Medicine Reports logoLink to Preventive Medicine Reports
. 2016 Jan 19;3:135–138. doi: 10.1016/j.pmedr.2015.11.015

What are the benefits and risks of using return on investment to defend public health programs?

Astrid Brousselle a,⁎, Tarik Benmarhnia b, Lynda Benhadj c
PMCID: PMC4929139  PMID: 27419005

Abstract

Return on investment (ROI) is an economic measure used to indicate how much economic benefit is derived from a program in relation to its costs. Interest in the use of ROI in public health has grown substantially over recent years. Given its potential influence on resource allocation, it is crucial to understand the benefits and the risks of using ROI to defend public health programs. In this paper, we explore those benefits and risks. We present two recent examples of ROI use in public health in the United States and Canada and conclude with a series of proposals to minimize the risks associated with using ROI to defend public health interventions.

Keywords: Return on investment, Public health, Savings, Economic evaluation

Highlights

  • •

    APHA and CPHA estimates of ROI on major public health range from 125% to 3900%.

  • •

    Assessing the economic value of public health programs using ROIs could provide robust arguments in their defense.

  • •

    Even if there are known benefits to using ROIs, there are also several risks.

  • •

    We offer recommendations for using ROI while acknowledging programs full range of impacts.

Introduction

Return on investment (ROI) is a ratio that has come into increasing use over recent years. ROI indicates how much economic benefit is derived from a program in relation to its costs. This ratio, calculated to demonstrate how relevant investments are, has the esthetic quality of being remarkably synthetic. Both private sector and non-profit firms use it to stimulate co-investments. Foundations, for example, seek to attract more funds for their activities by demonstrating to their funding partners that money is wisely invested. In recent years, there has been growing research interest in ROI use in the public health sector,(Baxter et al., 2014, APHA, 2013, CPHA, 2013) mainly to demonstrate the economic value of public health programs in a context of rationed public spending. In this paper, we explore the benefits and risks of using ROI to defend public health programs. The issue is salient because ROI-based public decisions will necessarily influence the public good and people's well-being and may have an impact on resource allocation and use. To orient our discussion, we present two recent examples of ROI use in public health and explore the context within which its use is expanding in public health economic evaluation. We then discuss the benefits and risks of using ROI to defend public health programs. Lastly, we offer proposals for using ROI information while acknowledging the full range of impacts of programs.

ROI in public health

Both the American Public Health Association (APHA) and the Canadian Public Health Association (CPHA) recently launched videos showing returns on investment for various programs, advocating that “investing in public health is smart” (APHA, 2013, CPHA, 2013). In these videos (see screenshots in Fig. 1), programs are compared in terms of their different ROI ratios. Some of the key CPHA messages are:

Every $1 spent on early childhood health and development saves up to $9 in future spending on health, social and justice services. Return on investment: 800%.

Every $1 invested in tobacco prevention programs saves up to $20 in future health care costs. Return on investment: 1900%.

Fig. 1.

Fig. 1

Open in a new tab

Screenshots from Public Health: A Return on Investment (left, CPHA) and Public Health is ROI (right, APHA).

The CPHA video concludes by saying, “Public health is a return on investment… Make investing in public health a priority.” The APHA video ends with the same message: “Public health is a return on investment: prioritize public health funding.”

Clearly, using ROI in the public sector is a way to defend public health programs in a context where funding is threatened, budgets are controlled, and the relevance of public health interventions still needs to be justified (Stine and Chokshi, 2012, Ifanti et al., 2013, Potvin, 2014). While this particular trend is new, comparing costs to economics benefits has a long-standing history in the field of economic evaluation. Cost-offset and ROI studies have been around for years (Kelly, 2005, Neumann et al., 2008, Chattopadhyay and Carande-Kulis, 2004, Drummond et al., 2008, Brent, 2003).

More often than not, public health interventions are complex undertakings (Shiell et al., 2008). A significant proportion of them target effects produced over the long term whose impacts are diffuse and may affect a variety of sectors (e.g. health, education, employment) (Alayli-Goebbels et al., 2013). Their adverse effects, which do exist, are also difficult to document (Drummond et al., 2008, Goldsmith et al., 2004, Brousselle and Lessard, 2011, Coast, 2004, Weatherly, 2007, Killoran et al., 2009). Many of these interventions also have externalities, in that they affect people not directly targeted by the intervention. Finally, the nature of these interventions makes it difficult to use experimental designs, and, when measuring the effects, researchers must contend with both contextual influences and those of concurrent interventions. These characteristics render causal relationships very complex, such that it is not often easy to attribute observed effects to a given intervention (Shiell et al., 2008). Identifying effects, measuring them, and attributing value to them are challenges in themselves (Alayli-Goebbels et al., 2013, Weatherly et al., 2009, Chalkidou et al., 2008). All these characteristics may explain why economic evaluations of public health programs and interventions have traditionally been under-represented in published economic evaluations (Drummond et al., 2008, Goldsmith et al., 2004, Weatherly, 2007, Killoran et al., 2009). Most economic evaluation studies deal with new technologies or drugs, while fewer than 10% address upstream interventions such as public health interventions,(Rush et al., 2002) with the majority (55%) of those targeting clinical prevention (Goldsmith et al., 2004).

In recent years, an emerging movement has provoked in-depth reflection on what economic evaluation methods are best suited to public health programs and interventions. The publication of the Wanless report (Allin et al., 2005) and the creation of the NICE Centre for Public Health Excellence in the United Kingdom have prompted considerable questioning and redeployment of economic evaluation methods for public health programs and interventions (Kelly, 2005). These reflections found concrete expression in the NICE report Supporting Investments in Public Health, published in 2011 (NICE, 2011). In Canada, a similar reflection occurred, (PHAC, 2009) resulting in a report that proposed a way to grasp the complexity of public health programs using economic evaluation methods (Husereau et al., 2014). It is important to note that all these published reports give new prominence to the valuation of impacts. They explicitly suggest including complementary information on ROIs (NICE, 2011) or “some estimate (or valuation) of the monetary value of added improvement in health outcomes” (Husereau et al., 2014). These publications are examples of the new trend of using ROIs when conducting economic evaluations of public health programs and interventions. They have also helped to institutionalize this approach, with the consequence that ROIs are now being used not only in advocacy efforts, but also, and more importantly, in decision-making on resource allocation. This new prominence given to ROI in public health raises the need for thoughtful reflection on the benefits and risks of this orientation.

Benefits and risks

Benefits

In 2008, Neumann et al. observed that:

“The value of governmental public health systems may seem obvious in light of progress in public health over the past century. The reality of chronic underfunding of these systems suggests that the general public is unaware of public health's value” (p. 2173)(Neumann et al., 2008).

Funding for public health has always been difficult to secure (Freedman et al., 2013, CPHA, 2010, Robert Wood Johnson Foundation, 2013). Even though public health programs and interventions may lead to important population health gains, in our society, medical care still receives the larger portion of funding (Robert Wood Johnson Foundation, 2013). In recent years, in Canada and in the United States, public health expenditures represented 5.3% and 3% of total health spending, respectively (CIHI, 2013, Harvard and Chan, 2012, Institute of Medicine, 2012). In contexts where public expenditures are being reduced, there is a real risk that the proportion of resources devoted to public health will shrink. Public health has historically contributed to changing the epidemiological profile of societies. Today's public health challenges, which include chronic diseases, life course epidemiology, and environmental health issues, call for multisectoral interventions.

ROI is a timely tool with which to advocate for public health interventions that have long term implications and require substantial investments. Assessing the economic value of public health programs and interventions using ROIs, cost-offsets, and profitability threshold analysis could provide robust arguments in their defense. First, it would turn the discourse around, such that the public health budget would no longer be seen as an expense, but rather as an investment that is often profitable over the short, medium, or long terms. Second, in a context of controlling public expenditures, such arguments might influence public opinion and ultimately help safeguard funding for public health programs and interventions. Third, for public health interventions with demonstrated savings, the issue would no longer be their worthiness as financial investments, but rather their affordability, as the question then becomes: even if we know the benefits outweigh the costs, can we afford to invest today in a public health intervention whose effects will only manifest over the longer term? Affordability might present the next serious obstacle to funding, and budget impact analysis may be necessary as complementary decisional information.

Risks

Even if there are known benefits to using ROIs in relation to public health interventions, there are also several risks. First, using ROIs inevitably engenders comparisons. For instance, using the two previously mentioned videos, we ranked the ROIs for various public health programs (Table 1). A cursory look at the results prompts questions such as this one: given that the ROI of water fluoridation is twice that of early education programs, should water fluoridation have priority over early education? If allocation decisions were based only on ROI, would it mean interventions with the lowest ROIs should not be funded?

Table 1.

Examples of public health intervention ROIs, ranked highest to lowest.

Intervention ROI Source
Child safety seat 3900% CPHA and APHA
Water fluoridation 3700% CPHA and APHA
Mental health and addiction 3600% CPHA
Tobacco prevention 1900% CPHA
Vaccination 1500% CPHA
Early education 1300% APHA
Biking and walking opportunities 1200% APHA
Food and nutrition 1000% APHA
Childhood health and development 800% CPHA
Workplace safety 500% CPHA and APHA
Cleaner vehicles 300% CPHA
Tobacco cessation 125% APHA
Open in a new tab

Taking ROIs into account in allocation decisions may considerably change which interventions are funded and which are not. Decision-makers may wish to stop funding interventions that do not show a positive ROI. What would these interventions be? Who would be the losers? And what would the social impacts be of reallocation decisions? Finally, how would innovation and experimentation be funded if showing a positive ROI became a condition of funding?

Second, there are important concerns related to the way ROIs are calculated. As discussed earlier, public health interventions are complex, with effects that are sometimes scattered and intangible and with externalities that are neither easily quantifiable nor easily convertible into dollars (Shiell et al., 2008, Goldsmith et al., 2004, Payne et al., 2013). ROI calculation readily accommodates medical costs averted with improved health (e.g. reduced hospitalizations related to asthma, shortness of breath, lung cancer treatment), but assigning dollar values to life-years saved, to intangible effects such as increased well-being or empowerment, and to externalities is not so easy. While some evaluators do not hesitate to attribute an economic value to life-years saved, this is still very much subject to ethical debate (Drummond et al., 2005, Pinkerton et al., 2002, Robinson, 1993, Kelman, 1981, Mooney, 1980, Weinstein and Fineberg, 1980, Mishan, 1971). On the other hand, not including these effects because of a conviction, on principle, that they should not be converted into dollars will inevitably favor interventions with more acute and easily measurable impacts. Finally, whereas public policies and organizations advocate for the importance of integrating equity considerations into public health evaluations (World Health Organization, 1986). ROI methodology has no way of taking into account equity benefits, such that these consequences do not figure in the calculation.

Discussion

The mission of public health is to prevent diseases and traumas and to protect and promote health (World Health Organization). In austerity contexts, ROIs can offer reassurance on the economic value of public investments in this sector (Stuckler & Basu, 2014). However, quantifying the economic benefits of public health programs can also lead to people being seen as inputs into the overall economic system, with no intrinsic value other than the economic advantages their health and activities can provide.

Public health has a value in itself, as it has the potential to positively impact the health of humans. Nevertheless, every public investment has an opportunity cost: when we decide to invest in one program, our capacity to invest in another is reduced. There is no reason why public health should be exempt from this predicament. How, then, are we to be guided in these considerations? Based on the above discussion, we propose the following guidelines to minimize risks associated with using ROIs to defend public health interventions:

  • (1)

    Defend public health programs for what they do. ROIs should not be central to the discussion, but rather should be seen as complementary information to be considered after effectiveness indicators. Furthermore, we must always keep in mind that cost and benefit data are eminently contextual and that, as such, ROIs cannot be exported to other contexts than those used for the study.

  • (2)

    ROIs should only be used for equivalent alternatives and not to compare interventions that are different in their objectives.

  • (3)

    To avoid ethical questions, it would be judicious and sufficient simply to assess whether benefits outweigh costs, without attempting to identify, calculate and translate into dollars all the effects of public health interventions and programs (Tchouaket et al., 2013).

  • (4)

    For programs that may be threatened by ROI comparison, it could be useful to anticipate what the impacts of stopping their funding would be.

  • (5)

    The effectiveness of public health programs should be systematically documented. There is a dearth of reliable data on public health programs; evaluative efforts should be intensified to adapt methodological approaches and compile more evidence.

  • (6)

    Finally, greater use should be made of logic models. Based on evaluation methodologies, the logic model shows the resources invested in an intervention, as well as the activities and the chain of effects, including direct effects (e.g. morbidity, mortality), indirect effects (e.g. reduced utilization of health care and of professional resources in schools, fewer work days lost) and externalities (protection of other individuals) (Funnell & Rogers, 2011). Logic models are a powerful tool to represent all the effects, both those included in ROI calculations and those that are not.

Conclusion

ROIs are increasingly being calculated to demonstrate the value of investments and ultimately to reinforce funding (Gargani, 2014, King, 2014). Consequently, careful reflection is needed on how their use influences allocation decisions. ROIs of public health interventions are helpful to introduce economic information into the political debate and are used mainly as an advocacy tool in the political arena. It is therefore crucial to understand the basics of how ROIs are calculated and to know their limitations and risks, rather than blindly accepting black-box numbers.

Conflict of interest statement

The authors declare that there is no conflict of interest.

References

  1. Alayli-Goebbels A.F., Evers S.M., Alexeeva D. A review of economic evaluations of behavior change interventions: setting an agenda for research methods and practice. J. Public Health. 2013;36(2):336–344. doi: 10.1093/pubmed/fdt080. [DOI] [PubMed] [Google Scholar]
  2. Allin S., Mossialos E., McKee M., Holland W. The Wanless report and decision-making in public health. J. Public Health. 2005;27(2):133–134. doi: 10.1093/pubmed/fdi014. [DOI] [PubMed] [Google Scholar]
  3. APHA . American Public Health Association; Washington, DC: 2013. Public Health is ROI. (Available at: https://www.youtube.com/watch?v=B5M9JefYxJ) [Google Scholar]
  4. Baxter S., Sanderson K., Venn A.J., Blizzard C.L., Palmer A.J. The relationship between return on investment and quality of study methodology in workplace health promotion programs. Am. J. Health Promot. 2014;28(6):347–363. doi: 10.4278/ajhp.130731-LIT-395. [DOI] [PubMed] [Google Scholar]
  5. Brent R.J. Edward Elgar; Northampton, MA: 2003. Cost–Benefit Analysis and Health Care Evaluations. [Google Scholar]
  6. Brousselle A., Lessard C. Economic evaluation to inform health care decision-making: promise, pitfalls and a proposal for an alternative path. Soc. Sci. Med. 2011;72(6):832–839. doi: 10.1016/j.socscimed.2011.01.008. [DOI] [PubMed] [Google Scholar]
  7. Chalkidou K., Culyer A., Naidoo B., Littlejohns P. Cost-effective public health guidance: asking questions from the decision-maker's viewpoint. Health Econ. 2008;17(3):441–448. doi: 10.1002/hec.1277. [DOI] [PubMed] [Google Scholar]
  8. Chattopadhyay S.K., Carande-Kulis V.G. Economics of prevention: the public health research agenda. J. Public Health Manag. Pract. 2004;10(5):467–471. doi: 10.1097/00124784-200409000-00015. [DOI] [PubMed] [Google Scholar]
  9. CIHI . Canadian Institute for Health Information; Ottawa, ON: 2013. National Health Expenditure Trends, 1975 to 2012. (Available at: https://secure.cihi.ca/free_products/NHEXTrendsReport2012EN.pdf. Accessed June 11, 2015) [Google Scholar]
  10. Coast J. Is economic evaluation in touch with society's health values? BMJ. 2004;329(7476):1233–1236. doi: 10.1136/bmj.329.7476.1233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. CPHA . Canadian Public Health Association; Ottawa, ON: 2010. Enhancing the Public Health Human Resource Infrastructure in Canada. Presentation by the Canadian Public Health Association to the House of Commons Standing Committee on Health. (Available at: http://www.cpha.ca/uploads/policy/enhance_ph_e.pdf. Accessed June 11, 2015) [Google Scholar]
  12. CPHA . Canadian Public Health Association.; Ottawa, ON: 2013. Public Health: A Return on Investment. (Available at: http://www.cpha.ca/en/internal/homepage/roi-video.aspx. Accessed June 8, 2015) [Google Scholar]
  13. Drummond M.F., Sculpher M.J., Torrance G.W., O'Brien B.J., Stoddart G.L. third ed. Oxford University Press; Oxford: 2005. Methods for the Economic Evaluation of Health Care Programmes. [Google Scholar]
  14. Drummond M., Weatherly H., Ferguson B. Economic evaluation of health interventions. BMJ. 2008;337(7673):770–771. doi: 10.1136/bmj.a1204. [DOI] [PubMed] [Google Scholar]
  15. Freedman A.M., Kuester S.A., Jernigan J. Evaluating public health resources: what happens when funding disappears? Prev. Chronic Dis. 2013;10:130130. doi: 10.5888/pcd10.130130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Funnell S.C., Rogers P.J. Jossey-Bass; San Francisco, CA: 2011. Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. [Google Scholar]
  17. Gargani J. Presentation at the 11th European Evaluation Society: Evaluation for an Equitable Society. 2014. Potential sources of bias when estimating the monetary value of impacts. (Dublin, Ireland; October 1–3) [Google Scholar]
  18. Goldsmith L.J., Hutchison B., Hurley J. McMaster University, Centre for Health Economics; Hamilton, ON: 2004. Economic Evaluation Across the Four Faces of Prevention: A Canadian Perspective. [Google Scholar]
  19. Harvard T.H., Chan School of public health . 2012. Public health and the U.S. economy. Available at: http://www.hsph.harvard.edu/news/magazine/public-health-economy-election/. (Accessed June 11, 2015) [Google Scholar]
  20. Husereau D., Jacobs P., Manns B., Hoomans T., Marshall D., Tamblyn R. Institute of Health Economics; Edmonton, AB: 2014. Economic Evaluation of Complex Health System Interventions: A Discussion Paper. [Google Scholar]
  21. Ifanti A.A., Argyriou A.A., Kalofonou F.H., Kalofonos H.P. Financial crisis and austerity measures in Greece: their impact on health promotion policies and public health care. Health Policy. 2013;113(1–2):8. doi: 10.1016/j.healthpol.2013.05.017. [DOI] [PubMed] [Google Scholar]
  22. Institute of Medicine . The National Academies Press; Washington, DC: 2012. For the Public's Health: Investing in a Healthier Future. [PubMed] [Google Scholar]
  23. Kelly M.P. Public health guidance and the role of new NICE. Public Health. 2005;119(11):960–968. doi: 10.1016/j.puhe.2005.08.007. [DOI] [PubMed] [Google Scholar]
  24. Kelman S. Cost–benefit analysis: an ethical critique. Regulation. 1981;5(1):33–40. [PubMed] [Google Scholar]
  25. Killoran A., White P., Millward L., Fischer A. NICE public health guidance. J. Public Health. 2009;31(3):451–452. doi: 10.1093/pubmed/fdp073. [DOI] [PubMed] [Google Scholar]
  26. King J. Julian King & Associates Ltd.; Auckland, NZ: 2014. Value for Investment: A Practical Evaluation Theory. (Available at: http://www.julianking.co.nz/wp-content/uploads/2014/09/140908-VFI-JK-web.pdf. Accessed June 11, 2015) [Google Scholar]
  27. Mishan E.J. Evaluation of life and limb: a theoretical approach. J. Polit. Econ. 1971;79(4):687–705. [Google Scholar]
  28. Mooney G.H. Cost–benefit analysis and medical ethics. J. Med. Ethics. 1980;6(4):177–179. doi: 10.1136/jme.6.4.177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Neumann P.J., Jacobson P.D., Palmer J.A. Measuring the value of public health systems: the disconnect between health economists and public health practitioners. Am. J. Public Health. 2008;98(12):2173–2180. doi: 10.2105/AJPH.2007.127134. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. NICE . National Institute for Health and Clinical Excellence; London, UK: 2011. Supporting Investment in Public Health: Review of Methods for Assessing Cost Effectiveness, Cost Impact and Return on Investment. Proof of Concept Report. [Google Scholar]
  31. Payne K., McAllister M., Davies L.M. Valuing the economic benefits of complex interventions: when maximising health is not sufficient. Health Econ. 2013;22(3):258–271. doi: 10.1002/hec.2795. [DOI] [PubMed] [Google Scholar]
  32. PHAC . Public Health Agency of Canada; Ottawa, ON: 2009. Investing in Prevention—The Economic Perspective. Key Findings from a Survey of the Recent Evidence. [Google Scholar]
  33. Pinkerton S.D., Johnson-Masotti A.P., Derse A., Layde P.M. Ethical issues in cost-effectiveness analysis. Eval. Program Plann. 2002;25(1):71–83. [Google Scholar]
  34. Potvin L. Canadian public health under siege. Can. J. Public Health. 2014;105(6):e401–e403. doi: 10.17269/cjph.105.4960. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Robert Wood Johnson Foundation . Robert Wood Johnson Foundation; Princeton, NJ: 2013. Investing in America's Health: A State-by-State Look at Public Health Funding and Key Health Facts. [Google Scholar]
  36. Robinson R. Cost–benefit analysis. BMJ. 1993;307(6909):924–926. doi: 10.1136/bmj.307.6909.924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rush B., Shiell A., Hawe P. University of Calgary, Centre for Health and Policy Studies; Calgary: 2002. A Census of Economic Evaluations of Primary Prevention Interventions in Population Health. [Google Scholar]
  38. Shiell A., Hawe P., L. G. Complex interventions or complex systems? Implications for health economic evaluation. BMJ. 2008;336(7656):1281–1283. doi: 10.1136/bmj.39569.510521.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Stine N.W., Chokshi D.A. Opportunity in austerity—a common agenda for medicine and public health. N. Engl. J. Med. 2012;366(5):395–397. doi: 10.1056/NEJMp1112543. [DOI] [PubMed] [Google Scholar]
  40. Stuckler D., Basu S. Éditions Autrement; Paris: 2014. Quand l'Austérité tue. Epidémies, Dépressions, Suicides: l'Économie Inhumaine. [Google Scholar]
  41. Tchouaket E., Brousselle A., Fansi A., Dionne P.A., Bertrand E., Fortin C. The economic value of Quebec's water fluoridation program. J. Public Health. 2013;21(6):523–533. doi: 10.1007/s10389-013-0578-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Weatherly H. Assessing the challenges of applying the standard methods of economic evaluation to public health programmes. iHEA 2007 6th World Congress: Explorations in Health Economics. 2007 (Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=991835. Accessed June 11, 2015) [Google Scholar]
  43. Weatherly H., Drummond M., Claxton K. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy. 2009;93(2):85–92. doi: 10.1016/j.healthpol.2009.07.012. [DOI] [PubMed] [Google Scholar]
  44. Weinstein M.C., Fineberg H.V. W.B. Saunders; Philadelphia, PA: 1980. Clinical Decision Analysis. [Google Scholar]
  45. World Health Organization . 1986. The Ottawa Charter for Health Promotion. Available at: http://www.who.int/healthpromotion/conferences/previous/ottawa/en/. (Accessed June 11, 2015) [Google Scholar]
  46. World Health Organization. Trade, foreign policy, diplomacy and health: Public health. Available at: http://www.who.int/trade/glossary/story076/en/. (Accessed June 11, 2015).

Articles from Preventive Medicine Reports are provided here courtesy of Elsevier

RESOURCES