Old Myths, New Myths: Challenging Myths in Public Health

Abstract

Myths are widely held beliefs and are frequently perpetuated through telling and retelling. We examined 10 myths in public health research and practice. Where possible, we traced their origins, interrogated their current framing in relation to the evidence, and offered possible alternative ways of thinking about them. These myths focus on the nature of public health and public health interventions, and the nature of evidence in public health. Although myths may have some value, they should not be privileged in an evidence-informed public health context.

Myths are “beliefs held to be true despite substantial refuting evidence.”^1(p447) They are frequently perpetuated through telling and retelling and periodically there is a need to “bust” myths by examining them in relation to current evidence. In the public health context, this has been done for myths about systematic reviews,^2,3 tobacco control,⁴ and obesity.¹

THE MYTHS

The myths discussed here are focused on the nature of public health and public health interventions, and the nature of evidence in public health.

1. Public Health Interventions Inevitably Represent a “Nanny State”

It has been argued that public health interventions that seek to “interfere” with freedoms or “intervene” in the lives of citizens represent “nanny statism.” Nanny statism arises as a result of governments “telling people how to live their lives.”^5(p146) The tension between libertarian and utilitarian perspectives on interventions has been discussed by the Nuffield Council on Bioethics, which recommends that the role of the state in relation to public health interventions should justify the intensity of intervention with the rationale and strength of evidence for intervening.^5(pxix)

There are alternatives; recent policy directions of the current British government has suggested that a preferred approach is to “nudge” citizens toward healthy choices⁶ rather than to “nanny” the public by overly regulating environments. Although there is some evidence that nudging may work in shaping consumer behavior, there are also cases in which increased regulation may be the more effective route to achieve the scale of change needed to support population health outcomes.⁷ Many of the greatest public health achievements globally have been the result of regulatory efforts of governments to shape environments and industry behavior^8,9 and should be components of the government’s role as a steward of the public’s health as opposed to being a “nanny.”¹⁰

Nanny statism is not inevitably a bad thing and, in some cases, state interventions are a more effective approach than more individualized approaches (which depend on individual choice) to problems that are not addressed, or addressed inequitably, when left to individual choice. For example, in a recent systematic review in the area of alcohol, Martineau et al. found that evidence of effectiveness existed across a number of different population-level interventions, many of which were policy-oriented, and that benefits generally outweighed harms, suggesting a role for government intervention.¹¹

2. Prevention Should Require Fewer Resources

Benjamin Franklin’s expression “an ounce of prevention is worth a pound of cure” has become a widely stated idiom for public health^12–14 with some making a corollary assumption that prevention should also require fewer resources than other areas of health spending.^15,16 Compared with the health care system, public health receives a small proportion of the overall investment in health and preventive services. In Canada, public health accounts for approximately 5% of health expenditures nationally¹⁷—it is closer to 3% in the United States.¹⁸ These figures capture a range of public health interventions beyond health promotion, disease prevention, and health inspection and may also include basic infrastructure and human resource costs.¹⁸

The comparatively small investment could result in a false conclusion that public health interventions are therefore “cheap” (particularly when compared with health care interventions) and should require fewer resources. Although it is true that there is relatively limited cost-effectiveness evidence for many public health interventions (e.g., Owen et al.)^19,20 and, of what evidence was examined, only cost savings in 15% of interventions, it is clear that public health interventions can benefit from significant and sustained upstream investments. Consider, for example, the relationship between large-scale and well-resourced public health interventions such as the Massachusetts and California tobacco control programs and the corresponding large-scale population-level outcomes.^21,22

3. The Only Job of Public Health Research Is to “Translate” Basic Science Discoveries

There is an absolutely critical role for public health in translational research and knowledge translation. Over the past 10 years, considerable literature has accumulated regarding the distinctions among diffusion, dissemination, and knowledge translation and the nature of the public health evidence base and how it is generated to achieve greater impact.²³ The increased focus on investing not only in basic biomedical research or in research on the efficacy of highly controlled interventions but also toward issues of implementation science (e.g., why public health interventions work, for whom, under what conditions, and at what cost) is imperative to seeing the fields of dissemination and implementation research flourish.²⁴

That said, translation of public health research to action is not the only role for public health research. If the pipeline model to research is taken literally, public health research may fall toward the end of a developmental pathway of research designed to move from bench to bedside and perhaps on to populations. In addition to the relevance and knowledge translation challenges that have already been described when one looks along the pipeline,^25,26 there is also the implication that the only role of public health research is to translate knowledge from earlier in the pipeline as opposed to also generating discoveries and new knowledge within the public health field in and of itself as opposed to scaling from basic biomedical research or clinical interventions.

Basic theoretical and methodological work within public health is also needed to drive forward the field,²⁷ and work that challenges existing assumptions underpinning population health interventions.²⁸ Furthermore, this myth may result in a devaluing of developmental public health research such as feasibility studies that may be at too early a stage to be translated or replication of intervention research, and may place too great an emphasis on the impact of single studies which have yet to become a part of a body of knowledge or synthesized in relation to other evidence within a systematic review, for example.

4. Ten Percent of an Intervention’s Cost Should Be Spent on Evaluation

One widely held and pervasive myth is that 10% of any program budget should be spent on evaluation. For the purposes of unpacking this myth, it is helpful to think of evaluation as an intervention, which has costs and benefits like any other intervention, at which point the 10% figure looks less convincing. We would not argue that 10% of a health budget would be spent on hip replacements or another medical intervention without a formal needs assessment. Some evaluations need more than 10%, and some less. The 10% rule of thumb is partly driven by funders’ expectations: the Economic Opportunities Studies office expects applicants to build in an evaluation budget that is 10% to 15% of the total budget.²⁹

However, the exact figure fluctuates. It could be more than 10%: according to the US Administration for Children and Families, “A useful rule of thumb is to estimate that your evaluation will cost approximately 15 to 20 percent of your total program budget.”^30(p30) It could be less: the W. K. Kellogg Foundation says that evaluation costs can be 5% to 7% of a project’s budget,³¹ and the International Labor Organization suggests a minimum of 2%.³² For some projects, the appropriate figure is not 10%, and may even be 0%—some interventions may not be worth evaluating. For example, if the intervention is implemented in such a way that an informative evaluation is not possible or if a decision has already been taken about the program’s future and findings are unlikely to have relevance.³³

Overall, it is more likely that the budget for evaluation will depend on the nature of the intervention, its stage of development, its size, the value of the information that the evaluation is expected to provide, and how much is already known about the intervention. European Union guidance suggests that for large-scale relatively routine programs the budgets required for evaluation will be a small proportion of the program resources (normally less than 1%). On the other hand, for interventions that are relatively innovative in character, and in which evaluation has a strong learning and participatory aspect, the costs could constitute a relatively high proportion of program resources. Recent work by Leviton et al.^34,35 has proposed a Systematic Screening and Assessment method to extend existing evaluability assessment methodologies by identifying which innovations are more likely to be effective and thereby most worth evaluating. The approach aims to improve allocation of evaluation resources, while also linking evaluation explicitly to issues of likely impact and scalability of interventions.

5. Benefits of Prevention Interventions Take a Long Time to Accrue

Although the burden of chronic disease is increasingly globally, a frequently cited reason for prevention investments being harder to “sell” politically is that the accrual of health impacts are too far off and may exceed the time horizon of a typical electoral cycle.³⁶ Furthermore, investment decisions may be necessarily influenced by crises such as the short-term impacts of something like an infectious disease outbreak compared with the sustained impacts that something like increases in chronic disease prevalence may have over time.

Although there is no doubt that the long-term health benefits of prevention interventions may take time to be demonstrated and that a longer-term perspective on health outcomes is critical to ensuring the long-term health of populations,³⁵ there are examples of prevention interventions that have had significant impacts over relatively short time-scales. The “top 10” public health achievements in the United States, from 2001 to 2010, all involved interventions that have demonstrated public health impact in a period of 10 years or less.⁹ Examples include the impacts of workplace smoking bans, which resulted in reduced respiratory and sensory symptoms, as well as reductions in hospital admissions because of acute myocardial infarction within a relatively short period following the introduction of such bans,³⁷ reductions in mortality following seat belt and air bag use,³⁸ and reductions in smoking prevalence following introduction of taxation measures.³⁹ Furthermore, repeals of public health measures can also result in rapid behavior changes such as in Finland where, within a 1-year period of a policy change that reduced the tax on alcohol, there were increases in alcohol-related deaths.⁴⁰

To provide evidence on health impacts attributable to interventions, appropriate surveillance mechanisms must be in place to detect both exposure to the intervention and outcome measures. Such monitoring is a key input to ensuring that premature conclusions are not made regarding effectiveness or broader harms and benefits of interventions.

6. “If It Only Helps 1 Person, Then It's Worth Doing”

It is sometimes asserted that public health and health care interventions that are obviously beneficial are worth doing anyway, irrespective of their effectiveness, because if they “help just 1 person. . . .” One of us heard this view expressed about the World Health Organization Healthy Cities program, where debate about what effect it might have had was rejected as unimportant by a practitioner who argued that if it “had saved just 1 life” then it was worth it.

The myth is particularly prevalent in the case of screening programs, where the logic of trying to identify early cases of disease seems unassailable. However, for the argument to be true, it would require that no expenditure is too great to save a life. Even ineffective, harmful interventions are likely to save just 1 life in a large enough population. This does not make them “worth” doing.

7. The Public Health Evidence Base Is Weak

The argument is often made that the public health evidence base is “weak” compared with other areas (medicine for example). The assumption is that a large part of this perceived weakness is attributable to the lack of studies using randomized controlled trial (RCT) designs that, when taken as a body of evidence, will enable causal inferences to be made. There are several assumptions here that can be challenged. One is that the lack of trials is an indicator of a weak evidence base. Another is that there is something different, and perhaps “softer,” about public health science in its inability to marshal a large body of experimental evidence to inform decision-making. Many fields, however, have been built largely on observational evidence—astronomy, for example. In others, observational evidence is often used with caveats to inform major policy decisions (such as transport and climate change).

In public health, the use of nonexperimental methods is often entirely appropriate and sufficient, particularly when causal chains are short and effect sizes are large.⁴¹ Valuable evidence about public health interventions can be gathered through RCTs, non-RCTs, and many other types of research, particularly mixed-methods approaches, depending on the question and the level of certainty required from the answer.⁴² The prevailing view that every nonrandomized study in public health is either weak, or is simply a failed attempt at an RCT, needs to be challenged, particularly in view of the need to better address issues of external validity, which persist within some highly controlled studies. Defining observational evidence by what it is not (“nonrandomized”) is also simplistic. Observational studies are not simply “nonrandomized studies,” any more than RCTs are “nonobservational” studies.

8. Every Gap in the Evidence Base Needs Filling

We often point to the gaps in the public health evidence base as a problem. However, some of these gaps are problematic and some are not. Every evidence base—even in the hard sciences—has, and should have, gaps. Some gaps are there for a reason and should never be filled because they represent minor or unimportant questions, or low priorities, or are simply not fillable—because they represent unanswerable questions—such as whether, if all societies were completely reorganized along egalitarian lines, they would be more healthy.

Pointing to such research gaps implies that every intervention needs to be evaluated, and this cannot be the case. The “payback” from some evaluations in terms of the knowledge likely to be gained is likely to be so low that they are not worth conducting.³² What is needed is a greater focus, not on identifying gaps, but on identifying gaps that, when filled, will yield the greatest payback in terms of public health.⁴³

9. There Is a Hierarchy of Evidence

One of the criticisms of the application of evidence-based thinking to public health is that it depends on the existence of a hierarchy of evidence, which may underprivilege public health research because of the relative absence of RCT evidence. This argument is often repeated and entirely spurious. The perhaps surprising fact is that there is no such thing as a hierarchy of evidence (outside such straw-man arguments).

What there is, however, is a hierarchy of evidence of efficacy or effectiveness.⁴⁴ This was developed by the Canadian Task Force on the Periodic Health Examination to help decide on priorities when searching for studies to answer clinical questions about effectiveness, and was subsequently adopted by the US Preventive Services Task Force. Its origins have been said to derive from Campbell and Stanley’s seminal text on evaluation.⁴⁵ It was not intended to be an overarching hierarchy of evidence for all types of questions. Yet, the original focus of the hierarchy—on effectiveness—is generally overlooked, and it is often assumed that it is a once-and-for-all hierarchy of evidence. Quite simply, there is no such thing.

10. Public Health Interventions Should Only Be Based on Research

The gap between research and its use in practice and policy exists in many disciplines, including public health.⁴⁶ The phrase “evidence-based” may imply to practitioners and policymakers that researchers expect decisions to be based only on research evidence and nothing else. Such a worldview is not only unrealistic, but also may in fact obstruct public health action that will occur in the absence of research. To paraphrase Muir Gray, decisions should be based on the best evidence available as opposed to the best evidence possible.⁴⁷

Thinking about evidence-based public health only in relation to research is inconsistent with the nuanced description of evidence-based public health, which focuses on the processes needed to ground public health decision-making in evidence and a wide evidence base to both inform and evaluate decisions and interventions before or once implemented.⁴⁸ This broadened understanding is more deliberately inclusive of the many inputs into public health decision-making alongside research evidence.^49–51

In public health, the adaptation of interventions to local context is also often important. It is conceivable that an intervention that is solely evidence-based and taken off the shelf and implemented without adequate consideration or adaptation for contextual considerations may see decreased effectiveness or not be adequately adjusted for critical implementation factors. As noted differently by Green and by Hawe et al., a shift toward greater appreciation for contextualized evidence is needed from best practices to “best processes”⁵² and understanding and maintaining the “active ingredients” within interventions to maintain the functional (rather than compositional) fidelity that contribute to their effectiveness and implementation.⁵³

REVIEW

We have presented a selection of salient and pervasive myths in public health research and practice, including their origins and arguments for why they may not be truisms. As the field continues to develop and debate, it is likely that more myths will be generated and it is fully acknowledged that there is a larger set of myths in our field.

Limitations

The main limitation of the article is that the selected myths are not comprehensive—they were chosen for their relevance by the authors, informed by informal consultations with colleagues, and drawing on relevant literature. Clearly, not all myths in public health have been covered and some other myths (not discussed here) have already been given thorough treatment elsewhere—for example, the myth that RCTs are only able to respond to questions of intervention effectiveness and not questions related to questions of external validity and “for whom” and “under what circumstances” interventions are effective^41,54; the myth that public health relates only to the health sector ⁵⁵; the myth that waiting lists should only exist for health care⁵⁶; and the myth that public health interventions are always beneficial.⁵⁷

As new data become available, we would encourage public health practitioners and researchers to dispel myths when opportunities present themselves to do so as part of a commitment to evidence-informed public health practice and the evolution of public health training programs and curricula.

Conclusions

Myths are perpetuated by telling and retelling. They also have value: in other contexts they are a source of comfort, and help people explain where they come from, why the world is as it is, and why things are the way they are. However they should have no such privileged place in evidence-informed public health.