Public Health Research: Lost in Translation or Speaking the Wrong Language?

Abstract

Public health leaders, like physicians, need to make decisions that impact health based on strong evidence. To generate useful evidence for public health leaders, research must focus on interventions that have potential to impact population-level health.

Often policy and environmental changes are the interventions with the greatest potential impact on population health, but studying these is difficult because of limitations in the methods typically used and emphasized in health research.

To create useful evidence for policy and environmental interventions, other research methods are needed, including observational studies, the use of surveillance data for evaluation, and predictive mathematical modeling. More emphasis is needed on these types of study designs by researchers, funding agencies, and scientific journals.

Whereas the goal of pure scientific research is to increase knowledge, the goal of health research is more practically oriented to develop tools to combat human disease. Health research findings are often compiled into guidance that can be used by physicians to make evidence-based decisions. Undeniably, the translation of research into such guidance has led to more effective treatment of patients. But, whereas physicians have the utility of this evidence to guide their decisions, public health practitioners, who must also make decisions that impact health but usually on a much larger scale, often do not. Some see this as a failure of translation of research into action and have called for greater attention and funding for translational research as a means to improve health.^1,2 But the problem is less failure to translate research than it is to conduct research that is relevant to public health. Thus, the solution lies less in translation and more in the reorientation of our research questions and methods.

GAPS IN CURRENT RESEARCH

More than half of National Institutes of Health (NIH) research funding goes toward basic biomedical research.³ This type of research has greatly increased our understanding of biological processes, from cellular mutations that cause various cancers to development of insulin resistance and diabetes. Clinical research, another significant portion of NIH-funded research,⁴ complements this biological research by examining the occurrence of disease in individuals, including risk factors for disease and the impact of drugs or surgery on outcomes. Both types of research are essential. They facilitate the recognition of disease processes, identification of at-risk populations, and implementation of treatment. However, although this knowledge may be important for physicians treating ill patients, it does not give public health practitioners, such as those making decisions in local, state, or federal public health agencies, solutions for improving the health of entire populations.

For example, an NIH-funded study published in 2002 of 3000 patients demonstrated that a diet and physical activity program in prediabetic individuals decreased the incidence of diabetes.⁵ It was a valuable study, but the program was intensive: it involved 150 minutes of physical activity per week; a healthy low-fat, low-calorie diet; and a minimum of 16 individual counseling sessions. Although the study is useful to physicians who can test individual patients and gauge motivation and access to the intervention components, to be useful to public health practitioners, research must identify interventions that can improve diet and physical activity across populations. Achieving this kind of behavior change in large numbers of people is an uphill battle when constrained by an unsupportive social and physical environment. According to the authors of this study, 10 million Americans resemble the original study participants.

Another example comes from research on asthma. Biological and epidemiological studies of asthma have shown that people with asthma have inflammation of the airways, residents of low-income communities are disproportionately affected, and inhaled corticosteroids can reduce exacerbations.⁶ Although a considerable amount of research went into creating this one line of knowledge, it does not alone provide a population-based solution to decrease asthma exacerbations. Implementing this knowledge requires individuals to visit a medical provider for diagnosis and to adhere to recommended treatment—a goal that is laudable but not always achieved. Just as corticosteroids reduce asthma symptoms in the individual, research questions must ask what interventions reduce asthma symptoms in populations. A true population-based strategy for addressing this extraordinarily common health problem has yet to be developed, but the failure is not from insufficient translation.

Interventions that have the potential for population-level health effects are those that reach very large numbers of people or that change the physical or social environment in which they live, and often must be done at a very low cost per person reached to be feasible. They also need to be within the power of policymakers to implement. For example, research that examines the impact of clean indoor air laws on asthma meets both of these criteria. Clean indoor air laws have broad population reach and can be implemented through legislative changes that public health leaders have the power to influence.

THE NEED FOR NEW METHODS

A key obstacle to gathering evidence on the impact of environmental interventions is that they are not easily amenable to study through methods typically emphasized in health research. Randomized controlled trials, in which individuals are the units of randomization, are the gold standard of health research, and journals, funding sources, and recognition gravitate toward these studies. This type of study design has been a major—even revolutionary—advance in determining the effectiveness of medical and surgical treatments to cure individuals of disease. However, a randomized controlled trial, along with other experimental designs, can almost never be used to study environmental or policy interventions and is inadequate to assess population impact. Controlled experiments measure statistically significant differences among cohorts of individuals that, because of the high costs of enrollment and data collection, are relatively small. Because of the small sample sizes, these studies can identify only those effect sizes that are relatively large; smaller effect sizes are not statistically significant and are nearly always interpreted as not clinically significant. However, small, seemingly insignificant shifts at the individual level may create massive impact if they occur across entire populations.⁷ Designing trials of individuals to capture such small but important effect sizes would be far too expensive and impractical, requiring the recruitment of tens or hundreds of thousands of persons.

For example, reducing sodium content of processed foods can decrease population sodium consumption. But it is difficult to design a clinical trial that could capture the impact of reductions in sodium consumption across the population on health outcomes such as stroke and cardiovascular disease—even in a trial considered large by current standards. Conducting this type of trial would require a massive sample size because the effects on any individual are small, and it may take years to see an effect. However, reductions in sodium consumption across the entire US population have the potential to save tens of thousands of lives as suggested by other research methods such as mathematical models.⁸

In addition, controlled experiments are often not practical because of ethical or design constraints. For example, it would be politically impossible to apply a tobacco tax or provide smoke-free air to a representative one half of any population. Ethical constraints may prevent the controlled study of certain interventions, such as highway safety regulations on motor vehicle crash fatalities. However, this does not mean these interventions should not be studied. It is still crucial that public health leaders have evidence to predict their effectiveness, and, if effective, to garner support for the policy changes they require.

OTHER RESEARCH TOOLS

Therefore, to accommodate the need for evidence to inform public health policies, not only the topics, but also the methods of research must shift. The goals of research methods particularly relevant to public health decision-making are twofold: (1) focus on interventions that can be practically applied to entire populations, and (2) estimate the health impact on the population as a whole. Research methods that can contribute to these goals include observational studies, the use of surveillance data for evaluation, and predictive mathematical modeling.

Observational studies of analogous, naturally occurring scenarios include cross-sectional analyses, time-series analyses, and combinations of these. These “natural experiments” rely on the presence of historical, geographical, or other differences in the variable of interest that create an opportunity to study the health impact. For example, firearms are the mechanism in about half of suicide deaths.⁹ This finding leads to the question of whether policies that decrease access to firearms decrease suicide rates. Ethical and legal constraints prevent the study of this question through controlled experiments. Also, the population required for this study to show significance would be large, because the current suicide rate is 11.5 per 100 000 persons.¹⁰ The relationship between suicide and firearm access, however, can be determined through cross-sectional, ecologic studies that examine naturally occurring differences in firearm access across political jurisdictions. For example, a study of different states demonstrated that those states with greater household firearm ownership had higher suicide rates.¹¹ Studies that examine changes in suicide rates across time as firearm policies change can also examine this association. Even better studies are those that incorporate both geographic and temporal differences in policies and outcomes. Although these studies cannot definitively prove causation, they can still inform policy in ways that controlled trials of small cohorts of individuals cannot.

Public health surveillance data can assist in the execution of ecologic studies by measuring the outcome of interest before and after an intervention to change it. Beyond just measuring outcomes, comprehensive public health surveillance can also be used and is often necessary to measure the presence of the intervention itself across jurisdictions, including policies, community efforts, and environmental changes. But the use of surveillance data to determine impact of an intervention requires the implementation first and the study after. Many public health interventions, such as highway safety regulations, are enacted with strong assumption of benefit and low likelihood of risk, only to have the research and study occur afterward. Only after institution of these policies did surveillance of injuries enable determination of true impact. However, researchers must work with policymakers to design the evaluation before the policy goes into effect, so that baseline data can be collected systematically to allow comparison with follow-up data, ideally in intervention and comparison communities.

Lastly, predictive mathematical modeling relies on available data sources to develop mathematical simulations of an intervention and is one way to estimate the potential impact when the actual impact cannot be measured directly because of design constraints or limitations of the existing environment. For example, in planning for pandemic influenza, public health leaders need to know the most effective way to minimize mortality in the population. Many factors interact to determine influenza-related mortality that cannot be evaluated until the event itself happens, but public health leaders need this information in advance. Simulation through mathematical models allows for the comparison of different interventions in a hypothetical but plausible situation such as this.¹² To do this, first, factors that affect mortality such as the prevalence of pre-existing immunity, likelihood of an individual getting infected and of transmitting to others, and case–fatality rate, are estimated. To determine impact, assumptions associated with the intervention, such as antiviral efficacy in preventing infection among the uninfected and preventing mortality in those who get infected, are estimated using the strongest available evidence and are then applied to the scenario. Then the end outcomes, such as hospitalizations and deaths, are compared with and without the intervention and with varying assumptions.

Models can also be used when ethical reasons prevent the study of the intervention. For example, models have been used to measure the impact of relaxing blood donor screening criteria on risk of transfusion-related blood infection.^13,14 The number of additional units introduced into the blood supply is multiplied by the probability that an infection would be missed or that a unit that did test positive would be accidentally released. Without models, public health practitioners would not be able to develop a sense of potential impact or range of impacts as assumptions change—especially for policies where risks must be gauged. The value of predictive mathematical models is dependent on the strength of the evidence underlying the assumptions used, but these assumptions can be varied to see how conclusions vary; if the key conclusions do not change, then the models are particularly valuable.

It is worth noting that research that generates evidence of population health impact is the most basic evidence needed, but to change practice and policy the results of this research may need to be presented in a form that is most relevant to public health practitioners and policymakers. For example, summary reports and systematic reviews that present results in plain language can be more useful than are primary studies of any type.^15,16

CONCLUSIONS

Public health actions involve making decisions about entire populations. Although differences in physiology and behaviors determine disease occurrence among individuals within a population, there are often far greater differences in disease incidence among populations (e.g., states, cities) because of variations in their size, environment, and social infrastructure.¹⁷ For that reason, the effectiveness of public health policies on the health of populations will never be proven with the certainty offered to physicians about an individual treatment by an individual randomized controlled trial. Nonetheless, public health decisions should be based on the strongest possible scientific evidence. That evidence will come from synthesis of studies of different designs rather than any single intervention trial.

Most important, though, is that studies should address questions of relevance to public health practitioners, particularly the potential effectiveness of policy and environmental changes that have broad population reach. This process of translating data to policy is dynamic and complex, but using these methods ensures that these policies will at least be built on a firm evidence base. Although these research methods produce data that can inform policy creation, the adoption of policies still depends on many other factors including politics, economics, advocacy, and timing.^18,19

The study of policy and environmental changes relevant to public health leaders must receive greater emphasis in the nation's research agenda. Although there have been some efforts to fund and conduct these types of studies, ^20–22 they represent a small proportion of current research efforts. Successful studies can only be achieved if public health researchers and leaders communicate—so researchers know what studies are useful, and public health practitioners implement policies in ways that can be studied. By focusing on interventions that can be applied to populations, the evidence is more likely to be utilized by public health leaders and policymakers. Only with the careful application of this research and evidence can public health truly become an evidence-based practice.