Asking the Right Questions: Research of Consequence to Solve Problems of Significance

“If they can get you asking the wrong questions, they don’t have to worry about the answers.”

-Thomas Pynchon

Despite the volumes of research studies published every week, public health practitioners often find a frustrating shortage of studies that offer practical solutions to our biggest public health problems. Until the last few years, for example, for all of the studies published on obesity, practitioners were hard-pressed to find research that defined policies with the potential to alter the trajectory of this epidemic across an entire population.

There are many reasons for this mismatch between published and actionable studies. Too often, observational studies are driven by the availability of data, rather than the importance of the research question. Studies that characterize risk factors often do not sufficiently emphasize modifiable, as opposed to immutable, risks. And intervention trials with budget-constrained sample sizes are designed to prove statistical significance, so they lean toward intensive, maximalist interventions that are too expensive to be brought to the scale of entire populations.

I offer four suggestions for how researchers can choose research questions that contribute more directly to solving population-wide health problems.

DO LESS FOR MORE PEOPLE

The biggest public health problems affect large fractions of populations. In trying to solve these problems, we can almost never afford interventions that are expensive per person reached. Whether the problem is smoking, obesity, substance use, or risky sexual behavior, interventions that involve individual in-person interactions costing hundreds or thousands of dollars per person reached have never been brought to scale and never will. Researchers should instead focus on minimalist interventions that can be scaled to reach large fractions of the population. Interventions with population potential may cost pennies per person reached (like mass media campaigns or mass condom distribution) or virtually nothing (like smoke-free air rules or healthy food standards). The work of public health is extensive, not intensive.

THINK FORWARD AND WORK BACKWARD

The tools to achieve population-level changes in risks are limited. Entire populations can be informed (though rarely “educated”); policies can recommend, prohibit, or mandate; and environments can be reshaped. But the tendency for humans to seek pleasure and to choose the path of least resistance cannot be changed, and our gene pool cannot be altered. In deciding what questions to ask, researchers should start by thinking forward; that is, cataloging the interventions that are feasible and might have a population-level benefit. Then researchers can work backward to fill in the many gaps in knowledge that make those ideas practical. For example, a vast research literature has shown that income inequality, poverty, and social disadvantage cause poor health. This literature is thin, though, on what exactly public health practitioners should do about it, leading practitioners to create programs (like increasing access to medical care) that are unlikely to reduce health inequalities. It would be far better for researchers, rather than further characterizing social determinants, to list policies with the potential to reduce poverty or social disadvantage—like increases in the minimum wage or affordable housing programs—and then work backward to answer questions likely to be raised if these were considered by a legislative body.

OBSERVE BY WATCHING

Remember Yogi Berra’s “you can observe a lot just by watching.” In a nation with 50 states and thousands of municipalities, we can learn much by studying variations across jurisdictions in policies or environments, or changes in health outcomes after policies or environments are altered. We know from observational studies like these that smoking rates fall when cigarette taxes are raised. Similar studies can measure the effect of increased access to healthy foods on diet, the effects of alcohol sales restrictions on alcohol-involved injuries, or the effects of welfare policies on rates of mental illness. Studies of this sort exist, but we need more of them and on more topics. While the most methodologically strong studies are those that include time series across multiple jurisdictions that have implemented similar interventions, even a single example can be very useful. In fact, in political debates that precede any meaningful policy change, the anecdote of what happened after another jurisdiction enacted a similar change is often the most persuasive evidence. Practitioners have an obligation to respond to public health crises before all questions about their attempted solutions have been answered, and researchers have an obligation to study those attempts to inform future actions.

MEASURE SIDE EFFECTS

Before any new policy is adopted, policymakers raise many questions about the policy’s potential effects far beyond those on health. They may ask about the effects of the policy across income strata, the effects on economic activity, or the likelihood of counter-productive consumer reactions. And most will want to know, even if few ask openly, the degree of public support for the policy. There is great value in solid evidence to answer all of these questions. Examples of “side-effect” studies are: What is the effect of mandatory calorie labeling on menus on restaurant sales? What is the effect of a tax on sugary drinks on employment in the food industry? After the legal drinking age was raised to 21 years, was there a backlash in consumer opinion against government or instead a greater acceptance on restrictions on alcohol sales?

AN EXAMPLE

Recently, the City of Philadelphia, Pennsylvania, enacted a tax on sweetened beverages, the revenue for which will help expand pre-Kindergarten, rebuild public parks and recreation centers, and offer health and social services in public schools. While the primary argument for the tax was the benefit of these services, the potential health value of the tax itself was widely discussed. Fortunately, there was a robust research literature on sweetened beverage taxes that informed this discussion. Economic studies established estimates of the price elasticity of sweetened beverages,^1,2 and a simulation model predicted that the tax would lead to substantial reductions in obesity and type 2 diabetes, along with savings in health care costs.^3,4 A study verified that Mexico’s sugary beverage tax led to a meaningful reduction in sales.⁵ Other studies refuted “side effects” arguments raised by opponents. When they argued that the tax would hurt retailers and distributors, proponents showed that the tax in Mexico was followed by an increase in sales of untaxed beverages large enough to compensate for the fall in sales of taxed beverages.⁵ When opponents called the tax regressive, proponents pointed to a study estimating that the tax to be paid by low-income households would exceed that paid by high-income households by less than five dollars per year.⁶ And when opponents argued that the tax would cause a loss of jobs, proponents had a published model suggesting that the tax would in fact cause a net increase in employment.⁷

CONCLUSIONS

Many types of research studies are needed to advance public health. We certainly need studies that simply identify or characterize new or worsening health risks, studies that describe biologic or social risk factors, and studies that measure the effect of a new technology on behavior and biology. But today’s most significant health risks (e.g., smoking, unhealthy food, physical inactivity, and substance use) are already well described. The greatest advances in public health will be alterations in policies and environments that shape these risks for entire populations. And in these circumstances, the studies of greatest consequence will be those that ask questions designed to enable intelligent policymaking.