What matters most: quantifying an epidemiology of consequence

Abstract

Risk factor epidemiology has contributed to substantial public health success. In this essay, we argue, however, that the focus on risk factor epidemiology has led epidemiology to ever increasing focus on the estimation of precise causal effects of exposures on an outcome at the expense of engagement with the broader causal architecture that produces population health. To conduct an epidemiology of consequence, a systematic effort is needed to engage our science in a critical reflection both about how well and under what conditions or assumptions we can assess causal effects and also on what will truly matter most for changing population health. Such an approach changes the priorities and values of the discipline and requires reorientation of how we structure the questions we ask and the methods we use, as well as how we teach epidemiology to our emerging scholars.

Our charge as epidemiologists and the limits of risk factor epidemiology

Individuals who are overweight might have longer life spans [1,2] (or not [3]), blueberry and strawberry consumption may decrease risk for heart attacks [4] (or not [5]), moderate alcohol consumption is good for cardiovascular health [6] (or not [7]), green tea consumption might prevent stomach cancer [8] (or not [9]), and on and on. The list of diet, lifestyle, environmental, and genetic factors that purportedly cause or prevent a wide range of chronic diseases is voluminous, and often in conflict. Is butter bad? [10] Or is it good [11,12]? Should we recommend multivitamin use to individuals in countries with a low prevalence of micronutrient deficiency? If you are not pregnant and are generally healthy, probably not [13], but maybe vitamin D? Is salt bad? The evidence is murky [14,15].

“Risk factor epidemiology” can be considered the process of identifying causes of disease and illness and has been the dominant analytic paradigm in epidemiology for several decades. Although conceptual approaches such as social and life course epidemiology have substantially broadened our understanding of the factors that we may consider to be “causes” of disease and illness [16,17], our dominant analytic paradigm has remained one where we use increasingly sophisticated analytic methods to identify causes that increase the risk of particular pathology. Although this approach has proven useful and met with considerable successes [18], the past 30 years of risk factor epidemiology have also presented us with a baffling and almost endless array of potentially causal observations.

The search for modifiable risk factors that prevent the onset of chronic disease is important, worthy of our epidemiologic time, effort, and resources. In many ways, one might argue, there is nothing inherently wrong with the discipline producing a long list of risk factors and it is the nature of scientific inquiry to articulate both arguments for, and against, particular observations [19]. We collect data, present findings, question them, refine hypotheses, incorporate refuting evidence, and slowly and systematically move our understanding forward until new data, new hypotheses, and new knowledge from across disciplines and designs upend our previously held paradigms [20]. Increasingly, causal models have been applied within epidemiology, improved upon, refined, and expanded to allow researchers to provide increasingly precise estimates of the “causal effect” of a single risk factor on a health outcome [21–24]. Yet, this proliferation of causal effects—typically identified through an approach that aims to isolate risk factors for particular outcomes—presents a conundrum for scientists, let alone the lay public, to synthesize and form evidence-based recommendations that can promote health. Furthermore, the methodologic debates regarding the validity of the evidence for causal effects is often based on associations for which there are small overall effects [25], suggesting that a decision one way or another (cause or not cause) may translate to little public health benefit.

The debate over the efficiency of risk factor epidemiology within the framework of our charge as population health scientists has raged for more than two decades. In his presidential address to the Society for Epidemiologic Research in 1993, for example, Milton Terris challenged the field by noting [26], p146:

“We cannot remain indefinitely in our ivory towers; they may crumble around us. We need to foster epidemiologic research, not only by improving our methodology and sharing our scientific experience, but by helping to convince the American public and its legislators that prevention is far more important than treatment, that our expanded agenda for research needs full legislative and financial support, and that the application of our findings to improve the health of the public must become the highest priority for health policy in the United States.”

Terris has certainly not been alone in calling to action a focus on translation and implementation of our science for public health improvement [27–30]. In fact, few would argue that a focus on research questions that have direct and demonstrable ability to shift the curve of population health is a necessary implication of our science. After the relatively “low hanging fruit” of factors such as cigarette smoking served as the benchmark through which modern epidemiologic methods were developed [31], the risk factor approach has been reproduced and reified for relatively small effects and less stable science [32].

Is it, however, reasonable to think that this approach is going to lead to further discovery of single risk factor “causes” of disease? As a field, we have a fairly robust set of factors for which there is good evidence that they cause at least one poor health outcome across the life course. Absence of nutritious and plant-based diets, sedentary lifestyle and body fat, substance use, exposure to chronic stress, exposure to environmental toxins, genetic vulnerability, and poverty are among a list of factors that are commonly associated with a range of adverse health outcomes. It has been noted in the past yet continues to be true that much of the bulk of our research continues on the path of pursuing ever more precise estimates of the causal effect of these exposures on an increasingly long list of health outcomes [33]. Creative and rigorous ways to control for confounding and test for mediators continue to proliferate and populate the epidemiologic literature [34–36].

Should we, rather, think about alternate ways in which we, as a discipline, go about our work? Several authors have suggested alternate frameworks that move beyond risk factor identification [28,33,37,38].We have argued for the need for a causal architecture approach that, building on these other approaches, can get us to consider the full set of causal dynamics that produce population health [39]. Irrespective of the approach or framework, we suggest there that the field is at a point where it needs some reorientation and that a “what matters most” lens can be helpful in that regard. A “what matters most” lens challenges us to consider not simply which risk factors are causal but also which of these matter most. The appeal of this approach may be intuitively obvious—we clearly want to know what matters and also understand which may be the most important levers for the purposes of public health improvement.

The connections between concepts underneath headings of population health, public health, and preventive medicine are often fluid, and epidemiological sciences inform all three. We consider population health to be the science of understanding the conditions that shape distributions of health in the aggregate, including the mechanisms through which these conditions manifest in the health of individuals within the population. We then endeavor to translate this population health science into policies and programs to improve the health of populations. Epidemiological science, then, informs the conversation that guides our effort to improve public health and prevent disease. To this end, examining epidemiologic science through the lens of improving population health science for public health benefit is a consequential part of our charge as a field.

A re-emphasis on what matters most

The emphasis on identifying risk factors within a paradigm that hunts for precise causal effects obscures what we argue is the broader goal of the field—an attempt to identify “what matters most.” This approach urges us to identify what we can do about those factors that do indeed matter most for the health of populations, which necessarily involves both theory-driven approaches and a pragmatic assessment of what is likely to make a difference. Instrumentally, this would mean a focus on the relative “weight” of risk factors in producing poor health within and across populations. This includes an explicit focus on the prevalence of the factor, the prevalence hypothesized causes that interact with the factor, and the causal structure which underlies the factor, all combined with the prevalence of the health condition under study; all these determine what is consequential for public health. Foundationally, we know that what matters most are factors that are going to affect the population on a large scale. And we also know that factors that will affect the population on a large scale are those that have very big effects and/or are very prevalent [30,40].

An approach to understanding what matters most inevitably has to rely on theory, hypothesis, and data-driven assessments of the multifactorial causal structures that produce population health, acknowledge that single causes do not act in isolation, and that understanding the nature of disease requires understanding the broader network of causal structures [33,41]. Such an approach requires us to model and test such structures, focusing less on the individual effects of exposures that penetrate the model, and instead, explication of which factors, from the ubiquitous to the rare, are likely to have a large population effect, and narrowing in on creative designs and broad comparisons to assess the public health effect of these factors as well as the factors for which they interact.

A more rigorous focus on the absolute change in population health associated with exposures of interest has captured the attention of the methodologic literature [42,43], and several measures and indices with which to build from the solid risk factor work produced by the field in the last several decades have been developed [44]. Yet, the field has remained, in large part, focused on producing methods which allow for more precise causal inference from individual exposures, “surgery on equations” as suggested by Pearl [45], rather than embracing the need for a pragmatic science that seeks as its goal to produce knowledge about the specific contexts and the needs of communities within those contexts.

We suggest that, in some respects, a focus on what matters most is necessary, and our absence of focusing on it has hurt the field and detracts from our capacity to influence population health consistent with the goals of the field. To help us move toward a “what matters most” lens, we propose here a formal approach to tackling what matters most as an agenda in epidemiology. In the following sections, we begin to articulate a basis for such an approach, including mathematically understanding what causes have the biggest effect on population health and showing how shifting exposure prevalence has a major effect on how much a particular risk factor is important for population health.

We do not intend to suggest that research on rare exposures and rare disease should be abandoned; such research can provide rich scientific insight. For example, vaginal tumors are quite rare, but the discovery that in utero diethylstilbestrol exposure is causally linked to the emergence of tumors at an early age was a breakthrough in our understanding of carcinogenesis and at the early life determinants of health [46]. Furthermore, equity considerations may suggest that particular conditions are important to consider for reasons that extend beyond their net effect on population health. In addition, rare conditions and exposures remain important to understand and prevent, both because of the toll on human life and because such conditions may be the prevalent condition of tomorrow. However, the calibration of focus on rare conditions versus prevalent conditions is worth careful consideration. Resources are not infinite and decisions must be made in the present regarding how best to improve public health. Thus, engagement in the question of what matters most does not take away from the need for research on all human disease and disability. Rather it suggests recognition on the part of the epidemiological community that we need careful calibration of our resource—including financial and intellectual—investments in a way that these investments correspond to the pressing challenges of the time.

Mathematically understanding the effect of prevalent versus rare causes

The magnitude of the risk ratios and risk differences we obtain in our studies for the effect of an exposure on an outcome is dependent on the prevalence of those causes that interact with the exposure of interest [39,47]. Thus, the idea that we can identify “the” causal effect of an exposure on an outcome is not only inefficient but also at odds with the very math of risk ratio and difference estimation when the exposure of interest is not sufficient to produce the outcome. We can clearly see this concept through a simple mathematical simulation.

To draw from an example from the literature, suppose we are interested in whether a set of genes influences cognitive ability (CA) [48,49]. At least some proportion of the variability in CA in populations is because of the genes that are passed down from parents to offspring [50,51]. However, a supporting and nurturing early childhood rearing environment is also important, and some evidence indicates that such environments potentiate the effects of what we might term the “genetic endowment” of the child [52,53]. Thus, both the genetic endowment we receive from our biological parents and a supportive environment to nurture our innate ability matter for the development of CA.

Suppose that we have two populations, one in which exposure to high-quality education is almost ubiquitous (99%) and one in which only the wealthiest 20% of children are exposed to high-quality education. We dichotomize CA, considering those with high scores on a standardized test to have demonstrated high CA. We know that cognitive ability exists on a continuous dimension, as do many if not most health outcomes that we study, but here we dichotomize cognitive ability for pedagogical purposes to create a simple example. The central themes illustrated here hold whether the health outcome is dichotomous or continuously observed in populations.

In both populations, approximately 15% of children are born with a genetic endowment that provides the potential to have high CA. In our example, we will assume that the only way in which genetic endowment can lead to high CA is in the context of high-quality education. However, 5% of children will have high CA no matter whether they have a genetic endowment or a supportive learning environment; in other words, among those with no high-quality education, some students will beat the odds and develop CA. Furthermore, among those with high-quality education, some students will fail to thrive no matter their genetic endowment. In Appendix 1 we provide a full compendium of methodological details regarding the simulation implementation.

Results of such an example are summarized in Table 1. Although genetic endowment is associated with higher CA in both populations, the contribution of genetic endowment to CA in population with near ubiquitous nurturing educational environment is dramatically higher than the contribution of genetic endowment to CA in the population with lower exposure to a nurturing educational environment. Those with high genetic endowment are almost 339 times more likely to have high CA among those in the ubiquitous nurturing learning environment; in contrast, when the prevalence of nurturing learning environment exposure is low, those with the genetic endowment are about 1.7 times more likely to have high CA.

Table 1.

An illustration: the association between genetic factors and cognitive ability in two populations^*

99% Of children in nurturing educational environment
RR	338.95
RD	0.20
PARP	1.00
	CA+	CA−	Total
GE+	303	1214	1517
GE−	5	8480	8485
Total	308	9694	10,002
20% Of children in nurturing educational environment
RR	1.68
RD	0.00
PARP	0.40
	CA+	CA−	Total
GE+	6	1511	1517
GE−	20	8465	8485
Total	26	9976	10,002

^*In population 1, 99% of children are exposed to a nurturing educational environment. In population 2, 20% of children are exposed to a nurturing educational environment.

For emphasis, the prevalence of genetic endowment and the structure of how CA is caused was the same in the two populations; all that changed was the environment. This suggests that when all children are in a nurturing environment, genetic endowments to high CA become the principal reason that separate high from low cognitive achievers. In contrast, when a minority of children are exposed to a nurturing environment, it is much more difficult to see contributions to CA from genetic endowment, and whether a child does or does not have such an endowment matters much less in explaining population health. If we focus on genetic endowment, we miss the fact that the learning environment is in fact driving CA across both populations. And although a risk ratio of 339 may seem astronomical, available simulation analyses indicate that measures of association more than at least 350 are needed for relative association measures to be good prognostic indicators of whether an individual will develop the outcome [40], further emphasizing the need for epidemiology to reconsider the role of individual prediction in a framework of an epidemiology that matters. Of note, the choice of studying genetic endowment, nurturing learning environment, or any other of the host of potential factors that may be of interest must be guided by theory and the available literature. We suggest that the formulation of hypotheses based on extant theory should also be guided by these considerations of potential interaction of associations across multiple domains of inquiry.

In fact, the empirical literature on differences in the heritability of CA across learning environments suggests the same results as we simulated. Using twin data, Turkheimer et al. [48] demonstrated that the proportion of variance in intelligence explained by genetics was highest among children in the highest socioeconomic strata; as socioeconomic stratum declines, contribution of additive genetic effects decreases, whereas the contribution of shared environment increases. Although follow-up studies have shown inconsistencies in these findings across development and context [49], it is clear that the contribution of specific causes to variation in CA will systematically differ across contexts in which the co-occurring causes also vary. Thus, understanding CA requires an approach that seeks to understand the way in which these causes work together, rather than estimating the specific causal effect of any one cause.

From simulation to the community: shifting exposure prevalence across geographic space and time

One could potentially write off our previously mentioned example as a convenient mathematical exercise, but there is substantial empirical literature to indicate that such variations in the magnitudes of our effect estimates occur frequently in the empirical literature. These variations are sometimes explained by random chance [54,55], faulty study design, or other methodologic bias, exposing our innate preference for exposures to have one true causal effect in the population.

Of course, the most obvious areas in epidemiology in which we see variations in the magnitudes of effect of exposures on disease across contexts are in infectious disease. For example, consider the basic and net reproductive rate, central measures of the average number of additional cases per infected person, in which the former is estimated among a completely susceptible population, whereas the latter can vary according to vaccination and other sources of immunity. The basic reproductive rate provides some information about the underlying pathogenesis and virulence of infection, whereas the net reproductive rate is expected to vary widely across geographic contexts, subpopulations, and time as immunity levels change within and across populations. Thus, built into the assessment of infectious disease transmission and incidence is explicit understanding of the way in which context and distributions of risk and protective factors alter our estimates of disease pathogenesis.

As another example, consider firearm violence in the United States. Death by firearm is among the most central public health threats currently facing health in the United States. One response to the threat has been to restrict firearms to individuals with mental illness. In fact, firearm-disqualifying mental health adjudications increased from 7% of firearm disqualifications in 2007 to 28% in 2013 [56]. When we step back from the United States and compare across countries, however, we see that by comparison, Canada has a similar prevalence of psychiatric disorders compared with the United States [57–59] but a much lower overall rate of gun violence and a much smaller proportion of homicides and suicides committed with guns [60]. In fact, empirical evidence that there is only a marginal association between mental illness and increased risk of violence [56,61], suggesting that the dynamics that underlie the epidemic of firearm injury in the United States require little focus on mental illness. Differences in firearm availability and gun culture likely underlie the differences in firearm injury across these two geographic contexts [62], and a focus on factors such as mental illness may misplace resources. Understanding the causal architecture of firearm violence and the factors that vary across geographic contexts (e.g., firearm availability) and those that do not (e.g., mental illness prevalence) provides a framework that draws attention to the areas in which the greatest public health effect could be found.

We could draw on many more examples from the literature to illustrate this fundamental point that the variation in the magnitudes of our associations across time and place are a critical part of the epidemiology of our outcomes, and by drawing on this variation, we may be able to acquire a stronger foothold into how we can shift population health more dramatically in the contexts in which we study.

Furthermore, understanding the shifting magnitudes of association across populations as a foothold into causal architecture is not the only way to increase our capacity for an epidemiology of consequence. A deeper engagement with the mechanisms through which exposures (either prevalent or rare) exert an effect can provide insight into the more generalizable processes through which health is distributed across populations [63,64]. The need to “look inside the black box” to formulate a science that explicates general mechanisms beyond context specific mechanisms underscores central philosophical tensions about the role of epidemiology as a science. However, more instrumentally, methods and applications of mediation within the epidemiological context are rapidly developing, providing us with an opportunity to shed light on mechanisms that can inform our science [36,65–67].We suggest that methods and applications of context-specific approaches that endeavor to outline the ways in which, and the reasons why, associations differ across context should be a useful adjunct to these developing methodologies.

Implications and conclusions for conducting an epidemiology of consequence

To conduct an epidemiology of consequence, we need to identify what matters most for population health so that we can guide public health stakeholders toward strategies that reduce the burden of these factors. It is hard to argue that we should not be thinking about what matters most as we endeavor to build our research questions and design studies to answer these questions, and our mathematical simulation demonstrates the importance of such an approach. The next question is, then, how do we identify what matters most?

We suggest that there are four key factors that we as a discipline need to grapple with as we move into the next era of epidemiology. First, we need to move beyond a risk factor approach, in which the effects of exposures on outcomes are estimated, to a causal architecture approach, in which the causal structure that underlies our exposures and outcomes is fully hypothesized. In this way, we can be more attentive to the potential for ubiquitous exposures with the potential for intervention, the way in which interactions among exposures may drive high-risk groups, and the way in which dynamic contexts within and across populations may influence causal pathways. Although prevalent exposures are just one part of a dynamic inquiry that shapes priorities for population health sciences, we suggest that such considerations could be brought to the fore in a more prominent way and that an emphasis on how research can translate into shifting population health may aid in the prioritization of potential research and action avenues.

Second, critically engaging in what matters most for population health almost inevitably suggests that we are going to be increasing assessments of upstream, early life, prevalent causes such as material deprivation, early childhood education, and child adversity. These are not easy to change, and some argue that it is not in our wheelhouse to change them [68]. However, disciplines across the social and economic sciences are increasingly forming consensus that it is these early life, upstream, and macro policy–related factors that are the critical drivers of many adult outcomes including social and economic well-being, and as mentioned previously, CA [69–72]. Consider the case of exposure to crack cocaine and offspring health. Although the crack epidemic of the 1980s lead to many sensational news stories about the health of offspring born to mothers addicted to crack cocaine, a long-term follow-up of children born to crack-addicted women, nonaddicted women in the same social class, and a control group found that there was no effect of crack exposure on any cognitive or behavioral outcome assessed [73] but a substantial effect of poverty. Those children born in deprived conditions, whether the mother used crack during pregnancy, were at long-term disadvantage on almost every outcome measured. This underscores the “what matters most” approach: On the basis of these data, what matters for ability and achievement is deprivation, resources, and support for families living in poverty. An emphasis on understanding and mitigating the effects of childhood poverty through the life course is thus, arguably, more important for public health than mitigating in utero exposures to substances. Although resources for pregnant women who are using substances should undoubtedly be allocated to achieving and maintaining sobriety, by focus on prenatal substance use we are, perhaps, missing what matters most.

Third, increasingly, epidemiologic innovation has focused on internal validity of our studies to estimate precise causal effects. Although internal validity is central to the conduct of science, there are and have always been concerns about generalizability beyond the study sample from tightly controlled experiments and analyses of specific subgroups chosen for comparability of groups [74–76]. If we are concerned about what matters most for population health, however, generalizability and external validity become more prominent concerns as we endeavor to understand the distributions of risk factors and causal exposures in the specific contexts in which the populations that we aim to study reside. As such, an elevation of external validity and representativeness of samples forces us to be concerned about the facts that are emerging from our studies and the extent to which they are providing insights that apply to large populations. Such concerns inevitably center on external validity. A causal architecture approach aids in this endeavor by promoting the application of theoretical models that explicitly take into account the prevalence and distributions of causes as well as their interactions. For example, although replication of study findings increases our confidence in their validity, nonreplication may be telling us something crucial about causal architecture across populations.

Fourth, a cornerstone of shifting paradigms within any discipline is education of up and coming scholars. Shifts in goals and framing of a discipline can be gradual and subtle, but they inevitably land at the feet of younger generations of scientists to proliferate and innovate. This is not to suggest that current and previous generations of scholars within our discipline have not been at the forefront of calls for recalibration of our thinking on the meaning and effect of population health science; however, a critical self-reflection on how we train and socialize epidemiologist scholars to focus on what matters most will be paramount to enacting an approach that focuses attention on curve shifting in population health.

Logistic and pragmatic barriers are as much an obstacle to adopting such an approach as philosophical objections. The reality is that most epidemiological science is funded through federal and foundation grants, which historically have tended to subscribe with more traditional approaches to scientific progress. However, the process through which we fund our science and a debate about how to best conduct our science need not be two separate conversations; both are critical to our survival as a field, and although they may not align at times, discourse about such alignment with our funders and through education of new scholars will increase our visibility as a field and as a science.

Rose’s [77,78] seminal contributions to defining population science have formed the backbone of much of our understanding of how to influence public health. Rose illuminated the foundational methodologic principle that those factors that contribute to variation within a population may differ from the factors that contribute to variation in incidence across populations. When we conceptualize research agendas and craft public health responses to health problems, we must not lose sight of the fact that causes may be ubiquitous in one context, and therefore invariant, yet may still drive the health of the society. Within this framework stands the current state of epidemiologic research. Much has been written about paradigms of science [79,80]. In epidemiology, the past 40 years have heralded the era of the risk factor, understanding and isolating insufficient and unnecessary yet important (hopefully) causes of complex health outcomes. Yet the potential for new risk factor discovery has yielded few novel insights of late, underscoring a call for new directions in the discipline, from integration of risk factor epidemiology with a broader platform of ecologic and environmental assessments [28,81], movement away from so-called “black box” epidemiology, to illumination so mechanisms through which risk factors are operative [22,36,65], and integration of epidemiologic principles with broader attempts to mine increasingly available biological and clinical data at the population level [82]. A “what matters most” lens has the potential to clarify our thinking, focus our scholarship, and help guide our pedagogic focus toward preparing the next generation of scientists to engage with an epidemiology of consequence.

Appendix 1

Methodological details of mathematical simulation

First, we created a simulated population with 10,000 individuals. Second, we created 100 series of 10,000 evenly distributed random numbers 0 or greater and up to 1. Third, we generated 100 variables with independent Bernoulli probability of success P. For each variable, the individual response was based on the random number generation set to the expectation of P. That is, if P =.01, the variable was set to be “1” if the random number was .99 or greater. If P =.02, the variable was set to be “1” if the random number was .98 or greater. We did this for every tenth P between .01 and.99 (e.g., .01, .02, .03 ….99) resulting in 100 random variables with prevalence from 1% to 99%.

We next stipulated a simple causal structure for the outcome, based on three hypothetical variables we will term X, Y, and Z. The outcome was positive if Z = 1 or if X = 1 and Y = 1.We then selected X, Y, and Z from our simulated data set based on P.

On the basis of this causal structure, we then created two scenarios choosing among our 100 variables based on prevalence, which again were generated to be independent and from a random number generator.

• Scenario 1:

  P(X) = 0.15.

  P(Y) = 0.99.

  P(Z) = 0.05.

• Scenario 2:

  P(X) = 0.15.

  P(Y) = 0.20.

  P(Z) = 0.05.

Thus, in both scenarios, P(X) and P(Z) remained constant but P(Y) changed.

Then, we aggregated the results into 2 × 2 tables, with X as the row variable and the outcome as the column variable. Thus, those who have the outcome are both those exposed to Z (around 5% in both scenarios 1 and 2) and those exposed to both X and Y (which will be higher in scenario 1 because 99% of people are exposed to Y compared with 20% in scenario 2).

Finally, we estimated standard measures of association for the 2 × 2 tables, including the risk ratio, risk difference, and population-attributable risk proportion.