Causal Inference in Public Health: A Call to Stop Causal Fishing Expeditions

Establishing causal relationships has major implications for public health. It reveals opportunities for interventions and underscores the need and/or potential to implement actions that either minimize exposure to a harmful agent or promote exposure to a beneficial one. ¹ In observational studies with a causal inference task (ie, the identification and estimation of causal effects between exposures and outcomes), ² public health researchers are concerned about the use of noncausal terminologies and approaches.^3,4 A PubMed search with the terms “factors associated,” “risk factors,” “associated factors,” “factors linked,” or “determinants” retrieves >1.6 million records published from 1907 to January 9, 2025, with an upward trend over time. The intended purpose of these terms—often used interchangeably—is to denote observational studies that aim to assess the predictive relationship of multiple independent variables on a single outcome using multivariable regression models, without testing valid hypotheses established a priori. Furthermore, their meanings differ depending on the intended use: diagnosis, prognosis, treatment effects, or etiology. ⁵ For more than a century, these goals have been approached with similar misunderstood methods, yielding estimates that often imply correlation, not causation, and leading to misinterpretations.

In this commentary, we define “causal fishing expeditions” as studies that test multiple associations simultaneously and attempt to interpret the associations present through a causal lens without a clear predefined hypothesis or appropriate methodological framework. While associations can be useful for description, prediction, and diagnosis, observational studies frequently conflate predictive and causal factors. Such confusion hampers scientific progress in public health, leading to the generation of unsupported hypotheses. The sole contribution of causal fishing expeditions is inordinately focused on producing underdeveloped hypotheses—a phenomenon referred to as “the hypothesis-generating machine”—rendering the epidemiology field a target of ridicule. ⁶ Researchers must transition beyond these terms and methods by adopting causal inference approaches and addressing causal questions.

Designing Observational Studies for Causal Inference

Regardless of the study design, adjusting for exposure–outcome confounders often, and sometimes unconsciously implies an interest in causal effects. Nonetheless, it is impossible to identify and isolate causal effects by fishing for associated factors. To make causal inferences from observational analyses, an explicit goal and study design aimed at identifying causal effects are required. ⁷ These conditions include a well-defined causal question, a precisely specified causal estimand (ie, the target causal quantity), a clearly articulated study design, explicit causal assumptions (eg, exchangeability, positivity, consistency), the use of appropriate statistical methods, and the tenability of the causal interpretation. Collectively, these are referred to as the conditions required for causal identifiability. ⁸ To express it plainly, causal inferences from observational data are valid only if we can emulate random assignment of well-defined interventions via statistical control of covariates or by using an instrumental variable. ⁹ Furthermore, causal etiologic research is not restricted to longitudinal data. Even cross-sectional data can yield causal insights when certain conditions are met and carefully examined. ¹⁰ Thus, inferring causality becomes possible even in limited situations if, and only if, the right assumptions are met and the underlying causal structure allows it.

Causal directed acyclic graphs (DAGs) are a useful tool to represent the empirical causal assumptions being assessed, facilitating the selection of appropriate controls while avoiding adjustment for inappropriate ones.^11,12 Nevertheless, when fishing for associated factors, DAG loses its capacity to represent correctly all the causal links due to the absence of causal ordering. ⁷ Apart from including multiple covariates in a single model, displaying multiple exposures in a table leads to interpretative difficulties and confounded effects. ¹³

Biases of Causal Fishing Expeditions

Causal fishing expeditions are unable to prevent systematic biases and ensure identifiability. They are well known for improper variable selection and their arbitrary inclusion in multivariable models, leading to overadjustment on mediators and colliders, all of which introduce bias into total effect estimates.^14,15 What is the purpose of adjusting for multiple factors that do not confound the effect in question? Or, more concerning, is there even a clear effect of interest? A key concern arises when scientific readers interpret these studies through a causal lens, mistakenly assuming that these factors, while correlated with the outcome, are causally linked. This misinterpretation risks conflating correlation with causation. Both authors and readers contribute to the problem by interpreting estimates as causal, which ultimately leads to misguided conclusions from such studies.

In addition to obscuring the distinction between correlation and causation, causal fishing expeditions have led to malpractices such as P-hacking, data dredging, HARKing, and publication bias, ¹⁶ which result in an increased probability of type 1 error (false-positive outcomes) and misleading conclusions. ¹⁷ Indeed, fishing expeditions are significantly associated with a 3-fold increase of a false-positive outcome. ¹⁸ The exploratory nature of the multivariable regression models constructed for fishing associated factors often uses automatic variable selection procedures (eg, stepwise algorithms), which inadvertently lead to P-hacking because of automatic selection, multiple testing, selective reporting, overfitting, manipulation of data (eg, post hoc adjustments), and, most important, lack of theory-driven selection.^19,20 Additionally, findings produced from causal fishing expeditions are unlikely to be replicated or validated with independent datasets, thereby further increasing the likelihood that reported associations are mere artifacts of data exploration. Poor replicability increases the likelihood of spurious findings entering the scientific literature without confirmation.

Modern Methods Enhance Validity of Causal Inferences

During the last 40 years, the development of methods for causal inference has enabled us to make valid causal estimations. The counterfactual theory and the structural causal model have provided tools such as DAGs, as well as frameworks consisting of target trial emulation, instrumental variables, and difference in differences, among others, that help us to respond to causal questions more credibly while mitigating common systematic biases such as confounding, selection bias, and measurement error. These approaches not only make causal assumptions transparent but also facilitate open discussions, strengthening the robustness of study findings. Enhancing the robustness of study findings is achieved by allowing researchers to account for the temporal sequence of events, isolate direct and indirect pathways, and better align their research designs to address causal questions. Furthermore, causal studies offer clear insights into interventions and policy decisions that can be implemented in real-world settings.^1,7,21

Motivating Example

To illustrate the importance of framing and answering causal questions, we simulated a dataset for a research study. The corresponding computer code (Supplemental Material) allows for replication of this example. This simulated dataset is intended solely for educational purposes. In our example, we simulated a dataset of 20 000 electronic health records of patients without baseline cardiovascular disease that included sociodemographic and clinical data. Here, using the same estimator (Poisson regression) and estimand (relative risk), we illustrate 2 scenarios.

Scenario 1: Causal Fishing Expedition

A group of researchers was interested in evaluating sociodemographic and clinical factors associated with 10-year incident heart events (myocardial infarction, coronary revascularization, and stroke). They applied a backward selection approach, excluding variables with P > .20 to determine which covariates to retain in the multivariable regression analysis.^22,23 After this data-driven selection process, the final model included the following covariates: 24-hour urinary sodium excretion (eNa^24h; in milligrams), age, sex, and mean arterial pressure (in millimeters of mercury). They presented the adjusted estimates for all covariates in Table 2 of their article and interpreted each covariate’s effect. Ultimately, they concluded that sodium intake was not associated with incident heart events.

Scenario 2: Framing and Answering a Causal Question

The harmful dose–response effect of sodium intake on cardiovascular events is well documented,^24,25 but a recent study found conflicting results. ²⁶ To resolve this discrepancy, a group of researchers designed an observational study to respond to the following question: What is the effect of sodium intake on 10-year incident heart events in adults without baseline cardiovascular disease? They used eNa^24h as a marker for sodium intake.

To approach this question systematically, the group developed the protocol for a target trial—that is, a hypothetical clinical trial that would be designed to answer the causal question of interest. This protocol included eligibility criteria, treatment strategies, treatment assignment, outcomes, follow-up, causal contrasts, and statistical analysis. Consequently, to isolate the effect of sodium intake (measured by eNa^24h) on 10-year incident heart events, they constructed a DAG informed by empirical knowledge and a comprehensive literature review. The DAG identified all potential confounders, which were then adjusted in a multivariable regression model. Based on their analysis, they concluded that sodium intake increases the 10-year risk of incidental heart events.

Results and Interpretations

For ease of analysis, eNa^24h was categorized into quartiles, with the first quartile used as the reference category. The DAG outlined the assumed causal relationships between sodium intake and cardiovascular events in both scenarios (Figure 1). The corresponding outputs are displayed in Figure 2. For simplicity, only age and sex were included as confounders.

Figure 1.

Directed acyclic graph for the effect of sodium intake on 10-year incident heart events among adults without baseline cardiovascular disease. (A) Scenario 1: causal fishing expedition. (B) Scenario 2: causal inference approach.

Figure 2.

Relative risks of the effect of sodium intake on 10-year incident heart events among adults without baseline cardiovascular disease: comparison of scenario 1 and scenario 2. Urinary sodium excretion was used as a marker for sodium intake. Quartile 1 served as the reference category. Relative risks were estimated by using Poisson regression. Scenario 1 (causal fishing expedition) was adjusted for confounders and the mediator, while scenario 2 (causal inference approach) was adjusted only for confounders. Horizontal lines indicate 95% CIs.

In scenario 1, when the focus was solely on the relationship between eNa^24h and cardiovascular events, no significant association was found. In contrast, scenario 2 showed that the effect size of eNa^24h on incidental heart events significantly increases across higher quartiles. These contrasting results raise a fundamental question: Why do the same data lead to different conclusions?

The answer lies in the design of the studies. In scenario 1, the researchers unknowingly conditioned on a mediator (mean arterial pressure) by relying on an automated P value–driven statistical selection process. Because blood pressure is a mediator that largely explains the well-established relationship, adjusting for it blocks the causal pathway and leads to the false conclusion that sodium intake has no effect. Notably, P values cannot establish or rule out causal relationships, regardless of the chosen threshold.^19,20 Additionally, when interpreting adjusted estimates from a single table, the authors fell into the Table 2’s fallacy, thereby further obscuring the interpretation of their findings. ¹³ That is to say, the authors incorrectly assumed that each coefficient represents an independent causal effect. In contrast, in scenario 2, the researchers explicitly formulated a causal question and used DAGs to guide variable selection. This approach ensured that the final model accounted for the confounders without conditioning on mediators, yielding an effect estimate that is consistent with previous literature.^24,25

If a researcher aims to isolate the total effect of exposure on an outcome, conditioning on mediators of the causal relationship introduces bias because it blocks the causal pathway. It changes the causal estimand of the exposure variable of interest, resulting in an estimand that does not align with the target causal effect—the researcher would be estimating the direct effect rather than the total effect. Moreover, mediator–outcome confounding can lead to collider stratification bias if the mediator is conditioned on. This type of bias occurs because mediators often have their own confounders and researchers frequently lack a DAG to guide proper variable selection. As a result, adjusting for a mediator inappropriately does not simply shift the estimand from the total to the direct effect—it often leads to an estimate that does not correspond to any meaningful causal effect at all. At best, the total effect is underestimated as the pathway through the mediator is removed (ie, the indirect effect). Often, confounders are confused with mediators, which, when adjusted, compromises the effect estimates. ¹² Scenario 1 illustrated the mediator bias; other previously described biases, not shown in our example, can arise from causal fishing expeditions.

Shifting Toward Transparent Causal Inference in Public Health

The lack of awareness about the hazards of causal fishing expeditions has tilted the playing field of public health research toward an inaccuracy that undermines the credibility of a substantial number of existing publications.^3,27 A shift in public health research is needed to prioritize answering well-framed causal questions. To make this goal feasible, certain steps must be taken, including adopting more transparent causal frameworks, particularly in designing studies and interpreting results. Causal inference training should be integrated into epidemiology and public health curricula to equip researchers with the necessary skills—particularly in the competencies of assessing the strength of evidence for a causal relationship, distinguishing between prediction and causality frameworks, and applying appropriate analytical approaches to make causal inferences based on implicit and explicit assumptions. ²⁸ Additionally, journal editors should require authors to clearly differentiate their research task: description, prediction, or explanation. ² Overall, focusing on causal inference can improve public health interventions by targeting causes and effects rather than correlations. Otherwise, continuing the current practices will undermine the trust of policy makers and the general population.

The role of the researcher is to obtain reliable results that can be applied to real-world phenomena. Statistical models seek to describe, explain, and predict reality; however, misuse can distort the interpretation of natural phenomena, leading to deceptive conclusions and jeopardizing the decision-making process. Therefore, what value do estimates hold for science and society if they are biased and fail to approximate the truth?