Authors’ reply to the Discussion of ‘Methods for estimating the exposure–response curve to inform the new safety standards for fine particulate matter’

The authors thank all the discussants for their insightful comments, constructive critiques, and proposals to expand our work. The discussion contributions revolve around the following key themes: (1) causal inference framework and terminology, (2) modelling assumptions and estimator selection, (3) diagnostics and validation of causal assumptions, (4) handling of missing data, (5) data application, and (6) future directions.

1. Causal inference framework and terminology

We begin by addressing concerns related to the causal inference framework and terminology used in our paper.

1.1. Defining causal inference vs. causal evidence

Pearce raises concerns about the use of the term causal inference to describe a specific class of estimators, arguing that causal inference, as he defines it, involves synthesizing findings in multiple studies rather than establishing causality within a single study. We believe that this critique mainly reflects a difference in terminology. Pearce appears to be referring to what we would call causality or causal evidence, which pertains to the broader establishment of a causal relationship between an exposure and an outcome.

In the context of air pollution epidemiology, Dominici and Zigler (Dominici & Zigler, 2017; Zigler & Dominici, 2014) described three complementary approaches to establish causal evidence. The first is biological plausibility, which considers mechanistic understanding and toxicological evidence. The second is consistency in multiple studies, supported by adherence to the Bradford-Hill criteria. The third is the use of causal inference methods in the context of the potential outcome framework (Imbens, 2000, 2004; Rubin, 1974). These methods aim to improve robustness to model misspecification and, under appropriate assumptions, isolate causal effects. We emphasize that all three approaches are essential to establish causal relationships. However, our study and simulation focus specifically on the third approach, evaluating how the methods framed within the causal inference potential outcome framework compare to traditional approaches to estimate the exposure–response functions.

A core strength of the potential outcomes framework is its clear separation between the design and analysis phases of an observational study (Hernán & Robins, 2016; Rubin, 2008). In the design phase, researchers specify the causal estimand, target population, comparison group (typically through matching or weighting), and explicitly state assumptions required to identify causal effects, aiming to emulate a randomized controlled trial (Rubin, 2007). When carefully implemented, this structure can replicate key advantages of randomized experiments and improve robustness against model misspecification compared to traditional regression approaches (Imbens & Rubin, 2015).

1.2. Critique of matching and weighting-based approaches

Pearce also critiques matching and weighting methods used in causal inference, arguing they limit the ability to infer causation by obscuring assessments of the strength and direction of confounding (Pearce & Vandenbroucke, 2023). This concern arises from the common practice in observational epidemiology of comparing two regression models: an unadjusted model, which includes only the exposure as a predictor, and an adjusted model, which includes the exposure along with potential confounders (often specified as linear terms). Differences between adjusted and unadjusted estimates are commonly interpreted as evidence of confounding, with the size and direction indicating the magnitude and direction of the confounding bias.

However, this traditional diagnostic relies on several strong and often untestable assumptions: (1) confounders relate linearly and additively to both exposure and outcome, potentially missing non-linearities or interactions; (2) only true confounders are adjusted for, excluding colliders or mediators whose inclusion can introduce bias (Greenland et al., 1999; Holmberg & Andersen, 2022); and (3) the exposure–outcome relationship is correctly specified, as incorrect functional forms can falsely indicate confounding. Violations of these assumptions render interpretations unreliable, complicating efforts to accurately determine confounding bias.

In contrast, modern causal inference methods explicitly state required assumptions (e.g. exchangeability, consistency, positivity), utilize theory-driven causal structures (e.g. directed acyclic graphs), and clearly distinguish the design and analysis stages. This structured approach ensures greater transparency in identifying the drivers of changes in estimated effects. For example, in a causal inference framework, one can explicitly assess whether observed changes in effect estimates arise from different exposure model specifications, the inclusion or exclusion of particular covariates, or violations of key identification assumptions. Given these tools and their associated clarity, we do not find Pearce’s critique to be a strong argument against causal inference methods. Instead, we view these methods as principled evolutions of earlier practices that clarify assumptions, promote transparency, and enable meaningful sensitivity analyses to evaluate the robustness of conclusions.

1.3. Comparing traditional regression and causal inference approaches

We also respond to Pearce’s assertion that our study does not compare traditional and causal inference approaches, but rather contrasts traditional regression methods with the use of entropy balancing weights. This characterization understates the scope of our analysis. Our study compares seven estimators in total: three traditional regression-based approaches and four estimators grounded in causal inference. Three of the causal inference methods rely on entropy balancing, which directly balances covariate distributions across the continuous exposure range by solving a globally convex constrained optimization problem in the design stage (Tübbicke, 2022; Vegetabile et al., 2021). We believe entropy balancing provides a meaningful illustration of an approach that explicitly separates the design and analysis stages, a fundamental principle in modern causal inference methodology. Additionally, we include the CausalGPS estimator, which applies machine learning to estimate generalized propensity scores for continuous exposures using calliper matching (Wu et al., 2022). Thus, our study represents a broad range of causal inference strategies for continuous exposures, whereas Pearce’s critique narrowly addresses only the entropy balancing component.

1.4. Clarification on the target estimand: exposure–response function vs. average treatment effect

Pearce contends that using entropy balancing weights in our analysis yields an estimate of the average treatment effect (ATE) in a different subpopulation—specifically, an equipoise population—rather than in the study population. This critique conflates concepts relevant to binary treatment settings with those for continuous exposures. In binary settings, the ATE refers to the expected difference in potential outcomes between treated and untreated individuals. For continuous exposures, however, the relevant estimand is the exposure–response curve (ERC), which describes the population-level dose–response relationship across the entire exposure range. In our study, we estimate the average potential outcome for the full population under hypothetical interventions assigning all individuals to each exposure level. This approach aligns with established causal inference frameworks for continuous exposures and allows direct comparison between traditional regression and causal inference estimators, including entropy balancing. Under the assumptions we outline, both classes of methods target the same population-level ERC—not an effect limited to a restricted or reweighted subpopulation.

2. Model assumptions and ERC estimators

This section addresses concerns regarding the choice of estimators in our study and the role of modelling assumptions in estimating the ERC.

2.1. Selection of ERC estimators

Singh raises an important question regarding our choice of estimators. Our study includes seven in total: three traditional regression-based methods and four grounded in causal inference. We selected these based on their availability in standard software, widespread use in applied research, and their representation of current practice. Given our goal to evaluate methods commonly used in real-world settings, we focused on established approaches rather than introducing novel methodologies.

However, we acknowledge the rapid development of new causal inference methods for continuous exposures. Doubly robust estimators, for example, offer protection against model misspecification by providing asymptotically unbiased estimates when either the outcome or generalized propensity score (GPS) model is correct (Kennedy et al., 2017; Schulz & Moodie, 2021). Independence weighting is another emerging method that aims to reduce exposure—confounder dependence directly (Huling et al., 2023). We did not include these newer approaches due to practical constraints: many lack mature software implementations or do not scale well to the large datasets used in our Medicare-based analysis. Nonetheless, we expect their performance to be comparable to other flexible causal estimators. Our use of entropy weighting illustrates one such method in practice. As computational tools improve, future work should explore and benchmark these emerging methods alongside those evaluated here.

2.2. Performance of GAM and entropy-GAM models

Ziang et al. provide a valuable investigation into our hypothesis regarding the suboptimal performance of the generalized additive model (GAM) and entropy-GAM models in capturing the sublinear behaviour of the ERC in our simulation study. We appreciate their careful exploration of our conjecture that the placement of knots may have contributed to this underperformance. Their analysis, which tested both unpenalized splines with a small number of basis functions and a highly flexible integrated Wiener process with 100 basis functions, demonstrates that altering the knot structure does not substantially improve model performance.

These findings suggest that, at least in our simulation, the limited flexibility in curve shape is not the main driver of the limitations of GAM-based models. Instead, their performance appears to be more sensitive to core modelling assumptions, such as the distributional properties of the exposure, including its density and support across the observed range, the nature of the confounding structure, the presence or absence of effect heterogeneity, and the validity of the underlying causal identification assumptions. This underscores that in many applied settings, fitting highly flexible ERCs may not be appropriate unless supported by strong prior knowledge or biological plausibility.

Nonetheless, increased flexibility may be warranted in specific contexts, such as when biological evidence supports non-linear responses. In such cases, domain-informed flexibility may offer a more principled modelling strategy. We thank Ziang et al. for clarifying the limited impact of knot placement and for emphasizing the importance of foundational modelling decisions in ERC estimation.

3. Diagnostics and validation of causal assumptions

Bind emphasizes the importance of embedding causal observational studies within a plausible hypothetical randomized experiment and underscores the need for extensive diagnostics when working with continuous exposures. A key point she raised is that covariate balance may not be maintained across the full distribution of the continuous treatment, as days with high versus low air pollution levels may differ across multiple dimensions. This raises the possibility that the structure of the confounding may vary between levels of exposure to ${\text{PM}}_{2.5}$, an issue that motivates the use of methods such as matching or other procedures to adequately account for this variation. This concern is particularly salient in studies using daily or short-term ${\text{PM}}_{2.5}$ measurements, where sharp fluctuations in exposure may coincide with time-varying confounders such as temperature, humidity and human activity patterns.

In contrast, our analysis relies on annual average ${\text{PM}}_{2.5}$ exposures, which may be less influenced by these extreme peaks and valleys and may offer greater stability in estimating long-term exposure effects. Nevertheless, the possibility of variation in the confounding structure across the exposure distribution remains an important consideration even in long-term settings. In our simulation study, we do not explicitly model this type of heterogeneity in confounding—commonly referred to as local confounding (Papadogeorgou & Dominici, 2020)—but we acknowledge it as a meaningful extension of our framework. Incorporating local confounding structures could allow for a more nuanced understanding of how causal effects vary between different exposure levels. This remains an active area of research, and some recent work is beginning to explore these dynamics in the context of exposure to air pollution (Papadogeorgou & Dominici, 2020). We plan to expand upon this point in future work, integrating alternative confounding structures to assess their impact on causal effect estimation in continuous treatment settings.

4. Handling of missing data

Swallow raises an important concern about the potential impacts of missing data on our results. We fully acknowledge that missingness in confounders, exposures, or outcomes can significantly affect both traditional and causal inference estimators. Addressing missing data remains a critical challenge in epidemiological and causal inference research. Our analysis currently assumes complete data to isolate differences in estimator performance under ideal conditions; however, real-world applications inevitably involve missing data, which can introduce bias if not appropriately handled.

Future research should systematically examine how various missing data mechanisms—completely at random, at random conditional on observed data, or not at random (MNAR)—affect estimator performance. MNAR missingness presents the greatest challenge for causal inference (Little & Rubin, 2019). Established methods such as multiple imputation (Rubin, 1987) and inverse probability weighting (Robins et al., 1994) can address these scenarios under different assumptions (Ding & Li, 2018).

Conceptually, causal inference itself is a special case of the missing data problem, since only one potential outcome per unit is observed (Ding & Li, 2018; Lee et al., 2023). This perspective has led to unified frameworks, including causal diagrams and graphical models, which represent both confounding and missingness in a single structure (Hernán & Robins, 2020; Mohan & Pearl, 2021). These tools clarify key assumptions and enhance the identifiability of causal effects under missingness. We appreciate Swallow’s insightful comment and agree that integrating missing data analysis with causal inference methods is a promising direction for future research, particularly to assess the robustness of estimators under realistic conditions.

5. Data application

Our data application estimates the ERC between long-term ${\text{PM}}_{2.5}$ exposure and all-cause mortality using data from more than 68 million Medicare beneficiaries in the United States. This large-scale analysis complements our simulation study by applying the methods under consideration to a real-world setting, with the aim of evaluating their performance in a policy-relevant context. Several discussants provided thoughtful critiques of this application, focusing on different aspects of the modelling choices and interpretation. In general, their comments fall into three categories: (1) the model structure, (2) the generalizability of the results, and (3) implications for public health policy.

5.1. Model structure and hierarchical considerations

Tian critiques the pseudo-outcome construction in Josey et al. (2023), arguing it overlooks the hierarchical structure of the data. However, our analytical objective differs from Tian’s critique. While Tian addresses modelling stratum-specific mortality rates, our analysis focuses explicitly on estimating a population-level exposure–response function at the ZIP code-year level. Following Josey et al. (2023), we apply confounding weights and design-stage adjustments at this aggregated level because stratum-specific covariates are unavailable at the required granularity. Although we use individual-level strata and ZIP code-level covariates to inform predictions in the analysis stage, we do not directly estimate stratum-specific mortality rates. Therefore, Tian’s critique regarding hierarchical structure does not directly apply to our methodological approach, which appropriately captures covariates at the relevant aggregation level to estimate a population-level relationship between ${\text{PM}}_{2.5}$ and mortality.

5.2. Generalizability of results

Singh questions the sharp increase in relative mortality at low levels of ${\text{PM}}_{2.5}$, its consistency with prior studies, and generalizability to other populations. The sharper increase at lower ${\text{PM}}_{2.5}$ levels we observed aligns with prior Medicare-based analyses of long-term exposure (Josey et al., 2023; Wu et al., 2020, 2022). However, direct comparisons to previous studies are limited by methodological differences; specifically, we model the logarithm of the mortality rate as a continuous outcome, consistent with our simulation framework.

Though our analysis uses only the US Medicare population, the findings remain broadly informative, given that older adults are among the most vulnerable to air pollution impacts (Ailshire & Brown, 2020), accounting for two-thirds of US deaths (Di et al., 2017). Future studies should assess generalizability by including younger populations, other geographic regions, or international cohorts, and evaluating impacts of population-specific health, socioeconomic, and exposure variations.

5.3. Implications for public health policy

Finally, we would like to address how our findings of the ERC translate into real-world public health policy. When this study was conducted, it was motivated in part by ongoing discussions about strengthening the US National Ambient Air Quality Standards (NAAQS) for ${\text{PM}}_{2.5}$-with particular attention to the importance of setting federal limits that lead to meaningful reductions in population-level exposure. Air pollution is a shared environmental burden that does not respect geographic or political boundaries. Pollutants often travel far from their source, affecting communities located downwind of industrial, urban, and transportation corridors. This reality underscores the need for large-scale policy interventions grounded in a collective responsibility to reduce harmful exposures and protect public health.

The US Environmental Protection Agency (EPA) relies heavily on the estimated shape of the ERC between air pollution and health outcomes to determine whether updates to air quality standards are warranted. On 7 February 2024, the EPA revised the annual NAAQS for ${\text{PM}}_{2.5}$ from 12 $\mu {\text{g/m}}^3$ to 9 $\mu {\text{g/m}}^3$, citing substantial epidemiological evidence demonstrating the health risks of higher levels of pollution—evidence built on many of the same methodological principles employed in this study (US Environmental Protection Agency, 2024). Although our analysis was completed prior to this regulatory change, its findings remain highly relevant to future policy debates, reinforcing the central role of empirical evidence in guiding environmental protections.

At the same time, it is important to recognize that scientifically grounded environmental regulations are not guaranteed to endure. The durability of public health protections often hinges on political leadership, and recent shifts in political priorities have signalled a troubling movement away from environmental health regulation, including efforts to weaken existing air pollution standards. These policy reversals carry real consequences not only for regulatory infrastructure but also for human health and environmental sustainability. In the world we are in today, the need for independent and rigorous scientific research is greater than ever. Although regulatory decisions may be influenced by political and economic considerations, the scientific community has a responsibility to continue producing credible, transparent evidence that informs decision-making at all levels of government. We must also remain clear-eyed and creative about how this evidence can shape public understanding and policy action beyond federal regulation alone. The path forward will require collective effort—not only from scientists and policy makers but also from communities, advocates, and institutions—to advance a broader cultural recognition of the health and climate harms associated with air pollution. In this way, continued research on the exposure–response relationship can contribute meaningfully to both public dialogue and long-term progress in environmental health.

6. Future directions

We appreciate the discussants’ thoughtful suggestions for future research directions. While beyond our current scope, these proposals highlight valuable opportunities for enhancing the robustness and flexibility of causal inference methods for continuous exposures.

6.1. Extensions to study design and data-generating mechanisms

Several discussants raised important points about expanding our study design and refining the underlying data-generating processes. Swallow suggests incorporating lagged effects between exposure and outcome, as well as exploring causal discovery methods. Including lagged effects would allow for an assessment of how exposure history influences outcomes over time, which is particularly relevant for chronic exposures such as ${\text{PM}}_{2.5}$. Given the dynamic nature of air pollution exposure, considering time-dependent causal effects would add an important dimension to future studies. Similarly, causal discovery methods could provide a data-driven approach for uncovering relationships between variables in observational data sets, reducing the reliance on strong parametric assumptions.

Swallow also proposes using marginal structural models (MSMs) as an alternative framework for studying continuous exposures, along with copula-based modelling to allow more flexible ERCs (Evans & Didelez, 2024). We agree that MSMs offer a compelling complement to our current approach. In this study, we employed a structural equation modelling framework—also known as a direct causal model—for our simulation, where variables are specified sequentially: covariates, then exposure conditional on covariates, and finally the outcome conditional on both. This approach fully defines the joint distribution through factorization, ensuring internal compatibility. In contrast, MSMs rely on a more parsimonious specification, focusing only on the causal contrast of interest without modelling the full data-generating process. This makes them particularly useful in settings with time-varying covariates, where they offer greater flexibility in addressing dynamic confounding. Incorporating MSMs in future work could yield important insights into how alternative frameworks affect ERC estimation and the robustness of causal conclusions.

Tian emphasizes the importance of testing robustness under alternative data-generating mechanisms and exploring how model choices influence policy conclusions. While our analysis already examines more than 72 distinct simulation scenarios reflecting realistic complexities, expanding scenario diversity in future studies would further enhance generalizability. Our existing simulation framework provides a foundation for such extensions, facilitating sensitivity analyses of ERC estimates to varying assumptions regarding confounding, effect heterogeneity, and exposure distributions.

6.2. Expanding methodological approaches

Srakar suggests exploring Bayesian, nonparametric, and machine learning approaches to address potential biases in ERC estimation. Bayesian frameworks are especially promising, as they incorporate prior knowledge and offer principled uncertainty quantification. However, most Bayesian causal methods have focused on binary treatments, with fewer applications to continuous exposures—though this is beginning to change (Antonelli et al., 2022; Papadogeorgou et al., 2019).

The CausalGPS methodology already illustrates how machine learning can aid causal inference by estimating the GPS (Wu et al., 2022). Additionally, ongoing research into doubly robust estimators aims to combine machine learning’s flexibility with the theoretical guarantees of causal inference (Kennedy et al., 2017; Schulz & Moodie, 2021). While these methods show great promise, we did not include them in our study due to practical constraints: many lack mature software implementations or do not scale efficiently to the large, national Medicare dataset we analysed. Nonetheless, incorporating such methods into future analyses could provide valuable comparative insights and help evaluate their performance alongside more established approaches. These extensions would further advance the development of flexible, reliable tools for estimating exposure–response relationships in large-scale environmental health research.

Funding

This research was supported by grants from the National Institutes of Health (Grant No. T32 ES007142, R01ES026217, R01MD012769, R01ES028033, 1R01ES030616, 1R01AG066793, 1R01ES029950, 1R01ES 034373-01) and the Alfred P. Sloan Foundation (Grant No. G-2020-13946, 1R01AG066793). We would like to thank these funding agencies for their support.