Regularizing Extrapolation in Causal Inference

Abstract

Many causal inference and machine learning estimators are linear smoothers, where the prediction is a weighted average of training outcomes. Whether weights are constrained to be non-negative creates a key tradeoff: non-negative weights (e.g., inverse propensity weighting, random forests) limit extrapolation but can worsen covariate imbalance, while unconstrained weights (e.g., OLS, kernel ridge regression) improve balance but increase dependence on parametric assumptions. We propose a unified framework that directly penalizes extrapolation via a soft constraint on negative weights, replacing the standard hard non-negativity restriction. We derive a worst-case error bound and introduce a novel “bias-bias-variance” tradeoff among distributional imbalance, model misspecification, and estimator variance; this tradeoff is especially pronounced in high dimensions with poor positivity. We develop a convex optimization procedure that regularizes this bound and outline how to use the extrapolation penalty as a sensitivity analysis for parametric assumptions. We demonstrate our approach on synthetic data and a real-world application generalizing randomized trial estimates to a target population.

1. INTRODUCTION

A core challenge in observational causal inference and domain adaptation is to adjust data distributions so that features are comparable across distinct groups, such as control and treated arms or source and target populations [Imbens and Rubin, 2015, Farahani et al., 2021]. Weighting estimators and linear smoothers, in which the prediction is a weighted average of training outcomes, are widely used for such adjustment; examples include implicit weighting estimators like ordinary least squares (OLS) and random forests and explicit weighting approaches like inverse propensity score weighting [Li et al., 2013] and importance sampling [Thomas and Brunskill, 2017].

An important divide among weighting estimators is whether weights are constrained to be non-negative, such as in traditional IPW, matching [Stuart, 2010], the synthetic control method [Abadie et al., 2010], and stable balancing weights [Zubizarreta, 2015, Ben-Michael et al., 2021a], as well as in the weighting component of popular doubly robust estimators like double machine learning [Chernozhukov et al., 2018]. This constraint limits extrapolation and dependence on parametric modeling assumptions, but typically at the cost of worse feature imbalance between re-weighted groups. This imbalance is especially pronounced in high-dimensional settings, when the curse of dimensionality means that positivity is less likely to hold, leading to further bias [D’Amour et al., 2021]. By contrast, linear smoothers like OLS and kernel ridge regression allow for arbitrarily negative weights [Robins et al., 2007], which can improve feature imbalance but at the cost of greater model dependence and higher estimator variance. Finally, augmented estimators that combine outcome modeling with explicit weighting strategies can be viewed as performing controlled extrapolation, balancing model dependence against feature imbalance. Pure weighting and pure outcome modeling thus represent the two extremes of no versus uncontrolled extrapolation.

In this paper, we leverage this geometric perspective to establish a general framework for systematically controlling extrapolation. In particular, we propose a unified approach that directly penalizes the level of extrapolation, replacing the current practice of a hard non-negativity constraint with a soft constraint and corresponding hyperparameter. Unlike prior research on extrapolation in machine learning that emphasizes predictions beyond observed covariate support, we conceptualize extrapolation through unit weights, a particularly natural framework for handling high-dimensional covariates [Ben-Michael et al., 2021b]. Specifically, our contributions are:

• Bias-bias-variance tradeoff. We propose a framework quantifying a “bias-bias-variance” tradeoff, decomposing error into bias from distributional imbalance, bias from outcome model misspecification, and estimator variance. This captures key tradeoffs encountered in common causal inference and distribution shift scenarios.

• Error bound and constrained optimization. We derive an error bound based on worst-case Hölder continuity deviations from linearity. We present an optimization approach to minimize this bound, explicitly controlling tradeoffs between biases. We characterize the finite-sample variance through our error bounds.

• Sensitivity analysis framework. We introduce a sensitivity analysis methodology integrated into our optimization framework, enabling systematic evaluation of distributional imbalance and outcome model misspecification impacts. We illustrate this using synthetic data and a practical application involving the transportation of causal estimates to a novel target population.

1.1. Related work

Extrapolation and generalization are core topics in causal inference and machine learning. Recent surveys by Degtiar and Rose [2023] and Johansson et al. [2022] provide comprehensive overviews on generalizability and transportability methods.

Extrapolation and the synthetic control method.

Extrapolating far from the support of the data is a longstanding concern in statistics and the social sciences especially; see King and Zeng [2006] for a seminal discussion of possible dangers of unchecked extrapolation. Methods that limit extrapolation are common; the synthetic control method [Abadie et al., 2010] is a particularly prominent example. Doudchenko and Imbens [2016] discuss the non-negativity constraint in this context, and explore possible regularization. Most relevant to our approach, Ben-Michael et al. [2021b] developed the augmented synthetic control method, which combines outcome modeling with constrained weights to reduce bias while controlling extrapolation.

Extrapolation in machine learning.

Within machine learning, there has been substantial recent progress on approaches for addressing extrapolation. Shen and Meinshausen [2024] introduced engression, a framework that views extrapolation through the lens of distributional regression, enabling principled uncertainty quantification outside the training distribution. Kong et al. [2024] developed a causal lens for understanding extrapolation, establishing theoretical connections between causal structure and extrapolation. Netanyahu et al. [2023] proposed a transductive approach for learning to extrapolate, leveraging unlabeled test points to guide the extrapolation process. Dong and Ma [2022] provided foundational analysis toward understanding the extrapolation of nonlinear models to unseen domains, establishing bounds on extrapolation error. Finally, Pfister and Bühlmann [2024] developed extrapolation-aware nonparametric statistical inference methods, with formal guarantees on validity beyond the support of training data.

Unlike this recent literature, we approach extrapolation from a weighting perspective, which offers particular advantages in high-dimensional settings. Rather than focusing on predictions outside the covariate support, we frame extrapolation in terms of the properties of unit weights, providing a natural parameterization for high-dimensional settings [Ben-Michael et al., 2021b]. This perspective allows us to directly quantify and regularize the degree of extrapolation without relying on complex directional derivatives or high-dimensional density estimation.

Positivity violations and shifting the target.

Our discussion is closely related to the literature on positivity violations in causal inference. Crump et al. [2006], Li et al. [2018], and Parikh et al. [2025] all proposed to avoid issues due to positivity violations by shifting the estimand to regions with greater overlap. By contrast, our approach directly incorporates the severity of positivity violations into the weight estimation process.

Weighting representations.

A growing literature highlights the connections between various causal estimators through their weighting representations [Chattopadhyay and Zubizarreta, 2023]. Knaus [2024] provided a unified framework for viewing treatment effect estimators as weighted outcomes. Bruns-Smith et al. [2023] showed that augmented balancing weights can be interpreted as a form of linear regression. Lin and Han [2022] examined regression-adjusted imputation estimators through their weighting properties. Our framework builds on these insights by explicitly parameterizing the degree of extrapolation through weight regularization, providing a continuum of estimators that navigate the bias-variance tradeoff.

2. PRELIMINARIES

2.1. Setup and notation

To ease exposition, we focus on the causal inference problem of estimating the missing control potential outcome for the Average Treatment Effect on the Treated (ATT). As we note below, however, these results hold for general linear estimands as well as for domain adaptation in ML [Johansson et al., 2022].

For each unit $i\in \left[n\right]$, we observe the tuple $(X_i,Y_i,Z_i)$, with covariates $X_i\in 𝒳$, outcome $Y_i\in \mathbb{R}$, and binary treatment $Z_i\in \{0,1\}$. Invoking SUTVA, let $Y_i(0)$ and $Y_i(1)$ denote the control and treated potential outcomes, respectively, for unit $i$. Our estimand of interest is the ATT, $\mathbb{E}\left[Y_i(1)-Y_i(0)\mid Z_i=1\right]$; we discuss generalizations below. Since we observe $Y(1)$ for the treated group, the key challenge is to estimate the missing control potential outcome mean, $\mathbb{E}\left[Y_i(0)\mid Z_i=1\right]$. Finally, define the density ratio $dQ/dP(x)$, where $Q$ and $P$ denote the populations of units assigned to treatment and control, respectively. We need the following key assumptions for nonparametric identification:

• A.1. (Exchangeability) $\mathbb{E}[Y(0)\mid X,Z=1]=\mathbb{E}[Y(0)\mid X,Z=0]$

• A.2. (Population overlap) $dQ/dP(x)<\infty$ for all $x\in 𝒳$

In our setup, we consider situations when the population overlap assumption A.2. might be violated. In that case, researchers can instead rely on parametric assumptions on $\mu (\mathit{x})=\mathbb{E}\left[Y(0)\mid X_i=\mathit{x}\right]$, such as linearity, to identify and estimate the expected outcomes.

Finally, following Chattopadhyay and Zubizarreta [2023], we will focus on estimating the mean at a target covariate profile, ${\mathit{x}}^{\star }\in 𝒳$, corresponding to our estimand of interest. For the ATT, this profile is simply the mean of the treated population, ${\mathit{x}}^{\star }=\mathbb{E}[X\mid Z=1]$, where $\mu \left({\mathit{x}}^{\star }\right)=\mathbb{E}[Y(0)\mid Z=1]$.

Linear in features.

Since we are focused on linear smoothers, we consider models that are linear in some features, but which could be complex functions of the underlying covariates. This is an extremely large model class that ranges from simple linear models to the last layer embedding from a pre-trained large language model. For our setup, we let $\mathit{x}$ be the features in the representation implied by the parametric model, rather than simply the raw covariates. We further assume:

Assumption 2.1. $\mu$ is Hölder continuous such that $\left|\mu (\mathit{x})-\mu \left({\mathit{x}}^{\prime }\right)\right|\le a\cdot {‖\mathit{x}-{\mathit{x}}^{\prime }‖}^{\alpha }$, with $a>0$ and $\alpha >0$.

Parameterizing $\mu$ in terms of its Hölder constants is useful for characterizing departures from linearity that directly affect the estimation error bound.

General linear estimands.

We can leverage recent work on the Riesz representer [Chernozhukov et al., 2022a] to immediately generalize our results to any linear functional of the data. Following Bruns-Smith et al. [2023], for each unit $i\in \left[n\right]$, we observe the tuple $(X_i,Y_i,Z_i)$, with covariates $X_i\in 𝒳$, outcome $Y_i\in \mathbb{R}$, and treatment $Z_i\in 𝒵$. The target functional is then $\mathbb{E}\left[h\left(X_i,Z_i,m\right)\right]$ for a function $h\in L_2$, where $m(x,z)=\mathbb{E}\left[Y_i\mid X_i=x,Z_i=z\right]$. Many common problems in causal inference and domain adaptation are special cases of this setup, including counterfactual quantities like the average derivative and the expected policy-specific outcome. Finally, define a feature map $\varphi :𝒳\times 𝒵\to {\mathbb{R}}^d$; the target feature profile is then ${\varphi }^{*}(x,z)=\mathbb{E}[h(X,Z,\varphi )]$. Our results below apply by replacing the simple covariate profile ${\mathit{x}}^{\star }$ with the much more general feature profile ${\varphi }^{\star }(x,z)$.

2.2. Weighting form of causal inference estimators

Our focus is on weighting estimators or linear smoothers [Buja et al., 1989] of the form:

$$\hat{\mu }\left({\mathit{x}}^{\star }\right)=\sum _{i=1}^n{w}_{\star \leftarrow i}\left({\mathit{x}}_i\right)Y_i,$$

with weights $w_{\star \leftarrow i}\left({\mathit{x}}_i\right)$, where $\star \leftarrow i$ emphasizes that the weights can depend both on the source covariates ${\mathit{x}}_i$ and the target covariates ${\mathit{x}}^{\star }$ [Lin and Han, 2022]. When there is no ambiguity, we suppress the dependence on the covariates ${\mathit{x}}_i$ and the target ${\mathit{x}}^{\star }$.

A broad class of estimators have this form. See Knaus [2024] for a comprehensive discussion of the weighting form for common causal inference estimators. We highlight several special cases here, with a focus on whether the implied weights are constrained to be non-negative.

Explicit weighting estimators.

The first class of methods estimate the density ratio $\widehat{dQ/dP}(x)$, either directly or indirectly.

• Traditional Inverse Propensity Score Weighting. In standard IPW [Rosenbaum, 1987], researchers first estimate a propensity score, $e(\mathit{x})=\mathbb{P}\left[Z_i=1\mid {\mathit{X}}_i=\mathit{x}\right]$ via a binary classifier like logistic regression, and then plug into a known functional form for $dQ/dP(x)$. For the ATT, $\hat{w}(\mathit{x})=\hat{e}(\mathit{x})/(1-\hat{e}(\mathit{x}))$; since $\hat{e}(\mathit{x})\in (0,1),{\hat{w}}_i(\mathit{x})>0$ for all $i$.

• Balancing weights, synthetic control, and matching. An alternative weighting approach instead directly estimates $dQ/dP(x)$ via constrained optimization [Ben-Michael et al., 2021a]. For example, consider the minimum variance weights that control imbalance in $\mathit{x}$ between $P$ and $Q$:

  $$\stackrel{\mathbf{ˆ}}{\mathbf{w}}\in \text{arg}\underset{\mathbf{w}\in 𝒲}{\text{min}}{‖\sum _{i=1}^n{w}_i{\mathit{X}}_i-{\mathit{x}}^{\star }‖}_p^2+\lambda \Vert \mathit{w}{\Vert }_2^2,$$ (1)

  where $\Vert \cdot {\Vert }_p$ is the $p$ vector norm and where $𝒲$ are possible constraints on the weights. Stable balancing weights [Zubizarreta, 2015] and the Synthetic Control Method [Abadie et al., 2010] are special cases where $𝒲$ is the simplex $(w_i\ge 0,\sum w_i=1)$ and the imbalance norm is $p=\infty$ and $p=2$, respectively. Matching is a special case where the weights are also constrained to be discrete.

• Riesz regression. A final weighting approach, also known as automatic estimation of the Riesz representer [Chernozhukov et al., 2022b] also finds weights via Problem (1), albeit without imposing the constraint that weights are non-negative. For example, minimum distance lasso Riesz regression in Chernozhukov et al. [2022b] solves Equation (1) with $𝒲={\mathbb{R}}^n$ and $p=\infty$.

Linear smoothers and implicit weighting estimators.

A wide range of popular outcome models are linear smoothers that implicitly estimate weights $w$, including (kernel ridge) regression, $k$-nearest neighbors, random forests, xgboost, and many implementations of neural networks; see Lin and Han [2022], Curth et al. [2024]. We highlight two prominent examples with and without a non-negativity constraint.

• (Kernel) ridge regression. For features $\mathit{X}$, the implied ridge regression weights are:

  $$w_{\star \leftarrow i}={\mathit{x}}^{\star \top }{\left({\mathit{X}}^{\top }\mathit{X}+\lambda \mathbb{I}\right)}^{-1}{\mathit{X}}_i,$$

  where $\lambda$ is a regularization parameter; ordinary least squares (OLS) as a special case when $\lambda =0$. Kernel ridge regression is instead based on the implied kernel features $\varphi (\mathit{x})$; see Bruns-Smith et al. [2023], Hirshberg et al. [2019]. As Bruns-Smith et al. [2023] discuss, the ridge regression weights are equivalent to solving optimization problem (1) with the imbalance norm set to $p=2$ and with $𝒲={\mathbb{R}}^n$, which does not include a non-negativity constraint.

• Random forests. As Athey et al. [2019] discuss in the context of causal inference, (honest) random forests is a locally adaptive linear smoother with non-negative weights:

  $${\hat{w}}_{\star \leftarrow i}=\frac{1}{B}\sum _{b=1}^B\frac{\mathbb{I}\left\{{\mathit{x}}^{\star }\in L_b(\mathit{x})\right\}}{\left|L_b(\mathit{x})\right|},$$

  where $L_b$ is the set of units that share a leaf node with the target ${\mathit{x}}^{\star }$ and $b=1,\dots ,B$ index the trees.

Augmented and hybrid estimators.

Finally, augmented or hybrid estimators combine initial weights ${\mathit{w}}^0$ and outcome model $\hat{m}$:

$$\begin{gathered} {\hat{\mu }}^{dr}\left({\mathit{x}}^{\star }\right)=\sum _{i=1}^N{\hat{w}}_i^0{Y}_i+\left(\hat{m}\left({\mathit{x}}^{\star }\right)-\sum _{i=1}^N{w}_i^0\hat{m}\left({\mathit{x}}_i\right)\right) \\ =\hat{m}\left({\mathit{x}}^{\star }\right)+\sum _{i=1}^N{\hat{w}}_i^0\left(Y_i-\hat{m}\left({\mathit{x}}_i\right)\right). \end{gathered}$$

When $\hat{m}$ is a linear smoother, then ${\hat{\mu }}^{dr}(\mathit{x})$ also has a weighting representation. Let $\hat{m}\left({\mathit{x}}^{\star }\right)=\sum {\hat{\omega }}_i(\mathit{x})Y_i$ for a weighting function $\hat{\omega }:{\mathbb{R}}^d\to {\mathbb{R}}^n$. Following Ben-Michael et al. [2021b]:

$$\begin{gathered} {\hat{\mu }}^{dr}\left({\mathit{x}}^{\star }\right)=\sum _{i=1}^N\left({\hat{w}}_i^0+{\hat{w}}_i^{\text{adj}}\right)Y_i \\ \text{where}{\hat{w}}_i^{\text{adj}}\equiv {\hat{\omega }}_i\left({\mathit{x}}^{\star }\right)-\sum _{j=1}^n{\hat{w}}_j^0{\hat{\omega }}_i\left({\mathit{x}}_j\right). \end{gathered}$$

For example, when the outcome model is ridge regression, the implied weights for the doubly robust estimator have the following form:

$${\hat{w}}_i^{dr}={\hat{w}}_i^0+{\left({\mathit{x}}^{\star }-{\mathit{x}}^{\prime }{\stackrel{\mathit{ˆ}}{\mathit{w}}}^0\right)}^{\prime }{\left({\mathit{x}}^{\prime }\mathit{x}+\lambda \mathbb{I}\right)}^{-1}{\mathit{x}}_i.$$

Importantly, even if the initial weights ${\mathit{w}}^0$ are constrained to be non-negative, such as in traditional IPW, the implied doubly robust weights ${\mathit{w}}^{dr}$ could be negative. In fact, the combined weights can be negative even if both the initial weights ${\mathit{w}}^0$ and the outcome model-implied weights $\hat{\omega }$ are non-negative.

There are many examples of combined estimators of this form: standard Augmented IPW [Chattopadhyay and Zubizarreta, 2023], bias correction for inexact matching [Lin et al., 2021], augmented synthetic control method [Ben-Michael et al., 2021b], and regression-adjusted imputation estimators more broadly [Lin and Han, 2022]. Finally, both debiased machine learning [Chernozhukov et al., 2018] and automatic debiased machine learning [Chernozhukov et al., 2022a] have this form. The former constrains the initial weights to be non-negative; the latter does not.

3. REGULARIZING WORST-CASE EXTRAPOLATION BIAS

Our goal is to bound the estimation error $\left|\mu \left({\mathit{x}}^{\star }\right)-\sum _{i=1}^n{w}_i{Y}_i\right|$. We begin by building intuition for our approach in three steps.

Reflection representation.

Under linearity, $\mu \left(-{\mathit{x}}_i\right)=-\mu \left({\mathit{x}}_i\right)$, so a negative weight $w_i<0$ on ${\mathit{x}}_i$ yields $w_i\mu \left({\mathit{x}}_i\right)=\left|w_i\right|\mu \left(-{\mathit{x}}_i\right)$; in other words, a negative weight $w_i$ is equivalent to applying a positive weight $\left|w_i\right|$ to the reflected point $-{\mathit{x}}_i$. We can then construct a “reflected” estimator, denoted by ‡, which reflects points with negative weights around the origin:

$$\begin{gathered} {\hat{\mu }}^{‡}\left({\mathit{x}}^{\star }\right)=\sum _{i=1}^n{w}_i\mathbb{1}\left(w_i\ge 0\right)\mu \left({\mathit{X}}_i\right)+\left|w_i\right|\mathbb{1}\left(w_i<0\right)\mu \left(-{\mathit{X}}_i\right) \\ =\sum _{i=1}^n\left|w_i\right|\mu \left({\mathit{X}}_i^{‡}\right),{\mathit{X}}_i^{‡}=\left\{\begin{array}{ll}{\mathit{X}}_i,& w_i\ge 0\\ -{\mathit{X}}_i,& w_i<0\end{array},\right. \end{gathered}$$

where $\hat{\mu }\left({\mathit{x}}^{\star }\right)={\hat{\mu }}^{‡}\left({\mathit{x}}^{\star }\right)$ if $\mu$ is an odd function, and where $w_i{\mathit{X}}_i=\left|w_i\right|{\mathit{X}}_i^{‡}$ for all $i$.

Measuring parametric model violations.

The difference between $\hat{\mu }\left({\mathit{x}}^{\star }\right)$ and ${\hat{\mu }}^{‡}\left({\mathit{x}}^{\star }\right)$ measures the degree to which the assumed parametric model is violated. Specifically, we can write $\mu \left(-{\mathit{x}}_i\right)=\delta \left({\mathit{x}}_i\right)-\mu \left({\mathit{x}}_i\right)$, where $\delta \left({\mathit{x}}_i\right)\equiv \mu \left(-{\mathit{x}}_i\right)+\mu \left({\mathit{x}}_i\right)$. If $\mu$ is an odd function (e.g., $\mu$ is linear through the origin), then $\delta \left({\mathit{x}}_i\right)=0$ for all $i$. Thus, $\delta \left({\mathit{x}}_i\right)$ is a point-specific measure of the extent to which the true outcome function violates the assumed parametric model; throughout, our discussion of “nonlinearity” should be understood as referring to such violations.

We use this representation to decompose the estimator $\hat{\mu }\left({\mathit{x}}^{\star }\right)$:

$$\begin{gathered} \hat{\mu }\left({\mathit{x}}^{\star }\right)=\sum _{i=1}^n{w}_i{Y}_i=\sum _{i=1}^n{w}_i\left(\mu \left({\mathit{X}}_i\right)+{ϵ}_i\right) \\ =\sum _{i=1}^n{w}_i\mathbb{1}\left(w_i\ge 0\right)\mu \left({\mathit{X}}_i\right)+\left|w_i\right|\mathbb{1}\left(w_i<0\right)\left(\mu \left(-{\mathit{X}}_i\right)-\delta \left({\mathit{X}}_i\right)\right)+w_i{ϵ}_i \\ =\underset{{\hat{\mu }}^{‡}\left({\mathit{x}}^{\star }\right)}{\underbrace{\sum _{i=1}^n\left|w_i\right|\mu \left({\mathit{X}}_i^{‡}\right)}}+\underset{\text{model violation}}{\underbrace{\sum _{i=1}^n{w}_i\mathbb{1}\left(w_i<0\right)\delta \left({\mathit{X}}_i\right)}}+\underset{\text{noise}}{\underbrace{\sum _{i=1}^n{w}_i{ϵ}_i}}. \end{gathered}$$

Error bound.

Although $\delta (\mathit{X})$ is unknown, we can bound it via Hölder continuity: $|\delta (\mathit{x})|=\mid \mu (\mathit{x})+\mu (-\mathit{x})\left|\le |\mu (\mathit{x})-\mu (\mathbf{0})|+|\mu (-\mathit{x})-\mu (\mathbf{0})|\le 2a\Vert \mathit{x}{\Vert }^{\alpha }\right.$, where the last step uses $\mu (\mathbf{0})=0$, which holds after centering.^* The resulting error bound is therefore

$$\left|\mu \left({\mathit{x}}^{\star }\right)-\hat{\mu }\left({\mathit{x}}^{\star }\right)\right|\le \underset{\text{error in}{\hat{\mu }}^{‡}\left({\mathit{x}}^{\star }\right)}{\underbrace{\left|\sum _{i=1}^n\right|w_i\left|\mu \left({\mathit{X}}_i^{‡}\right)-\mu \left({\mathit{x}}^{\star }\right)\right|}}+\underset{\text{error due to model violation}}{\underbrace{2a\sum _{i=1}^n\left|w_i\right|\mathbb{1}\left(w_i<0\right){‖{\mathit{X}}_i‖}^{\alpha }}}+\underset{\text{noise}}{\underbrace{\left|\sum _{i=1}^n{w}_i{ϵ}_i\right|}}.$$ (2)

The first term directly depends on the imbalance between the target point ${\mathit{x}}^{\star }$ and the re-weighted (reflected) training points $|\mathit{w}{|}^{\prime }{\mathit{X}}^{‡}$. The second term captures additional error due to model violation, corresponding to the $\delta (\mathit{X})$ term above; this is the key new term that our framework regularizes. The third term is the noise.

3.1. Characterizing asymmetry-induced bias

Thus far we have presented a conservative nonparametric bound. We now provide a slightly refined characterization by noting that the extent of the bias induced by negative weights is driven by the asymmetry in $\mu$. We do so by considering the decomposition of $\mu$ into its even and odd components, i.e., $\mu (x)={\mu }_e(x)+{\mu }_o(x)$. By the definition of odd functions, we have $-{\mu }_o(x)={\mu }_o(-x)$; we can then bound the worst-case risk of $\stackrel{\mathit{ˆ}}{\mathit{w}}$ using the assumed Hölder constants $a$ and $\alpha$ and isolate the effect of the even component. The formal statement is given below in Proposition 3.1; the proof is given in Appendix C.

Proposition 3.1. Let $\hat{\mu }\left(x^{*}\right)=\sum _{i=1}^n{\hat{w}}_i{Y}_i$ be the estimate of $\mu \left(x^{*}\right)$ with weights estimated via Equation (4) (defined below). Given $Y_i=\mu \left(X_i\right)+{ϵ}_i$ where ${ϵ}_i$ are independent random variables with $\mathbb{E}\left[{ϵ}_i\right]=0$ and finite second moment ${\sigma }^2=\mathbb{E}\left[{ϵ}_i^2\right]$, and $\mu$ is Hölder continuous with constants a and $\alpha$. If ${ϵ}_i$ are sub-Gaussian^† with parameter $\sigma$, then with probability at least $1-\delta$,

$$\left|\mu \left(x^{*}\right)-\widehat{\mu }\left(x^{*}\right)\right|\le \left|\mu \left(x^{*}\right)-\sum _{i=1}^n\left|{\widehat{w}}_i\right|\mu \left({\mathit{X}}_i^{‡}\right)\right|+2\sum _{i=1}^n\left|{\widehat{w}}_i\right|1\left({\widehat{w}}_i<0\right)a{\Vert {\mathit{X}}_i\Vert }^{\alpha }+\sigma {\Vert \widehat{w}\Vert }_2\sqrt{2\text{log}(2/\delta )}$$ (3)

where ${\mathit{X}}_i^{‡}=\text{sign}\left({\hat{w}}_i\right){\mathit{X}}_i$ as defined above. The first term is the imbalance of the reflected estimator ${\hat{\mu }}^{‡}\left(x^{*}\right)=\sum _i\left|{\hat{w}}_i\right|\mu \left({\mathit{X}}_i^{‡}\right)$; the second is the model violation bias due to negative weights (identical to the second term of (2)); the third is the noise.

The proof in Appendix C proceeds via an even-odd decomposition that shows that negative weights introduce additional bias only through the even component of $\mu$; the odd part is absorbed exactly into the first term via the reflection ${\mathit{X}}_i^{‡}$. The noise term in Proposition 3.1 requires a sub-Gaussian assumption.

Since ${\mu }_e$ is unidentifiable from a single dataset, we construct a conservative worst-case form that does not require access to ${\mu }_e$. For completeness, Proposition C.1 in Appendix C provides an empirical analog that approximates ${\mu }_e$ via nearest-neighbor matching when the data are approximately symmetric.

Finally, following Chattopadhyay and Zubizarreta [2023], we define negative influence as the fraction of total weight on units with negative weights, $\sum _i\mathbf{1}\left[w_i<\right.0]\left|w_i\right|/\sum _i\left|w_i\right|$. This is a useful summary of the extent to which the estimate relies on extrapolation.

3.2. Proposed Estimator

We now propose an estimator to learn weights $\mathit{w}$ that directly control the error bound in Equation (2). To do so, we modify the standard balancing weights optimization problem in Equation (1) by using the Lagrangian form of the non-negativity constraint, rather than the hard constraint. Thus, the combined estimator minimizes the error bound by controlling three terms: covariate imbalance, dispersion of the weights, and level of extrapolation:

$$\stackrel{\mathbf{ˆ}}{\mathbf{w}}\in \text{arg}\underset{\mathbf{w}}{\text{min}}\underset{(a)\text{imbalance}}{\underbrace{{‖{\sum }_{i=1}^n{w}_i{\mathit{X}}_i-{\mathit{x}}^{\star }‖}_2^2}}+\underset{(b)\text{variance}}{\underbrace{\lambda \Vert \mathbf{w}{\Vert }_2^2}}+\underset{(c)\text{extrapolation}}{\underbrace{\gamma {\sum }_{i=1}^n\mathbb{1}\left(w_i<0\right)\left|w_i\right|{‖{\mathit{X}}_i‖}^{\alpha }}}$$ (4)

where

• Term (a): Enforces balance between the target point ${\mathit{x}}^{\star }$ and the re-weighted training points $\left\{{\mathit{X}}_1,\dots ,{\mathit{X}}_n\right\}$, recalling that $w_i{\mathit{X}}_i=\left|w_i\right|{\mathit{X}}_i^{‡}$ for all $i$. We focus on $p=2$, but this generalizes to $p=\infty$. This corresponds to the first term of (2).

• Term (b): Regularizes the dispersion of the weights $\mathbf{w}$, controlling the noise term via $\Vert \stackrel{\mathbf{ˆ}}{\mathbf{w}}{\Vert }_2$ in Proposition 3.1.

• Term (c): Penalizes model violation bias: the penalty $\gamma \sum _i\mathbf{1}\left(w_i<0\right)\left|w_i\right|{‖{\mathit{X}}_i‖}^{\alpha }$ is proportional to the second term of (2), with $\gamma$ scaling the sensitivity to parametric model violations.^‡

Compared to the standard balancing weights problem (1), which trades off only imbalance and variance, the new objective (4) introduces term (c) to control extrapolation. When the target lies outside the convex hull of the training points, achieving balance requires some weights to be negative, which increases both $\Vert \mathit{w}{\Vert }_2$ and reliance on parametric assumptions. For $\gamma =0$, Equation (4) recovers unconstrained balancing weights; at the other extreme, $\gamma \to \infty$ is equivalent to a hard non-negativity constraint. Increasing $\gamma$ reduces extrapolation bias and $\Vert \mathit{w}{\Vert }_2$ but worsens imbalance in term (a).

Regularizing existing estimators.

Since many causal estimators have a weighting representation (Section 2.2), we can regularize extrapolation in any baseline estimator with implied weights ${\mathit{w}}^{\prime }$ by solving

$$\stackrel{\mathbf{ˆ}}{\mathbf{w}}\in \text{arg}\underset{\mathbf{w}}{\text{min}}{‖\mathbf{w}-{\mathbf{w}}^{\prime }‖}_2^2+\gamma {\sum }_{i=1}^n\mathbb{1}\left(w_i<0\right)\left|w_i\right|{‖{\mathit{X}}_i‖}^{\alpha }.$$

For example, the augmented synthetic control method [Ben-Michael et al., 2021b] first solves with non-negative weights, then augments with a ridge outcome model that implicitly introduces negative adjustment weights; the formulation above instead directly controls the degree of negativity through $\gamma$.

Convexity.

Despite the indicator function in term (c), the optimization problem (4) is strongly convex and admits a unique global minimizer. To see this, note that $\mathbf{1}\left(w_i<0\right)\left|w_i\right|=\text{max}\left(0,-w_i\right)$, which is convex as the pointwise maximum of two affine functions. Term (a) is a squared norm of an affine function of $\mathit{w}$, hence convex, and term (b) is strongly convex with parameter $2\lambda$. More generally, term (c) can be written using an ${\ell }_p$ norm over the vector of per-unit penalties. When $p=1$ (as written above and in Proposition 3.2), introducing slack variables $s_i\ge -w_i,s_i\ge 0$ reduces the problem to a quadratic program (QP), which can be solved exactly in polynomial time with standard solvers. When $p=2$, the problem becomes a second-order cone program (SOCP), which is likewise solvable in polynomial time.

Finally, we can specialize the error bound for our proposed estimator:

Proposition 3.2 (Regularized Bound). Let $\hat{w}$ solve Equation (4) and $\hat{\mu }\left(x^{*}\right)=\sum _{i=1}^n{\hat{w}}_i{Y}_i$ where $Y_i=\mu \left(X_i\right)+{ϵ}_i$. Under the assumptions of Proposition 3.1, with probability at least $1-\delta$:

$$\left|\mu \left(x^{*}\right)-\widehat{\mu }\left(x^{*}\right)\right|\le \left|\mu \left(x^{*}\right)-\sum _{i=1}\left|{\widehat{w}}_i\right|\mu \left(X_i^{‡}\right)\right|+2\sum _{i=1}^n\left|{\widehat{w}}_i\right|1\left({\widehat{w}}_i<0\right)a{‖X_i‖}^{\alpha }+\sigma {‖x^{*}‖}_2\sqrt{\text{log}(2/\delta )/\lambda }$$

where ${\mathit{X}}_i^{‡}=\text{sign}\left({\hat{w}}_i\right){\mathit{X}}_i$ as in Proposition 3.1.

The proof is provided in the appendix. The variance term $\sigma {‖{\mathit{x}}^{*}‖}_2\sqrt{\text{log}(2/\delta )/\lambda }$ does not depend on $\gamma$: increasing $\gamma$ further constrains the feasible set and cannot inflate $\Vert \hat{w}{\Vert }_2$. This means regularizing extrapolation reduces the bias terms without incurring additional variance cost. The full bias-bias-variance tradeoff is demonstrated empirically in Sections 4 and 5.

3.3. Practical guidance: $\gamma$ as a sensitivity parameter

We argue that $\gamma$ should be treated as a sensitivity parameter rather than a tuning parameter, and encourage researchers to examine the full set of estimates it spans. We recommend the following procedure:

1. Fix $\lambda$ via cross-validation on the standard balancing weights problem (i.e., with $\gamma =0$).

2. Sweep $\gamma$ over a grid from 0 to a value ${\gamma }_{\text{max}}$ at which all weights become non-negative.

3. For each $\gamma$, record the point estimate, covariate balance (RMSE), and negative influence.

4. Examine how estimates change across this range to assess sensitivity to parametric assumptions.

If the goal is a single point estimate, we can instead choose ${\gamma }^{*}$ in the spirit of Lepski’s method, selecting the largest $\gamma$ for which the change in the point estimate remains below a researcher-defined cutoff.

4. SYNTHETIC DATA STUDY

We evaluate our approach using synthetic data with both linear and nonlinear data generating processes (DGPs) where the target point lies outside the convex hull of training points (Figure 1), creating a challenging extrapolation scenario with limited sample size ($n=10$ training units, $n/p=5$). We also consider a high-dimensional setting $(p=5000,n/p=0.2)$ using the Friedman DGP. Full descriptions and additional figures are provided in Appendix D.

Figure 1:

Convex hull of source (training) and target units. The target point lies outside the convex hull, requiring extrapolation.

We consider two DGPs:

$$\text{Linear:}\mu (\mathit{X})={\beta }^{\top }\mathit{X},$$

$$\text{Nonlinear:}\mu (\mathit{X})=2X_1^2+X_2+X_1{X}_2.$$

For the linear DGP, where the parametric assumption holds, estimation error increases monotonically as we regularize extrapolation (i.e., increase $\gamma$): relying on correct parametric assumptions yields optimal estimates. However, for the nonlinear DGP (Figure 2), the quadratic and interaction terms violate the linearity assumption. The results illustrate the bias-bias-variance tradeoff predicted by our theory: small amounts of extrapolation remain beneficial due to the linear component, but excessive extrapolation leads to high error rates due to violations of the parametric model. The high-dimensional experiment further demonstrates that sweeping over $\gamma$ smoothly interpolates between parametric regression-like behavior (unconstrained extrapolation) and IPW-like behavior (no extrapolation).

Figure 2:

Estimation error (MSE) for the nonlinear DGP as a function of $\gamma$. The U-shaped curve illustrates the bias-bias-variance tradeoff: small $\gamma$ allows beneficial extrapolation, while large $\gamma$ incurs bias from poor balance.

5. GENERALIZING MEDICATION FOR OPIOID USE DISORDER TRIAL EVIDENCE

We now apply our framework to the problem of generalizing causal estimates from a randomized trial to a target population. Appendix B states the formal identification assumptions and maps this setting onto the general framework of Section 2.2.

The Starting Treatment With Agonist Replacement Therapies (START) trial $(S_i=1)$, initiated in 2006, was a multi-center study comparing buprenorphine versus methadone in treating opioid use disorder [Saxon et al., 2013, Hser et al., 2014]. The trial enrolled 1,271 participants, who were randomized in a 2:1 ratio to receive either buprenorphine or methadone. Methadone was found to have higher rates of patient retention in treatment compared to buprenorphine [Hser et al., 2014]. Our analysis focuses on the outcome of relapse to regular opioid use within 24 weeks of medication assignment, defined as non-study opioid use for four consecutive weeks or daily use for seven consecutive days.

Parikh et al. [2025] identified that Hispanic women with a pre-treatment history of amphetamine and benzodiazepine use were underrepresented in the START trial relative to the target population $(S_i=0)$, highlighting a practical violation of the positivity assumption T.4.. In this study, we estimate the target average treatment effect (TATE), $\tau =\mathbb{E}\left[Y_i(1)-Y_i(0)\mid S_i=0\right]$, for this underrepresented subgroup using our proposed framework alongside standard linear regression, gradient boosting regression (GBR), inverse probability weighting (IPW), and double machine learning estimators.

The target sample is drawn from the 2015–2017 Treatment Episode Dataset - Admissions (TEDS-A), which includes data on individuals entering publicly funded substance use treatment programs across 48 states (excluding Oregon and Georgia) and the District of Columbia. Our analysis focuses on Hispanic women with a pre-treatment history of amphetamine and benzodiazepine use.

We code methadone as $Z=1$ and buprenorphine as $Z=0$, with $Y=1$ representing relapse. Pretreatment covariates include age, race, biological sex, and substance use history (amphetamine, benzodiazepines, cannabis, and intravenous drug use) measured at the initiation of medication for opioid use disorder (MOUD) treatment. For each treatment arm $z\in \{0,1\}$, we estimate ${\mu }_z\left({\mathit{x}}^{\star }\right)=\mathbb{E}\left[Y_i\mid X_i={\mathit{x}}^{\star },S_i=1,Z_i=z\right]$ at the target profile ${\mathit{x}}^{\star }=\mathbb{E}\left[X_i\mid S_i=0\right]$ using the trial participants assigned to arm $z$ as source units. The estimated TATE is then $\hat{\tau }={\hat{\mu }}_1\left({\mathit{x}}^{\star }\right)-{\hat{\mu }}_0\left({\mathit{x}}^{\star }\right)$.

We then apply our proposed framework to this problem. By varying $\gamma$ from 0.01 to 10, we examine how treatment effect estimates shift with increasing regularization of negative weights. Without regularization, the point estimates converge to those from linear regression. As regularization intensifies, however, the estimates smoothly shift towards zero and occasionally change sign from negative to positive for smaller values of $\lambda$. This sensitivity underscores the influence of assumptions on the point estimates. While increasing $\gamma$ reduces negative influence (Figure 5), it worsens covariate balance, as reflected in higher RMSE values (Figure 4). Thus, our framework highlights a trade-off between minimizing reliance on parametric assumptions and achieving optimal covariate balance. Applied researchers should therefore interpret treatment effect estimates for this under-represented subgroup with caution given the sensitivity to modeling assumptions. As Parikh et al. [2025] emphasized, collecting more representative trial data is critical to credibly estimate treatment effects.

Figure 5:

Negative influence, defined as the contribution of negative weights in estimation, for different values of $\gamma$ and $\lambda$.

Figure 4:

Balance between the trial and the target samples measured as the root mean squared error (RMSE) for different values of $\gamma$ and $\lambda$.

6. CONCLUSION

This work proposes a framework for regularizing extrapolation in causal inference by replacing hard non-negativity constraints with soft penalties on negative weights. Our theoretical error bounds show a fundamental “bias-bias-variance” tradeoff between distributional imbalance, model misspecification, and estimator variance, decomposing extrapolation bias through a novel reflection perspective. Empirically, synthetic data experiments confirm that controlled extrapolation smoothly interpolates between fully constrained and unconstrained approaches. A real-world medication trial illustrates how sweeping over the regularization parameter provides a practical sensitivity analysis for transportability estimates under positivity violations.

Limitations and Future Work.

Our approach focuses on weighting-type estimators and relies on Hölder continuity and conditional ignorability, which may not hold in practice. Operationalizing sensitivity analysis for unmeasured confounding is a critical next step; existing proposals for balancing weights [Soriano et al., 2023] do not directly apply to our framework, and adapting such methods is an important direction. More broadly, future work should extend the bias-biasvariance tradeoff analysis to more flexible estimator classes and weaker continuity assumptions. A key open question is to characterize data-generating processes under which soft-constrained extrapolation $(\gamma >0)$ provably improves MSE relative to both unconstrained $(\gamma =0)$ and fully constrained $(\gamma \to \infty )$ estimators, for example when the density ratio is large near ${\mathit{x}}^{\star }$. Finally, our theoretical results assume sub-Gaussian noise, though analogous bounds follow under bounded outcomes via Hoeffding-type inequalities.

Figure 3:

Target Average Treatment Effects for the Target Sample for Hispanic Females who have a history of Amphetamine and Benzodiazepine use in TEDS-A population. Each hue corresponds to a value of $\lambda$ and the x-axis corresponds to different values of $\gamma$ (on log scale).