HAL-Based Plug-in Estimation with Pointwise Asymptotic Normality of the Causal Dose–response Curve

Abstract

Estimating and obtaining reliable inference for the marginally adjusted causal dose–response curve for continuous treatments without relying on parametric assumptions is a well-known statistical challenge. Parametric models risk introducing significant bias through model misspecification, compromising the accurate representation of the underlying data and dose–response relationship. On the other hand, nonparametric models face difficulties as the dose–response curve is not pathwise differentiable, preventing consistent estimation at standard rates and making the construction of confidence intervals challenging. The Highly Adaptive Lasso (HAL) maximum likelihood estimator offers a promising approach to this issue. In this paper, we introduce a HAL-based plug-in estimator for the causal dose–response curve, bridge between recent theoretical developments and empirical applications, and assess its empirical performance against other estimators. This work emphasizes the translation of existing HAL asymptotic theory to the causal dose–response setting and demonstrates its practical implications through comprehensive simulations, thereby filling an essential gap between theory and practice. Our comprehensive simulations demonstrate that the HAL-based estimator approaches pointwise asymptotic normality under the assumed smoothness conditions and yields confidence intervals that are consistently closer to nominal coverage than those of competing methods. While undercoverage can persist in more complex settings at moderate sample sizes, the HAL estimator consistently outperforms existing approaches for estimating the causal dose–response curve.

1. Introduction

A marginally adjusted dose–response curve, often referred to as a “causal” dose–response curve, provides a graphical representation of the population-level probability of a health outcome following a uniform application of a specific dose or, more generally, a dosage rule applied to the target population. Estimation of such a curve where the dose is ordinal or continuous requires accounting for potential confounders, missingness, and other typical complications in data-generating processes. Efficient estimators with valid inference are crucial to a wide range of scientific applications, including epidemiology, medical research, social sciences, and many other fields. For example, in a clinical trial investigating the impact of a continuous dose of drugs on blood pressure reduction, the dose–response curve can provide valuable insights into the specific reduction or increase in blood pressure attributed to each incremental change in dosage, accounting for confounding factors such as demographic and baseline clinical characteristics.

To estimate the causal dose–response curve, we need to define a parameter under the counterfactual or potential outcome framework in which the outcome would be observed for subjects in the target population if all of them received a specific treatment level [1–3]. This framework, widely used in causal inference, has predominantly focused on binary interventions [4–10]. Common approaches for estimating causal effects, such as the average treatment effect (ATE), include matching [11], estimating equations (e.g., inverse probability of treatment weighting (IPTW) [12] and augmented IPTW (A-IPTW) [13], and substitution estimators (e.g., G-computation [9] and targeted maximum likelihood estimation (TMLE) [4])). Furthermore, highly adaptive lasso (HAL) [14] and undersmoothed HAL [15] have been developed to provide reliable inference for the effects of binary treatment. Although significant progress has been made in the binary treatment context, many causal inference questions inherently involve continuous or ordinal interventions, such as doses in clinical treatments, for which consistent and efficient estimators remain scarce.

The primary challenge in estimating the causal dose–response curve for continuous treatments in a nonparametric model is that the parameter is not pathwise differentiable [16]. Consequently, uniform $\sqrt{n}$-rate inference for the entire curve is not available in the fully nonparametric model, and finite-sample calibration can be difficult, leading to reliance on estimators that depend on estimating densities rather than a smooth function of this density.

Considerable efforts have been dedicated to crafting estimators for the causal dose–response curve. The Marginal Structural Model (MSM) framework, originally introduced by Robins [12], is known for defining causal dose–response curves in parametric models. However, these methods rely on specifying parametric models of the dose–response curve (DRC) and thus are subject to incorrect model specification. To address potential model misspecification, enhancements to MSM have been proposed using outcome prediction models, data-adaptive estimation of MSM, and projecting the true causal dose–response curve onto a functional space defined by MSM [17–19]. Despite these advances, challenges related to robustness remain. Alternative estimators based on plug-in methods for conditional outcome models have been proposed [20–23], but these often struggle with dimensionality issues, restrictive assumptions, and the complexity of valid inference. Techniques such as cross-validated TMLE (Chapter 25 of [1]) and nonparametric causal inference [24] achieve asymptotic normality, facilitating inference, but are constrained by their reliance on smoothness assumptions. Additionally, their reliance on kernel smoothing introduces complexities in selecting the tuning parameter, particularly given the suboptimal performance of global cross-validation selectors for smoothness and bandwidth selection [25].

In this article, we present and enhance the highly adaptive lasso (HAL) based plug-in estimator for the marginally adjusted causal dose–response, with a focus on bridging the gap between theory and empirical application. HAL-MLE is a nonparametric minimum-loss estimator capable of efficiently estimating pathwise differentiable parameters. Importantly, it has been theoretically proven to be pointwise asymptotically normally distributed in settings where the parameter is non-pathwise differentiable [26], such as with continuous treatments. Specifically, the HAL-based plug-in estimator converges to the true curve at a dimension-free rate up to log n factors [26]. This work aims to implement the HAL-based estimator and validate its theoretical properties through empirical analysis, by specializing and operationalizing the generic results for causal dose–response estimation.

Through extensive simulation studies, we study cross-validation, undersmoothing, and smoothness-adaptive HAL, demonstrating improved finite-sample performance and faster approach to nominal coverage compared with commonly used alternatives. The remainder of the paper defines the target parameter, presents the estimator and delta-method inference, and reports simulation results across a range of dose–response complexities.

2. Method

2.1. Target Parameter and Statistical Model

Consider an observational study or a randomized experiment with $n$ independent and identically distributed observations of a continuous exposure or treatment variable $A$, an outcome $Y$, and covariates $W$, represented as $O=(W,A,Y)$. $O$ follows a true data-generating distribution (DGD) $P_0\in {ℳ}^{NP}$, where ${ℳ}^{NP}$ is a nonparametric model that places minimal assumptions on the data-generating process, thus allowing for a more flexible and realistic representation of complex relationships among variables. The goal is to estimate the marginal causal dose–response curve, which captures the causal relationship between treatment and outcome. The target parameter ${\Psi }_P\left(a\right)$ is defined as:

$${\Psi }_P\left(a\right)={\mathbb{E}}_P\left[Y\left(a\right)\right],$$

where ${\Psi }_P\left(a\right)$ represents the expected outcome if all subjects were treated with $A=a$. The parameter ${\Psi }_0=\Psi \left(P_0\right)$ represents a causal estimand that needs to be distinguished from the analogue statistical estimand that can be learned from the data defined below. It represents a feature of the full-data distribution of $(W,(Y\left(a\right):a\left)\right)$, while the observed data is linked to this full data through $O=(W,A,Y(A\left)\right)$.

Identification of ${\Psi }_P\left(a\right)$ from the observed data distribution relies on the standard causal assumptions: (i) Consistency, $Y=Y\left(A\right)$ almost surely; (ii) Conditional exchangeability (no unmeasured confounding), $Y\left(a\right)⫫A\mid W$ for all $a$; and (iii) Positivity, $P(A=a\mid W=w)>0$ for all $a$ in the support of $A$ and all $w$ in the support of $W$. Under these assumptions, the counterfactual mean ${\mathbb{E}}_P\left[Y\right(a\left)\right]$ is identified by the formula

$${\Psi }_P\left(a\right)={\mathbb{E}}_P\left[Y\left(a\right)\right]={\mathbb{E}}_P\left\{{\mathbb{E}}_P\left[Y∣W,A=a\right]\right\}.$$

When $Y$ is continuous, we define the conditional mean function $Q_P(W,A)≔{\mathbb{E}}_P[Y\mid W,A]$. With this notation, the target parameter can be written as ${\Psi }_P\left(a\right)={\mathbb{E}}_P\left[Q_P(W,a)\right]={\int }_{w}Q(w,a)dP\left(w\right)$. This leads to the plug-in estimator $\hat{\Psi }\left(a\right)=\frac{1}{n}\sum _{i=1}^n\hat{Q}\left(W_i,a\right)$, where $\hat{Q}$ is a nonparametric estimator of the conditional mean, such as the Highly Adaptive Lasso (HAL). When $Y$ is binary, we parameterize $E_P(Y\mid W,A)={\left(1+\text{exp}\left(-Q_P(W,A)\right)\right)}^{-1}$ so that $Q_P$ now plays the role of $\text{Logit}{E}_P(Y\mid W,A)$ as the conditional probability. This is a reparameterization that enforces the (0, 1) range of a Bernoulli mean; $Q_P$ itself is still estimated nonparametrically by HAL over a rich, infinite-dimensional function class.

Estimating this target parameter is challenging, since there is no efficient estimation method with reliable inference. Therefore, we adopt a flexible and realistic statistical model based on càdlàg functions with finite sectional variation norm. This approach allows for robust estimation without making unrealistic assumptions.

2.2. Flexible and Realistic Statistical Model: Càdlàg Functions

To model the relationship between $A,Y$, and $W$ without relying on strong parametric assumptions, we use the class of multivariate càdlàg (right-continuous with left limits) functions with finite sectional variation norms (FSVN) as the foundation of the statistical model. This class includes not only continuous functions but also functions with jumps and other non-linear features, providing high flexibility for approximating complex relationships in real-world data while avoiding strict smoothness assumptions.

For a $d$-variate càdlàg function $Q:[\mathrm{0,\; 1}{]}^d\to \mathbb{R}$, the function $Q\left(x\right)$ can be represented as:

$$Q\left(x\right)=Q\left(0\right)+\sum _{S\subset \left\{1,\dots ,d\right\}}\underset{\left(0\left(S\right),x\left(S\right)\right]}{\int }{Q}_s\left(du\right),$$

where for each subset $S\subset \{1,\dots ,d\},x\left(S\right)$ denotes the vector of coordinates in $S$, and $Q_S=Q(\cdot ,0(-S\left)\right)$ is the $S$-specific section of $Q$ that sets the coordinates in $S^c$ equal to zero. We will write $Q:[\mathrm{0,\; 1}{]}^d\to \mathbb{R}$ without loss of generality. In practice, continuous covariates are rescaled to [0, 1], and categorical variables are represented by indicator variables, which HAL naturally accommodates. The measure $Q_S\left(du\right)$ represents the measure on the lower-dimensional cube $\left(0\right(S),1(S\left)\right]$ generated by the càdlàg function $Q_S$. The sectional variation norm, which equals the sum over all sections of the variation norm of each section $Q_S$, is defined as:

$$\Vert Q{\Vert }_v^{*}=|Q(0)|+\sum _{S\subset \{1,\dots ,d\}}\underset{(0\left(S\right),1\left(S\right)]}{\int }\left|d{Q}_S\left(u\right)\right|.$$

Càdlàg functions with a universal bound on the sectional variation norm form a Donsker class, ensuring that empirical processes indexed by this class converge to Gaussian processes. Specifically, we have bounds on the covering number of this class presented in [27], which implies precise understanding of the so called entropy integral for these empirical processes, which can then be utilized to establish rates of convergence as in the above article. This fact forms an important ingredient in the proofs of optimal convergence rates for zero and higher order HAL; asymptotic normality of HAL plug-in estimators of pathwise differentiable features; and pointwise asymptotic normality of HAL itself [27]. These properties make càdlàg functions an attractive choice for constructing estimators in complex, high-dimensional data settings. In particular, Highly Adaptive Lasso (HAL) leverages this flexibility to estimate the target function $Q$ robustly, while achieving sub-optimal convergence rates under weak assumptions.

2.3. Highly Adaptive Lasso (HAL): From Statistical Model to Practical Estimator

The Highly Adaptive Lasso (HAL) is a nonparametric minimum-loss estimator over the class of multivariate càdlàg functions with bounded sectional variation norm [14]. In the present setting, HAL is used to estimate the conditional outcome regression

$$Q_P\left(W,A\right)={\mathbb{E}}_P\left(Y∣W,A\right),$$

or, for binary outcomes, its logit-scale representation. The HAL estimator minimizes an empirical loss subject to an $L_1$-type constraint on the coefficients of a rich spline basis expansion. This constraint corresponds to a bound on the sectional variation norm and controls the complexity of the fitted function.

In practice, HAL represents $Q$ as a finite linear combination of spline basis functions constructed from knot-points in the observed data. For the zero-order HAL, these basis functions are indicator functions indexed by subsets of coordinates and observed support points. Writing $X=(W,A)$, a zero-order HAL approximation takes the form

$$Q_{\beta }\left(x\right)={\beta }_0+\sum _{S\subset \left\{1,\dots ,d\right\}}\sum _{j=1}^n{\beta }_{S,j}{\varphi }_{u_{S,j}}\left(x\right),{\varphi }_{u_{S,j}}\left(x\right)=I\left\{u_{S,j}\le x\left(S\right)\right\},$$

with coefficient constraint

$$\left|{\beta }_0\right|+\sum _{S\subset \{1,\dots ,d\}}\sum _{j=1}^n\left|{\beta }_{S,j}\right|<M.$$

Here, $u_{S,j}$ denotes a knot point obtained from the observed data restricted to the coordinate subset $S$. Thus, HAL estimates $Q$ by solving

$$Q_{n,M}=\text{arg}\underset{Q\in D_M^{\left(0\right)}\left({ℛ}_N\right)}{\text{min}}{P}_{n}L\left(Q\right),$$

where $L\left(Q\right)$ is the chosen loss function and $D_M^{\left(0\right)}\left({ℛ}_N\right)$ denotes the finite-dimensional zero-order HAL working model induced by the observed knot-points and the $L_1$ constraint.

For continuous outcomes, we use squared-error loss,

$$L(Q)(O)=(Y-Q(W,A){)}^2.$$

For binary outcomes, we use the Bernoulli negative log-likelihood,

$$L(Q)(O)=-\left\{Y\text{log}{p}_Q\left(W,A\right)+\left(1-Y\right)\text{log}\left(1-p_Q(W,A)\right)\right\},p_Q(W,A)=\{1+\text{exp}[-Q(W,A)]{\}}^{-1}.$$

HAL can also be implemented with higher-order spline basis functions. The first-order HAL uses basis functions of the form

$${\varphi }_{u_S}^1\left(x\right)=\prod _{\ell \in S}I\left(x_{\ell }\ge u_S\left(\ell \right)\right)\left\{x_{\ell }-u_S\left(\ell \right)\right\},$$

which provides smoother approximations than zero-order splines while retaining the same general $L_1$-constrained minimum-loss framework. More generally, $k$-th order HAL estimators use $k$-th order spline basis functions and are appropriate when the target regression function belongs to a corresponding higher-order smoothness class $D^{\left(k\right)}\left([\mathrm{0,\; 1}{]}^d\right)$ [26].

In this paper, we use HAL as the outcome regression estimator in the plug-in dose–response estimator. Details on the càdlàg representation, sectional variation norm, zero-order and higher-order HAL basis constructions, and the theoretical approximation properties are provided in Appendix C.

2.4. Optimizing HAL Fit of Dose–response Curve

2.4.1. Sectional Variation Norm Bound (Penalty) Selector

The HAL estimator minimizes the empirical loss while selecting an appropriate bound for the sectional variation norm. The choice of the $L_1$-norm bound, $M$, is crucial for balancing bias and variance in the estimation. While cross-validation consistently selects an $M$ that is optimal for estimating the regression function $Q$ in loss-based norms, it often leads to oversmoothing when the estimand of interest is a lower-dimensional functional such as the marginal causal dose–response curve. To mitigate this bias, we employ an undersmoothing strategy that increases model complexity beyond the cross-validated choice [15].

Operationally, we implement undersmoothing using a global score-based criterion designed to ensure that the HAL estimator approximately solves the score equations required for asymptotic linearity of the plug-in estimator. Starting from the cross-validated penalty ${\lambda }^{\star }$, we consider a decreasing sequence of penalties (equivalently, increasing $L_1$-norm bounds $M$) and refit a lasso along this grid on the fixed HAL design matrix returned by the initial HAL fit. For each candidate penalty $\lambda$, we compute the maximal standardized empirical score across the active HAL basis functions,

$$\underset{j\in 𝒥}{\text{max}}\left|\frac{1}{n}\sum _{i=1}^n{r}_i(\lambda ){\varphi }_j\left(X_i\right)/{\hat{\sigma }}_j\right|,$$

where $𝒥$ denotes the active basis functions in the initial fit at ${\lambda }^{\star },r_i\left(\lambda \right)$ denotes the residual under penalty $\lambda ,{\varphi }_j$ is a basis function, and ${\hat{\sigma }}_j$ is the empirical standard deviation of $r_i\left({\lambda }^{\star }\right){\varphi }_j\left(X_i\right)$.

We select the smallest amount of undersmoothing, i.e., the largest penalty $\lambda \le {\lambda }^{\star }$, for which this maximal standardized score is bounded by $1/\left(\sqrt{n}\text{log}n\right)$. Under established conditions [15, 26], this threshold is sufficient to make the bias of the plug-in estimator asymptotically negligible relative to its sampling variability, while avoiding excessive overfitting that could inflate higher-order remainder terms. If no candidate penalty satisfies this criterion, we revert to the cross-validated penalty. A detailed description of this undersmoothing procedure, including the construction of the penalty grid and the computation of standardized scores, is provided in Appendix C (Algorithm 2).

2.4.2. Smoothness Order Selector

In practical scenarios, the exact shape of the dose–response curve is often unknown, posing challenges in determining the optimal level of smoothness and other crucial hyperparameters for the Highly Adaptive Lasso (HAL) fit, which depend on the curve’s characteristics. To address these challenges, the Smoothness-Adaptive HAL, introduced by van der Laan [26], leverages the discrete Super Learner [28] to identify the appropriate smoothness order and other relevant hyperparameters, such as the number of knots, within a cross-validation framework. This approach offers a robust data-driven method for selecting the hyperparameter that yields the best empirical performance in modeling the causal dose–response curve.

Pseudo-code for the undersmoothing criterion and the Super Learner smoothness-order selection is provided in Appendix D.

2.4.3. Dose–response Curve Estimation with HAL

Let $\hat{Q}$ denote the final HAL fit obtained using the smoothness-order selection in Section 2.4.2 and the (optional) undersmoothing strategy in Section 2.4.1. We estimate ${\Psi }_P\left(a\right)$ with the plug-in estimator

$$\hat{\Psi }\left(a\right)=\frac{1}{n}\sum _{i=1}^n\hat{Q}\left(W_i,a\right).$$

2.5. Inference: Pointwise Asymptotic Normality of Delta-Method-Based Estimators and Confidence Intervals

The HAL estimator has been shown to be pointwise asymptotically normally distributed [26]. This property allows us to conduct inference using the delta method, providing a mechanism for constructing confidence intervals (CIs) for the estimated dose–response curve. The variance of $Q_{{\beta }_n}\left(x\right)$, where $Q_{{\beta }_n}$ is the HAL-based estimator, is estimated by the influence curve of the coefficients ${\beta }_n$.

A relaxed HAL that refits the selected working model with a non-penalized MLE solves the score equation:

$$P_n\frac{d}{d\beta }L\left(Q_{\beta }\right)=0,$$

where ${\beta }_n=\text{arg}{\text{min}}_{\beta }{P}_{n}L\left(Q_{\beta }\right)$. In practice, HAL is computed via penalized empirical risk minimization,

$${\hat{\beta }}_{\lambda }=\text{arg}\underset{\beta }{\text{min}}\left\{P_{n}L\left(Q_{\beta }\right)+\lambda \Vert \beta {\Vert }_1\right\},$$

which is equivalent, via the standard Lagrange duality between $\lambda$ and $M$, to empirical risk minimization over the constrained class

$${\hat{\beta }}_M=\text{arg}\underset{\beta :\Vert \beta {\Vert }_1\le M}{\text{min}}{P}_{n}L\left(Q_{\beta }\right).$$

That is, the penalized and constrained formulations define the same estimator. The asymptotic results in [26] are stated for the constrained empirical risk minimizer, and therefore apply directly to the HAL estimator implemented via penalization.

For the purpose of variance estimation for the plug-in HAL estimator of the causal dose–response curve, we treat the HAL fit as a non-penalized maximum likelihood estimator within the selected working model and apply standard delta-method–based inference. Although an unconstrained MLE would generally exhibit larger variance than its constrained counterpart, this comparison ignores approximation bias induced by the working model. As a result, the resulting variance estimates are not conservative. Moreover, in practice, the HAL estimator approximately satisfies the score equations, further justifying the use of MLE-based influence-function calculations for inference. We consider the continuous outcome case with squared error loss. Using the delta method, the plug-in estimator $\hat{\Psi }\left(a\right)$ is asymptotically linear, with the influence curve:

$$I{C}_{\Psi \left(a\right),P}\left(O_i\right)={\mathbb{E}}_W\left[\Phi (a,W{)}^{T}I{C}_{\beta ,P}\left(O_i\right)\right],$$

where $\Phi (a,W)$ is the basis vector for treatment $A=a$ and covariates $W$, and $I{C}_{\beta ,P}\left(O_i\right)={\mathbb{E}}_P{\left[{\Phi }^T\Phi \right]}^{-1}{\Phi }_i^T\left(Y_i-\right.\left.{\Phi }_i^T\beta \right)$ is the influence curve of the MLE of the coefficient vector. The variance of the estimator ${\hat{\Psi }}_n\left(a\right)$ is estimated by computing the empirical variance of the estimate of this influence curve:

$${\hat{\sigma }}_n^2\left(a\right)=\frac{1}{n}\widehat{\text{Var}}\left(I{C}_{\Psi \left(a\right),P}\left(O\right)\right).$$

where $\widehat{\text{Var}}$ denotes the sample variance computed from the estimated influence curve evaluated on the observed data. Because ${\hat{\sigma }}_n^2\left(a\right)$ estimates the variance of ${\hat{\Psi }}_n\left(a\right)$, it can be directly used to construct confidence intervals.

The corresponding 95% confidence interval for the dose–response curve is then given by:

$${\hat{\Psi }}_n\left(a\right)\pm z_{1-\frac{\alpha }{2}}{\hat{\sigma }}_n\left(a\right),$$

where $z_{1-\frac{\alpha }{2}}$ is the $1-\frac{\alpha }{2}$ quantile of the standard normal distribution $(\alpha =0.05)$.

These delta-method confidence intervals are asymptotically reliable according to the theoretical results in [26]. While they do not account for potential bias, they provide inference for the projection $Q_{{\beta }_n}$. However, with appropriate undersmoothing, this bias becomes asymptotically negligible relative to the standard error. The inverse of $E_P\left({𝚽}^{\top }𝚽(W,A)\right)$ can be unstable in finite samples, so that we generally recommend to add a small number to the diagonal of this matrix to control such instability.

2.6. Simulation settings

To assess the effectiveness of HAL-based estimations in determining the marginal dose–response curve $\mathbb{E}Y\left(a\right)$, Monte Carlo simulations were conducted for each of four different DGDs, each with qualitatively different marginal dose–response curves, $\Psi \left(a\right)\left(P_0\right)$. These curves varied in complexity, continuity, and the presence of discontinuities.

Figure 1 illustrates the true marginal dose–response curve $\Psi \left(P_0\right)\left(a\right)$ for each simulation. In Simulation 1, the curve is smooth and monotonically increasing, albeit non-linear. Simulation 2 presents a curve with several oscillations. Simulation 3 combines characteristics from simulations 1 and 2: the first half of the curve is smooth, resembling that of simulation 1, while the segment where $a$ lies between 2 and 5 exhibits oscillations. In simulation 4, the curve is characterized by two discontinuities at a = 2 and a = 4, with non-zero values exclusively within this interval.

Figure 1. True causal dose–response curves across four simulations.

The ground-truth dose–response relationships exhibit increasing complexity, ranging from monotonic trends (Simulation 1) to highly non-linear and discontinuous patterns (Simulations 2–4).

These four simulations were executed under a common setting. For each simulation, $n$ independent and identically distributed (i.i.d.) copies of the data $O=(W,A,Y)$ were simulated. Here, $W$ represents a baseline covariate, $A$ denotes the treatment level, which depends on $W$ and varies continuously between 0 and 5, and $Y$ is the outcome of interest, denoting the occurrence of an event.

In each simulation, we generated 500 datasets with sample sizes of 200, 500, 1000, and 5000, following the data-generating processes outlined in Table 3 in Appendix A. These datasets were fitted by six variations of Highly Adaptive Lasso (HAL), obtained by crossing two modeling dimensions: the smoothness order, including zero-order, first-order, or adaptive (selected via Super Learner), and penalty regime, including cross-validated (CV) or undersmoothed. This yields the following six estimators: (1) zero-order smoothness with cross-validation (CV) selector, (2) zero-order smoothness with undersmoothing, (3) first-order smoothness with CV selector, (4) first-order smoothness with undersmoothing, (5) adaptive smoothness with CV selector, and (6) adaptive smoothness with undersmoothing. The zero-order models were configured using the default number of knots from the hal9001 R package [29], catering to scenarios with indeterminate smoothness orders and knot configurations. In contrast, first-order models were trained with a smaller number of knots, suited to simulated scenarios with lower dimensionality and complexity. The adaptive models dynamically adjusted to the data’s complexity, providing increased flexibility in the fitting process. For each simulation setting, empirical standard deviations were calculated across the 500 repetitions, with corresponding (so-called oracle) confidence intervals derived, to evaluate coverage under correct estimation of the sampling variability and thereby assess how well the sampling distribution of the estimator is approximated by a normal distribution centered at the truth.

These variants are evaluated based on five metrics from both perspectives of delta-method-based inferences and Monte Carlo empirical performances: coverage rates of 95% confidence intervals, bias, SE, bias/SE ratio, and mean squared error (MSE). To estimate coverage, we report both that based on the estimated delta-method-based standard error and on the empirical standard error, calculated as the standard deviation of the estimator over the repeated simulations. We did this to separate biases in estimating the standard error from non-normal finite sampling distributions. When oracle Monte Carlo intervals attain close-to-nominal coverage but delta-method intervals undercover, this points to variance-estimation/calibration error rather than an intrinsic impossibility of achieving nominal coverage at that sample size. Conversely, when oracle coverage is also low, residual finite-sample bias and/or non-normality remains a limiting factor, particularly near discontinuities. To evaluate the selected penalties by CV and global undersmoothing criteria, comparisons were made against a grid of penalties. Furthermore, we compared the performance of globally undersmoothed smoothness–adaptive HAL against established models, including the generalized additive model (GAM), polynomial regression model, and the nonparametric causal inference (npcausal) model by (author?) [24].

3. Results

In this section, we present our key findings across four perspectives: (1) Cross-Validation (CV) and the implications of Undersmoothing, (2) Evaluation of Undersmoothing Criteria, (3) Analysis of the Smoothness-Order Adaptive HAL, and (4) Comparative Assessment with other methods. To substantiate our findings, we reference one or two simulations as examples. Detailed results shown in comprehensive plots, including Q–Q plots assessing finite-sample normality, for all simulations and sample sizes are provided in online supporting materials.

3.1. CV and Undersmoothing

Table 1 summarizes the average pointwise performance of estimators at two points on the dose–response curve, using first-order Highly Adaptive Lasso (HAL) with $\lambda$ selected by both Cross-Validation (CV) and Undersmoothing criteria, for a sample size of 1000. In simpler scenarios, such as simulation 1, differences between the two selection methods were minimal. The delta-method-based confidence intervals achieved average coverage rates of 94.2% and 95.7% for CV and undersmoothing selectors, respectively. Both methods show an average bias of approximately 0.02. Figure 2 illustrates that the CV selector slightly outperforms the undersmoothing selector, though the difference is negligible.

Table 1. Average pointwise performance of the HAL-based plug-in estimator at $A=1.0$ and $A=4.2$ across four simulation settings (n = 1000).

The table compares the performance of the HAL estimator under two selection strategies: cross-validation (CV) and undersmoothing (U). For each simulation scenario and evaluation point, we report absolute bias, mean squared error (MSE), confidence interval (CI) coverage, bias-to-standard error ratio (Bias/SE), and standard error, using both delta-method-based and Monte Carlo (empirical) inference. In general, undersmoothing improves CI coverage and yields smaller Bias/SE ratios, particularly in more complex simulation settings (Simulations 3 and 4), at the cost of a modest increase in standard error. Empirical results serve as a benchmark for evaluating the accuracy of delta-method-based inference. SE denotes the delta-method standard error; SD denotes the Monte Carlo empirical standard deviation.

Simu	A	Selector	|Bias|	MSE	Delta Method			Empirical
					CI cov.	Bias/SE	SE	CI cov.	Bias/SD	SD
1	1.0	CV	0.019	0.001	0.940	0.835	0.024	0.954	0.851	0.022
		U	0.021	0.001	0.974	0.689	0.031	0.954	0.795	0.026
	4.2	CV	0.020	0.001	0.944	0.830	0.025	0.954	0.785	0.025
		U	0.026	0.001	0.946	0.813	0.033	0.946	0.770	0.034
2	1.0	CV	0.027	0.001	0.834	1.095	0.027	0.912	0.924	0.029
		U	0.027	0.001	0.878	0.964	0.032	0.943	0.797	0.034
	4.2	CV	0.027	0.001	0.530	2.065	0.014	0.888	1.057	0.025
		U	0.022	0.001	0.669	1.739	0.015	0.931	0.880	0.025
3	1.0	CV	0.021	0.001	0.958	0.807	0.028	0.944	0.803	0.026
		U	0.025	0.001	0.964	0.719	0.037	0.946	0.781	0.032
	4.2	CV	0.062	0.005	0.614	1.817	0.037	0.768	1.323	0.047
		U	0.055	0.004	0.728	1.476	0.041	0.862	1.094	0.050
4	1.0	CV	0.025	0.001	0.960	0.769	0.033	0.938	0.826	0.030
		U	0.028	0.001	0.958	0.692	0.044	0.948	0.789	0.035
	4.2	CV	0.112	0.017	0.482	2.134	0.052	0.636	1.643	0.068
		U	0.071	0.009	0.758	1.313	0.052	0.870	0.957	0.075

Figure 2. Performance comparison in Simulation 1 and Simulation 3 (n = 1000).

The blue lines represent estimated dose–response curves and corresponding 95% confidence intervals (CIs) using cross-validated HAL (CV-HAL), while the orange lines represent those using undersmoothed HAL. Solid lines denote the Monte Carlo (empirical) CIs derived from 500 Monte Carlo repetitions, and dashed lines indicate delta-method-based CIs. For the simpler curve in Simulation 1, both CV-HAL and undersmoothed HAL perform similarly. In contrast, for the more complex curve in Simulation 3, undersmoothing improves performance by yielding higher CI coverage rates and lower mean squared error (MSE). Additionally, delta-method-based CIs closely approximate the empirical CIs, demonstrating their accuracy.

In more complex settings, such as simulation 3, the undersmoothing selector significantly outperformed the CV selector, especially at points of increased complexity along the curve. The CV selector yielded an empirical coverage rate of 83.1% with an average bias of 0.048, while the Undersmoothing selector achieved a 90.4% empirical coverage rate with a lower bias of 0.044. As shown in Figure 2 and Table 1, the undersmoothed HAL estimators reduced bias and improved confidence interval coverage at points of increased complexity.

Furthermore, delta-method-based variance estimates closely approximated the Monte Carlo empirical variance across most treatment values, with the largest discrepancies concentrated at highly non-smooth regions of the curve (e.g., near discontinuities or high curvature), as demonstrated in Figure 2. The dashed lines represent the pointwise performance of delta-method-based estimators and their inference, while the solid lines denote the Monte Carlo (empirical) performance, highlighting the consistency between the two.

3.2. Evaluation of Undersmoothing Criteria

Figure 3 illustrates the estimator’s performance across a range of penalty candidates, delineated at five points along the dose–response curve in simulation 3, with a sample size of 1000. Each subplot corresponds to a different performance metric and treatment value $a$, showing how performance changes as the penalty values are adjusted. The orange vertical dashed lines represent penalties selected by the undersmoothing criteria, while the blue vertical dashed lines represent penalties selected by cross-validation (CV).

Figure 3. Pointwise performance across a grid of penalty values at five example locations in Simulation 3 (n = 1000).

Each column corresponds to a target point along the dose–response curve, with increasing complexity from left to right. The rows display: (1) estimated values and 95% CIs; (2) CI coverage rates; (3) log bias-to-standard error ratio; (4) absolute bias; (5) standard error; and (6) mean squared error (MSE). Results aggregated over 500 Monte Carlo repetitions are shown in green (empirical) and pink (delta-method-based). The dashed blue vertical lines indicate the penalty selected by cross-validation (CV), and the dashed orange lines indicate the penalty selected by the undersmoothing criterion. The undersmoothing approach consistently selects smaller penalties and often strikes a better balance across bias, variance, and MSE, particularly in regions with higher curvature.

As penalty values decrease, we observe an increase in standard errors (SE) and coverage rates of 95% confidence intervals (CIs). At certain points along the curve, such as $a=0.4$ and $a=1.4$, there is a monotonic increase in both bias and mean squared error (MSE) as penalties exceed the CV-selected value. In other cases, bias and MSE decrease initially, reaching a minimum before increasing again as the penalty further decreases.

Across the various treatment points ($a$ values), the undersmoothing selector achieves balanced performance, neither excessively penalizing nor underfitting. This balance is evidenced by the convergence of bias, standard error, and MSE toward optimal values at the selected penalties.

3.3. Analysis of the Smoothness-Order Adaptive HAL

In simulation 1, across all sample sizes, the first-order smoothness estimators consistently outperformed those with other smoothness orders, as shown in Appendix Figure 6 and 7. The adaptability of the method is evident, with the Smoothness-Order Adaptive HAL selecting first-order smoothness in 70.6% of the 500 Monte Carlo iterations for a sample size of 200, and in 98.4% of the iterations for a sample size of 5000.

In simulation 3, interesting dynamics emerge. For smaller sample sizes (200, 500, and 1000), zero-order HAL slightly outperformed first-order HAL on the dose–response curve. However, the average CV risks were comparable or marginally higher for zero-order HAL relative to first-order HAL. As the sample size increased, first-order smoothness estimators consistently outperformed all other smoothness orders across the evaluation metrics. Furthermore, the smoothness level selected by the discrete Super Learner adapted to the sample size of the training data. For smaller sample sizes of 200, 500, and 1000, the method based upon adaptive selection of spline order chose first-order smoothness 56%, 62.1%, and 67.4% of the time, respectively. For the largest sample size of 5000, the HAL demonstrated a definitive preference, selecting first-order smoothness in all iterations (100%). Such adaptive choices of smoothness order consistently align with the smoothness order that performed best for the available data.

3.4. Comparison with Other Methods

Figure 4 presents the empirical pointwise performance of four estimators in estimating the dose–response curve across four different simulations with a sample size of 5000. The estimators include the undersmoothed Highly Adaptive Lasso-based plug-in estimator (HAL), the Generalized Additive Model-based plug-in estimator (GAM), the polynomial model-based estimator (Poly), and the nonparametric causal inference estimator (npcausal). In addition, Table 2 summarizes the average performance of the four estimators based on the empirical SD and delta method, reporting absolute bias, mean squared error (MSE), confidence interval (CI) coverage, bias-to-SE ratio, and SE across all simulations.

Figure 4. Empirical performance of four estimators across four simulation settings (n = 5000).

Each column corresponds to a simulation scenario with increasing complexity from left to right. The rows display: (1) point estimates and 95% confidence intervals (CIs), (2) CI coverage rates, (3) log bias-to-standard error (SE) ratio, and (4) mean squared error (MSE). The four estimators include HAL (green), GAM (orange), polynomial regression (pink), and nonparametric causal regression (npcausal, purple). In Simulation 1, all methods approximate the true curve well, with HAL and npcausal slightly outperforming the others. In Simulations 2–4, polynomial regression fails to capture the complex curve shapes, while HAL consistently demonstrates superior performance across all metrics, including accuracy, CI coverage, and MSE.

Table 2. Average performance of four estimators across four simulation settings (n = 5000).

This table summarizes the average bias, mean squared error (MSE), confidence interval (CI) coverage, bias-to-standard error ratio (Bias/SE), and standard error (SE) for four methods: Highly Adaptive Lasso (HAL), nonparametric causal inference (npcausal), generalized additive models (GAM), and polynomial regression (Poly). Results are reported for both empirical and delta-method-based inference procedures. HAL consistently achieves the best overall performance, with low bias, high CI coverage, and balanced Bias/SE across all simulation scenarios. While npcausal performs competitively in some settings, both GAM and polynomial regression exhibit deteriorating performance as the complexity of the data-generating process increases, particularly in Simulations 3 and 4. Polynomial regression fails to maintain accurate CI coverage and has the highest Bias/SE ratios and MSEs under complex scenarios.

Simu	Method	|Bias|	MSE	Empirical			Delta Method
				CI cov.	Bias/SD	SD	CI cov.	Bias/SE	SE
1	HAL	0.012	0.000	0.946	0.792	0.015	0.969	0.710	0.017
	npcausal	0.014	0.001	0.927	0.841	0.017	0.868	1.065	0.014
	GAM	0.019	0.001	0.653	1.621	0.013	0.704	1.524	0.015
	Poly	0.021	0.001	0.498	2.153	0.010	0.593	1.848	0.012
2	HAL	0.010	0.000	0.930	0.860	0.012	0.889	0.958	0.012
	npcausal	0.018	0.001	0.850	1.112	0.018	0.706	1.499	0.013
	GAM	0.009	0.000	0.907	0.919	0.010	0.876	0.998	0.009
	Poly	0.100	0.015	0.070	12.769	0.008	0.065	14.020	0.007
3	HAL	0.019	0.001	0.909	0.916	0.020	0.903	0.898	0.022
	npcausal	0.034	0.003	0.764	1.260	0.028	0.614	1.900	0.020
	GAM	0.031	0.002	0.658	1.745	0.017	0.672	1.712	0.017
	Poly	0.073	0.010	0.214	7.528	0.009	0.253	6.068	0.011
4	HAL	0.042	0.008	0.843	1.522	0.024	0.840	1.483	0.027
	npcausal	0.048	0.008	0.708	2.490	0.021	0.662	2.923	0.017
	GAM	0.087	0.014	0.281	3.807	0.022	0.338	3.243	0.025
	Poly	0.158	0.042	0.072	11.135	0.015	0.096	11.458	0.015

The first row of Figure 4 shows the estimated dose–response curves and corresponding empirical confidence intervals (CIs). HAL, GAM, and npcausal estimators closely follow the trend of the true dose–response curves across all simulations, whereas the polynomial estimator demonstrates noticeable misalignment. The second row of Figure 4 shows the CI coverage rates for each estimator. In Simulation 1, the HAL estimator achieved an average empirical CI coverage rate of 94.6% and a delta-method-based CI coverage rate of 96.9%. In contrast, the other methods were more anti-conservative: npcausal had an average empirical CI coverage rate of 92.7% and a delta-method CI coverage rate of 86.8%, with GAM and polynomial regression performing even worse in terms of CI coverage.

Performance varied between simulations with differing complexities. In the more complex Simulation 3, the HAL estimator maintained strong performance with average empirical and delta-method CI coverage rates of 90.9% and 90.3%, respectively. The other methods, however, showed more pronounced anti-conservatism in this setting.

Across all simulations, the HAL estimator consistently demonstrated lower average absolute bias, with values of 0.012, 0.010, 0.019, and 0.042. While the npcausal estimator exhibited slightly higher absolute bias, it still outperformed the GAM and polynomial estimators. For both MSE and bias-to-SE ratio, HAL estimators had superior results, consistently achieving lower variance and bias ratios, as detailed in Table 2.

4. Application to the Right Heart Catheterization Study

We illustrate the proposed method using observational data from the Right Heart Catheterization (RHC) study [30]. We estimate the causal effect of continuous age on (i) continuous length of hospital stay and (ii) binary in-hospital mortality, adjusting for demographic and clinical covariates, including sex and severity measures. Age is treated as a continuous exposure, and the undersmoothed HAL plug-in estimator described in Section 2 is employed with Gaussian and Bernoulli loss functions for the two outcomes, respectively.

Figure 5 displays the estimated causal dose–response curves based on the full cohort. Figure 5(a) shows the effect of age on expected length of stay. The curve rises gradually from early adulthood through midlife, plateaus with mild oscillation between approximately ages 40 and 70, and then declines sharply at older ages, indicating a strongly nonlinear relationship.

Figure 5.

Estimated causal dose–response curves for age in the Right Heart Catheterization study. (a) shows the estimated effect of age on expected length of hospital stay (days), and (b) shows the estimated effect of age on the probability of in-hospital mortality. Solid lines denote point estimates from the undersmoothed HAL plug-in estimator; shaded bands represent pointwise 95% confidence intervals. Both outcomes exhibit nonlinear and non-monotone relationships with age, with increased uncertainty at the extremes of the age range.

Figure 5(b) presents the corresponding curve for in-hospital mortality. The relationship is again non-monotone: mortality risk increases through midlife, exhibits a sharp local rise around the early 60s, and remains elevated at older ages with a slight decline at the upper extreme. Such structure would be obscured by parametric or monotone models.

In both subfigures, shaded regions denote pointwise 95% confidence intervals. Uncertainty is lowest in the central age range and widens toward the extremes, reflecting reduced effective sample size. These results demonstrate that the HAL-based estimator can recover flexible causal dose–response relationships for both continuous and binary outcomes, together with principled measures of uncertainty.

5. Discussions

In this paper, we explored the performance of different methods for causal dose–response estimation through a series of simulations, with a novel emphasis on the Highly Adaptive Lasso (HAL) plug-in estimator. Our contribution lies in implementing HAL with inference for dose–response estimation and systematically evaluating its empirical robustness and adaptability across a range of complex scenarios, areas which have not been thoroughly addressed in previous work. We investigated the reliability of the HAL estimator and examined estimation criteria, including penalty selection and smoothness order choices.

Our results highlight the importance of the choice between cross-validation (CV) and undersmoothing when selecting HAL’s penalty parameter. In simple scenarios, such as in simulation 1, where the dose–response curve is smooth, CV and undersmoothing approaches exhibit comparable performances. However, as curve complexity increases and with a limited sample size, such as in simulation 3, the undersmoothing approach stands out by achieving higher empirical coverage rates and lower biases. This finding indicates that undersmoothing is particularly beneficial for accurately capturing complex dose–response relationships, striking an optimal balance between bias and variance. Importantly, undersmoothing enables HAL to adjust across various treatment levels on the curve effectively. Additionally, smoothness-adaptive HAL selects the optimal smoothness orders given the dose–response curve’s complexity and available sample size. With sufficiently large sample sizes, first-order HAL demonstrates the flexibility to handle both simple and complex (non-smooth) curve structures.

When comparing the four estimators—HAL, Generalized Additive Models (GAM), polynomial regression, and the nonparametric causal inference approach (npcausal)—our results show that the HAL-based plug-in estimator consistently delivers superior performance. It achieves the highest CI coverage rates, lowest absolute biases, and smallest mean squared error (MSE) values, especially in scenarios with more complex dose–response curves. In contrast, other estimators, particularly the polynomial model, show considerable variability and fail to maintain the same level of performance. These patterns suggest that HAL is particularly well suited for settings with complex, non-smooth dose–response relationships.

The estimator’s ability to provide unbiased estimates of complex dose–response relationships and deliver confidence intervals with coverage closer to nominal than competing approaches, particularly as sample size increases, further enhances its robustness and practical applicability. However, in the most complex settings (Simulations 2–4), undercoverage remains at moderate sample sizes across all methods, consistent with slow approach to the asymptotic regime for this non-pathwise differentiable problem. Nevertheless, its adaptability to increasing complexity and varying sample sizes, facilitated by smoothness-order adjustments, highlights its potential for broad application in diverse practical contexts.

While HAL shows considerable promise in modeling dose–response curves, there are computational challenges. Specifically, its implementation demands substantial computing resources, both in terms of time and memory, as it incorporates a comprehensive set of basis functions to flexibly fit a wide range of true curves without assuming specific parametric forms. Moreover, solving the inverse of a matrix with many basis functions, especially those selected with the undersmoothing criteria, may lead to invertibility issues. Ongoing research aims to address these concerns by improving HAL’s scalability and computational efficiency.

Our approach presents HAL’s capability of providing statistical inference for non-pathwise differentiable parameters, as demonstrated for the conditional mean and the conditional average treatment effect [31, 32]. A brief overview of this capability is also provided in [33]. Moreover, advances in the adaptive targeted maximum likelihood framework may further improve robustness for nonpathwise differentiable functions [34]. These developments, coupled with the adoption of high-dimensional parametric working models revealed by an initial HAL fit, suggest substantial future improvements in estimation and inference for complex causal parameters.

6. Conclusion

In this paper, we presented the HAL-based plug-in estimator for continuous causal dose–response curves, highlighting both its theoretical underpinnings and empirical performance. By leveraging the flexibility of càdlàg functions and the adaptivity of HAL, the estimator is theoretically guaranteed to achieve pointwise asymptotic normality under weak conditions, even for parameters that are non-pathwise differentiable. Consistent with this theory, our simulations show strong progress toward nominal coverage in many regions, with the most challenging behavior concentrated near highly non-smooth features of the dose–response curve. Through extensive simulations, we demonstrated its robustness across diverse settings, consistently yielding improved coverage, reduced bias, and superior accuracy relative to established alternatives such as GAM, polynomial regression, and npcausal estimators. Importantly, undersmoothing and smoothness-adaptive selection further enhance performance, enabling the method to adapt to increasing complexity with growing sample sizes.

Together, these findings position the HAL plug-in estimator as a practical, theoretically grounded, and empirically validated approach for dose–response estimation in continuous treatment settings, bridging the gap between asymptotic theory and applied causal inference.

The halDRC R package, which includes both real-data application examples and simulated sample demos, is available at https://github.com/SeraphinaShi/halDRC. The source code implementing our methods can be found at https://github.com/SeraphinaShi/HAL-DoseResponseCurve. These resources support reproducibility and are intended to facilitate further research in this area.

Appendix

A. Data Generating Distributions (DGDs) in 4 Simulations

Table 3.

List of Underlying Data Generating Distributions (DGDs)

Simulation	DGD
1	Exogenous variables: $U_W~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=1^2\right)$ $U_A~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=2^2\right)$ $U_Y~\mathit{\text{Uniform}}(min=0,max=1)$ Structural equations F and endogenous variables: $W=U_W$ $A=\mathit{\text{bound}}\left(2-0.5W+U_A\right.,min=0,max=5)$ $Y=\mathbf{I}\left[U_Y<\mathit{\text{expit}}\left(-3+0.5W+1.25A-0.5WA\right)\right]$
2	Exogenous variables: $U_W~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=1^2\right)$ $U_A~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2={1.3}^2\right)$ $U_Y~\mathit{\text{Uniform}}(min=0,max=1)$ Structural equations F and endogenous variables: $W=U_W$ $A=\mathit{\text{bound}}\left(2.5-0.5W+U_A,\mathit{\text{min}}=0,\mathit{\text{max}}=5\right)$ $Y=\mathbf{I}\left[U_Y<\mathit{\text{expit}}\left(-5+3W+5\mathit{\text{sin}}\left(1.25A^{1.5}\right)+3WA\right)\right]$
3	Exogenous variables: $U_W~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=1^2\right)$ $U_A~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=2^2\right)$ $U_Y~\mathit{\text{Uniform}}(\mathit{\text{min}}=0,\mathit{\text{max}}=1)$ Structural equations F and endogenous variables: $W=U_W$ $A=\mathit{\text{bound}}\left(2-0.5W+U_A\right.,min=0,max\left.=5\right)$ $Y=\mathbf{I}\left[U_Y<\mathit{\text{expit}}\left(-4-2W+1.5A+\mathbf{I}\left(A>2\right)1.5\mathit{\text{sin}}\left((0.8A{)}^2-2.56\right)\right)\right]$
4	Exogenous variables: $U_W~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=1^2\right)$ $U_A~\mathit{\text{Normal}}\left(\mu =0,{\sigma }^2=1^2\right)$ $U_Y~\mathit{\text{Uniform}}(\mathit{\text{min}}=0,\mathit{\text{max}}=1)$ Structural equations and endogenous variables: $W=U_W$ $A=\mathit{\text{bound}}\left(2.5-0.5W+U_A,\mathit{\text{min}}=0,\mathit{\text{max}}=5\right)$ $Y=\mathbf{I}\left[U_Y<\mathit{\text{expit}}(-2+W+A\mathbf{I}(A\ge 2)-A\mathbf{I}(A\ge 4)-0.5WA)\right]$

B. Compare performances of estimators based on HAL with different smoothness order splines

Figure 6.

Empirical performance of estimators based on HAL trained with different smoothness orders across simulations 1 and 3 with sample sizes of 200 and 5,000

Figure 7.

Average CV risks of estimators based on HAL trained with different smoothness orders across 4 simulations with all tested sample sizes

C. Highly Adaptive Lasso (HAL) and Representation of $Q$: From Statistical Model to Practical Estimator

Building on the flexibility of càdlàg functions, the Highly Adaptive Lasso (HAL) is a nonparametric minimum-loss estimator that can flexibly estimate complex, high-dimensional target parameters. In its most nonparametric form, HAL only assumes that the true target function belongs to the class of $d$-variate real-valued càdlàg functions with bounded sectional variation norms. This allows HAL to fit a wide range of functions, including those with discontinuities and nonlinear relationships, making it a powerful tool for data-driven estimation without imposing unrealistic parametric assumptions [14].

At the heart of HAL is the minimization of an empirical loss function, subject to a constraint on the function’s sectional variation norm, which effectively controls the complexity of the model. This combination allows HAL to achieve near-optimal convergence rates for the regression function Q in loss-based norms. Specifically, if the true $Q$ belongs to a $k$-th order smoothness class $D^{\left(k\right)}\left([\mathrm{0,\; 1}{]}^d\right)$, the $k$-th order HAL-MLE converges at rate $n^{-k^{*}/\left(2k^{*}+1\right)}$ up to logarithmic factors, where $k^{*}=k+1$ [26]. These rates depend on the complexity and smoothness of the true function and are generally slower than $n^{-1/2}$. Importantly, however, [26] shows that, under appropriate undersmoothing, plug-in estimators of fixed functionals such as the causal dose–response curve $\Psi \left(a\right)=\mathbb{E}\left[Q\right(W,a\left)\right]$ are nevertheless pointwise asymptotically normal, which forms the basis for the inference in Section 2.5.

C.1. Representation of $\mathit{Q}$ in HAL: Zero-Order Approximation

For a $d$-dimensional càdlàg function $Q:[\mathrm{0,\; 1}{]}^d\to \mathbb{R}$, HAL approximates $Q$ using a finite linear combination of indicator (zero-order spline) basis functions. This corresponds to replacing the increments $d{Q}_S$ in the above representation of $Q$ by a pointmass (negative or positive). Such a discrete approximation, which essentially corresponds with approximating a cumulative distribution function (i.e., $Q_S$) by a discrete cumulative distribution function with point-masses on a set of knot-points, can be represented as:

$$Q_M\left(x\right)=Q\left(0\right)+\sum _{S\subset \left\{1,\dots ,d\right\}}\sum _{j}I\left(s_j\left(S\right)\le x\left(S\right)\right)d{Q}_{M,S,j},$$

where $I\left(s_j\left(S\right)\le x\left(S\right)\right)$ is an indicator function that captures whether the observation $x$ falls within a certain region of the domain defined by support points $s_j$, and $d{Q}_{M,S,j}$ are the coefficients. The sectional variation norm of such a discrete approximation $Q$ reduces to:

$${‖Q_M‖}_v^{*}=\left|Q\left(0\right)\right|+\sum _{S\subset \left\{1,\dots ,d\right\}}\sum _{j=1}^n\left|d{Q}_{M,S,j}\right|,$$

where $M>0$ denotes the bound on the sectional variation norm, and the index $j=1,\dots ,n$ ranges over support (knot) points drawn from the observed data in each subspace indexed by $S$. This ${‖Q_M‖}_v^{*}$ measures the total variation of the function across all sections, controlling the complexity and smoothness of the estimator.

Let us denote the set of all such functions $Q_M$ by $D_M^{\left(0\right)}\left({ℛ}_N\right)$, where this represents all linear combinations of the basis functions $x\to I\left(s_j\left(S\right)\le x\left(S\right)\right)$ across $j$ and subsets $S$ under the constraint that the $L_1$-norm is bounded by $M$. ${ℛ}_N$ represents the set of these basis functions. Note that $D_M^{\left(0\right)}\left({ℛ}_N\right)\subset D^{\left(0\right)}\left([\mathrm{0,\; 1}{]}^d\right)$ and with the chosen approximation it has been shown that it approximates the full space at rate $1/n^{1/2}$ making it a fine enough approximation of $D^{\left(0\right)}\left([\mathrm{0,\; 1}{]}^d\right)$ so that this HAL approximates the full MLE over $D_M^{\left(0\right)}\left([\mathrm{0,\; 1}{]}^d\right)$ up till an asymptotically negligible approximation error. HAL solves for the estimator by minimizing the empirical loss over all linear functions in $D_M^{\left(0\right)}\left({ℛ}_N\right)$:

$$Q_{n,M}=\text{arg}\underset{Q\in D_M^{\left(0\right)}\left({ℛ}_N\right)}{\text{min}}{P}_{n}L\left(Q\right).$$

where $P_{n}L\left(Q\right)$ is the empirical loss. This formulation ensures that the estimated function adheres to the constraint $\Vert Q{\Vert }_v^{*}<M$, preventing overfitting. We denote the resulting estimator by $\hat{Q}\equiv Q_{{\hat{\beta }}_M}$, where ${\hat{\beta }}_M$ is the coefficient vector in the basis expansion defined in Section 2.3.

For continuous outcomes, we use the squared-error loss $L\left(Q\right)\left(O\right)=(Y-Q(W,A){)}^2$. For binary outcomes, we use the Bernoulli negative log-likelihood, $L\left(Q\right)\left(O\right)=-\left\{Y\text{log}{p}_Q(W,A)+\left(1-Y\right)\text{log}(1-\right.\left.\left.p_Q(W,A)\right)\right\},p_Q(W,A)=\{1+\text{exp}(-Q(W,A)){\}}^{-1}$. In both cases, HAL computes a nonparametric minimum-loss estimator over the constrained function class.

C.2. Approximation in Practice: Basis Expansion with Knot-Points

In practice, HAL approximates $Q$ by constructing a large set of zero-order spline basis functions using knot-points from the observed data. Specifically, knot-points are the observed covariate values and coincide with the support points used to index the spline basis functions. Given the input vector $X=(W,A)$, the basis functions are the zero-order splines with knot-points in $\left\{X_i\left(S\right):1,\dots ,n,S\subset \{1,\dots ,d\}\right\}$. The resulting approximation is expressed as follows:

$$\begin{gathered} Q_{\beta }(x)={\beta }_0+\sum _{s_1\subset \{\mathrm{1,\; 2},\dots ,d\}}\sum _{j=1}^n{\beta }_{s_1,j}{\varphi }_{u_{s_1,j}}(x)={𝚽}^0(x{)}^{\top }\mathit{\beta }, \\ \text{with}\Vert \beta {\Vert }_1=\left|{\beta }_0\right|+\sum _{s_1\subset \{\mathrm{1,\; 2},\dots ,d\}}\sum _{j=1}^n\left|{\beta }_{s_1,j}\right|<M. \end{gathered}$$

Here $S\subset \{1,\dots ,d\}$ indexes coordinate subsets, $s_1$ denotes a particular subset, and $u_{s_1,j}$ denotes a support (knot) point in the set $\left\{X_i\left(s_1\right):i=1,\dots ,n\right\}$. The vector ${𝚽}_i=𝚽\left(X_i\right)$ collects all basis functions evaluated at the $i$th observation.

This approximation is a linear combination of zero-order spline basis functions $x\to {\varphi }_{u_{s_1,j}}\left(x\right)\equiv I\left(u_{s_1,j}\le x\left(s_1\right)\right)$, defined by support points $x_{u_{s_1,j}}$ from the observed data, along with their corresponding coefficients ${\beta }_{s_1,j}$.

C.3. Higher-Order HAL: First-Order and Higher

HAL can be extended to estimate smoother functions by incorporating higher-order splines. The first-order HAL assumes that the true function $Q$ belongs to the class $D^{\left(1\right)}\left([\mathrm{0,\; 1}{]}^d\right)$, which consists of functions whose sections are absolutely continuous with respect to the Lebesgue measure. The derivative of the $S$-specific section $Q_S$ with respect to $x\left(S\right)$, denoted as $Q_S^{\left(1\right)}\equiv d{Q}_S/dx\left(S\right)$, is assumed to be itself a $\left|S\right|$-dimensional càdlàg function with bounded sectional variation so that the above representation for cadlag functions can be plugged in, resulting in a new representation. This new representation can be rewritten as an infinite linear combination of first order spline basis functions resulting in the first order spline representation of any function in $D^{\left(1\right)}\left([\mathrm{0,\; 1}{]}^d\right)$.

The first-order HAL first approximates $Q$ using a finite set of these first-order spline basis functions, which allow smoother approximations of the target function relative to linear combinations of zero-order splines as in the zero-order HAL. The precise first order HAL representation is presented in [26], and involves a parametric part of first order splines with knot-point at 0 (i.e., if the dimension $d=2$, this would be $x_1,x_2,x_1{x}_2$) and infinite linear combination of first order splines with interior knot-points. In this applied article, we focus on a simplified version of that representation given by:

$$Q_{\beta }(x)={\beta }_0+\sum _{S\subset \{1,\dots ,d\}}\sum _j{\beta }_{S,j}{\varphi }_{s_j(S)}^1(x(S)),$$

where ${\varphi }_{s_j\left(S\right)}^1\left(x\right(S\left)\right)$ represents tensor products of the univariate first-order spline functions:

$${\varphi }_{s_j\left(S\right)}^1\left(x\left(S\right)\right)=\prod _{l\in S}I\left(x_l\ge s_j\left(S\right)\left(l\right)\right)\left(x_l-s_j\left(S\right)\left(l\right)\right),$$

with ${\beta }_0$ and ${\beta }_{S,j}$ being the model coefficients. The $L_1$-norm of the coefficients corresponds to the first-order sectional variation norm $\Vert Q{\Vert }_{v,1}^{*}$. This norm now controls the variation norm of the first order derivatives. This first order HAL now assumes smoothness of the function but still allows great flexibility.

As discussed in detail in van der Laan (2023) [26] the above simplified representation assumes that derivatives of the sections $Q_S$ equal zero when evaluated at a point with some zero components. Due to this assumption, the representation simplifies to the above since $d{Q}_S/dx\left(S\right)$ is now approximated by our zero-order representation that only includes the integral over the full $x\left(S\right)$ while all its sections are not contributing.

Extending beyond the first order, HAL can be generalized to accommodate $k$-th order smoothness classes $D^{\left(k\right)}\left([\mathrm{0,\; 1}{]}^d\right)$, where $k=\mathrm{1,\; 2},\dots$. Functions in $D^{\left(k\right)}\left([\mathrm{0,\; 1}{]}^d\right)$ can be approximated by a finite linear combination of $k$-th order splines. The $L_1$-norm corresponds with the sum over all $k$-th-order derivatives of its sectional variation norms, referred to as the $k$-th order sectional variation norm. See works by (author?) [26] for detailed theoretical development of these higher order smoothness classes $D^{\left(k\right)}\left([\mathrm{0,\; 1}{]}^d\right)$; their precise $k$-th order spline representation; the implied finite linear combination of $k$-th order spline basis functions and their approximation error, and theoretical analysis of the resulting $k$-th order HALs.

D. Implementation Pseudocode

Notation.

Let $O_i=\left(Y_i,X_i\right)$ for $i=1,\dots ,n$, where $X$ includes treatment/exposure $A$ and other covariates $W$. Let ${\widehat{Q}}_{\lambda }\left(x\right)$ denote the HAL regression fit at penalty $\lambda$ and let ${\lambda }^{\star }$ be the CV-selected penalty from the initial HAL fit object fit_init. Let $\Phi \left(X\right)$ denote the HAL basis/design matrix returned by hal9001 (stored as x_basis). Let $𝒜=\left\{a_1,\dots ,a_M\right\}$ be evaluation points for the curve $\psi \left(a\right)=\mathbb{E}\left\{Y\right(A=a\left)\right\}\approx \mathbb{E}\left\{\widehat{Q}\right(a,W\left)\right\}$.

Algorithm 1:

SL3 Smoothness-Order Selector for HAL (CV selector metalearner)

Input: dataset obs; covariate names x_names; outcome name y_name; outcome family y_type; covariate matrix X (used only to set max_degree);

Output: trained SL3 object dSL_fit and candidate metadata (smoothness orders, knot settings).

Create SL3 task: $$\mathtt{\text{task}}\leftarrow \mathtt{\text{make\_sl}}\mathtt{3}\_\mathtt{\text{Task}}\left(\mathtt{\text{data}}=\mathtt{\text{obs}},\mathtt{\text{outcome}}=\mathtt{\text{y\_name}},\mathtt{\text{covariates}}=\mathtt{\text{x\_names}}\right).$$ Construct HAL learner library. For each smoothness order $s\in \left\{\mathrm{0,\; 1,\; 2,\; 3}\right\}$, define a “default knots” HAL learner: $$L_{s,\mathtt{\text{def}}}\leftarrow \mathtt{\text{Lrnr\_hal9001}}(\mathtt{\text{smoothness\_orders=s}},\mathtt{\text{family=y\_type}},\mathtt{\text{return\_x\_basis=TRUE}}).$$ Define a “smaller knots” version for each $s\in \left\{\mathrm{0,\; 1,\; 2,\; 3}\right\}$ by setting $$\mathtt{\text{max\_degree}}\leftarrow \mathbf{1}\{p\ge 20\}\cdot 2+\mathbf{1}\{p<20\}\cdot 3,p=\mathtt{\text{ncol}}\left(\mathtt{\text{X}}\right),$$ and generating num_knots via hal9001:::num_knots_generator with base_num_knots_0=20 and base_num_knots_1=20: $$L_{s,20}\leftarrow \mathtt{\text{Lrnr\_hal9001}}(\mathtt{\text{smoothness\_orders}}=\mathtt{\text{s}},\mathtt{\text{family}}=\mathtt{\text{y\_type}},\mathtt{\text{return\_x\_basis}}=\mathtt{\text{TRUE}},\mathtt{\text{num\_knots}}=\dots )$$ Stack the 8 learners in the fixed order: $$\left\{L_{0,\text{def}},L_{\mathrm{0,20}},L_{1,\text{def}},L_{\mathrm{1,\; 20}},L_{2,\text{def}},L_{\mathrm{2,20}},L_{3,\text{def}},L_{\mathrm{3,20}}\right\}.$$ Train SL with CV selector metalearner. Define a CV selector metalearner using binomial log-likelihood loss: $$\mathtt{\text{cv\_selector}}\leftarrow \mathtt{\text{Lrnr\_cv\_selector}}\left(\mathtt{\text{eval\_function}}=\mathtt{\text{loss\_loglik\_binomial}}\right).$$ Fit Super Learner as a selector over the stacked library: $$\mathtt{\text{dSL}}\leftarrow \mathtt{\text{Lrnr\_sl}}\left(\mathtt{\text{learners}}=\mathtt{\text{stack}},\mathtt{\text{metalearner}}=\mathtt{\text{cv\_selector}}\right),\mathtt{\text{dSL\_fit}}\leftarrow \mathtt{\text{dSL\$train}}\left(\mathtt{\text{task}}\right).$$ Return: $$\left(\mathtt{\text{dSL\_fit,smoothness\_orders}}=\left(0,0,1,1,2,2,3,3\right),\mathtt{\text{num\_knots}}=\left(\mathtt{\text{def,}}20,\mathtt{\text{def,}}20,\mathtt{\text{def,}}20,\mathtt{\text{def,}}20\right)\right).$$

Algorithm 2:

Global-criterion undersmoothing for HAL via glmnet refit on HAL basis

Input: covariates X; outcome Y; initial HAL fit object fit_init (contains lambda_star, coefs, x_basis); number of lambdas Nlam; family.

Output: undersmoothed penalty lambda_under and optional complexity summaries along the grid.

Extract active basis from initial HAL. Let $\widehat{\beta }$ be fit_init$coefs (including intercept). Define the active set $$𝒥\leftarrow \left\{j:{\widehat{\beta }}_j\ne 0,j\ne \text{intercept}\right\}.$$ If $\left|𝒥\right|=0$, set ${\lambda }_{\text{under}}={\lambda }^{\star }$ and stop. Construct refined lambda grid (less penalized than ${\lambda }^{\star }$). $${\lambda }^{\left(k\right)}\leftarrow {\lambda }^{\star }\cdot {10}^{t_k},t_k=\mathtt{\text{seq}}(0,-3,\mathtt{\text{length=Nlam}}),k=1,\dots ,\mathtt{\text{Nlam}}.$$ Refit lasso on the fixed HAL design matrix. Fit glmnet on $\Phi \left(X\right)$ stored as fit_init$x_basis: $$\mathtt{\text{us\_fit}}\leftarrow \mathtt{\text{glmnet}}\left(\mathtt{\text{x\_basis}},Y,\mathtt{\text{lambda}}={\mathtt{\text{lambda}}}^{\left(k\right)},\mathtt{\text{family}}=\mathtt{\text{family}},\mathtt{\text{standardize}}=\mathtt{\text{FALSE}}\right)$$ If all degrees of freedom are zero (no selected coefficients), set ${\lambda }_{\text{under}}={\lambda }^{\star }$ and stop. Compute residuals. Compute initial residuals using the initial HAL predictor: $$r_i^{\star }\leftarrow {\widehat{Q}}_{{\lambda }^{\star }}\left(X_i\right)-Y_i.$$ For each $k$, compute fitted values from us_fit and residuals: $${\widehat{Q}}_{{\lambda }^{\left(k\right)}}\left(X_i\right)\leftarrow \mathtt{\text{predict}}(\mathtt{\text{us\_fit}},\mathtt{\text{x\_basis}})(\text{or type}=\text{response for binomial}),r_i^{\left(k\right)}\leftarrow {\widehat{Q}}_{{\lambda }^{\left(k\right)}}\left(X_i\right)-Y_i.$$ Estimate direction-specific standard deviations from initial fit. Restrict the basis matrix to active columns ${\Phi }_{𝒥}\left(X\right)$. For each $j\in 𝒥$: $${\widehat{\sigma }}_j\leftarrow \text{sd}\left(r_i^{\star }\cdot {\varphi }_j\left(X_i\right)\right).$$ Compute global standardized score for each candidate lambda. For each $k$, standardize basis columns $u_{ij}={\varphi }_j\left(X_i\right)/{\widehat{\sigma }}_j$ and compute $$S_j^{\left(k\right)}\leftarrow \left|\frac{1}{n}\sum _{i=1}^n{r}_i^{\left(k\right)}{u}_{ij}\right|,S_{\text{max}}^{\left(k\right)}\leftarrow \underset{j\in 𝒥}{\text{max}}{S}_j^{\left(k\right)}.$$ Select undersmoothed lambda by the global criterion. Define threshold $${\tau }_n\leftarrow \frac{1}{\sqrt{n}\text{log}\left(n\right)}.$$ Choose the first (largest) lambda along the grid satisfying the criterion: $${\lambda }_{\text{under}}\leftarrow {\lambda }^{\left(k^{†}\right)},k^{†}=\text{min}\left\{k:S_{\text{max}}^{\left(k\right)}\le {\tau }_n\right\}.$$ If no $k$ satisfies, set ${\lambda }_{\text{under}}={\lambda }^{\star }$. (Optional) Summarize model complexity along the grid. For each $k$: $${‖{\widehat{\beta }}^{\left(k\right)}‖}_1=\sum _j\left|{\widehat{\beta }}_j^{\left(k\right)}\right|,\#\left\{j:{\widehat{\beta }}_j^{\left(k\right)}\ne 0\right\}.$$

Algorithm 3:

IC-based standard errors for $\psi \left(a\right)=\mathbb{E}\left\{\widehat{Q}\right(a,W\left)\right\}$

Input: fitted HAL object hal_fit; training data $(X,Y)$; evaluation points $𝒜$; family; optional X_unpenalized.

Output: pointwise standard errors $\widehat{\text{se}}\left(a_m\right)$ for $m=1,\dots ,M$.

Form design matrix and active set. Construct $B=[1,\Phi (X\left)\right]$ where $\Phi \left(X\right)$ is hal_fit$x_basis. Let $\widehat{\beta }=\mathtt{\text{hal\_fit\$coefs}}$ and $ℐ=\left\{j:{\widehat{\beta }}_j\ne 0\right\}$. If $\left|ℐ\right|=0$, return NA. Compute fitted values $\widehat{Y}$ on training data. Use predict(hal_fit, new_data=X) (or type="response" for binomial; include X_unpenalized if provided) to obtain ${\widehat{Y}}_i$. Compute IC for coefficients ${\widehat{\beta }}_{ℐ}$. Let $X_{ℐ}$ be columns of $B$ restricted to $ℐ$ and ${\widehat{\beta }}_{ℐ}$ the corresponding coefficients. Compute score contributions: $s_i=X_{ℐ,i}\left(Y_i-{\widehat{Y}}_i\right)$. Compute empirical Jacobian $J_n=n^{-1}\sum _i{X}_{ℐ,i}{\partial }_{\beta }\varphi {\left({\widehat{\beta }}_{ℐ}^{\top }{X}_{ℐ,i}\right)}^{\top }$ (logistic derivative for binomial; linear for gaussian as in code). Set ${\text{IC}}_{\beta }\left(i\right)=-J_n^{-1}{s}_i$. Delta method to get IC for $\psi \left(a\right)$. For each $a_m\in 𝒜$: Create $X^{\left(a_m\right)}$ by setting the exposure column $A=a_m$ for all $i$. Build basis rows $\Phi \left(X^{\left(a_m\right)}\right)$ using hal9001::make_design_matrix(., hal_fit$basis_list) and form $B^{\left(a_m\right)}=\left[1,\Phi \left(X^{\left(a_m\right)}\right)\right]$ (append X_unpenalized if used). Restrict to active columns: $B_{ℐ}^{\left(a_m\right)}$. Compute ${\text{IC}}_{\psi \left(a_m\right)}\left(i\right)$ by averaging the derivative-weighted coefficient ICs (as implemented in cal_IC_for_EY). Empirical SE. For each $m$: $$\widehat{\text{se}}\left(a_m\right)=\sqrt{\frac{{\text{Var}}_n\left\{{\text{IC}}_{\psi \left(a_m\right)}\right\}}{n}.}$$

Remark.

Algorithm 2 matches the implemented “global criterion” undersmoothing: it refits glmnet [35] on the fixed HAL basis x_basis, computes standardized mean scores using SDs from the initial residuals, and selects the largest $\lambda$ meeting the threshold $1/\left(\sqrt{n}\text{log}n\right)$.