Abstract
Background
Non-compliance with randomization is pervasive in surgical randomized controlled trials (RCTs), introducing unmeasured confounding that undermines valid causal inference. Conventional methods, such as intention-to-treat (ITT), per-protocol (PP), as-treated (AT) and instrumental variable (IV) analysis, exhibit significant limitations.
Methods
We conducted a simulation study to compare the performance of negative control methods (NCM), including difference-in-differences (DiD) and double negative controls (DNC), with traditional approaches (ITT, PP, AT, IV). Parameters from influential lumbar discectomy trials informed realistic crossover scenarios with crossover rates ranging from 10% to 40%. Performance was assessed via bias, mean squared error (MSE), and coverage probability (CP) for the method-specific treatment effect estimator across varying sample sizes (n = 100–500) and confounding structures.
Results
ITT, PP, and AT estimators showed severe bias, with CP ≤ 0.1% across the evaluated scenarios. The IV estimator showed small average bias but high estimation variability (MSE = 25.612 at a crossover rate of 40%). The DiD estimator showed low bias and near-nominal coverage under equal confounding (|bias| ≤ 0.003, CP ≥ 94.3%) but showed poor performance when this assumption was violated. In contrast, the DNC estimator remained stable, with |bias| ≤ 0.415, MSE ≤ 2.515, and near-nominal coverage across the evaluated parameter combinations. At n = 100 with a crossover rate of 40%, DNC reduced MSE by > 90% relative to IV while maintaining CP = 97.7%.
Conclusions
The DNC approach effectively mitigates unmeasured confounding arising from non-compliance with randomization in surgical RCTs, offering superior accuracy and reliability compared to conventional methods.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12874-026-02897-y.
Keywords: Double negative controls, Unmeasured confounding, Instrumental variable analysis, Non-compliance, Randomized controlled trials
Background
Randomized controlled trials (RCTs) are considered as the gold standard for intervention evaluation, yet treatment crossover frequently undermines their validity. In surgical RCTs, non-compliance rates tend to exceed those in pharmacological studies due to procedural complexity, surgeon expertise, and patient-related factors [1]. Recent meta-analyses have reported crossover rates of 7–13% in cardiac surgery RCTs [2], a mean of 12.6% (range 0–45%) in gastrointestinal cancer trials [3], and 8–42% in spine surgery trials (Supplemental Table S1) [4]. In the Spine Patient Outcomes Research Trial (SPORT), 43% of patients crossed over from conservative treatment to surgery and 42% vice versa within 24 months [5]. Similarly, in the anterior cruciate ligament surgery necessity in non-acute patients (ACL SNNAP) trial, 41% of patients allocated to rehabilitation crossed over to surgical reconstruction within 18 months, and 28% assigned to surgery did not receive their allocated intervention [6].
Conventional approaches to non-compliance with randomization exhibit critical limitations [7, 8]. Intention-to-treat (ITT) analysis preserves randomization integrity but attenuates effect estimates by diluting results with non-compliant participants. Per-protocol (PP) and as-treated (AT) analyses introduce bias by violating randomization principles. While methods like regression adjustment, propensity score matching, or inverse probability weighting can mitigate bias from measured confounders, they require complete confounder measurement and correct model specification. Instrumental variable (IV) methods leverage randomization to address unmeasured confounding under strict assumptions (Fig. 1), yet often yield imprecise estimates in surgical settings where these assumptions are frequently violated [9].
Fig. 1.
Directed acyclic graph for the instrumental variable framework under non-compliance with randomization. Z, randomized treatment assignment; D, treatment received; Y, outcome; U unmeasured confounder. IV condition (1): IVs are related to the treatment (relevance). IV condition (2): IVs are related to the outcome only through the treatment (exclusion restriction). IV condition (3): IVs are independent of unmeasured covariates (independence)
Negative control methods (NCM) offer a promising alternative by leveraging a priori null associations to detect and adjust for unmeasured confounding [10, 11]. These approaches utilize two proxy variables: negative control outcomes (NCOs), which are unaffected by treatment, and negative control exposures (NCEs), which lack causal effects on the outcomes of interest (Fig. 2). Such methods are increasingly adopted in epidemiology and observational studies for bias detection [10, 12, 13], with growing recognition of their utility in RCTs [14]. The double negative control (DNC) framework integrates both NCEs and NCOs to enable non-parametric identification of treatment effects [15]. By modeling a confounding bridge function, DNC isolates causal effects from pervasive unmeasured confounders [16]. Notably, difference-in-differences (DiD), a specialized form using baseline outcomes as NCOs, adjusts for time-invariant confounding when the assumption of additive equi-confounding holds [17]. Both approaches are well-suited for surgical RCTs where temporal proxies (e.g., preoperative outcomes as NCOs, delayed exposures as NCEs) are naturally available.
Fig. 2.
Directed acyclic graph for the negative control exposure and negative control outcome. D, the treatment; E the negative control exposure; Y, outcome; W the negative control outcome, U unmeasured confounder
Given their abilities to detect and correct bias with fewer parametric constraints, we conducted a simulation study to compare the performance of negative control methods (including DNC and DiD) with standard approaches (including ITT, AT, PP, IV) under settings of realistic surgical trials.
Methods
The section begins with a brief overview of the SPORT trial, which informed the simulation design. It then describes the simulation framework, which emulates key features of surgical RCTs with treatment crossover. Next, details of the six analytical methods compared are provided: ITT, PP, AT, IV, DiD, and DNC. Finally, the performance metrics used to evaluate bias, precision, and coverage are outlined.
SPORT study
Our simulation framework was inspired by the SPORT trial, a landmark multicenter RCT conducted between 2000 and 2004 across 13 spine clinics in America [8]. The study enrolled 501 surgical candidates with imaging-confirmed lumbar intervertebral disk herniation and persistent radicular symptoms lasting ≥ 6 weeks despite nonoperative care. Participants were randomly assigned to receive either standard open discectomy or individualized nonoperative treatment (including physical therapy, education, medications). Despite its rigorous design, substantial non-compliance occurred: only 57% of participants assigned to surgery underwent surgery within 24 months, while 42% of those assigned to nonoperative treatment crossed over to surgery. This high two-way crossover introduced significant unmeasured confounding, limiting the interpretability of the ITT analysis. Although both groups improved in the primary outcomes, between-group differences in SF-36 bodily pain and physical function scores were small and not statistically significant.
Notation and variable definitions
We index participants by 
. We use capital letters to denote variables and lowercase letters for their realizations. Let 
denote the randomized treatment assignment indicator (0: no surgery, 1: surgery). Let 
denote the treatment actually received (0: no surgery, 1: surgery). Treatment crossover is reflected by 
. Let 
denote the outcome of interest measured after treatment assignment and let 
denote a negative control outcome (NCO) used in methods that leverage auxiliary outcomes. Let 
denote an unmeasured confounder that affects both received treatment D and outcomes.
We adopt the potential outcomes framework. Let 
denote the potential outcomes under received treatment 
for 
. Let 
denote the potential treatment received under assignment 
for 
. Under consistency, the observed data satisfy 
and 
. We define the primary estimand as the average treatment effect (ATE) of treatment received on the outcome,
 |
We also define the complier average causal effect (CACE),
 |
For each method 
, we denote its reported treatment-effect estimate by 
, as described in the corresponding method subsection.
Simulation structure
Motivated by the SPORT trial, we simulated hypothetical RCTs involving patients with lumbar disc herniation, randomly assigned at a 1:1 ratio to either standard surgical treatment (open discectomy) or conservative treatment. Key similarities to and departures from SPORT design are summarized in Supplemental Table S2. Let 
denote the randomization indicator, and 
denote the potential treatment received under assignment 
. As a sensitivity analysis, we generated treatment assignments by randomly permuting a prespecified allocation vector with exact final 1:1 group-size balance to examine whether the main findings were driven by random group-size imbalance. Specifically, for a total sample size of n, treatment assignment was generated by randomly permuting a prespecified allocation vector containing n/2 control assignments and n/2 treatment assignments.
To emulate realistic non-compliance, treatment crossover was governed by an unmeasured confounder 
, such as health-seeking behavior or other patient-related factors. Six scenarios (labeled A–F) were designed to represent distinct patterns of crossover, with actual treatment received (
) determined by whether 
fell above or below certain quantiles, given varying degrees of inconsistency between assigned and received treatments. For example, in some scenarios, patients with extreme values of 
(very high or very low) either always opted for surgery or always avoided surgery, whereas other scenarios involved bidirectional crossover (patients with high 
always taking surgery and those with low 
always avoiding it). The details of each scenario’s crossover mechanism are summarized in Table 1, and the rationale for the corresponding parameter choices is provided in Supplemental Table S3. The overall crossover rate (proportion of participants whose actual treatment 
differed from assignment 
) varied at 10%, 20%, 30%, and 40% to reflect increasing non-compliance severity.
Table 1.
Data-generating parameters under simulation scenarios A–F
| Definition | Description | Data generation mechanism |
|---|---|---|
Randomized grouping ( ) |
1= “surgical treatment”, 0= “conservative treatment” |
|
Unmeasured confounding ( ) |
Health-seeking behavior |
|
| Crossover rate (δ) | Proportion of inconsistency between actual treatment and assigned treatment |
|
Actual treatment ( ) |
Scenario A: patients with would never receive the surgery. |
 |
Scenario B: patients with would always receive the surgery. |
|
|
Scenario C: patients with would always receive the surgery. |
 |
|
Scenario D: patients with would never receive the surgery |
|
|
Scenario E: patients with would never receive the surgery, while patients with would always receive the surgery |
|
|
Scenario F: patients with would always receive the surgery, while patients with would never receive the surgery |
 |
|
Primary outcome ( ) |
SF-36 bodily pain score, with higher scores indicating milder symptoms; the surgery is estimated to improve the score by an average of 5 points |
|
Negative outcome control ( ) |
Baseline SF-36 bodily pain scores unrelated to treatment |
|
| Equal confounding assumption | Under the equal confounding assumption |
|
| Under the unequal confounding assumption |
|
|
 denotes the empirical -quantile of Uin each simulated dataset; similarly, denotes the empirical -quantile. | ||
The primary outcome 
is the 2-year SF-36 bodily pain score (higher values indicate milder symptoms). Assuming surgery provides an average improvement of 5 points, outcomes were generated as 
, where 
and 
varies with the confounding structure. The intercept was set to 37 to anchor the simulated outcome to the mean 2-year outcome observed in the nonoperative group in SPORT. The treatment-effect parameter was set to 5 to represent a moderate and clinically interpretable additional benefit of surgery, lying between the crossover-attenuated 2-year ITT treatment effect reported in SPORT and the 10-point difference used in SPORT’s design-stage sample size calculation.
A negative outcome control 
was introduced, which was defined as the baseline SF-36 bodily pain score and not causally affected by treatment. It is generated as 
. The intercept was set to 27 to anchor the simulated baseline level to the mean bodily pain score observed at baseline in the randomized SPORT cohort.
Two confounding structures were considered:
Equal confounding: The confounder
influences 
and 
equally (
), corresponding to a confounder effect approximately twice the treatment effect;Unequal confounding: The confounder’s impact on
and 
differs (
).
Sample size varied across five levels (n = 100,200,300,400,500), with 2000 simulations conducted for each combination of sample size, crossover rate, and confounding setting. The assumed causal structure is illustrated in Fig. 3.
Fig. 3.
Data-generating structure for the simulation study. Z, randomized treatment assignment; D, treatment received; Y, outcome; W, negative control outcome; U, unmeasured confounder
Supplemental Figure S1 and Figure S2 show clear imbalance in the distribution of 
between the actual treatment groups across scenarios. As crossover rates increased from 10% to 40%, the absolute standardized mean difference (SMD) in 
between the surgery and conservative treatment groups also increased (Figure S2), indicating progressively stronger unmeasured confounding bias as non-compliance worsened.
Analytical Methods
Intention-to-treat (ITT) analysis
The ITT approach estimates the effect of treatment assignment 
on the outcome 
. We fit the ordinary least squares (OLS) model
 |
and define the ITT estimator as 
.
Per-protocol (PP) analysis
The PP analysis is restricted to participants who adhered to their assigned treatment (
). In this adherent subset, we fit the OLS model
 |
and define 
.
As-treated (AT) analysis
The AT approach estimates the association between actual treatment received D and outcome Y, ignoring randomized assignments. We fit the OLS model
 |
and define 
.
Instrumental variables (IV) analysis
The IV method uses assignment 
as an instrument for treatment received 
[18], under the standard IV assumptions of relevance, exclusion restriction, and independence. The estimation follows a two-stage least squares (TSLS) procedure:
Stage 1:
A linear regression model of the treatment variable D on the instrumental variables (
) is fitted to obtain the predicted treatment variable (
).
 |
Stage 2:
The outcome variable (Y) is regressed on the predicted treatment variable 
to estimate the effect.
 |
The coefficient 
obtained from Stage 2 is the IV estimate. We denote this IV estimate as 
. For a binary instrument Z and a binary treatment D, under assumptions A1-A5, CACE is nonparametrically identified as
 |
1 |
The IV estimate 
is a consistent estimate of the CACE in (1).
Under our data-generating model, the treatment effect 
is constant on the mean scale, that is, 
. The CACE corresponds to the average effect in the subgroup of compliers, which is determined by U. Because the treatment effect is homogeneous across all values of U, the CACE equals ATE, implying that ATE = CACE = τ. Consequently, the IV estimate 
also provides a consistent estimate of the ATE.
Difference-in-Differences (DiD)
The DiD approach leverages the NCO as a proxy for unmeasured confounding that equally affects baseline and outcome [17]. The key assumption is that the effect of confounder on the outcome is the same as on the NCO (additive equal confounding).
The relationships were formalized through the following structural equations:
Primary outcome model:
 |
-
(2)
Negative control outcome model:
 |
-
(3)
Confounder-treatment relationship:
 |
The observed associations are derived through marginalization:
 |
 |
Let 
, 
, the target causal estimand is: 
.
When the confounder effects on outcomes are equivalent (
), the DiD estimator identifies the true treatment effect: 
.
Double negative controls (DNC)
The DNC method utilizes an outcome confounding bridge function that characterizes the relationship between the confounding effects on the primary outcome and the NCO [16]. To identify this bridge function and estimate the average treatment effect (ATE), the method incorporates the NCE. Specifically, under Assumptions I–V provided in the Supplemental Material, the following equality holds:
 |
In practice, the bridge function 
is often specified using a parametric linear model, such as:
 |
The parameter 
can then be estimated using the generalized method of moments (GMM), with moment conditions constructed as:
 |
We define the DNC estimator as 
.
Performance measures
Under our data-generating model, the treatment effect 
is constant on the mean scale (i.e. 
). The CACE corresponds to the average effect in the subgroup of compliers, which is defined by U. Because the treatment effect is homogeneous across all values of U, the CACE equals the overall ATE, implying that ATE = CACE = τ. For each method m, the estimation performance of 
was evaluated using bias, empirical standard error (SE), mean squared error (MSE), and coverage probability (CP). Bias was defined as the average deviation of estimated effect from true effect across simulations and SE was empirical standard deviation of estimates. MSE was calculated as the average squared difference between the estimated effect and the true effect. CP was defined as the proportion of 95% confidence intervals (CI) containing the true effect.
The null hypothesis 
was tested using a Wald statistic
 |
Here, SE
denotes the standard error of the estimated effect 
. The null hypothesis was rejected when 
, where 
denotes the 
-quantile of the standard normal distribution. Based on this statistic, we assessed statistical power and type I error. Statistical power is the probability of rejecting the null hypothesis when the true effect is non-zero. Type I error rate is the probability of rejecting the null hypothesis when the true effect is zero.
With 2000 repetitions, the Monte Carlo standard error is approximately 0.0049 for a nominal type I error rate of 0.05 or a nominal coverage probability of 0.95, corresponding to 95% Monte Carlo confidence intervals of (0.040, 0.060) and (0.940, 0.960), respectively. To further assess whether 2000 repetitions were sufficient, we conducted an additional repetition-stability analysis under the most challenging setting of Scenario A (n = 100, crossover rate = 40%), with each repetition count (100, 500, 1000, 1500, 2000, 2500, and 3000) independently repeated 50 times.
All simulations and analyses were performed with R 4.4.3 (R Foundation).
Results
Figures 4, 5, 6, 7 and 8; Table 2 summarize the comparative performance of the six analytical methods (ITT, AT, PP, IV, DiD, DNC) under Scenario A, defined according to the data-generating mechanism described in Table 1.
Fig. 4.
Bias of treatment effect estimates under scenario A. Results are based on 2,000 simulation replicates. Panels show sample sizes of 100, 200, 300, 400, and 500 and crossover rates of 10%, 20%, 30%, and 40%. The horizontal reference line indicates zero bias
Fig. 5.
Mean squared error (MSE) of treatment effect estimates under scenario A. Results are based on 2,000 simulation replicates. Panels show sample sizes of 100, 200, 300, 400, and 500 and crossover rates of 10%, 20%, 30%, and 40%
Fig. 6.
Coverage probability (CP) under scenario A. Results are based on 2,000 simulation replicates. For a nominal coverage probability of 0.95, the Monte Carlo standard error (MCSE) is approximately 0.0049, corresponding to 95% Monte Carlo confidence intervals of (0.940, 0.960). Panels show sample sizes of 100, 200, 300, 400, and 500 and crossover rates of 10%, 20%, 30%, and 40%. The horizontal reference line indicates nominal 95% coverage
Fig. 7.
Type I error rate under scenario A. Results are based on 2,000 simulation replicates under the null treatment effect. For a nominal type I error rate of 0.05, the Monte Carlo standard error (MCSE) is approximately 0.0049, corresponding to 95% Monte Carlo confidence intervals of (0.040, 0.060). Panels show sample sizes of 100, 200, 300, 400, and 500 and crossover rates of 10%, 20%, 30%, and 40%. The horizontal reference line indicates the nominal 0.05 level
Fig. 8.
Statistical power under scenario A. Results are based on 2,000 simulation replicates under a non-null treatment effect. Panels show sample sizes of 100, 200, 300, 400, and 500 and crossover rates of 10%, 20%, 30%, and 40%. The horizontal reference line indicates 80% power
Table 2.
Simulation results for each method under Scenario A (n = 100)
| Method | Crossover rate = 10% | Crossover rate = 20% | Crossover rate = 30% | Crossover rate = 40% | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bias | SE | MSE | CP | Bias | SE | MSE | CP | Bias | SE | MSE | CP | Bias | SE | MSE | CP | |
| ITT | -1.019 | 0.794 | 1.603 | 0.774 | -2.017 | 0.741 | 4.541 | 0.214 | -3.014 | 0.741 | 9.561 | 0.014 | -4.018 | 0.793 | 16.713 | 0.001 |
| AT | -2.491 | 0.873 | 6.680 | 0.148 | -4.263 | 0.873 | 18.531 | 0 | -5.578 | 0.965 | 31.444 | 0 | -6.611 | 1.348 | 44.037 | 0 |
| PP | -1.504 | 0.868 | 2.873 | 0.597 | -2.990 | 0.903 | 9.447 | 0.063 | -4.465 | 1.040 | 20.403 | 0.002 | -5.955 | 1.448 | 35.936 | 0.004 |
| IV | 0.008 | 1.118 | 1.218 | 0.953 | 0.058 | 1.506 | 2.239 | 0.954 | 0.171 | 2.305 | 5.379 | 0.957 | 0.549 | 4.916 | 25.612 | 0.958 |
| DiDa | 0.003 | 0.289 | 0.083 | 0.946 | -0.003 | 0.309 | 0.096 | 0.954 | 0.003 | 0.354 | 0.133 | 0.952 | 0.003 | 0.476 | 0.239 | 0.943 |
| DNCa | 0.097 | 0.386 | 0.190 | 0.941 | 0.154 | 0.514 | 0.352 | 0.936 | 0.222 | 0.829 | 0.760 | 0.958 | 0.415 | 4.598 | 2.515 | 0.981 |
| DiDb | -0.827 | 0.405 | 0.812 | 0.480 | -1.422 | 0.418 | 2.151 | 0.048 | -1.857 | 0.471 | 3.606 | 0.011 | -2.200 | 0.645 | 5.099 | 0.046 |
| DNCb | -0.009 | 0.464 | 0.190 | 0.954 | -0.013 | 0.624 | 0.351 | 0.954 | 0.001 | 1.004 | 0.772 | 0.959 | -0.348 | 4.088 | 1.060 | 0.977 |
Each result is based on 2000 simulation repetitions. For a nominal coverage probability of 0.95, the Monte Carlo standard error (MCSE) is approximately 0.0049, corresponding to 95% Monte Carlo confidence intervals of (0.940, 0.960)
ITT Intention-to-treat, AT as-treated, PP Per-protocol, IV Instrumental variables, DiD Difference-in-differences, DNC Double negative control, SE Standard error, MSE Mean squared error, CP Coverage probability
1 Satisfying the equal confounding assumption
2 Not satisfying the equal confounding assumption
Conventional approaches exhibited increasing bias as the crossover rate increased. At n = 100 (Table 2), the ITT estimator’s bias changed from − 1.019 to − 4.018 as the crossover rate increased from 10% to 40%. Bias was larger for the AT and PP estimators, ranging from − 2.491 to − 6.611 and − 1.504 to − 5.955, respectively. The IV estimator showed smaller bias (0.008–0.549) than AT and PP, with SE increasing 4.4-fold from 1.118 to 4.916 as the crossover rate increased. Under the equal confounding assumption, the DiD estimator had negligible bias (|bias| ≤ 0.003). When this assumption was violated, bias increased, reaching − 2.200 in the scenario with a 40% crossover rate and n = 100. In contrast, the DNC estimator showed bias close to zero across the settings presented in Table 2 (|bias| ≤ 0.415). As sample size increased, the variability of the IV estimator decreased, whereas ITT, AT, and PP exhibited persistent bias (Fig. 4).
Regarding MSE, the ITT, AT, and PP estimators exhibited consistently large values (a maximum of 44.037), consistent with their large bias (Table 2). The IV estimator had smaller bias but showed an increase in MSE at a crossover rate of 40% (MSE = 25.612 at a crossover rate of 40% with n = 100). The DiD estimator maintained low MSE under equal confounding (MSE ≤ 0.239) but showed pronounced deterioration when this assumption was violated (MSE = 5.099 at a crossover rate of 40% with n = 100). The DNC estimator achieved consistently low MSE across settings (MSE ≤ 2.515), including scenarios with high crossover. As sample size increased, MSE decreased for the IV estimator, but it remained higher than that of DNC across sample sizes; ITT, PP, and AT continued to exhibit large MSE with limited improvement (Fig. 5).
Conventional methods (ITT, AT, PP) showed poor CP, which declined toward 0% as the crossover rate increased (Table 2; Fig. 6). The IV estimator maintained CP close to the nominal 95% level; however, confidence intervals became wider as the crossover rate increased. Under the equal confounding assumption, the DiD estimator achieved CP ≥ 94.3%. When this assumption was violated, CP decreased to 4.6% in the scenario with a 40% crossover rate and n = 100. In contrast, the DNC estimator maintained generally near-nominal coverage across all simulated settings.
Type I error and statistical power further distinguished the methods (Figs. 7 and 8). Under the null treatment effect, ITT and IV generally showed type I error closer to the nominal 0.05 level, whereas AT and PP showed inflated type I error across settings. DiD maintained low type I error under equal confounding but type I error increased when the equal confounding assumption was violated. DNC maintained type I error close to the nominal 0.05 level in most settings, although it was slightly conservative when the crossover rate was 40% and the sample size was 100, with type I error around 0.02. For statistical power, ITT, PP, AT, and IV generally showed lower power, with the lowest values observed at a crossover rate of 40% and a sample size of 100. DiD retained high power across settings. DNC also showed power exceeding 80% in most settings, falling below 80% only when the crossover rate was 40% and the sample size was 100.
Sensitivity analyses under randomization with exact final 1:1 group sizes yielded consistent results (Supplemental Figures S3–S7). Across Scenarios B–F, the overall ranking of methods remained broadly similar to that in Scenario A, with DNC generally showing greater robustness than ITT, PP, AT, and IV, and DiD remaining sensitive to violation of the equal-confounding assumption. However, the direction of bias under the non-null setting varied across scenarios. In Scenarios A, C, and F, PP and AT showed estimates that tended to fall below the true positive treatment effect, together with inflated type I error and lower power. In Scenarios B, D, and E, PP and AT showed estimates that tended to exceed the true positive treatment effect, together with inflated type I error and correspondingly higher power. These scenario-specific differences are shown in Supplemental Figures S8–S32.
We also performed an additional repetition-stability analysis under the most challenging setting of Scenario A (n = 100, crossover rate = 40%) to evaluate whether 2000 simulation repetitions were adequate. Across independent reruns, the variability of the estimated bias, MSE, coverage probability, type I error, and statistical power decreased as the number of repetitions increased (Supplemental Figures S33–S37). The distributions were visibly more concentrated at larger repetition counts, with comparatively limited changes beyond 2000 repetitions.
Discussion
This simulation study comprehensively evaluates methods for addressing unmeasured confounding due to non-compliance with randomization in surgical RCTs. Consistent with prior literature, conventional approaches (ITT, AT, PP) yielded severely biased estimates and inadequate CP under the non-compliance settings evaluated in this study. ITT estimates were diluted towards the null due to non-complier outcome averaging, risking false-negative conclusions. Despite this bias, ITT analyses better reflect real-world clinical scenarios and remain widely recommended as the primary analytic approach in randomized trials [19]. Naive PP and AT analyses, ignoring randomization, also produced bias because protocol-deviating patients differed systematically from adherers.
IV methods used to estimate CACE exhibited lower bias than AT and PP in the presence of unmeasured confounding, provided their assumptions (relevance, exclusion restriction, and monotonicity) were met. This aligns with recent simulation results by Papazoglou et al. (2025) [20], showing IV recovers the CACE without bias under standard assumptions, whereas PP estimators are unbiased only under completely random non-compliance. However, our study and others [3, 21] highlight important limitations of IV. First, the local effect (CACE) is restricted to compliers and may not generalize. Second, IV estimates suffer from imprecision and wide CIs in moderate sample sizes or with weak instruments (e.g. low compliance). Inflated SEs and reduced coverage were observed when compliance dropped or sample size was limited, consistent with IV’s large-sample power requirement. Third, IV assumptions may be violated in surgery trials: for example, if randomization (the instrument) has any direct effect on outcomes beyond via treatment received (for example, due to patient or provider behavior changes upon knowing the assignment), or if there exist always-takers/never-takers outside the complier strata, standard IV estimators become biased. While novel estimators relaxing these assumptions exist, they are uncommon in clinical research [22].
An additional finding from our simulations is that type I error and statistical power must be interpreted jointly, because both depend on the direction and magnitude of estimator bias under treatment crossover. In our study, PP and AT frequently showed inflated type I error under the null, indicating that these estimators were often shifted away from 0 even when no true effect was present. However, this did not imply uniformly high power under a non-null effect. Instead, the relationship between type I error and power was scenario-dependent. In Scenarios A, C, and F, the estimates from PP and AT tended to underestimate the true positive treatment effect under the alternative (Table 2; Fig. 4, Figure S13, Figure S28), so although these estimators were shifted away from 0 under the null, they were pulled back toward 0 under the non-null setting, resulting in limited power. In Scenarios B, D, and E, by contrast, the estimates tended to overestimate the true positive treatment effect under the alternative (Figure S8, Figure S18, Figure S23), and the corresponding power was therefore more consistent with the inflated rejection behavior observed under the null.
In contrast to the inconsistent performance of ITT, PP/AT, and IV methods, the NCM demonstrated more robust performance across the evaluated settings. The DiD method performed well under the ideal condition that the equal confounding assumption held. When valid, DiD produced unbiased estimates with improved precision. Its advantage lies in canceling time-invariant confounders influencing baseline and follow-up equally by analyzing within-patient change. Baseline measurements, often available in RCTs, serve as patient-specific controls. However, DiD performed poorly when the equi-confounding assumption was violated. In simulations, modest deviations caused greater DiD bias than IV in some cases. This aligns with theory: DiD is not robust to time-varying confounding or differential confounder changes [17]. Practically, baseline outcomes cannot always capture all confounding relevant for post-randomization compliance, especially in longer trials where new confounders may arise. Thus, while performed well when the equi-confounding assumption held, DiD can be misleading if violated.
Findings reinforce the conclusion that no single conventional method is universally reliable for addressing non-compliance. A 2023 systematic review of simulation studies [2] noted that while IV methods are commonly employed, they often show bias under certain settings, such as in the presence of an increasing hazard or unmeasured confounding, highlighting the need for further studies on “G-methods” (e.g., inverse probability weighting, g-estimation) in realistic trials. Earlier overviews similarly concluded that CACE and IV methods, which adjust for non-adherence, rely on strong and potentially vulnerable assumptions [1, 23]. For example, the exclusion restriction is a particularly strong assumption and is often difficult to verify in practice, and the monotonicity assumption is also strong and may not hold in many real-world settings. As Bang and Davis famously noted, neither ITT nor IV are perfect solutions, particularly in complex surgical trials [21]. The persistent risk of bias/imprecision with ITT, PP, AT, and IV methods highlights the need for new approaches. Zhu et al. [24] proposed augmented inverse probability weighting for partial compliance with ordinal outcomes, showing improved performance relative to ITT, PP, and IV under selection bias. These studies underscore active research on mitigating non-compliance bias. Within this landscape, the present evaluation adds new evidence: to our knowledge, this study provides one of the first simulation-based evaluations showing that the DNC framework can reduce bias relative to ITT, AT, PP, and IV under crossover-induced unmeasured confounding in RCT settings.
The DNC method showed robust performance across the simulated non-compliance setting. Compared with the other approaches, it generally achieved lower bias and better precision, particularly in settings where the competing methods performed poorly. Across settings including crossover rates up to 40%, DNC produced nearly unbiased estimates with correct or near-correct CI coverage. Its performance remained favorable even in challenging settings, such as a sample size of 100 and a crossover rate of 40%, where DNC reduced MSE by more than 90% relative to IV while maintaining coverage of at least 95%. In contrast, PP, AT, and ITT were mainly limited by bias, whereas IV and DiD were mainly limited by imprecision.
DNC capitalizes natural proxies: (1) pre-randomization outcome (baseline) as an NCO, and (2) randomization assignment (assuming no direct outcome effect) as an NCE. By simultaneously modeling the relationships between unmeasured confounding, these controls and the primary outcome, DNC “differences out” confounding more flexibly and nonparametrically than IV or DiD. DNC requires weaker assumptions and remains accurate even when “equal confounding” was violated—a condition causing DiD breakdown. Practically, DNC leverages more information than a single instrument or baseline adjustment, enhancing robustness to complex confounding.
DNC’s resilience aligns with proximal causal inference literature on negative controls [25]. Originally developed for observational studies, these techniques nonparametrically identify causal effects under unobserved confounding given valid negative control pairs. Our results affirm their utility in RCTs with post-randomization deviations. In surgical trials, common unmeasured factors (e.g., patient frailty, health-seeking behavior) influence both adherence likelihood and outcomes. DNC effectively accounts for these proxies: pre-surgery functional scores might act as NCOs reflecting frailty, and random assignment (or a delayed treatment offer) can serve as an NCE. By incorporating these, DNC isolates the causal effect of surgery itself. Unlike IV, which excludes non-compliers, DNC uses data from all patients and adjusts via the confounding bridge function, achieving bias reduction and greater efficiency. DNC also produced stable estimates under low compliance, scenarios where IV estimates become unstable or non-identifiable. Thus, DNC offers more robust adjustment for crossover bias than conventional methods, contingent on identifying suitable negative controls.
Table 3 summarizes the main strengths and limitations of the evaluated methods. Based on our findings and broader evidence, we propose practical recommendations:
Table 3.
Summary of the strengths and limitations of the evaluated methods for treatment non-compliance
| Method | Strengths | Limitations |
|---|---|---|
| ITT |
(1) Preserves randomization (2) Standard primary analysis in RCTs (3) Simple to implement and interpret |
(1) Estimates the effect of assignment rather than treatment received (2) Diluted by crossover (3) Showed large bias and poor coverage when non-compliance was substantial |
| AT |
(1) Directly compares treatment actually received (2) Simple to implement |
(1) Breaks randomization (2) Vulnerable to unmeasured confounding under crossover (3) Showed large bias, poor coverage, and inflated type I error |
| PP |
(1) Focus on protocol-adherent participants (2) Commonly reported in practice |
(1) Introduces selection bias (2) Vulnerable to unmeasured confounding under crossover (3) Showed large bias, poor coverage, and inflated type I error |
| IV |
(1) Using randomization as a natural instrument (2) Lower bias when assumptions hold (3) Targets the complier average causal effect |
(1) Relies on strong assumptions, especially exclusion restrictions and monotonicity (2) Low efficiency in challenging settings (3) Confidence intervals became wider as crossover increased |
| DiD | Performed well when the equal confounding assumption held | Strongly dependent on the equal confounding assumption |
| DNC |
(1) Uses randomization as a negative control exposure (2) Does not require the equal confounding assumption (3) Showed low bias, near-nominal coverage, and good power in most settings |
(1) Requires valid negative control proxies and additional identification assumptions (2) Type I error was slightly conservative under high crossover (3) Power may decrease in small samples with very high crossover |
ITT Intention-to-treat, AT as-treated, PP Per-protocol, IV Instrumental variables, DiD Difference-in-differences, DNC double negative control
Anticipate and Mitigate Crossovers in Trial Design
Recognize during planning that a proportion of patients may not adhere to their assigned treatment. Refine eligibility/protocols to minimize non-compliance (e.g., exclude patients unlikely to accept allocation, enhance surgeon training to reduce conversions, use ethical early switching rules). Adjust sample size calculations to account for expected non-compliance (e.g., inflation based on anticipated compliance rates). However, simply increasing sample size is not a panacea; combining moderate increases with robust analyses (below) is often more cost-effective.
-
(2)
Employ Advanced Methods Beyond Naive ITT/PP
Plan a priori to use methods accounting for non-compliance. Interpret ITT, AT, and PP results with extreme caution when crossover rates are high. At minimum, consider IV analysis to estimate CACE, while explicitly reporting its limitations (wider CIs) and conducting sensitivity analyses to assess the impact of potential violations of the exclusion restriction. Present both ITT and adjusted estimates (e.g., IV or DNC) to provide a comprehensive picture. ICH E9 recommends presenting analyses based on both the full analysis set and the per-protocol set, with any differences explicitly discussed [26]. More recently, ICH E9(R1) emphasizes that non-adherence should be handled through clearly defined estimands and appropriately aligned estimators, rather than naïve per-protocol analyses [27].
-
(3)
Leverage DNC Methods with Thoughtful Data Collection
Our study demonstrates the promise of the DNC methods, which is encouraged for use in future surgical trials facing compliance issues, given its fewer parametric assumptions and ability to handle unmeasured confounders. To implement DNC, prospectively collect high-quality proxies: key baseline outcomes or risk scores (candidate NCOs) and variables influencing treatment uptake that are unrelated to outcomes except via confounding (candidate NCEs, e.g., randomization assignment, scheduled delayed interventions). Proactive planning for these proxies is crucial and requires minimal additional effort. Verify the validity of negative controls, ensuring that the NCO remains unaffected by the treatment and that the NCE has no independent effect on the outcome. Additionally, provide clinical justification for these assumptions. Collaborate with biostatisticians familiar with causal inference for proper application; user-friendly software implementations are emerging.
-
(4)
Plan Sensitivity Analyses and Transparent Reporting
Accompany all non-compliance analyses with sensitivity analyses to assess the impact of residual bias (e.g., vary assumptions about unmeasured confounder effects, test direct effects in IV, assess different negative controls in DNC). Report the extent of non-compliance in CONSORT flow diagrams and discuss implications in line with CONSORT extensions [28]. Explicitly acknowledge potential primary analysis bias and present adjusted analyses alongside them. Transparent reporting and rigorous sensitivity checks enhance the reliability of conclusions.
Limitations
This study has several limitations. First, it focuses on continuous outcomes and specific additive confounding; while the methods are applicable to binary outcomes, their performance in such contexts requires verification. Integrating DNC with survival models is an important extension. Second, the analysis assumed a simple unmeasured confounding structure, whereas real-world treatment compliance may be influenced by multiple unmeasured and time-varying factors. Although DNC performed well in these settings, its performance in more complex settings requires further study. Third, the success of DNC hinges on the availability of valid negative control. Identifying truly valid NCOs and NCEs is challenging and demands substantive clinical knowledge. If controls are invalid (e.g., baseline outcome affected by pre-randomization treatments, NCE has slight outcome effect), DNC estimates could be biased. Investigators must carefully justify the validity of proxies.
Conclusion
Treatment crossover remains a critical threat to causal inference in surgical trials. These findings support the use of DNC to address unmeasured confounding in scenarios where conventional approaches are biased and IV methods perform poorly. In trials at risk of crossover, researchers should consider prospectively collecting high-fidelity negative control proxies (NCOs/NCEs) and report DNC-based sensitivity analyses alongside primary analyses when non-compliance is substantial. Embracing such robust methods will strengthen the evidence base for surgical interventions.
Supplementary Information
Supplementary Material 1. Table S1–S3. Figure S1–S37.
Acknowledgements
Not applicable.
Abbreviations
- ACL SNNAP
Anterior cruciate ligament surgery necessity in non-acute patients
- AT
As-treated
- ATE
Average treatment effect
- CACE
Complier average causal effect
- CI
Confidence interval
- CONSORT
Consolidated standards of reporting trials
- CP
Coverage probability
- DiD
Difference-in-differences
- DNC
Double negative controls
- GMM
Generalized method of moments
- ITT
Intention-to-treat
- IV
Instrumental variable
- MSE
Mean squared error
- NCE
Negative control exposure
- NCO
Negative control outcome
- OLS
Ordinary least squares
- PP
Per-protocol
- RCT
Randomized controlled trials
- SE
Standard error
- SF-36
Medical outcomes study 36-item short form health survey
- SMD
Standardized mean difference
- SPORT
Spine patient outcomes research Trial
- TSLS
Two-stage least squares
Authors’ contributions
Zhenzhen Lu: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Methodology, Formal analysis, Conceptualization. Wenwen Li: Writing – review & editing, Validation, Software, Methodology, Conceptualization. Mengyao Wang: Writing – review & editing, Methodology, Conceptualization. Lihong Huang: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing. Feng Chen: Conceptualization, Methodology, Project administration, Supervision, Validation, Visualization, Writing – review & editing.
Funding
This study was funded by the National Natural Science Foundation of China (grant no. 82273733), and Xuhui District Hospital-Local Collaborative Projects (25XHYD-04). The funding bodies had no role in the study design, data, collection, data analysis, data interpretation, manuscript preparation or review, and the decision to submit the manuscript for publication.
Data availability
No real data was used. The R code and original simulation results of this simulation study are available on reasonable request.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Zhenzhen Lu and Wenwen Li contributed equally to this work.
References
- 1.Adewuyi TE, MacLennan G, Cook JA. Non-compliance with randomised allocation and missing outcome data in randomised controlled trials evaluating surgical interventions: a systematic review. BMC Res Notes. 2015;8:403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gaudino M, Fremes SE, Ruel M, Di Franco A, Di Mauro M, Chikwe J, Frati G, Girardi LN, Taggart DP, Biondi-Zoccai G. Prevalence and Impact of Treatment Crossover in Cardiac Surgery Randomized Trials: A Meta-Epidemiologic Study. J Am Heart Association. 2019;8(21):e013711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Garas G, Markar SR, Malietzis G, Ashrafian H, Hanna GB, Zacharakis E, Jiao LR, Argiris A, Darzi A, Athanasiou T. Induced Bias Due to Crossover Within Randomized Controlled Trials in Surgical Oncology: A Meta-regression Analysis of Minimally Invasive versus Open Surgery for the Treatment of Gastrointestinal Cancer. Ann Surg Oncol. 2018;25(1):221–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chan VKY, Darsaut TE, Bailey CS, Raymond J. Understanding crossovers and potential ways to mitigate the problem: Lessons from influential trials on lumbar microdiscectomy. Neurochirurgie. 2023;69(5):101461. [DOI] [PubMed] [Google Scholar]
- 5.Weinstein JN, Tosteson TD, Lurie JD, Tosteson AN, Hanscom B, Skinner JS, Abdu WA, Hilibrand AS, Boden SD, Deyo RA. Surgical vs nonoperative treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT): a randomized trial. JAMA. 2006;296(20):2441–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Beard DJ, Davies L, Cook JA, Stokes J, Leal J, Fletcher H, Abram S, Chegwin K, Greshon A, Jackson W, et al. Rehabilitation versus surgical reconstruction for non-acute anterior cruciate ligament injury (ACL SNNAP): a pragmatic randomised controlled trial. Lancet (London England). 2022;400(10352):605–15. [DOI] [PubMed] [Google Scholar]
- 7.Mostazir M, Taylor RS, Henley W, Watkins E. An overview of statistical methods for handling nonadherence to intervention protocol in randomized control trials: a methodological review. J Clin Epidemiol. 2019;108:121–31. [DOI] [PubMed] [Google Scholar]
- 8.Abell L, Maher F, Jennings AC, Gray LJ. A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials. BMC Med Res Methodol. 2023;23(1):300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Edouard P, Steffen K, Navarro L, Mansournia MA, Nielsen RO. Methods matter: instrumental variable analysis may be a complementary approach to intention-to-treat analysis and as treated analysis when analysing data from sports injury trials. Br J Sports Med. 2021;55(18):1009–11. [DOI] [PubMed] [Google Scholar]
- 10.Lipsitch M, Tchetgen Tchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiol (Cambridge Mass). 2010;21(3):383–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shi X, Miao W, Tchetgen ET. A Selective Review of Negative Control Methods in Epidemiology. Curr Epidemiol Rep. 2020;7(4):190–202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Arnold BF, Ercumen A, Benjamin-Chung J, Colford JM Jr. Brief Report: Negative Controls to Detect Selection Bias and Measurement Bias in Epidemiologic Studies. Epidemiol (Cambridge Mass). 2016;27(5):637–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Dusetzina SB, Brookhart MA, Maciejewski ML. Control Outcomes and Exposures for Improving Internal Validity of Nonrandomized Studies. Health Serv Res. 2015;50(5):1432–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Arnold BF, Ercumen A. Negative Control Outcomes: A Tool to Detect Bias in Randomized Trials. JAMA. 2016;316(24):2597–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Shi X, Miao W, Nelson JC, Tchetgen EJT. Multiply robust causal inference with double-negative control adjustment for categorical unmeasured confounding. J Royal Stat Soc Ser B Stat Methodol. 2020;82(2):521–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Miao W, Shi X, Li Y, Tchetgen EJT, Wei Z. A confounding bridge approach for double negative control inference on causal effects. 2024.
- 17.Sofer T, Richardson DB, Colicino E, Schwartz J, Tchetgen Tchetgen EJ. On negative outcome control of unobserved confounding as a generalization of difference-in-differences. Stat science: Rev J Inst Math Stat. 2016;31(3):348–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Peugh JL, Strotman D, McGrady M, Rausch J, Kashikar-Zuck. SJJosp. Beyond intent to treat (ITT): A complier average causal effect (CACE) estimation primer. 2017, 60:7–24. [DOI] [PubMed]
- 19.Ahn E, Kang H. Intention-to-treat versus as-treated versus per-protocol approaches to analysis. Korean J anesthesiology. 2023;76(6):531–9. [DOI] [PMC free article] [PubMed]
- 20.Papazoglou T, Waddingham E, Young A. Estimating the Complier Average Causal Effect in Randomised Controlled Trials with Non-Compliance. A Comparative Simulation Study of the Instrumental Variables and Per-Protocol Analyses; 2025. [Google Scholar]
- 21.Bang H, Davis CEJSM. On estimating treatment effects under non-compliance in randomized clinical trials: are intent-to-treat or instrumental variables analyses perfect solutions? 2009, 28(5):1967–8. [DOI] [PubMed]
- 22.Chen G, Lu C, Chan T, Qi Z, Zhang D, Jin Y. Yang MJAaS: A New Estimator for Randomized Controlled Trials with Non-Compliance. Theory and Empirical Evidence; 2024. [Google Scholar]
- 23.Angrist JD, Imbens GW, Rubin DBJJAA. Identification of causal effects using instrumental variables. 1996, 91(434):444–55.
- 24.Zhu J, Li J, Richards AM, Chan MY, Tai B-CJBMRM. Accounting for extent of non-compliance when estimating treatment effects on an ordinal outcome in randomized clinical trials. 2025, 25(1):52. [DOI] [PMC free article] [PubMed]
- 25.Tchetgen EJT, Ying A, Cui Y, Shi X, Miao W. An Introduction to Proximal Causal Learning. 2020.
- 26.International Conference on Harmonisation (ICH). ICH E9: Statistical Principles for Clinical Trials. 1998.
- 27.International Council for Harmonisation (ICH). ICH E9(R1) Addendum: Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials. 2019.
- 28.Hopewell S, Chan AW, Collins GS, Hróbjartsson A, Moher D, Schulz KF, Tunn R, Aggarwal R, Berkwits M, Berlin JA, et al. CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ (Clinical Res ed). 2025;389:e081123. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Material 1. Table S1–S3. Figure S1–S37.
Data Availability Statement
No real data was used. The R code and original simulation results of this simulation study are available on reasonable request.