Adherence-adjustment in placebo-controlled randomized trials: An application to the candesartan in heart failure randomized trial

Abstract

Background:

The per-protocol effect provides important information in randomized trials with incomplete adherence. Yet, because valid estimation typically requires adjustment for prognostic factors that predict adherence, per-protocol effect estimates are often met with skepticism. In placebo-controlled trials, however, the validity of adjustment can be indirectly verified by demonstrating no association between adherence and the outcome among the placebo arm. Here, we describe a two-stage procedure in which we first adjust for time-varying adherence in the placebo arm and then use a similar procedure to estimate the per-protocol effect.

Methods:

We use the Candesartan in Heart Failure: Assessment of Reduction in Mortality and Morbidity (CHARM) randomized trial. First, we compare adherers versus non-adherers in the placebo arm, adjusting for pre- and post-randomization variables. Second, we use models validated in the placebo arm to estimate the per-protocol effect of adherence to candesartan versus placebo in the full trial.

Findings:

We successfully estimated no association between adherence and mortality in the placebo arm; hazard ratio: 0.91 (95% CI: 0.51, 2.52). We then estimated the per-protocol effect under two sets of protocol-defined stopping criteria after adjustment for post-randomization confounders. The mortality hazard ratio estimates ranged from 0.91 to 0.93 for the per-protocol effect estimates, similar to the intention-to-treat effect estimates.

Interpretation:

Adherence adjustment in the CHARM trial is feasible when appropriate assumptions about missing data and confounding are made. These assumptions cannot be verified but can be supported through the use of placebo-arm adherence assessment.

1. Introduction

The per-protocol effect—the effect that would have been estimated in the trial under full adherence to the protocol—is of interest in randomized trials with incomplete adherence to the assigned treatment [1,2]. However, valid estimation of the per-protocol effect generally requires that adjustment for pre- and post-randomization prognostic factors that predict adherence uses a method which can account for feedback between adherence and these prognostic factors [3]. That is, valid per-protocol analyses deviate from the intention-to-treat principle and require unverifiable assumptions similar to those made in observational studies (Table 1). As a result, per-protocol analyses are generally viewed as suspect.

Table 1.

Assumptions required for valid estimation of a per-protocol effect via covariate adjustment in a randomized controlled trial.

Assumption	Explanation	Justification for CHARM
Conditional exchangeability for adherence	Sufficient common causes of adherence and mortality are measured and modeled to control for all confounding. In addition, appropriate analytic methods are used to avoid inducing selection bias through time-varying confounders affected by prior treatment.	Our findings were robust to a range of confounder sets (Supplementary Appendix), and we used inverse probability weighting for time-varying confounder-treatment feedback.
Conditional exchangeability for loss to follow-upa	Sufficient common causes of loss to follow-up and mortality are measured and modeled to control for all bias and analytic appropriate methods are used.	Our findings were robust to a range of loss to follow-up modeling approaches (Supplementary Appendix).
Positivitya	All types of individuals must have the possibility of following each treatment protocol.	Positivity for trial arm at baseline is ensured by randomization. During follow-up, positivity for adherence is achieved by specifying a clinically relevant adherence protocol which allows discontinuation.
Well-defined interventiona	The causal contrast must be clearly specified so that the protocol of interest can be identified in the data and reproduced.	We allowed participants to alter adherence based on specific clinical diagnoses. We assessed robustness to this assumption by allowing any clinically advised discontinuation.
Correct model misspecificationa	The parametric models for the inverse probability of adherence and missingness weights, and for the outcome must all be correctly specified, including appropriate functional forms for continuous covariates and sufficient flexibility in the model for the baseline hazard where applicable.	Our findings were robust to a range of model specification sensitivity analyses (Supplementary Appendix).

^aThese assumption are also required for valid estimation of the intention-to-treat effect. Conditional exchangeability for loss to follow-up is not guaranteed for the intention-to-treat effect whenever there is informative drop-out.

However, in placebo-controlled trials, there is an indirect test of the validity of adherence adjustment: estimating the effect of adherence to treatment in the placebo arm only. When adherence to placebo is not expected to affect the outcome (i.e., mortality), this effect estimate is expected to be null if the adjustment for prognostic factors that predict adherence was successful. Several randomized trials have used this indirect test and have found better outcomes among placebo adherers than among non-adherers, even after adjustment for multiple prognostic factors, which has been widely interpreted as evidence that the available data are insufficient for adherence adjustment [4–10].

The findings of these placebo-controlled trials fueled concerns about the validity of adherence-adjusted estimates, and therefore of per-protocol effect estimates. However, we have demonstrated that the survival difference between placebo adherers and non-adherers could be virtually eliminated in one of these trials—the Coronary Drug Project [10]—by adjusting for time-varying confounders via inverse probability weighting and by relaxing the strong assumption that adherence was zero during all missed visits [11]. It remains an open question whether the survival differences between placebo adherers and non-adherers in the other trials can be similarly eliminated, which would increase our confidence in adherence-adjusted per-protocol estimates from those trials.

Another one of those placebo-controlled trials is the Candesartan in Heart Failure: Assessment of Reduction in Mortality and Morbidity (CHARM) trial [4]. The intention-to-treat mortality hazard ratio for candesartan versus placebo, which was 0.91 (95% CI: 0.83, 1.00) [4,12], measures the benefit of being assigned to candesartan versus placebo in the trial [2,3]. However, the intention-to-treat estimate may underestimate the true benefit of using candesartan as prescribed because 11% of individuals decided to discontinue the drug without documented adverse events, and an additional 7% were less than 80% adherent by the end of follow-up. Yet adherence-adjusted analyses to estimate the per-protocol effect in the CHARM trial were discouraged because better survival was found in placebo adherers than in non-adherers, even after adjustment [4].

Here, we revisit adherence-adjustment in the CHARM trial in two stages. First, we compare the survival between adherers and non-adherers in the placebo arm after adjustment for pre- and post-randomization variables. Second, we apply the same approach to adjust for adherence in the full trial and estimate the per-protocol effect. We start by reviewing the design and analysis of the CHARM trial.

2. CHARM randomized trial

Between March 1999 and March 2001, the CHARM trial [12] recruited patients with symptomatic congestive heart failure, randomized them to candesartan (3803 patients) or placebo (3796 patients) within three subgroups defined by baseline left-ventricular ejection fraction and history of angiotensin-converting-enzyme inhibitor use, and followed them for a median of 38 months. The primary outcome was all-cause mortality at 3.5 years (Fig. 1). Some analyses used a combined outcome of cardiovascular disease-related death and hospitalization.

Fig. 1.

CONSORT diagram.

Patients in the candesartan arm were assigned to a starting dose of 4 or 8 mg candesartan taken once per day and increased, in a stepwise fashion with a minimum of 14 days between increments, to a maximum of 32 mg per day. The starting dose of 4 or 8 mg was determined by baseline health status, including hypovolemia, diuretics, New York Heart Association (NYHA) functional class, blood pressure, creatinine, or frailty. After the 6-week dosage adjustment phase, study visits were scheduled every 4 months for a planned minimum of 2 years (Figs. A1–A2). According to the CHARM protocol, study medication could be decreased or discontinued at any visit if (a) the participant developed hypotension; (b) the participant developed indications of renal dysfunction including an increase of more than ≥0.5 mg/dL in serum creatinine; and (c) at the discretion of the study clinician.

The original computer code was not available. We therefore used the reported estimates and description of the methods to reverse engineer the code using SAS 9.4 (Cary, NC) (Appendix Tables A.1–A.3).

2.1. Intention-to-treat analysis

We were able to exactly replicate the original CHARM intention-to-treat estimates for all-cause mortality (Table 2). 10 individuals who discontinued all study visits and had unknown vital status at the end of follow-up were excluded from the original CHARM analyses and from our analyses. The mortality hazard ratio for candesartan vs. placebo was estimated via a Cox model stratified by randomization subgroup, with and without adjustment for baseline covariates (age, sex, number of comorbidities, New York Heart Association functional class, ejection fraction, heart rate, systolic and diastolic blood pressure, body-mass index, having a pacemaker, current smoker, and number of other medications).

Table 2.

Intention-to-treat effect estimates for all-cause mortality, CHARM randomized trial.

Method	Published Estimate [12], HR (95% CI)	Replicated Estimate, HR (95% CI)
Cox proportional hazards regression
Stratified by sub-group	0.91 (0.83, 1.00)	0.91 (0.83, 0.99)
Stratified by sub-group with adjustment for baseline covariates	0.90 (0.82, 0.99)	0.90 (0.82, 0.99)
Pooled logistic regressiona
Unadjusted	–	0.90 (0.82,0.99)
+ standardization across sub-group	–	0.90 (0.82, 0.99)
+ standardized across sub-group & baseline covariates	–	0.89 (0.82, 0.97)

^a95% confidence intervals (CIs) for pooled logistic regression estimated from 500 bootstrap samples.

We also estimated the hazard ratio using a pooled logistic model [13], which can be easily extended to estimate the standardized (by the baseline variables) absolute risk. The hazard ratio estimates did not materially change regardless of whether we used a Cox or pooled logistic model, and regardless of whether we did or did not standardize for the baseline variables age, sex, number of comorbidities, New York Heart Association functional class, ejection fraction, heart rate, systolic and diastolic blood pressure, body-mass index, having a pacemaker, current smoker, and number of other medications (Table 2). We obtained 95% confidence intervals using 500 bootstrap samples. All intention-to-treat effect hazard ratio estimates were in the range 0.89–0.91. After adding a product term between arm and time to the pooled logistic model, the absolute mortality risk at 3.5 years was estimated to be 1.6 percentage points lower for the candesartan group than for the placebo group (95% CI: −3.5, 0.1).

3. Adherence to placebo and mortality

Adherence information was collected as a categorical variable (< 20%, 20–80%, > 80% of pills taken) and assessed by study clinicians at each study visit. In the original analysis, adherence was set to 0% when data on adherence was missing and after dropout, and the association between adherence to placebo and mortality in the placebo arm was estimated in two separate analyses, as described in the methods section of the CHARM paper [4]: (i) a Cox model with most recent adherence level as a time-varying variable yielded a mortality hazard ratio of 0.64 (95% CI: 0.53, 0.78) for adherence > 80% vs. ≤80%, and (ii) a logistic model for 3.5-year mortality risk with cumulative average adherence by the end of follow-up yielded an odds ratio of 0.54 (95% CI: 0.43, 0.70) for average adherence to placebo > 80% vs. ≤80%. Here we focus on analysis (i).

After having approximately replicated the published estimates (Appendix Table A.1–A.3), we then made the following modifications to our analysis incrementally. For all analyses, we excluded 71 individuals with no adherence information in the placebo arm (for comparison, 65 in the candesartan arm had no adherence information).

First, in an attempt to better adjust for baseline confounding, we added randomization subgroup as a covariate, and modeled comorbidities (history of atrial fibrillation, angina pectoris, coronary artery bypass grafting, cancer, diabetes mellitus, hypertension, percutaneous coronary revascularization, myocardial infarction, and pacemaker) and medications (current use of lipid lowering drugs, aspirin, angiotensin converting enzyme inhibitor, anti-arrhythmic agents, oral anticoagulants, other anti-platelet drugs, and other cardiovascular drugs) separately rather than as count variables. The hazard ratio was 0.71 (95% CI: 0.62, 0.82) (Table 3, analysis 1).

Table 3.

Mortality hazard ratios for adherence vs. no adherence to placebo over 3.5 years, placebo arm of the CHARM randomized trial.

Analysis		Hazard ratio (95% CIs)a
0.	Original reported results [12]	0.64 (0.53, 0.78)
1.	Original analysis plus additional confounding control: (a) include comorbidities and medications as individual variables, (b) control for sub-group, and (c) categorical New York Heart Association functional class	0.71 (0.62, 0.82)
2.	+ standardization over baseline variables	0.73 (0.65, 0.83)
3.	+ alternative handling of missingness: Binary adherence carried forward up to 3 visits then censored when missing (but no adjustment for censoring)	0.58 (0.48, 0.73)
4.	+ censor when adherence changes from baseline	0.94 (0.52, 2.63)
5.	+ adjustment for post-baseline variables	0.91 (0.51, 2.52)
6.	Dose response model for adherence, with baseline adjustment only	1.12 (0.60, 2.59)
7.	Dose response model for adherence, with baseline and post-baseline adjustment	1.08 (0.56, 2.88)

^aEstimated via pooled logistic regression. 95% CIs from 500 bootstrap samples. Original reported results used Cox proportional hazards regression.

Second, we standardized the estimate to the baseline variables. The hazard ratio was 0.73 (95% CI: 0.65, 0.83) (Table 3, analysis 2).

Third, rather than assuming that adherence was ≤80% at the missed visits, we set missing adherence values to the most recent prior adherence value (up to three missed visits) and censored anyone who missed more than 3 visits at the expected time of the 4th visit, as in our previous placebo arm adherence analyses. The hazard ratio estimate moved to 0.58 (95% CI: 0.48, 0.73) (Table 3, analysis 3) [11,14]. This approach relies on a strong assumption that adherence remains constant when not recorded, but our estimates did not change in sensitivity analyses where we varied the time of censoring from 1 to 4 consecutive missed visits (see Appendix Table A.4). An advantage of using this last observation carried forward approach is that the imputation of adherence at missed visits does not depend on any future information, therefore preventing the “reverse causation” bias that would be introduced if the imputation procedure used future information.

Fourth, we censored individuals when their cumulative average adherence level (dichotomized as > 80% vs ≤80%) first differed from their first reported value. That is, we restricted the analysis to individuals who continuously adhered to either > 80% or ≤ 80% of placebo. The hazard ratio estimate moved to 0.94 (95% CI: 0.52, 2.63) (Table 3, analysis 4).

Finally, we adjusted for available post-randomization prognostic factors that predict adherence. We used inverse probability (IP) weighting to adjust for the following time-varying variables: diastolic blood pressure; heart rate; NYHA functional class; current medication use: diuretics, calcium channel blockers, other vasodilators, anti-arrhythmic agents, lipid lowering drugs, aspirin, angiotensin converting enzyme inhibitor, other anti-platelets, and other cardiovascular drugs; and updated medical conditions: coronary artery bypass grafting, diabetes, percutaneous coronary revascularization, stroke, and pacemaker (Appendix Table A.5). The hazard ratio estimate was 0.91 (95% CI: 0.51, 2.52) (Table 3, analysis 5). IP weighting uses a two-stage estimation procedure to appropriately adjust for measured confounders even if those confounders are affected by prior adherence to treatment (Fig. 2) [15]. The stabilized IP weight estimates had a mean of approximately 1, as expected (Appendix Table A.6).

Fig. 2.

Causal directed acyclic graph showing the relationships between adherence, confounders, and mortality over time. The red node (U) and arrows from this node to post-baseline covariates and vital status at the end of the trial indicate potential unknown or unmeasured covariates which would lead to bias in a traditional regression model adjusting for health at visit t, but do not cause bias when health at visit t is adjusted for via inverse probability of weighting. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

As an alternative approach to estimate the mortality hazard ratio, we fit a dose-response model for the average proportion of visits with adherence > 80% through each person-visit. This approach does not require censoring but relies on a correct specification of the dose-response function. The hazard ratio estimates for 3.5 years of adherence was about 1.08 (95% CI: 0.56, 2.88) (Table 3, analysis 7) after adjustment. Sensitivity analyses on the functional form gave similar results, except when dose-response was unrealistically assumed to be linear (HR: 0.5–0.6; see Appendix Table A.4).

In summary, we found no evidence that adherence to placebo is associated with mortality in the CHARM trial, after appropriate adjustment for baseline and post-baseline variables. Although our finding does not guarantee that one can validly estimate the per-protocol effect involving both arms of the trial, it increases confidence in such adjustment. This is because, in a trial in which placebo is not expected to impact the outcome, confirming the null effect of adhering to placebo is a prerequisite for valid estimation of the per-protocol effect.

4. Estimating the per-protocol effect in CHARM

To estimate the per-protocol effect, we need a precise definition of adherence to the protocol. The protocol of the CHARM trial is consistent with several interpretations of what the per-protocol effect should be. We therefore defined two per-protocol contrasts of interest: (a) the effect of being assigned to and then continuously adhering to candesartan versus placebo until incident hypotension and abnormal renal function, and (b) a similar effect with the exception that the treating clinician is allowed to determine if and when it was appropriate to discontinue. Acceptable reasons for discontinuation under this extended protocol include hypotension, abnormal renal function, hyperkalemia, or other physician-recorded events, and exclude discontinuation due to patient’s choice or withdrawal of consent.

To estimate these per-protocol effects we used a similar approach as that described in the placebo arm. At each visit, we considered individuals to be adherent to their assigned treatment (candesartan or placebo) if they had adherence > 80% or if they had discontinued for a protocol-allowed reason at any time up to, and including, that visit. The protocol-allowed reasons differed between the two per-protocol contrasts of interest: for the first, only incident hypotension or abnormal renal function were allowed; for the second, any clinician-approved reason was allowed. Individuals who discontinued their assigned treatment for any other reason were considered no longer adherent and were censored at the time of non-adherence. To adjust for post-randomization factors, we estimated IP weights as for the placebo arm only analysis, except that we fit models for adherence separately in each trial arm and only in the person-time during which an individual was eligible to be censored under the new adherence definition, and we used unstabilized weights because whether an individual was considered adherent to the treatment protocol at each time was dependent on time-varying health status. After an allowed treatment cessation event, an individual’s weight contribution becomes 1 for the remainder of their follow-up.

The mortality hazard ratio estimates ranged from 0.91 to 0.93 for the per-protocol effect estimates (Table 4, analyses 1–3 and 4–6), similar to the intention-to-treat effect estimates.

Table 4.

Mortality hazard ratio for candesartan to placebo over 3.5 years, CHARM randomized trial.

Effect	Method	Risk in candesartan arm, % (95% CI)	Risk in placebo arm, % (95% CI)	Risk difference, % (95% CI)†	Hazard Ratio, (95% CI)
Intention-to-treat	Pooled logistic standardized across sub-group & baseline covariates	23.6 (22.3, 25.1)	25.2 (23.9, 26.6)	−1.6 (−3.5, 0.1)	0.89 (0.82, 0.97)
Per-protocol [#1]	1. Unadjusted	20.8 (19.3, 22.4)	21.5 (19.9, 23.2)	−0.8 (−2.7, 1.5)	0.92 (0.83, 1.03)
Adherence unless protocol-defined discontinuation event*	2. + Standardized across sub-group & baseline covariates	21.1 (19.5, 22.7)	22.1 (20.4, 23.7)	−1.1 (−3.0, 1.0)	0.91 (0.83, 1.01)
	3. + Inverse probability weighting using sub-group, baseline and post-baseline covariates	21.1 (19.7, 22.8)	22.0 (20.3, 23.7)	−0.9 (−3.0, 1.4)	0.93 (0.83, 1.04)
Per-protocol [#2]	4. Unadjusted	22.1 (20.6, 23.7)	23.1 (21.5, 24.6)	−1.0 (−3.1, 1.0)	0.92 (0.84, 1.02)
Adherence unless protocol-defined discontinuation event* or treating clinician recommends discontinuation	5. + Standardized across sub-group & baseline covariates	22.2 (20.7, 23.8)	23.3 (21.8, 24.8)	−1.1 (−3.1, 0.9)	0.92 (0.83, 1.01)
	6. + Inverse probability weighting using sub-group, baseline and post-baseline covariates	22.5 (21.0, 24.3)	23.9 (22.3, 25.6)	−1.3 (−3.5, 0.8)	0.92 (0.83, 1.02)

95% CIs from 500 bootstrap samples.

^*Protocol-defined discontinuation events: incident hypotension, Δcreatinine ≥0.5 mg/dL.

^†Models for estimating risk difference include product term between trial arm and time.

See Appendix Table A.7 for sensitivity analyses in which we varied the time of censoring due to loss to follow-up, used IP weights for loss to follow-up, and used IP weights with stabilization based on baseline covariates and adherence only. Note that an alternative per-protocol analysis based on a dose-response model is not advisable when the treatment strategies are, as in this application, dynamic [16]. Correctly specifying a dose-response model of adherence for dynamic strategies is difficult because there are multiple ways of obtaining the same level of adherence. For example, both an individual who adheres to their full assigned dose of candesartan for the entire follow-up and an individual who discontinues after 6 months due to incident hypotension have 100% adherence to their assigned protocol.

Fig. 3 gives the standardized cumulative mortality risk over follow-up from the intention-to-treat analysis and the two per-protocol analysis using inverse probability weighting to compare continuous protocol adherence to non-adherence. In general, the absolute mortality estimates in the per-protocol analysis where discontinuation was only allowed for hypotension or abnormal renal function were lower than those in the intention-to-treat analysis and the less restrictive per-protocol analysis.

Fig. 3.

Cumulative incidence of mortality over 3.5 years for Candesartan versus placebo (standardized over sub-group and baseline covariates), CHARM randomized trial. Solid lines: Candesartan; Dashed lines: Placebo. (a) Intention-to-treat analysis; (b) Per-protocol analysis, when discontinuation is allowed following incident hypotension, or abnormal renal function only; (c) Per-protocol analysis, when discontinuation is allowed following incident hypotension, abnormal renal function, or treating clinician recommends discontinuation. Intention-to-treat effect estimate adjusted for baseline covariates and sub-study; per-protocol effect estimates adjusted for baseline and post-baseline covariates.

Finally, we also estimated per-protocol effects for secondary outcomes of interest reported in the CHARM sub-trials [17–19]. The per-protocol estimates for these outcomes were similar to the intention-to-treat estimates (Appendix Table A.8).

5. Discussion

Adjusting for adherence in randomized trials is often viewed with skepticism because of the perception of intractable confounding by post-randomization predictors of adherence and trial outcomes. To allay some of these concerns, we have presented a two-stage procedure to estimate per-protocol effects in placebo-randomized trials in which the outcome of interest is expected to be unaffected by the placebo. First, we exploit the information about adherence in the placebo arm to demonstrate that the available data are sufficient for confirm that the estimated effect of adherence to placebo is null. Second, we estimate the per-protocol effect for active treatment vs. placebo using a similar analytic approach as in the placebo arm only analysis.

The use of inverse probability weighting to estimate the per-protocol effect requires data on, and assumptions about, prognostic factors which are also predictive of non-adherence. When estimating per-protocol effects in randomized trials with point interventions, these assumptions may be unnecessarily strong, and the use of randomization as an instrument for the treatment effect is sometimes preferred. However, in a randomized trial such as CHARM where the assigned treatment is intended to be sustained throughout follow-up methods the required assumptions for instrumental variable analyses cannot be met. Whenever the assigned treatment is intended to be delivered over time, methods such as inverse probability weighting which adjust for time-varying treatments are required for estimating the per-protocol effect [3].

In the CHARM trial, like in the CDP trial before [10,11,14], reasonable assumptions about missing adherence data and appropriate adjustment resulted in a null association between adherence and mortality in the placebo arm. Although successfully estimating a null adherence effect in the placebo arm does not guarantee that our per-protocol effect estimate will be unbiased, it provides evidence in support of some of the assumptions that are needed for the validity of the per-protocol effect estimate which, in CHARM, was similar to the original intention-to-treat estimate.

In the CHARM trial, unlike in the CDP trial, relaxing the assumptions about adherence missingness used in the original analysis was sufficient to reduce the association between placebo arm adherence and mortality nearly to the null. Adjustment for post-randomization factors played a minor role (and, consistently, unadjusted per-protocol estimates were similar to the adjusted ones) probably because protocol deviations were not explained by variations in prognostic factors. Other prognostic factors shown to be associated with adherence in the literature were not available in CHARM, including behavioral factors, social support (for example “living alone”), education level, or social determinants of health and other socio-economic factors.

On the other hand, the importance of appropriately handling missing adherence data reflects the high correlation between missingness and mortality. In CHARM, the mortality odds ratio comparing visits without versus with adherence measurements was 2.2 (95% CI: 2.0, 2.5). Trials which found a non-null association between adherence and mortality in the placebo arm [4–10] tended to make extreme assumptions about missing data whereas the only other trial that, to the best of our knowledge, found no association between the outcome and adherence in the placebo arm relaxed the strong assumptions about missing data via multiple imputation [9].

Our re-analysis of the CHARM trial suggests several ways to improve the reporting of randomized trials.

First, to ensure that trial results are replicable, authors need to share their annotated computer code for data cleaning and analysis [20]. A description of the statistical approach in the Methods section of a published paper was insufficient for us to exactly replicate some components of the analysis of CHARM. This is not unexpected as the analysis of complex trial data requires multiple decisions regarding the definition and handling of missing values, dropout, adjustment variables, adherence, etc.

Second, estimation of per-protocol effects in randomized trials requires careful specification of the protocol-defined rules for treatment discontinuation. We used the information provided in the protocol of CHARM to propose two possible definitions of per-protocol effect. Ideally, the treatment strategies should be clearly specified in the trial protocol.

Third, estimation of per-protocol effects in randomized trials generally requires the collection of pre- and post-randomization data on adherence, clinical events, and prognostic factors. Since the CHARM investigators collected rich data on adherence and post-randomization covariates, we were able to successfully adjust for baseline and time-varying confounders for non-adherence and estimate a per-protocol effect. Ideally, trials would also collect information on reasons for non-adherence from participants. Reasons for non-adherence can provide important information for individuals and clinicians outside the trial when engaging in shared decision-making [21]. However, the types of data needed to understand reasons for non-adherence may be challenging to specify a priori and only those reasons which are also prognostic of the outcome are likely to cause confounding. Therefore, when limited resources for data collection are available, focusing on prognostic factors may be preferred [2].

In return for the extra effort, estimation of the per-protocol effect can provide valuable supplementary information to help patients and clinicians understand the benefits and risks of treatment decisions [21].

Research in context

Evidence before this study

Per-protocol effect estimates are often viewed with skepticism, because of a belief that adherence adjustment is hopelessly biased. In 1980, a landmark paper from the Coronary Drug Project found a large survival advantage for placebo arm adherers compared with non-adherers. This analysis, which had a chilling impact on the estimation of per-protocol effects, has been replicated in other placebo-controlled trials over the past 40 years, including the Candesartan in Heart Failure: Assessment of Reduction in Mortality and Morbidity (CHARM) trial. However, a recent re-analysis of the Coronary Drug Project showed that state-of-the-art analytic methods can successfully adjust for post-randomization predictors of adherence and remove the so-called ‘healthy adherer bias’.

Added value of this study

Here, we present a re-analysis of the CHARM trial that, contrary to previous methodological approaches, shows a null association between adherence and mortality in the placebo arm. We then apply a similar analytic approach to estimate the per-protocol effect in the whole trial. We argue that trial placebo arms provide an indirect test of the validity of the assumptions required for valid estimation of per-protocol effects. The per-protocol effect provides important context about the effect of treatment if everyone had adhered to the protocol, and may be more generalizable than the intention-to-treat effect which depends on the specific level of adherence observed in a given trial.

Implications of all the available evidence

Per-protocol effects add valuable information to randomized trial reports. Despite previous studies suggesting that per-protocol effects are intractably confounded, we provide evidence to support its validity when using appropriate analytic methods.

Availability of data and material

SAS code for all analyses is available at https://github.com/eleanormurray/CHARM_reanalysis. Data is available upon application to the CHARM coordinating center at the Department of Non-invasive Cardiology, Harvard Medical School, Boston MA, USA.